More investigation similar to alog80358
NLN 1412064049 lockloss tool, no tag, FSS chans looks stable.
AS_A_DC and IMC-TRANS loose lock within ~250 ms
PREP_DC_READOUT_TRANSITION 1412066248 lockloss tool, FSS tag
AS_A_DC and IMC-TRANS loose lock within 1 ms of each other, but the FSS chans look stable before hand
NLN 1412077192 lockloss tool, tool failed
AS_A_DC and IMC-TRANS loose lock within ~5 ms, small FSS glitching starts almost 2 mins before LL, large glitch 1/2 a sec before
NLN 1412084422 lockloss tool, No FSSery
AS_A_DC and IMC-TRANS loose lock within ~250 ms
NLN 1412092628 lockloss tool, FSS tag
AS_A_DC and IMC-TRANS loose lock within ~5 ms, FSS glitching starts 5 secs before LL
NLN 1412101888 lockloss tool, FSS tag
AS_A_DC and IMC-TRANS loose lock within ~250 ms
TRANSITION_FROM_ETMX 1412105312 lockloss tool, FSS tag
AS_A_DC and IMC-TRANS loose lock within ~ 13 ms, the FSS starts glitching 40 seconds before LL, FSS_PC_MON_OUTPUT starts rising ~40 seconds before LL
LOW_NOISE_ESD_ETMX 1412113952 lockloss tool, no FSSery
AS_A_DC and IMC-TRANS loose lock within ~250 ms
NLN 1412131865 lockloss tool, FSS tag
AS_A_DC and IMC-TRANS loose lock within ~1 ms, there's a FSS glitch 1 second before the LL
To summarize, Ryan looked at 8 locklosses from today, 6 of them had the FSS tag, 5 of them had the IMC loose lock withing 15ms of the IFO losing lock, and only 3 of them had the pattern that Ian and Camilla found was normal in O4a, where the IMC looses lock ~250 ms after the IFO losses lock. This FSS issue seems to be having a serious impact on our duty cycle.
I had a helpful conversation with Rick this afternoon about this issue, he sugested that we trying looking for the glitches in other places in the PSL, so that we could think about tests where we don't run some of the feedback servos and see if we still see the issue. I made an ndscope template with all the PSL fast channels I could find, it is saved in sheila.dwyer/ndscope/PSL/PSL_fast_channels.yaml
Looking at 1412111658 with this scope, it looks like several of these channels that we are saving at a fast rate might not be connected, for example the OSC and HPL channels are around 0. The only other channel that clearly shows these glitches is H1:PSL-PWR_NPRO_OUT_DQ.
Request for operators: If you are having locking difficulties over the weekend, it may be due to the PSL/FSS problems. To help narrow down the cause of the problem one could sit for an hour or two with the FSS unlocked, and the ISS unlocked. If the glitches show up in the NPRO channel even without these servos locked, then we will know that the problem is not from the FSS, but from the NPRO.
