Jenne asked me to check a line around 27 Hz, this does show up in max power  (today see brown trace) but also was there in max power before maintenance Tuesday this week (pink trace).

It doesn't seem to really show up in the NLN state (red trace is before maintenance, blue is one of our locks from early this morning).

My conclusion from Jennie's plot is that that 27 Hz mode isn't a major cause for concern; it's not something new that's giving us trouble.

Locking attempt 1 this morning:  Locked to full power, Oli ran the asc high bandwidth script (userapps/isi/h1/scripts/revert_asc.py) premptively to avoid having problems with the 1 Hz ring up.  

The last thing this script does is sets the LSC FF gains to 0, 45 seconds after this finished we had a sudden large increase in noise first at 300 Hz in DARM (ETMX ESD) and DHARD then 0.1 second later in CHARD, which unlocked the IFO.  This happened in the close beam diverters state, but is probably not related to the beam diverters.  This is a different problem than what we've been seeing overnight. 

No gains or anything else in the CHARD or DHARD filter banks were changed in those 45 seconds either, which rules out some other rogue change

Elenna and I commented out the DHARD boost from low noise ASC, because it takes 45 degrees of phase at 1Hz and so leaving it off may help our improve our 1Hz problems.  This boost was added to improve the stationarity of SRCL to DARM coupling by reducing the DAHRD RMS 77857.  We are leaving it off for now because we can probably go to observing without it if we can stay locked.   

Next locking attempt we sat for over an hour at full power, without doing the LOWNOISE ASC step.  

We had ETMY saturations when we ran through the ESD transitions, and they continued after DARM was switched, until Elenna turned off LSC FF.  When we turned the feedforward back on ETMY didn't start saturating.  However, at the time that the PRCL and SRCL feedforward came on, the roll mode started ringing up.  We lost lock a few minutes into this ring up, but the amplitude was much smaller than the locklosses from this mode overnight (about a factor of 100 lower).  

We aren't sure if the roll mode was the cause of the lockloss or not.

Called Rahul for help with the EY Roll Mode

The plan:

• Move to NLN very very slowly stopping at every stage of powerup and watching for violins
• Once at NLN watch for the 1Hz ringup and activate High Gain settings

The issue with this plan is that it seems that without the high gain on, the oscillation prompts a lockloss but we'll see what happens as I watch it closely this time.

Lost lock at OMC_WHITENING after spending ~10-15 minutes at each powerup state to ensure EY Roll mode and Violins were low. They were. The 1Hz ringup started on queue as soon as LOWNOISE_ASC turned on and got worse. I think I turned on the ASC High Gain too early - confused as to what caused this lockloss. Trying again. It seems that the slow method works to avoid the EY Roll Mode ringup.

Additionally, Elenna's alog 87658 talks of her plan for how to try to avoid the 1Hz ringup and it involves DHARD_P to HBW using turning off FM8, then FM4. I will do this instead of the entire HBW ASC EQ script.

Lockloss tool sees EY glitching 1.2 before lockloss.

After some DRMI troubles, finally back to where I was before. In about 10 minutes, EY Roll Mode permitting, I will attempt to go to NLN and watch the 1Hz ringup, turning on HBW DHARD_P when I see the ringup.

It worked! (for now). We're in NLN with bad SQZ though similar range to last time we were locked (which isn't that good). The EY Roll Mode is on the rise though and if this lock is like the ones Ryan C (EVE) had, then usually they rise until lockloss in ~45 mins to an hour. I'm going to give IFO at least 10 minutes before re-engaging the DHARD_P LBW. The ringup comes back thereafter according to my last lock and Ryan's too. So, let's see what happens.

Really doesn't look great especially that as soon as we're near NLN, the EY Roll mode begins to trend upwards. Violin modes are going down though.

Well... The mode has rung up again after switching FM4 and FM8 back on... I don't know what to do here. Meanwhile EY Roll Mode is still ringing up despite locking veeery slowly per Rahul's advice.

EY Mode 15 is rung up slightly and damping it per the settings has made things worse - resetting its gain to 0 seems to calm it down.

My not-very-creative-out-of-options plan now is to wait here in these settings and attempt to go back to LBW DHARD P in another 20/30 or so... However, by then we will probably have lost lock due to Roll EY ::(

As expected when I went back to LBW, DHARD P rung up again so that doesn't work either.

While violins are high, they've been much higher before so we're not really ringing them up - they're all reporting as damping now anyway.

OMC DCPDs are going quite wild I assume due to this Roll Mode (attached).

If we lose lock here, I'm just going to IDLE IFO due to inability to stay locked and to avoid any more violin mode ringup. On the bright side, it's nearing DAY shift, involving well-rested experts.

Lockloss 11:51 UTC - IFO in IDLE.

Thanks for catching this, Oli. I made a note in FRS ticket 35410 reminding us to make an installation script which automatically sets all the epics variables so we are more likely to get all the right settings from the beginning. 

Here's the measurements taken for MICH that show the phase margins at different points in the locking sequence

We have been thinking that maybe the MICH UGF is too low (I had a memory that it was supposed to be 8 Hz). Then I found this alog from Evan where he designed it to be 5.5 Hz and the phase margin looks about the same 76631. Note that Oli's measurement was taken after we DOUBLED the gain, so somewhere along the way we lost 6 dB of gain.

I held us at high bandwidth ASC for 9 minutes and there was no 1 Hz oscillation. A few minutes after running lownoise ASC the 1 Hz oscillation came back. It begins first in DHARD P. I stopped the oscillation by running the ASC HI GAIN script that we set up for earthquakes. It looks like the oscillation stopped as soon as DHARD P was put into HBW state. After waiting a few minutes, I put us back into the lownoise state. I have previously been suspicious of DHARD P for being the culprit causing the 1 Hz ring up. My recommendation is that if we see this again, try turning OFF FM8 in the DHARD P filter module (labelled GVBoost). If that doesn't work, just run the HI GAIN ASC script like you would for an earthquake, on the IS sensor config screen.

EDIT: I posted this comment too soon- the 1 Hz ringing came back. I tried just turning off DHARD P FM8, but that did nothing. However, going to Hi Gain ASC worked again, so that seems to be a good way to get around this (for now).

I've been flipping back and forth between high and low bandwidth DHARD P. The 1 Hz definitely begins when we are in LBW, and it seems to actually be the lowpass/reshaping in FM4 that is the issue.

I also noticed that there is a glitch in DARM whenever FM4 is turned on or off, so I increased the filter ramp time from 15 to 20 seconds.

To take DHARD P to HBW, disengage FM8, wait for the filter to ramp, then disengage FM4.

To take DHARD P to LBW, engage FM4, wait for the filter to ramp, then engage FM8.

I tried to measure DHARD P to understand what the instability is but we had a lockloss. Looks like it's an ETM glitch?

My recommendation is to put DHARD P in HBW if the 1 Hz starts oscillating again.

00:43 UTC We reached OMC_WHITENING where we will sit to damp violins for a bit.

IX13 has been the problem mode, I've set its' gain to 0 in lscparams and reloaded the VIOLIN node. I think I have found a new setting for IX13, I changed the phase from -60 to-30 and used a negative gain. Its damping, slowly...

We've also noticed that the ETMY roll mode is ringing up, turning off the damping made it ringup faster.

19:31 UTC Observing

FYI -- we went into observing with the ITMY PUM sat amp OSEM PD's still dead. We turned OFF the ITMY L2 (PUM) to R0 "damping" feedback loop, so the failed sensors *should* not have any impact on the IFO. However, an obvious, loud ~92-93 Hz feature remained new and glaringly obvious in the DARM sensitivity. 

@DetChar / @CW Group Probably best to just veto this observation stretch, unless we -- in the future -- find some amazingly exceptional event. 
     2025-10-16 19:31 UTC to 19:54 UTC.

ITMY L2 (PUM) Sat Amp S1100148 installed on 10/14/2025. Replaced on 10/16/2024 with S1100080.

https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=87469

Here's the characterization data and fit results for S1100148, assigned to ITMY L2's ULLLURLR OSEMs.

This sat amp is a UK 4CH sat amp, D0900900 / D0901284. The data was taken per methods described in T080062-v3, using the diagrammatic setup shown on PAGE 1 of the Measurement Diagrams from LHO:86807.

The data was processed and fit using ${SusSVN}/trunk/electronicstesting/lho_electronics_testing/satamp/ECR_E2400330/Scripts/
plotresponse_S1100148_ITMY_L2_ULLLURLR_20250917.m

Explicitly, the fit to the whitening stage zero and pole, the transimpedance feedback resistor, and foton design string are:

The attached plot and machine readable .txt file version of the above table are also found in ${SusSVN}/trunk/electronicstesting/lho_electronics_testing/satamp/ECR_E2400330/Results/
2025-09-17_UKSatAmp_S1100148_D0901284-v5_fitresults.txt

Per usual, R_TIA_kOhm is not used in the compensation filter -- but after ruling out an adjustment in the zero frequency (by zeroing the phase residual at the lowest few frequency points), Jeff nudged the transimpedance a bit to get the magnitude scale within the ~0.25%, shown in the attached results. Any scaling like this will be accounted for instead with the absolute calibration step, i.e. Side Quest 4 from G2501621, a la what was done for PR3 and SR3 top masses in LHO:86222 and LHO:84531 respectively.

Here's the characterization data and fit results for S1100119, assigned to ETMX L2's ULLLURLR OSEMs.

This sat amp is a UK 4CH sat amp, D0900900 / D0901284. The data was taken per methods described in T080062-v3, using the diagrammatic setup shown on PAGE 1 of the Measurement Diagrams from LHO:86807.

The data was processed and fit using ${SusSVN}/trunk/electronicstesting/lho_electronics_testing/satamp/ECR_E2400330/Scripts/
plotresponse_S1100119_ETMX_L2_ULLLURLR_20250916.m

Explicitly, the fit to the whitening stage zero and pole, the transimpedance feedback resistor, and foton design string are:

The attached plot and machine readable .txt file version of the above table are also found in ${SusSVN}/trunk/electronicstesting/lho_electronics_testing/satamp/ECR_E2400330/Results/
2025-09-16_UKSatAmp_S1100119_D0901284-v5_fitresults.txt

Per usual, R_TIA_kOhm is not used in the compensation filter -- but after ruling out an adjustment in the zero frequency (by zeroing the phase residual at the lowest few frequency points), Jeff nudged the transimpedance a bit to get the magnitude scale within the ~0.25%, shown in the attached results. Any scaling like this will be accounted for instead with the absolute calibration step, i.e. Side Quest 4 from G2501621, a la what was done for PR3 and SR3 top masses in LHO:86222 and LHO:84531 respectively.

Here's the characterization data and fit results for S1100118, assigned to ITMX L2's ULLLURLR OSEMs.

This sat amp is a UK 4CH sat amp, D0900900 / D0901284. The data was taken per methods described in T080062-v3, using the diagrammatic setup shown on PAGE 1 of the Measurement Diagrams from LHO:86807.

The data was processed and fit using ${SusSVN}/trunk/electronicstesting/lho_electronics_testing/satamp/ECR_E2400330/Scripts/
plotresponse_S1100118_ITMX_L2_ULLLURLR_20250916.m

Explicitly, the fit to the whitening stage zero and pole, the transimpedance feedback resistor, and foton design string are:

The attached plot and machine readable .txt file version of the above table are also found in ${SusSVN}/trunk/electronicstesting/lho_electronics_testing/satamp/ECR_E2400330/Results/
2025-10-14_UKSatAmp_S1100118_D0901284-v5_fitresults.txt

Per usual, R_TIA_kOhm is not used in the compensation filter -- but after ruling out an adjustment in the zero frequency (by zeroing the phase residual at the lowest few frequency points), Jeff nudged the transimpedance a bit to get the magnitude scale within the ~0.25%, shown in the attached results. Any scaling like this will be accounted for instead with the absolute calibration step, i.e. Side Quest 4 from G2501621, a la what was done for PR3 and SR3 top masses in LHO:86222 and LHO:84531 respectively.

Here's the characterization data and fit results for S1100127, assigned to ETMY L2's ULLLURLR OSEMs.

This sat amp is a UK 4CH sat amp, D0900900 / D0901284. The data was taken per methods described in T080062-v3, using the diagrammatic setup shown on PAGE 1 of the Measurement Diagrams from LHO:86807.

The data was processed and fit using ${SusSVN}/trunk/electronicstesting/lho_electronics_testing/satamp/ECR_E2400330/Scripts/
plotresponse_S1100127_ETMY_L2_ULLLURLR_20250916.m

Explicitly, the fit to the whitening stage zero and pole, the transimpedance feedback resistor, and foton design string are:

The attached plot and machine readable .txt file version of the above table are also found in ${SusSVN}/trunk/electronicstesting/lho_electronics_testing/satamp/ECR_E2400330/Results/
2025-09-16_UKSatAmp_S1100127_D0901284-v5_fitresults.txt

Per usual, R_TIA_kOhm is not used in the compensation filter -- but after ruling out an adjustment in the zero frequency (by zeroing the phase residual at the lowest few frequency points), Jeff nudged the transimpedance a bit to get the magnitude scale within the ~0.25%, shown in the attached results. Any scaling like this will be accounted for instead with the absolute calibration step, i.e. Side Quest 4 from G2501621, a la what was done for PR3 and SR3 top masses in LHO:86222 and LHO:84531 respectively.

ITMY L2 (PUM) Sat Amp S1100148 installed on 10/14/2025. Replaced on 10/16/2024 with S1100080.

https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=87515

Note that the satamp for ETMY L2 has now been swapped back to a S1100137, which has been modified for ECR E2400330. 

alog: 87722