Turns out that I had forgotten to turn the input on for the PR3 P estimator contribution from L (H1:SUS-PR3_M1_EST_P_FUSION_MODL_DRV_L_2GAP) (thanks to Edgard and Brian for having us even consider the estimators!), and that was causing the 0.6 Hz ASC ringups (87658). Even though we're pretty sure we found the cause, we've decided to temporarily turn off the PR3 and SR3 L and P estimators until we're back up and can verify that they are now working correctly.

TITLE: 10/23 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
INCOMING OPERATOR: Ryan C
SHIFT SUMMARY: Currently relocking and at LOWNOISE_COIL_DRIVERS. Still struggling with a 1 Hz oscillation(87658), but we've solved issues with MICH phase margin and with a 0.6 Hz oscillation caused by incorrect estimator settings, so 2/3 of the issues solved?
LOG:

14:30UTC Started relocking
    Initial alignment
    Lockloss from OMC_WHITENING due to 1 Hz ringup
    Multiple locklosses from DRMI and under due to ALSX getting noisy and ALSY having some glitchy drops in power
    Locklosses from LOWNOISE_LENGTH_CONTROL, MOVE_SPOTS, etc.                                                                                                                                                                

We managed to track down the source of the 0.6 Hz oscillation that we see at move spots and full power. It seems to be the result of the new pitch and length estimators on PR3 and SR3 that were installed on Tuesday. We tested this by sitting at move spots (after the spots converged), and turning on and off the estimators for a few minutes at a time. When turning ON the estimators, the 0.6 Hz ringing is not immediately apparent- it will slowly grow. However, once the 0.6 Hz oscillation is visible in the controls, turning OFF the estimator immediately stops the ringing. Therefore, we think this is the culprit for this instability. Oli will be turning off the length and pitch estimators for PR3 and SR3 for now. The yaw estimators are still on because they have been operating stably for a long time and were not changed this Tuesday.

We have also put the factor of 2 increase of MICH2 gain into the guardian, since we kept forgetting to add it in. Several locklosses that we thought were due to various other things were actually due to the MICH loop ringing up.

Finally, we just had a lockloss after moving to camera servos because the 1 Hz oscillation began to grow. Here is everything I tried to stop the ringing (none of it worked):

• CSOFT P gain was already at 30. I dropped it to 25 and re-increased to 30 several times, no change either way
• INP1 P gain was at -1, I doubled it and then tripled it, no change
• INP1 Y gain was at -0.1, doubled it, no change
• DSOFT P gain was at 5, I increased to 10, then 15, no change
• DSOFT Y gain was at 5, increased to 10, no change

I then had an idea to go to high bandwidth ASC using the seismic script, but we lost lock as soon as I thought of that.

Next plan: go to lownoise length control, skip lownoise asc, go to the final spots and camera servos, stay in HBW ASC and wait for the 1 Hz ring up and see if we can ride it out this way.

Attached scope screenshot shows the 0.6 Hz oscillation disappearing as soon as Oli flipped the length and pitch estimator switches OFF.

TITLE: 10/22 Eve Shift: 2330-0500 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
OUTGOING OPERATOR: Oli
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 21mph Gusts, 17mph 3min avg
    Primary useism: 0.05 μm/s
    Secondary useism: 0.30 μm/s 
QUICK SUMMARY:

• Microseism is decreasing over the past ~14 hours
• Wind has been rising the past 2.5 hours
• We're at MAX_POWER currently

Jennie W, Sheila

I noticed yesterday that the power reported as being reflected from the OMC is more than the power reported entering HAM6 (see this alog #87629). This does not affect any controls, but it does affect anyone who is using this channel to do comissioning tests.

Sheila checked this alog for the calibration of OMC-REFL_A PD, alog #65082. It looks like there should be a -20dB switch on in the filter bank when the gain is set to 20dB:

FM7/8 should be the inverse of H1:OMC-REFL_A_DC_GAINSETTING (e.g. enable "-20dB" if the gain is 20dB)

Checking the screen we found the gain setting was set to 0 on the front-end side but the '-20dB filter bank is still turned on. We have turned this filter bank off (see image with relevant parts circled), but the front-end H1:OMC-REFL_A_LF_OUT value still does not match the Beckhoff value, H1:OMC-REFL_A_DC_POWER.

Looking back at our last long lock (when the -20dB filter was still on) you can see that the OMC-REFL_A_LF_OUT16 and OMC-REFL_A_DC_POWER are off by a factor of 100 even though they are both supposed to be in mW. Therefore our switch off of the -20dB filter will not correct this problem and the rest of the calibration of this PD in the front-end filters needs more investigation.

Wed Oct 22 10:08:38 2025 INFO: Fill completed in 8min 34secs

Gerardo confirmed a good fill curbside

Sheila, TJ, Oli, Matt

We reverted the ASC gain changes (87582) setting INP1 P to 1 and CSOFT P to 25 at MAX power (rather than 30).  Most of our locklosses overnight were from states before Max power, so the CSOFT P gain can't have been the problem, but we are undoing it for now just for the sake of reverting recent changes. 

We looked at one of the overnight locklosses, it seems that MICH is ringing up at 4.7 Hz.  Since Oli is sitting at 2W DCreadout to damp violin modes, we measured the MICH OLG there, and we see it is pretty close to unstable at 5Hz, with low coherence there.  We added a factor of 2 gain in LSC MICH2, to bring the UGF to 10 Hz similar to the references, and give us 35 degrees phase margin. 

Next we checked the SRCL UGF, which is also locked on POP45, and it's gain looks low by a factor of 5 compared to the reference.  This loop has a large phase bubble so it's still stable. POP 45 whitening gain and filters seem to be the same as a few days ago, and the POP LF power is the same as it was a few days ago at 2W.  PRCL gain also looks low compared to the reference, but is stable.  

We were able to get to OMC_WHITENING (almost to low noise), but while we were sitting there we had the old ring up.  We tried manually increasing the CSOFT P gain to 30, but the osciallation had already grown large and we lost lock anyway.  I've now put the csoft P gain back to 30 in max power in the guardian.  

Something happened with BRSY this morning during maintenance that caused it to ring up more than normal and now the damping is not behaving quite as expected. For now, I have paused the ETMY sensor correction guardian with the BRSY out of loop and turned off the output of the BRS so it won't be used for eq mode, should that transition happen.

So far today, I did a bunch of "recapturing frames" in the BRS C# code, which has usually fixed this issue in the past. We also restarted the beckhoff computer, then the plc, C# and epics ioc. This did not recover the BRS either. Marc, Fil and I went to EY and looked at the damping drive in the enclosure and I think it was behaving okay. When the drive came on, the output would reach ~1.8V, then go to 0V when it turned off.

I've contacted UW and we will take a look at this again tomorrow. 

[Joan-Rene Merou, Alicia Calafat, Anamaria Effler, Sheila Dwyer, Robert Schofield, Jenne Driggers]

We have looked at the near-30 Hz and near-100 Hz combs (Detchar issue 340) in all of LHO Fscan channels (Full 148 channels list can be found in O4_H1_Fscan_ch_info.yml) to find witnesses and also channels where the amplitude and coherence change at the same dates as DARM.

The list of combs is the following one:


  

    

      
spacing (Hz)
      
offset (Hz)
    
  
  

    
29.9695138888
0
    
99.99845486125
70.02888889
    
99.99845679
0
    
99.99846
29.9694
    
99.9984722225
89.90847222
    
99.99865
0
    
99.99845
10.08992
    
29.96952
520.17208
    
29.9695211
589.9007589
    
29.96951374
760.22840625
  


As reported by Xuanye Fan in Detchar issue 296, "it can be determined that the high-order harmonics disappeared on April 8, 2024, and reappeared on July 7, 2024". Hence, we have compared coherence with DARM in all the channels and the difference between July 1st and July 7th 2024.

Following the findings by Xuanye, we have redone the comb time-evolution plots for all of these combs in order to check for dates where their amplitude changed. As can be seen in the plots below, there are two clear dates where changes in the behavior of the combs took place, April ~4th 2024 and July ~7th 2024. The attached plots to this post are similar to the ones generated by Fscan using FscanFineToothPlot plots, however, they have two differences: They use data from the non-normalized spectras instead of from the time average plots. Additionally, they compute the relative harmonic amplitude with respect to the median of the surrounding 1 Hz spectrogram. The attached plots confirm that there is a significant change in comb amplitude between April 4th and July 7th 2024. One clear example is shown here:



We list here the channels that do show most of these Combs. These same channels do show changes in coherence between July 1st and July 7th 2024, but do not show changes in the amplitude of the combs.

- H1_IMC-F_OUT_DQ
- H1_LSC-MCL_IN1_DQ
- H1_LSC-MICH_IN1_DQ
- H1_LSC-SRCL_IN1_DQ
- H1_PEM-CS_MAG_EBAY_LSCRACK_X_DQ
- H1_PEM-CS_MAG_EBAY_LSCRACK_Y_DQ
- H1_PEM-CS_MAG_EBAY_LSCRACK_Z_DQ
- H1_PEM-CS_MAG_LVEA_INPUTOPTICS_X_DQ
- H1_PEM-CS_MAG_LVEA_INPUTOPTICS_Y_DQ
- H1_PEM-CS_MAG_LVEA_INPUTOPTICS_Z_DQ

In most channels, the comb amplitude tends to get quite low after ~1500 Hz. The following sets of channels show differences between X, Y and Z:

- H1_PEM-CS_MAG_EBAY_SUSRACK_X_DQ (Higher amplitudes and towards higher frequencies)
- H1_PEM-CS_MAG_EBAY_SUSRACK_Y_DQ (Lower comb amplitudes)
- H1_PEM-CS_MAG_EBAY_SUSRACK_Y_DQ (Lower comb amplitudes)

Regarding CS_MAG_LVEA_OUTPUTOPTICS, these combs can be seen best in X, weaker in Y and almost non-existent in Z. (In CS_MAG_LVEA_INPUTOPTICS they look roughly the same height)

- H1_PEM-CS_MAG_LVEA_OUTPUTOPTICS_X_DQ (Strongest)
- H1_PEM-CS_MAG_LVEA_OUTPUTOPTICS_Y_DQ (Weaker lines)
- H1_PEM-CS_MAG_LVEA_OUTPUTOPTICS_Z_DQ (Almost no lines)

Same behavior at:
- H1_PEM-CS_MAG_LVEA_VERTEX_X_DQ (Strongest lines)
- H1_PEM-CS_MAG_LVEA_VERTEX_Y_DQ (Weaker lines)
- H1_PEM-CS_MAG_LVEA_VERTEX_Z_DQ (Almost no lines)

We can see that these combs mostly appear in the corner station. The combs do not appear in neither EX nor EY channels. However, comb 99.99865 Hz offset 0.000 appears in many EX, EY channels and does become more coherent after July 7th. However, it is very close to 100 Hz so it may be influenced by other round-number combs (?)