TITLE: 10/24 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
OUTGOING OPERATOR: None
CURRENT ENVIRONMENT:
    SEI_ENV state: USEISM
    Wind: 8mph Gusts, 6mph 3min avg
    Primary useism: 0.04 μm/s
    Secondary useism: 0.62 μm/s 
QUICK SUMMARY: H1 was down overnight due to ongoing locking issues. I'll start by running through an initial alignment while I get caught up on events of the past couple days.

• Wind low, microseism steady around 90th percentile
• DR1 dust monitor reports no change in counts
• All other systems nominal

TITLE: 10/24 Eve Shift: 2330-0500 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
INCOMING OPERATOR: Tony
SHIFT SUMMARY: Microseism is still slowly trending up, I got up past MAX_POWER twice, I confirmed the Roll mode damping works at 62Ws.
LOG: No log.

Lock^#1

• We sat in ADJ_POWER for a little over an hour at 50Ws as Rahul damped the Roll mode. It seems to be on ETMX. We manual-ed  to LOWNOISE_COIL_DRIVERS and later up to LOWNOISE_LENGTH_CONTROL and ETMY started to saturate (it looked to be coming from SUSETMY DAC  1, channels 1, 2, 3. We noticed that the DCPDs were growing as well and PIMON was showing some red. We then went back to ADJ_POWER and lowered the power down to 45Ws, the saturations stopped but the PI kept growing till we lost lock. Lowering the power looks to have made the PI ringup even faster?
• I set the gain of the ETMY roll mode to zero in ISC_LOCK and reloaded it. The Thermalization GRD went into error as we went into LOWNOISE_LENGTH_CONTROL since the SRCL1 offset was not at the nominal, I set the nominal back and reloaded it and it did not fix the error.

Lock^#2

• 02:42 UTC IA
• 03:48 UTC I got back to ADJ_POWER and brought us down from 62 to 40W
• 04:17 UTC Shortly after I brought us back up to 62 at Sheilas' suggestion and tried Rahuls new Roll mode damping settings and they seemed to work at MAX_POWER, after 6 or 7 minutes we tried to go to LOWNOISE_LENGTH_CONTROL. We lost lock shortly after during TRANSITION_FROM_ETMX. Sheila accepted the filter changes for the new Roll mode settings.
• 04:41 UTC Holding in DOWN

I modified Oli's script  make_EST_model.m that gets the OSEM estimator filters into the front-end models. This script is now the preferred way to get any estimator model filters into the real-time computers. The change was made to add failsafes to avoid further errors on the translation of fitted filters into foton ( see LHO: 87689 ).

For the script to work, any .mat files with estimator filters must have the following syntax:

The format for the filters can be seen in any of the .mat files listed below that live in (svnroot)/sus/trunk/HLTS/Common/FilterDesign/Estimator/. The files come with a README that indicates at how the fits inside were created

est_model_H1PR3_L_2025-10-07.mat  -- contains slightly tweaked versions of the PR3 L (and P) models reported in [LHO: 87593]
est_model_H1PR3_P_2025-10-07.mat  -- contains slightly tweaked versions of the PR3 P (and L) models reported in [LHO: 87593]
est_model_H1PR3_Y_2025-08-19.mat  -- contains the PR3 Y fits from [LHO: 86563]

est_model_H1SR3_L_2025-10-07.mat -- contains slightly tweaked versions of the SR3 L (and P) models reported in [LHO: 87612]
est_model_H1SR3_P_2025-10-07.mat -- contains slightly tweaked versions of the SR3 P (and L) models reported in [LHO: 87612]
est_model_H1SR3_Y_2025-08-05.mat -- contains the cleaned SR3 Y fits from [LHO: 86366]

These are the new .mat files that have the current OSEM estimator models. Until further notice, all previously existing .mat and .m files for OSEM estimator models/fits have been deprecated. We will later talk with Oli to standardize the addition of the OSEM estimator blends.

The updated make_EST_model.m, as well as the rest of the .mat files with the new syntax have been updated to the SUS svn under revision 12754.

TITLE: 10/23 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
INCOMING OPERATOR: Ryan C
SHIFT SUMMARY: Multiple locklosses today while continuing to troubleshoot the ifo. Multiple initial alignments were ran (it seems like we're needing them more often the past few days). Locking attempts and locklosses are documented in 87679 and 87701. There are multiple issues so it's been hard trying to distinguish them from each other and investigate one at a time. Secondary microseism has been heading back up too unfortunately.
LOG:

14:30UTC Started trying to relock
    - Ran initial alignment
    - Various locklosses

TITLE: 10/23 Eve Shift: 2330-0500 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
OUTGOING OPERATOR: Oli
CURRENT ENVIRONMENT:
    SEI_ENV state: USEISM
    Wind: 9mph Gusts, 6mph 3min avg
    Primary useism: 0.04 μm/s
    Secondary useism: 0.51 μm/s 
QUICK SUMMARY:

• We're still fighting to stay locked
  • Rolls mode and the ASC oscillation are still present
• Microseism is increasing back to around the 90th percentile

Looking at the last few locklosses from higher states from today and looking for any similarities:

2025-10-23_15:50:38 UTC (CLOSE_BEAM_DIVERTERS) quads, asc
MAX_POWER to LL: 00:08:30
- DHARD increase right before, DARM oscillation @ 10 Hz

2025-10-23_18:23:42 UTC(LOWNOISE_LENGTH_CONTROL) quads, asc
MAX_POWER to LL: 01:14:00
- DARM oscillation @ 6 Hz

2025-10-23_20:59:21 UTC (LOWNOISE_ESD_ETMX) quads, asc
MAX_POWER to LL: 01:13:00
- DARM oscillation @ 10 Hz

2025-10-23_22:12:25 UTC (TRANSITION_FROM_ETMX) quads, asc -> maybe secondary useism?
MAX_POWER to LL: 00:02:30
- IMC-REFL_SERVO_SPLITMON saturated right before the lockloss (possible cause - can high microseism do this?)

Maybe the roll mode we're fighting today isn't ETMY after all?

In the attached plot, the traces are 20 min apart, and you can see that the mode we were watching at 13.75 Hz gets bigger.  Throughout this time, Rahul was changing setttings on SUS-ETMY_M0_DARM_DAMP_R, trying to get it to stop increasing.  However, when the output of the damping filter was a very small number of counts, we weren't really seeing big changes in the slope of the increase in the monitor filter H1:SUS-ROLL_ETMY_RMS_LOG10_OUTMON.  When Rahul increased the gain to 1200 (from the original of 40), we started ringing up a different mode quite quickly.  That other mode is 13.94 Hz.  

13.94 Hz doesn't match any of the roll mode monitor filters we have in place (second attachment) according to the names of the filters. But, maybe this means that ETMY is actually at 13.94 Hz, and some other optic is at 13.75 Hz?  Back in July (alog 84982 that Elenna pointed me to), we originally thought that 13.7 Hz was ETMX, but had found at that time that actuating ETMY seemed to improve things. 

Anyhow, we may also consider trying to actuate on other optics, to see if we can do anything to that 13.75 Hz mode?  While the monitor filters are all quite narrow (as they should be), the actuator filters appear to all be the same between the optics.

To help with troubleshooting H1 locking, I have compiled a list of Guardian code changes which have been made since the last robust locking on Monday 20th October 2025.

30 nodes have been reloaded with modifed code since Monday. Note that some of these reloads were to just resync the nodes to the latest lscparams.py and clear the GRD-CFC.

A summary of the nodes is given in the attached txt file.

Full details of the code changes can be found in /ligo/home/david.barker/tuesdaymaintenance/23oct2025/grd_diffs/

Looking back at this alog #87678 from Ibrahim, I looked at the lockloss tool for this time and could see the 1Hz growing oscillation on DHARD_P but also three impulse/glitchy things on ETMY drive and showing up in DARM1 loop.

Thu Oct 23 10:08:16 2025 INFO: Fill completed in 8min 13secs

Gerardo confirmed a good fill curbside.

Attached is comparison of the LSC FF drives before the HAM1 vent (reference traces) and today (live traces). The first time we noticed the roll mode ringing was around June 11, one of the first days we were back into observing after the HAM1 vent (84982).

It doesn't look like there is significantly any difference in the drive between 10-20 Hz for any of the feedforwards. I have already previously checked the filters for high Q features around 13 Hz that might drive the roll mode and there are none. Sheila previously noticed that the roll mode started to ring up after the SRCL and PRCL feedforwards were engaged (87690).

Following up on [LHO: 87659], we found an error on the file that transfers the SR3/PR3 OSEM estimator fits posted in [LHO: 87612 LHO: 87593] into foton. Maybe this error explains the 0.67 Hz ringing that we had with the estimator ON.

The make_PR3_estimator_LP.m file was jamming the ISI fits into the M1 drive model. This was a copy-paste type error that I have resolved by now. The corrected code was committed to the SVN under revision 12752.

Next update I will make these pieces of code that commission the estimator less hardcoded to hopefully avoid repeating this mistake in the future.

There seem to maybe be lines in DARM that I don't recall from the past, so we're looking in to a few of those.  

The line at 843 Hz seems to be created in the calibration process, since it is present in GDS-CALIB_STRAIN, but not present in CAL-DELTAL_EXTERNAL.  Probably not a problem (unless it's there in NLN), but worth noting that it is *not* related to locking problems, since it's not in the raw DARM data.

TITLE: 10/23 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
OUTGOING OPERATOR: Ibrahim
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 6mph Gusts, 5mph 3min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.39 μm/s 
QUICK SUMMARY:

IFO was in IDLE when I got in due to issues last night with a high EY Roll mode. I just started an initial alignment so we can be further along in locking once commissioners arrive onsite.