TITLE: 10/30 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
INCOMING OPERATOR: Oli
SHIFT SUMMARY: 3 lock losses during my shift today, the first two looked vaguely similar, but we haven't come to any conclusion as to what the cause was. The last one was due to a ADC card failure at SEI EY. During troubleshooting of this the interlocked was tripped and we had to recover all of the site lasers after the ADC card was fixed. We ran an initial alignment and we are now back to NLN. For both of the relocks in the morning the 1Hz ringup didn't show up, but we just saw it come up for this lock. Oli pressed the new high ASC button.
LOG:

• 1540 Lock loss - 87848
• 1738 Lock loss - 87856
• 1913 Observing
• 2006 Lock loss
  • h1iopseiey ADC card error
  • Interlocked tripped during EY troubleshooting
  • More info for ADC card failure - alog87867
  • PSL recovery - alog87866                                                                     

M. Todd, E. Capote, J. Wright, S. Dwyer

We are attempting to understand discrepancies between model estimates and measurements of various parameters in the corner cavities. In particular, we have measurements of the PRC and SRC Gouy phase [alog 66215, alog 66211], along with input mode (IMC) overlap with the OMC mode from OMC scans [alog 87342], and beam profile measurments at the REFL and POP port in single bounce, with ITMY misaligned [alog 84307].

Using the LHO O4 yaml file in the modeling repo, we can model what we expect these values to be given our understanding of the optics geometries and lengths between optics. We've already identified a few issues with the model as several estimates disagree with the measurements. The first example of this was the SR2 position relative to the SRM and SR3. It was discovered that the model had not accounted for a 5mm shift of the SR2 towards the BS, adding roughly 5mm to each of the corrresponding lengths in the model.

This does not explain all of the discrepancies with measurements, however, so we are exploring how uncertainty in optic geometries can affect the model. Below is a table summarizing this effort so far.

The PR3/SR3 uncertainty in the RoC is +/- 6mm. The PR3 LIGO reported RoC is 36.021, the vendor reports 36.006 mm. The SR3 LIGO reported RoC is 36.013, the vendor reports 36.00. At each bound of these uncertainties, we analyze the beam overlap between the Input mode propagated to the POP port with the q-parameter fit from the measurements. We also analyze the mode overlap between the Input mode and the OMC mode, with ITMX misaligned (this was measured to be 91.7%).

After remotely rebooting h1seiey and seeing that the 4th ADC is now completely absent from the PCI bus, the next test was a power cycle of the IO Chassis.

Procedure was:

Stop h1seiey models, fence from Dolphin and power down the computer.

Dave (@EY):

power down the IO Chassis (front switch, then rear switch)

Power down the 16bit-DAC AI Chassis (prevent overvoltage when IO Chassis is power up)

Power up the IO Chassis (rear switch, then front switch).

The chassis did not power up. Tracking it back, the +24V-DC power strip was unpowered, the laser interlock chassis which is also plugged into this was powered down. This tripped all the lasers.

Fil came out to EY with a spare ADC

Dave & Fil (@EY):

We opened the IO Chassis and removed the 4th ADC. With this slot empty Fil powered up the DC power supply with the IO Chassis on, it did not trip.

We powered down the IO Chassis to install a new ADC. We are skipping the slot the old ADC was in, because it could be a bad slot.

The second DAC was moved from A2-slot4 to A3-slot1, the new ADC was installed in A2-slot4, leaving the suspect A2-slot3 empty.

We powered the IO Chassis on with no problems, then we powered up the h1seiey computer. The models started with no issues, I was able to reset the SWWD.

The chassis was buttoned up, pushed into the rack, and the AI Chassis were powered back up.

Marc is fixing a cracked cap on the old ADC so we can test it offline.

ADCs:

The PSL is now back online after the interlock trip.  During recovery I did a quick beam alignment into the PMC, as PMC Refl came back at ~26.5W, 1.2W higher than before the interlock trip.  After the alignment I was able to get PMC Refl down to ~25.4W.  I then changed the pump diode current for power supply 3 (Amp2, pump diodes 1 and 2), to 9.4A from 9.3A.  This made no change to Amp2 Out but brought PMC Trans back to ~106.2W and PMC Refl back to ~25.2W.  The FSS and ISS engaged without issue.  I had to change the ISS RefSignal to -2.02V from -1.99V to get the % diffracted power back to ~4.0; with the ISS engaged PMC Trans is now ~106.0W.  The % diffracted power will likely slowly change as things reach thermal equilibrium, as it usually does, so I'll monitor this over the weekend.

The watchdogs are back on and the PSL is good to go.

Thu Oct 30 10:07:57 2025 INFO: Fill completed in 7min 53secs

Gerardo confirmed a good fill curbside.

front end server h1seiey went down at 13:05 with an ADC timeout on the last ADC.  Restart of the server did not bring the ADC back.  Dave is headed out with a replacement.

This could be the cause of the last lockloss.

3rd lock loss of the day. This one was not like the others though, it was very fast.

A little late to observing since we thought that the estimator was injecting noise, but it's possible that it was some of Robert's equipment. Still not entirely sure but the scattering that we saw in DARM looks to be gone for now.

J. Kissel

Comparison of local performance metrics for H1 SUS PR3 after turning on the LP estimators this morning.
2025-10-30 13:00 UTC -- only Y estimator ON
2025-10-30 17:00 UTC -- all LPY estimators ON.

Both times are when IFO is in nominal low noise with no commissioning happening.

In short -- L is far more limited by residual suspension point motion that P or Y, so the reduction in off resonance drive is not *as* awesome as in P or Y. But there is reduction in off-resonance drive, and damping on resonance is about the same.
Nice!

More plots to come...

There was a small amount of ground motion at the time, but this lock loss seemed very similar to the previous one.

Some shots that were shared on MM by me and Elenna attached below.

With the great success of the ASC high gain seismic guardian state that TJ has been testing, 87788, we have decided to use the ASC high gain script for a different purpose, to damp the 1 Hz oscillation at the beginning of the lock.

From various testing, it appears that the pitch ASC is the likely culprit of this problem, but it's still not clear which loop in particular. So, now this script is designed to transition the pitch ASC only to the high bandwidth configuration. This is done in all one step, where CHARD P, DHARD P and DSOFT P are all transitioned to HBW. However, there is a 20 second time.sleep at the end of the script because that is the longest ramp of the filters that are changed. After this sleep, the script will print "Done!". The reverse happens for the "go back" script.

I would prefer if we could use this script for now to avoid the 1 Hz oscillation, as a way to test what effects the oscillation. If the script does not work, then I recommend using the seismic guardian high gain state, which will also transition the yaw ASC and turn of the LSC FF.

Jim created templates for exciting the HAM7 ISI in jim.warner/ham7/isi_injections

I copied these over to /ligo/home/sheila.dwyer/SEI/HAM7, and ran the x and y ones.  While I was getting ready to run the Z one (but not making any excitation), we lost lock for an unknown reason. 

Ran a broad band and then started simulines for 4 minutes before we had a lock loss.

Start:

PDT: 2025-10-30 08:36:06.772736 PDT
UTC: 2025-10-30 15:36:06.772736 UTC
GPS: 1445873784.772736
 

Lock loss at 154010UTC

In the middle of the simulines portion of the calibration sweep we lost lock. Robert was out on the floor at the time, but we're unsure of that was the cause at this time.

Summary

Round 2 of testing the guardianization of turning on and off the high ASC gains (Round 1 - alog87462). SEI_ENV will now automatically move us into the high gain ASC state when a.) we are in the earthquake state b.) there is an incoming or ongoing earthquake that is at or below the dotted line on the "rasta plot". The transition takes 11 seconds to complete, and it will transition back when the ground motion is low enough to bring us out earthquake state.

Details from today

I started testing with a few 10 and 5 second waits between steps, just as is done in the script that we currently use. Once those ran successfully a few times I started to decrease the wait times between steps. Eventually, I had success transitioning all the ASC at the same time, then the FF 10 seconds after. since this was the same configuration that I had last time I tried this, tried to reporduce the lock loss by requesting the High ASC state, then immediately requesting the Low ASC state. This did, again, cause a lock loss. To avoid this I have a wait timer in the High state so it won't switch quickly from one to the other.

Transitioning back out of the high ASC state has the same thresholds as the earthquake state currently. We didn't want to transition back and then have to do it all over again, or wait in earthquake for another 10 minutes for it to calm down. We might make this a bit shorter or smarter after we've seen it work a few times.

M. Todd

Today I ran another OMC scan (following last week's instructions alog 87316 and 87342) after the morning lockloss to see if could a better measurement of the hot state overlaps.

I was still limited to about 12 minutes after the lockloss, so we expect some amount of difference in the full hot state.

I've plotted the results on top of last week's scans, in yellow. Analysis of this will follow in a comment.