Here are the steps Sheila and I took that almost got us to prep ASC for full ifo:

• take guardian to carm_5_picometers
• change H1:LSC-TR_CARM_OFFSET to -52
• ramp H1:LSC-TR_CARM_GAIN to 2.1
• change H1:LSC-TR_CARM_OFFSET to -56

We measured all the LSC gains and found that maybe PRCL gain was dropping and causing locklosses so

• increase PRCL2 gain from 1 to 1.6
• increase DHARD P gain to -30, DHARD Y gain to -40
• run CARM_TO_REFL state in guardian
• reduce H1:LSC-TR_REFLAIR9_OFFSET (make sure to set a ramp time first!) slowly to zero
• ran RESONANCE state in guardian
• ran CARM_TO_ANALOG

We then went to prep_ASC_for_Full_ifo but we forgot to reduce the PRCL2 gain, so this probably caused a lockloss since DRMI moved to POP

We will keep trying this and maybe do full IFO ASC next

On the next attempt, I added this additional step:

• before DRMI to POP, I realized that I should drop the PRCL2 gain
• by hand, I lowered the PRCL2 gain slightly, and then increased the input matrix value for REFLAIR27 I accordingly until that was 1.6 times higher than its nominal value
• then, I ran DRMI to POP, which went fine.

Hi y'all!

Link to full report here.

Summary:

• 73%, 44.6%, 71.8%, 43.02%, 0%, 0%, 0% Observing per day
• Large EQ off coast of OR, as well as an electrical storm, caused lockloss/glitches on Tues
• Power outage on Wednesday caused many issues with IFO
• Greatly reduced range on Thursday, as well as extremely increased glitchrate
• IFO down for corrective maintenance from Thursday onwards. 
• One HVeto repeated winner for Mon, Tues, Wed:H1:LSC-REFL_A_LF_OUT_DQ. On Thursday, following the power outage, there were 12 round winners, 9 of which belonged to IMC
• Only three days of FScan-Observing: Mon - Wed. Tuesday was the lowest and Wednesday was the highest. All three hovered around ~2500 lines

Mon Sep 15 10:07:58 2025 INFO: Fill completed in 7min 54secs

I** have written a program to compare slow controls channels before and after with Wed 10sep2025 power glitch to see if there are any which look like they may have been broken and need further investigation.

(** Full disclosure, it was actually written by AI (Claude-code) and runs on my GC laptop safely contained in a virtual machine running Deb13 Trixie)

As a first pass, the code is looking for dead channels. These are channels which were active before the glitch, and are flat-line zero following.

The code gets its channels to analyze from the slow controls INI files.

Using Jonathan's simple_frames python module, it reads two minute trend GWF frame files. For before I'm using the 10:00-11:00 Wednesday file (an hour before the glitch) and for after I'm using the Thu 00:00-01:00 file. In both cases H1 was locked.

I'll post the results as comments to this alog.

TITLE: 09/15 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
OUTGOING OPERATOR: None
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 6mph Gusts, 3mph 3min avg
    Primary useism: 0.01 μm/s
    Secondary useism: 0.09 μm/s
QUICK SUMMARY: I'll start by taking H1 through an initial alignment then relock up through CARM_5_PICOMETERS, since it sounds like that's still the latest stable state H1 can get to.

TITLE: 09/15 Eve Shift: 2330-0500 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
INCOMING OPERATOR: None
SHIFT SUMMARY:

IFO is in IDLE for CORRECTIVE MAINTENANCE

The mission today was to get to RESONANCE. We got really close!

The idea was that the calculated TR_CARM offset was likey incorrect so we had to step the offset to maintain stability as we went into resonance. So attempts below are all from CARM_TO_5_PICOMETERS.

While sitting in CARM_TO_REFL during handoff, took RF_DARM TF (attached).

I'm stealing Ryan S's format for his IFO troubleshooting today since it was so easy to read! (Thanks Ryan!)

Reference for healthy CARM to RESONANCE transition pre-outage: 2025/09/09 20:01 UTC, GPS: 1441483301

Relocked,

• TR_CARM_OFFSET set -52
• Lockloss ~2-3s into CARM_TO_REFL
• Could be a 5.3 EQ but unlikely

Relocked,

• TR_CARM_OFFSET set -52
• Lockloss ~2-3s into CARM_TO_REFL
• Alignment looked bad judging from INP1_INMON, buildups and AS_AIR so touched PRM to make better, which worked.
• Lockloss at 10s after CARM_TO_REFL, but the state finished
• Conclusion: alignment was still bad and more touching didn't help

Ran Initial Alignment - fully auto

Relocked,

• TR_CARM_OFFSET to -52
• CARM_TO_REFL Successful
• Sheila's advice was to step slowly into RESONANCE with the first line of code and see what happens, which I did
• SRM Saturated at 02:21:18 UTC (attached)
• Lockloss at 02:21:20 (DRMI)
• Looked at OPLEVS but no indication that they saw anything (attached)

Relocked,

• Took DHARD_P SS 2W OLG TF per Shela's instruction - no indication and the measurement didn't really look good due to low coherence. Phase was ok.
• Sheila came online
• TR_CARM_OFFSET to -52
• CARM_TO_REFL reached but PRG unstable
• Lockloss ~3s after

Plan now was to try to step the TR_CARM_GAIN up as we were stepping the TR_CARM_OFFSET down further from -52 to keep the UGF consistent and not cause a LL. Sheila plannd to measure this each time we stepped it.

Relocked,

• TR_CARM_OFFSET to -52, UGF measured, TR_CARM_GAIN to 1.2 (from 1)
• TR_CARM_OFFSET to -53, UGF measured, TR_CARM_GAIN to 1.4
• TR_CARM_OFFSET to -54, UGF measured, TR_CARM_GAIN to 1.7
• TR_CARM_OFFSET to -56, UGF measured, TR_CARM_GAIN to 2.1
• Sheila stepped into resonance by trying to split up the offset reduction (1st line of RESONANCE) but this failed either due to the problem or because there was 0 ramp time
• Lockloss

IFO in IDLE per Sheila's Instruction. We will try again tomorrow.

LOG:

None

TITLE: 09/14 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
INCOMING OPERATOR: Ibrahim
SHIFT SUMMARY: More troubleshooting today on H1 locking, primarily focused on getting through the CARM_TO_REFL transition and RESONANCE states. See my morning alog for details on the first half of the day; the rest I list here:

• After a discussion online between people, decided using a different TR CARM offset may be the path forward
• Reverted my earlier change of LSC_TR_REFLAIR9_CARM_TO_REF gain in lscparams back to -0.8
• Relocked - held in CARM_5_PICOMETERS and stepped TR_CARM offset
  • Took TR CARM transfer function between steps to check UGF doesn't get too low
  • Settled on -52 as a good offset
  • Moved to CARM_TO_ANALOG, PRG got noisy, waited here about 10 sec
  • Moved to RESONANCE, PRG less noisy, but DHARD control signals got glitchy and lockloss
  • 2 Hz ringup seen in DHARD and DARM
• Relocked - after CARM_5_PICOMETERS lowered TR CARM offset to -52
  • Requested CARM_TO_ANALOG, lockloss soon after entering RESONANCE
• Relocked - lowered TR CARM offset again after CARM_5_PICOMETERS
  • Turned off DHARD loops
  • Requested CARM_TO_ANALOG; lockloss at same place in RESONANCE
  • Realized RESONANCE turns DHARD gains back on, temporarily commented these lines out for next attempt
• Relocked - did NOT lower TR CARM gain
  • Turned off DHARD loops
  • Requested RESONANCE, lockloss at same place
• Relocked - lowered TR CARM offset again after CARM_5_PICOMETERS
  • Turned off DHARD loops
  • Requested RESONANCE, lockloss at same place
• Relocked - lowered TR CARM offset even lower to -55
  • Left DHARD on
  • Lockloss before even leaving CARM_5_PICOMETERS
  • -55 is too low of an offset
• Relocked - lowered TR CARM offset to -52
  • Left DHARD on
  • Lockloss at start of RESONANCE again
• Relocked - lowered TR CARM offset to -52
  • Uncommented DHARD gains in RESONANCE
  • Moved to CARM_TO_ANALOG, PRG noisy but otherwise stable
  • Ibrahim starts taking measurements

Generally locking has been very easy today up through CARM_5_PICOMETERS and alignment has been good.

TITLE: 09/14 Eve Shift: 2330-0500 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
OUTGOING OPERATOR: Ryan S
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 10mph Gusts, 6mph 3min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.12 μm/s
QUICK SUMMARY:

IFO is DOWN for CORRECTIVE MAINTENANCE

Ryan S has tried an exhaustive combination (I will link the alog when posted) of lock attempts to get beyond where we were yesterday, which was losing lock upon loading the matrix necessary to get from CARM_5_PICOMETERS to CARM_TO_REFL to RESONANCE.

I don't have any ideas of what to try but will be in communication with comissioners.

Below is my outline of locking attempts so far this morning, with any changes I made in Guardian or otherwise in bold:

• Plan to start w/ initial alignment then allow ISC_LOCK to go up to 'RESONANCE' if possible
  • Sheila messages saying CARM_TO_ANALOG won't work; changed ISC_LOCK line 2826 from 2 to 8
• Alignment goes without issue, just needed some SRM adjustment
• Lockloss early in 'RESONANCE', maybe actually something at end of CARM_TO_REFL like Ibrahim found last night
  • DHARD control signals got very glitchy
• Relocked - lockloss while waiting in CARM_TO_REFL after it finished; definitely end of this state (matrix loading?) is the issue
• While relocking, lockloss in DHARD_WFS, seems like alignment was just bad
  • Added SRC loops back into DRMI ASC since they should work after Elenna's dark offset updates
  • They do
• Relocked - holding in CARM_5_PICOMETERS while stepping CHARD P and Y to try and increase LSC-REFL
  • Not seeing much appreciable improvement
  • Reverted Sheila's REFLAIR phase changes from yesterday
  • Tried CARM_TO_REFL again; lockloss
  • SDF_REVERT reverts my phase reversion
• Relocked - held in CARM_5_PIVOMETERS to measure TR CARM transfer function
  • Gain looks too high compared to Feb 2023 reference (screenshot 1), changed LSC_TR_REFLAIR9_CARM_TO_REF gain in lscparams from -0.8 to -0.46 (might undo this later)
  • Remeasured (screenshot 2)
  • Tried CARM_TO_REFL again with new gain; lockloss

Summary of what we've changed so far since the overnight lock where IMC visibility degraded: (strting some notes, plan to add links later).

• reduced power on IMC refl diode
• rephased REFLAIR 9 + 45 for DRMI acquisition
• dark offsets for AS_C and AS 72, this makes DRMI ASC work again
• reduced tr_REFLAIR gain to match a reference from 2023, this did not help us make it through that transition, might be something we want to undo.

I posted the following message in the Detchar-LHO mattermost channel:

Hey detchar! We could use a hand with some analysis on the presence and character of the glitches we have been seeing since our power outage Wednesday. They were first reported here: https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=86848 We think these glitches are related to some change in the input mode cleaner since the power outage, and we are doing various tests like changing alignment and power, engaging or disengaging various controls loops, etc. We would like to know if the glitches change from these tests.