It seems like it would be useful to have this recipie written in one place.  After the gate valves are opened and closed, we often have a hard time relocking because of alignment.  

Here's what we think we should do to recover after gate valves open.  

1. Do initial alignment without using green cameras in the loop, as the cameras are right next to the gate valves. 
   1. Start by using the baffle PD dither alignment script to set the TMS alignment: use the suspension guardians to misalign both ITMs and ETMs, run dither align script for both TMSs found on the INIT ALIGN MEDM, (dither align scripts, TMSX and TMSY, can run at the same time).  Open the ITM baffle PD scope and watch the script find the pointing of TMS that sets the beam to PD1, PD4, and finally centered on the optic.  When this finishes sucsesfully for both TMSs, there should be a dim green beam visible on both ALS cameras, if there is not check the alignment of PR3 (affects both arms) and the BS (only moves the Y arm beam).  This TMS alignment has to be set before you can move onto the next step, otherwise the ITM alignment won't be right. 
   2. Use the suspension guardians to align both ITMs, but leave the ETMs misaligned.  Run the dither alignment script for ITMX and ITMY, this time watching the ETM baffle PD scope to watch the script's progress.  Once this is finished, the ITMs should be set, and we should not move them based on the cameras. 
   3. Use the suspension guardians to set the ETMs to aligned, and you will hopefully see mulitple beams or cavity flashes on the ALS cameras.  You can manually align the ETMs to increase the flashes until they are good enough to lock the arm.  Use the ALS guardians (in auto mode might be helpful) to set them to ETM_TMS_WFS_OFFLOADED, this will not run the green camera servos, it runs green WFS to the ETM and TMS alignment only to increase build up. Don't run INITIAL_ALIGNMENT or INITIAL_ALIGNMENT offloaded. 
   4. Align the vertex to these ITMs: set the ALIGN_IFO guardian to manual, init it to control the suspension guardians.  Choose INPUT_ALIGN_OFFLOADED, once that finishes PRC_ALIGN_OFFLOADED, then MICH_BRIGHT_OFFLOADED, then SRC_ALIGN_OFFLOADED.  
2. Make two guardian changes before you start locking the full IFO
   1. Don't use the SRC1 + SRC2 ASC loops in DRMI, these don't seem to work until the alignment is set well for full lock.  You can turn these off by going to lscparams, and searching for use_DRMI_ASC, setting SRC1+ SRC2 to false.  Load ISC_DRMI for this change to take effect. 
   2. Don't shutter the ALS beams once they are no longer used for locking.  In ISC_LOCK, find the edge     ('IDLE_ALS',  'CARM_OFFSET_REDUCTION', 5), and reduce the weight which is normally 5 to 1, so that this path which avoids shuttering the ALS becomes the default path.  Load ISC_LOCK for this change to take effect.
3. Start to relock as usual, going through PRMI.  Stop at CHECK_DRMI_ASC, where all the DRMI ASC loops will be running other than SRC1+SRC2, and manually adjust SRM sliders to reduce POP90 and increase POP18. 
4. You can try to lock all the way to ENGAGE_ASC for full IFO, but this may not work.  If it does not, you may need to manually align DHARD, CHARD, and SRM. 
   1. If you are adjsuting alignments by hand, you want to keep the green beams locked to the arms as the alignment changes.  To get the green alignment to follow any changes you make, stop at IDLE_ALS and turn on the scripts that servo the green TMS QPD offsets based on the green WFS.  You can access this script by going to either ALS end station overview and clicking the button that says QPD OFFSETs, then using the pink drop down button to start the servo for each arm, and open the ndscope that lets you watch the progress of these servos (screenshot)
   2. If the power in the two arms is mismatched (visible on the PRMI SB4 min ndscope on the front wall), this could be a sign that DHARD is misaligned. 
      1. If you notice this in a state before DHARD WFS are closed, RF_DARM or CARM_OFFSET reduction , you can use the move_arm_dev script to step DHARD pitch and yaw to bring the arms closer to each other.  userapps/asc/h1/scripts/move_arm_dev.py DH P 0.01, or similar.  
      2. If you notice the arm powers diverging from each other after DHARD WFS are closed, this might be fixed by moving SRM while watching to make sure POP 90 stays low.  
   3. If the power recycling gain is low, you may need to adjust CHARD.  You can step through CARM_OFFSET reduction or stop after it finishes and use 62110 to check if the refl power is too low for the arm transmission that you are at, and try to move CHARD while trying to increase the refl power (watch build_up_slow ndscope linked from the ISC guardians overview and other places).  
5. Once you are all the way locked (congratulations), stop at PREP_ASC_FOR_FULL_IFO before you try turning on ASC.  It may be a good idea to note the green QPD offsets with a screenshot, if you loose lock they wil be reverted with SDF revert but you would need to reset them to these values to relock.   Also look at the ITM green camera offsets, from each ALS end station overview in the green WFS section there is a button that says CAM ITM YAW or PIT, you can take a screenshot of this screen for both ITM cameras and compare the current location of the beam centroid to the setting (screenshot)
6. Use a guardian shell (guardian -i ISC_LOCK) to copy and paste lines from ENGAGE_ASC_FOR_FULL_IFO manually, skipping SRC1 +SRC2.  (you might grab another screenshot of the green QPDs and cameras at this point)
7. Then you can manual the guardian to TMS_SERVO, engage_soft_loops, and DARM offset, and take another green QPD offset + camera screenshot.  
8. Having the DARM offset on as well as the soft loops seems to help the SRC ASC work, so try engaging those once those steps are done.  (Congratulations on getting all the ASC controlled)
9. Important step: reset the green references.  In the past we have reset the green QPD offsets by accepting them in SDF as well as the camera offsets, but it doesn't make sense to be changing the QPD offsets as they shouldn't be impacted by the gate valve moving the camera.  Today once we had all the ASC engaged, we stepped the QPD offsets back to their original settings from SDF, watching the arm transmissions in the ndscope.  The arm transmissions seemed fine with the original QPD offsets, so we left those there and only updated the green camera offsets.  I think this should be the way we do things in the future, not reseting the QPD offsets by default.  

TITLE: 12/04 Eve Shift: 0030-0600 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Planned Engineering & LOCKED
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 10mph Gusts, 6mph 3min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.28 μm/s 
QUICK SUMMARY:

SQZ_OPO_LR was giving us this error: 
"CLF frequency out of range, check CLF CMB".
I do not recall how this was resolved by Daniel while he was checking out the CMB. 

Once that was resolved we got the classic: "pump fiber rej power in ham7 high, nominal 35e-3, align fiber pol on sqzt0."  issue.
The Quarter & Half wave plates on SQZT0 were touched up to minimize H1:SQZ-SHG-FIBR_REJECTED_DC_POWER_MON. 
OPO was locked. Great!

Then the FC wasn't locking: 
So FC2 was moved from its nominal location. only after I accidentally moved ZM1 and reverted it.
Fumbled with the FC2 sliders for a while, then used the clear history button.
Fumbled around some more with the FC2 sliders. 
Locked the the FC with FC2 P @ 246.1 Y @ 51.1

And now the SQZ has been SQuoZe.

Forcast for tonight's wind looks good for a stable lock.

[Sheila, Jeff, Elenna, Corey]

Today we relocked with some issues. We think that the soft close of the gate valves shook the ITM green cameras enough that the green references were no longer good, which caused many alignment problems through carm offset reduction.

The SRC ASC is OFF in DRMI ASC at this time, by having the use_DRMI_ASC['SRC1'/'SRC2'] flag set to False. When locking, SRM and SR2 might need to be moved by hand before offloading DRMI ASC.

Sheila will write a more detailed alog about the process of updating the green references, because we think we have a better method now. For the summary, we have updated the green camera references, SDFed them in safe, and run an initial alignment after the lockloss.

Other guardian changes:

• DARM OFFSET now occurs after DRMI TO POP and befor PREP ASC FOR FULL IFO. This is how it used to be before the OFI problems. We had some problems with SRC1 P during full IFO engagement, but the problems seemed to go away as soon as we went through DARM OFFSET
• CLOSE BEAM DIVERTERS is now out of the ladder. While we troubleshoot, we want these open, and to avoid lockloss they are closed at 2 W. I removed them from the ladder completely for now.

Tony, Oli and I have been watching violin mode damping. So far, there doesn't seem to be any problems.

By eye, Jeff and I think the PCAL X crosses do not match the line height in CAL DELTA L, so there may be some calibration mismatch. I will run a cal measurement when we are thermalized.

Squeezer is not working, Daniel and others are trying to troubleshoot now.

TITLE: 12/03 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Planned Engineering
INCOMING OPERATOR: None
SHIFT SUMMARY:

Main goal for today was to continue to see troubleshoot H1 to see if it survived the CDS work from earlier this week (by trying to get H1 back to NLN)---Elenna and Sheila worked on this.  There continued to be issues with ASC SRC---Elenna said this is common for H1 recoveries.  Sheila also mentioned that the alignment issue most likely is a result of Gate Valve closures affecting the Green Camera alignment.

Recovery work above had to wait until about 11amPDT due to remaining LVEA crane inspections.  

During Recovery work 20-bit DAC card was swapped out for one which was installed yesterday that had issues.
LOG:

• 1532-1605 Initial Alignment
• 1541 Crane inspection begins in South Bay [randy & CraneTech contractor]----> done in LVEA
  • After lunch will do Staging and then FCES
• 1606-1613 Restart all of the models on h1iscex (this is for the the issue with the PCal for EX).  [dave]
• 1640-1730 EY, EX trips to check on Dustmon Pumps (tj)
• 1659-1706 lvea walkthru (richard)
• 1707-1714 meet w/ crane inspector (tyler)
• 1729-1901 Moving magnetometer around HAM1 (rene, alicia)
• 1740-1755 Swapping in new Corner Station Dust Monitor pump & then checking flow  (tj)
• 1756-1808 Testing out new dust monitors in the diode room (ryanC)
• 1803-1854 Power supply measurements at EX related to PCal & new DAC (tony)
• 1819-1840 Looking for picomotor mount assys in optics lab (jennie)
• 1839-1852 Parts hunt in West Bay (mitch)
• 1852-1857 LVEA walkthru (sheila)
• 1850 Restart locking H1 
  • darm to rf -> idle als -> dhard wfs
  • Thought is GV soft closes affect green camera alignment so this causes recovery grief.
• 1941-2111 LVEA Tour for Michael Turner (cosmologist) & Dennis Overbye (NYT reporter) which was in CR and now moves to LVEA (mike, jenne)
  • 2127-2156 Driving to EY
  • 2206-2218 Drive to Overpass & then Back to LExC
• 2040-2139 Gantry crane inspections at Mechanical Room & FCES.  This COMPLETES crane inspection. (randy, cranetech)
• 2140-2348 OPO testing + laser Hazard in the optics lab (KarMeng)
• 2145-2157 magnetometer tests in CER (alicia,rene)
• 2158 CHETA table work in the CHETA/JAC Lab (matt)
• 2200-2207 Hi-Bay trip (ibrahim)
• 2230-2308 h1iscex being shut down for the to investigate issues with 20-bit DAC (dave, tony)
• 2250 JAC table work in CHETA/JAC Lab (jennie)
  • 2342 Looking for JAC parts in the LVEA (while we are locking H1). [jennie]
• 2310 Initial Alignment #2 of the day (post EX 20bit DAC swap)
•  

J. Kissel, D. Sigg, D. Barker
ECR E2100485
WP 12901

Continuing on the deprecation of 18-bit DAC path (ECR E2100485), we upgraded the h1iscex's DAC0 card yesterday from an 18- to a 20-bit DAC (LHO:88333). One of the front-end models that use that DAC is the h1calex model.

In this aLOG, I discuss the front-end model changes we implemented to support this change.

I show the differences in before vs. after in the following screenshots. All changes were made at the top level of the model, no library parts were changed.
    - Top level model BEFORE vs. AFTER.
        Functional Changes:
        :: Changed the CDS part representation of the DAC from a 18- to 20-bit DAC.
        :: Added a constant of 4.0 ***, whose output is piped into a goto tag, CAL_FACTOR.
        :: The CAL_FACTOR from tag is first piped into a EPICs read back for display on MEDM (and for recording in frames), which will manifest as the channel H1:CAL-EX_DAC_CAL_FACTOR
        :: Then CAL_FACTOR is piped into multiplication blocks downstream of the DAC outputs such that the two channels used by the model CH5 and CH6 (with the count starting at 0; CH6 and CH7 in the analog world where counting starts at 1)

        Aesthetic Changes:
        :: Added the modern labels for direct IO chassis and inter-(computer)-process communication
        :: Detached the RFM IPC that ships the PCAL excitations to the corner station from its direct wire pick-off and "changed" it's input to be the tag that already exists to ship the signal to the DAC
        :: Moved that IPC system down to the left under the "new" RFM section, which allows for the model to conform to modern convention of having clear/clean separate sections for "input" "controls" and "output."
        :: Changed the color of the CDS parameters block to be orange, as is standard.

     - Another view of the top level to better highlight which signals are multiplied by four, and where in the signal change BEFORE vs. AFTER

The h1calex model has been committed, and rev-updated to to 
    /opt/rtcds/userapps/release/cal/h1/models/
        h1calex.mdl         r23189 --> r34056


*** Because this model houses both the PCAL system, Hardware Injection and the DuoTone system -- critical flywheels for the detector calibration, we decided to implement our usual gain scaling of (exactly) 4.0x to compensate for the change in calibration (from 20/2^18 to 20/2^20 [ Vpp_differential / ct]) the DAC card upgrade as a hard-coded constant in the front-end, rather than in an "adjustable/changeable" filter module in a filter bank just upstream of the DAC output.

Wed Dec 03 10:09:57 2025 INFO: Fill completed in 9min 54secs

Gerardo confirmed a good fill curbside. Plot looks strange because temps were starting positive.

WP12901 Daniel, Marc, Fil, Jonathan, Richard, Jeff, Oli, EJ, Dave:

Following from Monday's success in upgrading h1susey to the new LIGO-DACs on Tuesday 02dec2025 we upgraded h1susex to use LIGO-DACs.

This was a slightly different upgrade compared to h1susey because for the past 18 months h1susex ESDs have been driven using an early version of the LIGO-DAC. The h1susetmx model was using the GenStd 18/20-bit DACs for the upper stages and the LIGO-DAC for the L3 and ESD. The other models (h1sustmsx and h1susetmxpi) continued to use the Gen Std DACs.

The ESD and PI drives were being driven by a single 20bit-DAC using the special PI-AI chassis (first 6 chans through standard filters and output via DB15, last 2 chans through PI filters and output via DB9). When we upgraded the ESD drives to the LIGO-DAC we retained the original PI-AI chassis for these signals and temporarily installed the spare PI-AI chassis for h1susetmxpi 20bit-DAC's exclusive use.

The original LIGO-DAC used the 8-channel interface card. This card took the 32 channels from the LIGO-DAC and outputed the first 8 using its onboard DB25 connector. The remaining 24 channels were distributed using 3 Header Slot Plates (each with a DB25) ribbon cabled to the interface card. To fit these into the chassis, we displaced the two BIO cards to A4-3 and A4-4. 

The IO Chassis changes were then:

Remove first-gen LIGO-DAC card, its 8chan interface card and the 3 header slot plates.

Remove the 3 18bit-DACs and their interface cards

Remove the 2 20-bit DACs and their interface cards

Install 2 LIGO-DACs in A1-4 and A2-2 with their new LIGO-DAC interface cards

Move the 2 BIO cards back to A4-1 and A4-2.

Fil upgraded all the AI chassis for LIGO-DAC SCSI Connection, duplicating what was done at EY. The second LIGO-DAC drives the PI-AI chassis, the first LIGO-DAC drives the 2 standard AI chassis, now daisy chained together.

Temporary cabling from the original LIGO-DAC chans 16-31 looped over to h1iscex were removed.

LIGO-DACs ADDED

Cards Removed

* 20bit-DAC installed into h1iscex, now suspect.

WP12901. Daniel, Jeff, Oli, Fil, Marc, Jonathan, EJ, Tony, Dave:

Summary:

Yesterday I replaced h1iscex's 18bit-DAC with a 20bit-DAC as part of the project to eliminate all 18bit-DACs from production. There is currently no indication that the 20bit-DAC is driving the AI chassis with any voltage, investigation is ongoing.

Details:

The 18/20 bit DACs use the same interface card, so for the upgrade I was able to pull the IO Chassis out half way with cables still attached to access the PCIe bus. I removed the old 18bit-DAC and replaced it with on of the 20bit-DACs from h1susex upgrade

I do not know if the 20bit-DAC from h1susex was the one driving the PI or the one which had been idle since the LIGO-DAC went in mid 2024.

The original interface card and ribbon cable were reused. No field cabling was disconnected, no AI chassis were powered down.

Upgrade was done between 12:00 and 13:00 Tue 02dec2025.

Later that afternoon Tony found that the PCAL readbacks were not responsive. Tony and I went to EX around 17:00 to verify the 20bit-DAC was installed and connected correctly, it was.

This DAC has a special AI Chassis, D1101785 "aLIGO 18 Bit AI Chassis" which has handy BNC pickoffs for all 8 channels. We put a scope on the two channels being driven by h1calex (6,7) and could see no signal.

Also the last channel on the AI is loop-backed to the first ADC AA chassis with a DB9 cable, and the IOP is configured by SDF to drive the duotone from DAC0-chan7 back to ADC0-chan30. No signal is seen there either since the upgrade.

Tue Dec 02 10:01:28 2025 INFO: Fill completed in 1min 27secs

Does not look like a good fill. Will try again tomorrow

TITLE: 12/03 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Planned Engineering
OUTGOING OPERATOR: None
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 5mph Gusts, 3mph 3min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.27 μm/s 
QUICK SUMMARY:

H1 was IDLE overnight (had troubles getting through DRMI and a few steps beyond yesterday afternoon/eve---see Oli's log).  In the middle of an Initial Alignment.  Then we'll see about locking...

Day #2 of Crane Inspection work continues:  This morning will be work for the South Bay crane, but will also need pendant work; this should go until about noon.  (Then remaining cranes would be:  Filter Cavity End Station, Staging Bldg, and some gantry cranes.)  Yesterday, rest of LVEA cranes were inspected as well as End & Mid Stations.

(it's been almost 15min and Green Arms are almost aligned from Initial Alignment.)

The IFO has been trying to relock for a bit now, but we were having multiple locklosses, from DHARD_WFSx3, BS_STAGE2, and CARM_OFFSET_REDUCTION. Most of the locklosses looked sudden (based on quick look at buildups ndscope), but the last attempt, we started getting SRM saturations and the PR gain was moving all over the place. I remembered that Jenne and RyanS had been saying that SRC1 wasn't being cooperative yesterday (88298), so I turned off the SRC1 and SRC2 loops and that stopped the oscillation. Unfortunately, we lost lock soon after in DHARD_WFS. I was (am) about to go home so I've put the detector in IDLE for the night.

Tuesday Maintenance day impact notes.

Due to Beckhoff upgrades and restarts the PCAL lasers were shutoff.
When Beckhoff came back up the PCAL Lasers oddly didn't come back up this time.
The H1:CAL-PCALX_LASERPOWERCONTROL voltage was set to 0. I took that back to 5V for both arms.
Turned H1:CAL-PCALX_LASERENABLE back on. 
H1:CAL-PCALX_SHUTTERPOWERENABLE back on.

Arm specific Settings : 

PCALX: 
H1:CAL-PCALX_OPTICALFOLLOWERSERVOOFFSET was set back to 3.65V. 
H1:CAL-PCALX_OPTICALFOLLOWERSERVOGAIN : 39.60 dB

PCALY: 
H1:CAL-PCALY_OPTICALFOLLOWERSERVOOFFSET : 3.80 V
H1:CAL-PCALY_OPTICALFOLLOWERSERVOGAIN : 38.56 V

Hey future Tony,  Today is the day the X Arm 18bit DAC was changed out for a 20bit as well.