[Sheila, Eric, Karmeng]
We checked the NLG of the OPO without cavity lock, threshold power is roughly at 3.7mW, at the same order as the test performed at MIT (E2500270)
Checked the crystal alignment, found 9 good dual resonant position, from -2689um to 1223um, with 490um separation between each point. Everything seems to be in a good position.
Plus a clipped dual resonance position at -3190um. Crystal edge is roughly at 1500um and -3200um.
Red alignment does not shift between the various crystal position.
It's been a few minutes so far. There is emergency lighting. Luckily since it was lunchtime there were no people out on the floor. Recovery Begins!
FAMIS 38886, last checked in alog88025
Both HAM5 and the BS chambers were in 'ISI_DAMPED_HEPI_OFFLINE' at the time this was run, but all other chambers were nominal.
There are 15 T240 proof masses out of range ( > 0.3 [V] )!
ETMX T240 2 DOF X/U = -1.565 [V]
ETMX T240 2 DOF Y/V = -1.633 [V]
ETMX T240 2 DOF Z/W = -0.886 [V]
ITMX T240 1 DOF X/U = -2.256 [V]
ITMX T240 1 DOF Z/W = 0.467 [V]
ITMX T240 3 DOF X/U = -2.386 [V]
ITMY T240 3 DOF X/U = -1.073 [V]
ITMY T240 3 DOF Z/W = -2.895 [V]
BS T240 1 DOF Y/V = -0.323 [V]
BS T240 2 DOF Z/W = 0.327 [V]
BS T240 3 DOF X/U = -0.547 [V]
BS T240 3 DOF Z/W = -0.401 [V]
HAM8 1 DOF X/U = -0.638 [V]
HAM8 1 DOF Y/V = -0.729 [V]
HAM8 1 DOF Z/W = -1.11 [V]
All other proof masses are within range ( < 0.3 [V] ):
ETMX T240 1 DOF X/U = 0.007 [V]
ETMX T240 1 DOF Y/V = -0.049 [V]
ETMX T240 1 DOF Z/W = -0.017 [V]
ETMX T240 3 DOF X/U = -0.005 [V]
ETMX T240 3 DOF Y/V = -0.056 [V]
ETMX T240 3 DOF Z/W = -0.051 [V]
ETMY T240 1 DOF X/U = 0.045 [V]
ETMY T240 1 DOF Y/V = 0.198 [V]
ETMY T240 1 DOF Z/W = 0.249 [V]
ETMY T240 2 DOF X/U = -0.073 [V]
ETMY T240 2 DOF Y/V = 0.218 [V]
ETMY T240 2 DOF Z/W = 0.037 [V]
ETMY T240 3 DOF X/U = 0.268 [V]
ETMY T240 3 DOF Y/V = 0.076 [V]
ETMY T240 3 DOF Z/W = 0.147 [V]
ITMX T240 1 DOF Y/V = 0.254 [V]
ITMX T240 2 DOF X/U = 0.151 [V]
ITMX T240 2 DOF Y/V = 0.258 [V]
ITMX T240 2 DOF Z/W = 0.223 [V]
ITMX T240 3 DOF Y/V = 0.104 [V]
ITMX T240 3 DOF Z/W = 0.094 [V]
ITMY T240 1 DOF X/U = 0.074 [V]
ITMY T240 1 DOF Y/V = 0.123 [V]
ITMY T240 1 DOF Z/W = -0.013 [V]
ITMY T240 2 DOF X/U = 0.017 [V]
ITMY T240 2 DOF Y/V = 0.22 [V]
ITMY T240 2 DOF Z/W = 0.148 [V]
ITMY T240 3 DOF Y/V = 0.091 [V]
BS T240 1 DOF X/U = 0.184 [V]
BS T240 1 DOF Z/W = -0.223 [V]
BS T240 2 DOF X/U = 0.094 [V]
BS T240 2 DOF Y/V = -0.224 [V]
BS T240 3 DOF Y/V = 0.004 [V]
There are 2 STS proof masses out of range ( > 2.0 [V] )!
STS EY DOF X/U = -4.625 [V]
STS EY DOF Z/W = 2.321 [V]
All other proof masses are within range ( < 2.0 [V] ):
STS A DOF X/U = -0.498 [V]
STS A DOF Y/V = -0.915 [V]
STS A DOF Z/W = -0.485 [V]
STS B DOF X/U = 0.154 [V]
STS B DOF Y/V = 0.939 [V]
STS B DOF Z/W = -0.337 [V]
STS C DOF X/U = -0.685 [V]
STS C DOF Y/V = 0.77 [V]
STS C DOF Z/W = 0.505 [V]
STS EX DOF X/U = -0.195 [V]
STS EX DOF Y/V = -0.111 [V]
STS EX DOF Z/W = 0.112 [V]
STS EY DOF Y/V = 1.271 [V]
STS FC DOF X/U = 0.071 [V]
STS FC DOF Y/V = -1.254 [V]
STS FC DOF Z/W = 0.567 [V]
Last night I took a calibration measurement (broadband and simulines) after the IFO was locked for about 3.5 hours. This generated calibration report 20251204T035347Z.
The results from broadband and simulines on the front page look reasonable. The plot label indicates that these are from PCAL X and Y but I think the broadband and simulines are run on PCAL Y only, so I don't know if PCAL X agrees with Y. I believe as of last night, the DAC for PCAL X had been swapped.
I made no changes to the pydarm ini file to account for the DAC change before running the report.
Jeff and I looked over the report and have concluded that the changes we see are very minimal, and do not require any updates to the calibration at this time.
There is a 1% change in the L3 actuation strength, comparing the measurement from 11/18 and 12/04. This could be from the DAC change or it could be from charging. Either way, it is small enough that kappa TST is likely correcting this properly. The overall systematic error is still around 1%, as it was before the DAC change.
We don't think there needs to be any changes to the pydarm ini file at this time.
Thu Dec 04 10:15:45 2025 INFO: Fill completed in 15min 42secs
Plot has zoomed y-scale to reduce number of divisions in order to show detail.
FAMIS 27645
pH of PSL chiller water was measured to be between 10.0 and 10.5 according to the color of the test strip.
Summary of where we are in the upgrade:
h1susey
All Gen Std DACs replaced with 2 LIGO-DACs. All AI-Chassis upgraded to SCSI daisy-chain.
Timing card replaced with one having latest firmware
New models [h1iopsusey, h1susetmy, h1sustmsy, h1susetmypi]
All models running RCG-5.5.2
h1susex
All Gen Std DACs and first-gen LIGO-DAC replaced with 2 LIGO-DACs. All AI-Chassis upgraded to SCSI daily-chain.
Timing card firmware upgraded in place
New models [h1iopsusex, h1susetmx, h1sustmsx, h1susetmxpi]
All models running RCG-5.5.2
h1iscex
18bit-DAC replaced with 20bit-DAC.
Timing card firmware upgraded in place
New models [h1iopiscex, h1calex, h1pemex]
All models running RCG-5.5.2
h1iscey
Timing card firmware upgraded in place
h1seiex, h1seiey
Timing card firmware upgraded in place
h1susauxex, h1susauxey
Timing card firmware upgraded in place.
All models running RCG-5.5.2 (first test of new RCG)
h1pemmx, h1pemmy
Timing card firmware upgraded in place.
At approx 8am the cleanroom above HAM 5/6/7 was powered on in prep for vent activities and initial cleanings. As to be expected, zone 4g temp has spiked. No action on behalf of FAC is required at this time. T. Guidry K. Stewart N. Sanchez
On Tuesday 02dec2025 Marc upgraded the firmware version of the IO Chassis timing cards at both end stations and mid stations. This upgrade was done "in place" using a laptop connected to the JTAG port on the timing card.
The current timing card firmware versions for H1 are:
All are running 0xfe0 except the following which are running 0x635
h1omc0: upgraded some time ago during O4 to move its DUOTONE frequencies from [960/961Hz] to [1920/1921Hz]
h1susey: upgraded Mon 01dec2025 when its timing card was replaced as part of the LIGO-DAC upgrade
Upgraded by Marc Tue 02dec2025:
h1iscex
h1iscey
h1pemmx
h1pemmy
h1seiex
h1seiey
h1susauxex
h1susauxey
h1susey
Jonathan, Jeff, Tony, Dave:
Following the upgrade of h1iscex 18bit-DAC upgrade to a 20bit-DAC Tuesday 02dec2025 we discovered the 20bit-DAC was not outputting any voltage.
Wed 03dec2025 Tony and I went to h1iscex to replace the suspect 20bit-DAC with the second DAC which was removed from the h1susex upgrade Tue02dec2025.
All while H1 was locked, we powered down h1iscex and its IO Chassis, pulled the chassis from the rack, took its lid off, replaced the 20bit-DAC and powered the chassis up while it was open.
I powered up the computer and auto started the models. H1 lost lock soon after the IOP model started running (possible Dolphin IPC minor break in transmission?).
We verified that the replacement 20bit-DAC was working, looking at both the DAC_chan7->ADC_chan30 DUOTONE inteface card loop back and h1calex drive of DAC_chan6.
We closed the chassis and pushed it back into place.
| Suspect broken 20bit-DAC (removed) | 190219-10 |
| Replacement 20bit-DAC (installed) | 150311-01 (S1700286) |
Note that both of these 20bit-DACs came out of h1susex during its upgrade to LIGO-DACs Tue02dec2025. I'm not sure which one was driving the PI signals, and initially I chose the 2019 card for h1iscex over the older 2015 card.
Also note that the 2015 card has a LIGO S-number tag, the first I've seen on a GenStd IO card.
TITLE: 12/04 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Planned Engineering
OUTGOING OPERATOR: None
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 3mph Gusts, 1mph 3min avg
Primary useism: 0.02 μm/s
Secondary useism: 0.33 μm/s
QUICK SUMMARY:
H1 has been locked almost 15hrs....but H1 is going DOWN in prep for the HAM7 vent and will be set to IDLE.
There will be more CDS (have heard Beckhoff) work, but no "proof of life" for H1 after that work will be done.
Now the focus is on prep to vent with Kim & Nellie commencing shortly with thorough technical cleaning of HAM chambers. Other possible items on the list for today:
FYI: Elenna ran a Calibration last night (she noted in Mattermost and files were generated at 818pmPDT).
It seems like it would be useful to have this recipie written in one place. After the gate valves are opened and closed, we often have a hard time relocking because of alignment.
Here's what we think we should do to recover after gate valves open.
Thank you Sheila for writing up this process in very helpful detail.
I wanted to add a few more notes based on many iterations of this process.
#2 and 3 add on: often, if the vent was invasive enough, the other DRMI loops besides SRC ASC may also give some trouble. There are options in the use_DRMI_ASC and use_PRMI_ASC to also disengage PRC1 (PRMI and DRMI), INP1 (DRMI), PRC2 (DRMI) and MICH (PRMI and DRMI). I recommend disengaging PRC1, INP1 and PRC2 if the DRMI ASC is not working well and PRC1 for the PRMI ASC. This will then require moving PRM (PRC1), PR2 (PRC2) and INP1 (IM4) by hand. We rarely disengage MICH. Generally MICH needs to stay on during CARM offset reduction so the beamsplitter can follow DHARD. Also note that SRC2 controls both SRM and SR2, so SR2 may need to be moved as well as SRM.
#4 add on: the DHARD error signal may be very bad if the arms are too far away from resonance, and the error signals can flip sign as the CARM offset is reduced. To make DHARD engagement easier, it might be helpful to step the CARM offset by hand while moving DHARD manually, even taking smaller steps than the guardian code does. Then, DHARD can be engaged. Note: if you find yourself flipping the sign of the DHARD loops to engage them, this is a problem, and you will likely have a lockloss during CARM offset reduction if the DHARD loops are closed with the wrong sign!
#6 and 7 add on: if the alignment is poor enough, it might be necessary to walk the IFO alignment further before engaging the loops in ENGAGE_ASC_FOR_FULL_IFO by putting the guardian code in the terminal. This is the process I follow:
I like the recommendation of noting the green camera references throughout the process. If you lose lock before you get a chance to reset the references, you will have to repeat the whole process (alignment does not offload after DRMI!). Jenne's metaphor is that the green references are like video game save points, so noting them throughout the process can be helpful in case a lockloss happens, you can return to where you were before. The final green reference setting should be after all alignment loops are closed (including soft loops!).
Another offset that is useful to reset is the POP A offset. In DRMI, PRC1 runs on the POP A QPD and the alignment is offloaded. If the POP A offsets are set to the PRM's final location in ENGAGE_ASC_FOR_FULL_IFO, the ADS convergence for DOF3 will be much faster, as DRMI will put the PRM where it already needs to be. The convergence of this loops is so slow it sets how long this state takes, which can be many minutes if the PRM is far off from where it needs to be.
Created a wiki page for Recovering the IFO After Gate Valve Closures and added a link to that wiki in our ops Troubleshooting the IFO wiki. This way we can make changes next time we exercise this procedure and hopefully it leaves more bread crumbs to find it again.
TITLE: 12/04 Eve Shift: 0030-0600 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Planned Engineering & LOCKED
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 10mph Gusts, 6mph 3min avg
Primary useism: 0.02 μm/s
Secondary useism: 0.28 μm/s
QUICK SUMMARY:
SQZ_OPO_LR was giving us this error: "CLF frequency out of range, check CLF CMB". I do not recall how this was resolved by Daniel while he was checking out the CMB. Once that was resolved we got the classic: "pump fiber rej power in ham7 high, nominal 35e-3, align fiber pol on sqzt0." issue. The Quarter & Half wave plates on SQZT0 were touched up to minimize H1:SQZ-SHG-FIBR_REJECTED_DC_POWER_MON. OPO was locked. Great!
Then the FC wasn't locking:
So FC2 was moved from its nominal location. only after I accidentally moved ZM1 and reverted it.
Fumbled with the FC2 sliders for a while, then used the clear history button.
Fumbled around some more with the FC2 sliders.
Locked the the FC with FC2 P @ 246.1 Y @ 51.1
And now the SQZ has been SQuoZe.
Forcast for tonight's wind looks good for a stable lock.
[Sheila, Jeff, Elenna, Corey]
Today we relocked with some issues. We think that the soft close of the gate valves shook the ITM green cameras enough that the green references were no longer good, which caused many alignment problems through carm offset reduction.
The SRC ASC is OFF in DRMI ASC at this time, by having the use_DRMI_ASC['SRC1'/'SRC2'] flag set to False. When locking, SRM and SR2 might need to be moved by hand before offloading DRMI ASC.
Sheila will write a more detailed alog about the process of updating the green references, because we think we have a better method now. For the summary, we have updated the green camera references, SDFed them in safe, and run an initial alignment after the lockloss.
Other guardian changes:
Tony, Oli and I have been watching violin mode damping. So far, there doesn't seem to be any problems.
By eye, Jeff and I think the PCAL X crosses do not match the line height in CAL DELTA L, so there may be some calibration mismatch. I will run a cal measurement when we are thermalized.
Squeezer is not working, Daniel and others are trying to troubleshoot now.
TITLE: 12/03 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Planned Engineering
INCOMING OPERATOR: None
SHIFT SUMMARY:
Main goal for today was to continue to see troubleshoot H1 to see if it survived the CDS work from earlier this week (by trying to get H1 back to NLN)---Elenna and Sheila worked on this. There continued to be issues with ASC SRC---Elenna said this is common for H1 recoveries. Sheila also mentioned that the alignment issue most likely is a result of Gate Valve closures affecting the Green Camera alignment.
Recovery work above had to wait until about 11amPDT due to remaining LVEA crane inspections.
During Recovery work 20-bit DAC card was swapped out for one which was installed yesterday that had issues.
LOG:
WP12901 Daniel, Marc, Fil, Jonathan, Richard, Jeff, Oli, EJ, Dave:
Following from Monday's success in upgrading h1susey to the new LIGO-DACs on Tuesday 02dec2025 we upgraded h1susex to use LIGO-DACs.
This was a slightly different upgrade compared to h1susey because for the past 18 months h1susex ESDs have been driven using an early version of the LIGO-DAC. The h1susetmx model was using the GenStd 18/20-bit DACs for the upper stages and the LIGO-DAC for the L3 and ESD. The other models (h1sustmsx and h1susetmxpi) continued to use the Gen Std DACs.
The ESD and PI drives were being driven by a single 20bit-DAC using the special PI-AI chassis (first 6 chans through standard filters and output via DB15, last 2 chans through PI filters and output via DB9). When we upgraded the ESD drives to the LIGO-DAC we retained the original PI-AI chassis for these signals and temporarily installed the spare PI-AI chassis for h1susetmxpi 20bit-DAC's exclusive use.
The original LIGO-DAC used the 8-channel interface card. This card took the 32 channels from the LIGO-DAC and outputed the first 8 using its onboard DB25 connector. The remaining 24 channels were distributed using 3 Header Slot Plates (each with a DB25) ribbon cabled to the interface card. To fit these into the chassis, we displaced the two BIO cards to A4-3 and A4-4.
The IO Chassis changes were then:
Remove first-gen LIGO-DAC card, its 8chan interface card and the 3 header slot plates.
Remove the 3 18bit-DACs and their interface cards
Remove the 2 20-bit DACs and their interface cards
Install 2 LIGO-DACs in A1-4 and A2-2 with their new LIGO-DAC interface cards
Move the 2 BIO cards back to A4-1 and A4-2.
Fil upgraded all the AI chassis for LIGO-DAC SCSI Connection, duplicating what was done at EY. The second LIGO-DAC drives the PI-AI chassis, the first LIGO-DAC drives the 2 standard AI chassis, now daisy chained together.
Temporary cabling from the original LIGO-DAC chans 16-31 looped over to h1iscex were removed.
LIGO-DACs ADDED
| DAC0 S2500451 | Interface card SN007 |
| DAC1 S2500456 | Interface card SN005 |
Cards Removed
| First Gen LIGO-DAC | S2400042 |
| LIGO-DAC Interface Card | "2" in marker pen |
| 18bit-DAC | 110425-32 |
| 18bit-DAC | 110425-22 |
| 18bit-DAC | 101208-74 |
| 20bit-DAC | 190219-10* |
| 20bit-DAC | 150311-01 (S1700286) |
| 18/20-DAC Interface card | S1200225 |
| 18/20-DAC Interface card | S1104340 |
| 18/20-DAC Interface card | S1201759 |
| 18/20-DAC Interface card | S1102690 |
| 18/20-DAC Interface card | S1102687 |
* 20bit-DAC installed into h1iscex, now suspect.
Tagging SUS and CAL, as these are the DAC cards for SUS ETMX. We have done the logical thing of applying a gain factor of 1024x in the COIL and ESDOUTF banks to again mimic the calibration function of a 18- bit, like the factor of 4x that's been in play for a while after we upgraded to a 20-bit DAC. But, of course at the calibration group's level of precision / accuracy the DAC cards / and channels within will NOT have the same calibration as 2^18 / 20 [ct/Vpp_differential] * 2^10 = 2^28/20 [ct/Vpp]. This will change the open loop gain magnitude of the top mass damping loops, but to a much less important degree. I expect a similar level of impact on ISC loops wrapped around ETMX too. Also now that these LIGO 32CH DAC have had their maximum output voltage tuned to spit out 10.0 Vpp differential at the output of the AI chassis. This is unlike the 18-bit DAC who's output saturated at ~9.7 Vpp, or the 20-bit DACs who's output saturated at ~9.3Vpp.
Model Changes:
h1iopsusex (Dave) modified to new DAC configuration. SWWD section updated to new DAC cards
h1susetmx, h1sustmsx, h1susetmxpi (Jeff, Oli) updated to new DAC configuraiton.
All models built and installed with RCG5.5.2
The following AI Chassis were modified for installation of the LIGO 32 CH DAC:
AI Chassis D1000305 S1104367 (SUSEY-C1, slot U32, new assembly drawing D2500353)
AI Chassis D1000305 S1001202 (SUSEY-C1, slot U31, new assembly drawing D2500353)
AI Chassis D1500177 S1500299 (SUSEY-C1, slot U26, new assembly drawing D2500400)
The rear panel and DAC AI Interface Board D1000551 were removed. A new D2400308 LIGO DAC Anti Image Chassis Rear Panel and LIGO DAC AI Interface D2500097 were installed.
Tuesday Maintenance day impact notes.
Due to Beckhoff upgrades and restarts the PCAL lasers were shutoff.
When Beckhoff came back up the PCAL Lasers oddly didn't come back up this time.
The H1:CAL-PCALX_LASERPOWERCONTROL voltage was set to 0. I took that back to 5V for both arms.
Turned H1:CAL-PCALX_LASERENABLE back on.
H1:CAL-PCALX_SHUTTERPOWERENABLE back on.
Arm specific Settings :
PCALX:
H1:CAL-PCALX_OPTICALFOLLOWERSERVOOFFSET was set back to 3.65V.
H1:CAL-PCALX_OPTICALFOLLOWERSERVOGAIN : 39.60 dB
PCALY:
H1:CAL-PCALY_OPTICALFOLLOWERSERVOOFFSET : 3.80 V
H1:CAL-PCALY_OPTICALFOLLOWERSERVOGAIN : 38.56 V
Hey future Tony, Today is the day the X Arm 18bit DAC was changed out for a 20bit as well.
I was not able to get PCALX excitations in DARM. The new 20 bit H1IOPSCEX DAC is then plugged into an 18 bit AI chassis. Dave and I went out to the End station and put an Oscilloscope on it to try and see any sort of excitation coming out of the 18 Bit AI chassis. No dice.
We will need to do more troubleshooting tomorrow.
Relvent Alog: https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=88333
Dave and I went back down to EX and Swapped out a DAC in H1ISCEX, and the Excitations and Duotone started to work again.
DAC card that was put in on Tuesday : SN 190219-10 <- bad DAC
DAC card that was swapped in today : SN150311-01 This is the working DAC.
I have since put back the OFSPD offset and Gain back to their nominal settings listed above.