IFO is in NLN and OBSERVING as of 19:26 UTC

Mini-Events Today:

• Lost lock around time when CP1 Fill just ended - Dave informed us that while the last part of the fill is loud, it doesn't usually cause a lockloss
• Sheila edited the A2L Script TJ made earlier this week (adjusting wait times to allow gain to settle - alog 78438
• Had to run and rerun Align SQZ script due to High Freq Noise being higher than usual
• Accepted and reverted certain diffs per Sheila's instruction (from aforementioned edits). The diffs are linked below.

Since the yaw ASC cross couplings to DARM were high in our last lock, Ibrahim and I took a couple minutes before going into observing to run the script that TJ edited yesterday  (78419) again.  The attached screenshot shows that in our first time running the script the ADS values were still drifting when the script moved on to measuring the next step.  This might have been because the IFO had locked recently, or because it needs to wait longer.  We added a 30 second wait after we change the A2L gains before checking the values by adding +30 on line 185, after this it looked like things had settled well. 

We ran the script for all test masses yaw only from the terminal using: python a2l_min_multi.py --quads ETMX ETMY ITMX ITMY --dofs Y 

After the script ran, there were many SDF diff that Ibrahim has screenshots of.  It would be good to edit the end of the script to have fewer of these diffs each time we run it. We added these new values to lscparams and loaded ISC_LOCK.

Here are our first and second run results. 

*************************************************
RESULTS
*************************************************
          ETMX Y
Initial:  4.94
Final:    4.95

     ETMY Y
Initial:  0.94
Final:    1.01
          ITMX Y
Initial:  2.89
Final:    2.89

          ITMY Y
Initial:  -2.51
Final:    -2.39
Diff:     0.11999999999999966

*************************************************
RESULTS
*************************************************
          ETMX Y
Initial:  4.95
Final:    4.91
Diff:     -0.040000000000000036

          ETMY Y
Initial:  1.01
Final:    1.0
Diff:     -0.010000000000000009

          ITMX Y
Initial:  2.89
Final:    2.85
Diff:     -0.040000000000000036

          ITMY Y
Initial:  -2.39
Final:    -2.45
Diff:     -0.06000000000000005

Here are some plots that show the fuctioning of the fast shutter, before, during and after the June 7th pressure spikes.  The fast shutter functions the same way before and after the pressure spikes.  However, in the locking attempts with the HAM6 alignment different the beam going to AS_C was clipped, and this meant that the fast shutter didn't block the beam on AS_A and AS_B as quickly as it normally does. 

The first attachment shows a lockloss from June 6th, the lockloss before the pressure spikes started.  The fast channel that records power on the shutter trigger diode (H1:ASC-AS_C_NSUM_OUT_DQ) is calibrated into Watts arriving in HAM6.  Normally these NSUM channels are normalized by the input power scaling, but as the simulink screenshot shows that is not done in this case.  Using the time that the power on AS_A is blocked, the shutter triggered when there was 0.733 Watts arriving in HAM6, and the light on AS_A which is behind the shutter is blocked.  There is a bounce, where the beam passes by the shutter again 51.5 ms after it first closes, this bounce last 15ms and in that time the power into HAM6 rises to 1.3W.  In total, AS_A (and AS_B) where saturated for 24ms.  This pattern is consistent for 4 locklosses in several locklosses from higher powers, with the normal alignment on AS_C, both before and after the pressure spikes in HAM6. 

In the first lockloss with a pressure spike, 78308, the interferometer input power was 10W, rather then the usual 60W, and the alignment into HAM6 was in an unusual state.  The shutter triggered when the input power was 0.3Watts according to AS_C, which was clipped at the time and so is underestimating the power into HAM6.  The trend of power on AS_A and AS_B was different this time, with what looks like two bounces and a total of 35ms of time when AS_A was saturated.  The first bounce is about 14ms after the shutter first triggers, but the beam isn't unblocked enough to satrate the diode, and a second bounce saturates the diode 36ms after the shutter first closed, and lasted 26ms, during which time the power into HAM6 rose to 0.55W.  The power on AS_C peaked about 250ms after the shutter triggered, at about 1W onto AS_C.  Keita is going to look at energy deposited into HAM6 in typical locklosses, where AS_C is not clipping as that will be more accurate. The pressure spike shows up on H0:VAC-LY_RT_PT152_MOD2_PRESS_TORR about 1.5 seconds after the power peaks on AS_C, and peaks at 1.1-7 Torr.

At the next pressure spike, the interferometer was locked with 60W input power, and AS_A was saturated for 80ms before the shutter triggered, and there was no bouncing.  This time the pressure rise was recorded 2 seconds after the lockloss, and 1.8 seconds after the power on AS_C peaked, this was a larger pressure spike than the first at 1.3e-7 Torr.  

The third pressure spike was also a 60W lockloss, with AS_A saturated for 80ms and no bounce from the AS shutter visibile.  The pressure rise was recorded 2.1 seconds after the lockloss, and was 3.1e-7 Torr. 

The final attachment shows the first lockloss after we reverted the alignment in AS_C, when the fast shutter behavoir follows the same pattern seen before the pressure spikes happened.

Fri Jun 14 10:09:56 2024 INFO: Fill completed in 9min 53secs

Jordan confirmed a good fill curbside.

Closes FAMIS 26252

Laser Status:
    NPRO output power is 1.813W (nominal ~2W)
    AMP1 output power is 66.77W (nominal ~70W)
    AMP2 output power is 137.7W (nominal 135-140W)
    NPRO watchdog is GREEN
    AMP1 watchdog is GREEN
    AMP2 watchdog is GREEN
    PDWD watchdog is GREEN

PMC:
    It has been locked 16 days, 20 hr 27 minutes
    Reflected power = 21.32W
    Transmitted power = 106.0W
    PowerSum = 127.3W

FSS:
    It has been locked for 0 days 18 hr and 48 min
    TPD[V] = 0.8414V

ISS:
    The diffracted power is around 2.7%
    Last saturation event was 0 days 18 hours and 48 minutes ago

Possible Issues:
    PMC reflected power is high

TITLE: 06/14 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 152Mpc
OUTGOING OPERATOR: Corey
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 9mph Gusts, 5mph 5min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.12 μm/s
QUICK SUMMARY:

IFO is in NLN and OBSERVING (17 hrs 28 mins)

Woke up to see that the SQZ_OPO_LR Guardian had the message:

"disabled pump iss after 10 locklosses. Reset SQZ-OPO_ISS_LIMITCOUNT to clear message"

Followed 73053, but did NOT need to touch up the OPO temp (it was already at its max value); then took SQZ Manager back to FRE_DEP_SQZ, and H1 went back to OBSERVING.

TITLE: 06/14 Eve Shift: 2300-0800 UTC (1600-0100 PST), all times posted in UTC
STATE of H1: Observing at 150Mpc
INCOMING OPERATOR: Corey
SHIFT SUMMARY: Quiet shift with H1 locked and observing for the duration. There was one errant Picket Fence trigger (see midshift log) and we rode out a M5.9 EQ from the southern Atlantic, but otherwise uneventful. H1 has now been locked for 11 hours.
LOG: No log for this shift.

State of H1: Observing at 157Mpc, locked for 6.5 hours.

Quiet shift so far except for another errant Picket Fence trigger to EQ mode just like ones seen last night (alog78404) at 02:42 UTC (tagging SEI).

SheilaD, FranciscoL, ThomasS [Remote: LouisD]

Today, we tried to evaluate the effect on H1:CAL-CS_TDEP_KAPPA_TST_OUTPUT from changing H1:SUS-ETMX_L3_DRIVEALIGN_L2L_GAIN. In summary, we were not able to collect enough data and we want to do this again with a quiet IFO.

We used the script KappaToDrivealign.py to change the value of H1:SUS-ETMX_L3_DRIVEALIGN_L2L_GAIN. The script worked as intended but has to round off the the value that it uses to change the drivealign to the digits that EPICS can take. H1:SUS-ETMX_L3_DRIVEALIGN_L2L_GAIN changed from 184.65 to 186.53576 (1.012%). We are chainging the gain to reduce the difference of the model of the actuation function and the measurement of \DeltaL_{res}. We did not change H1:CAL-CS_DARM_FE_ETMX_L3_DRIVEALIGN_L2L_GAIN because that would have a net-zero change between measurement and model. We were not able to assess the effect of our change at the end of the commissioning period so we reverted our changes.

kappa_tst_crossbar.png shows a trend of the channels we are concerned of. The crossbar on KAPPA_TST shows that the value drops closely after the gain of DRIVEALIGN_L2L was reverted. However, the uncertainty from H1:CAL-CS_TDEP_PCAL_LINE3_UNCERTAINTY increased during these values, which makes them less reliable. The data between the vertical dashed lines on KAPPA_TST from kappa_tst_deltat.png shows that there was a period of time at which KAPPA_TST was closer to 1 along with a lower uncertainty of PCAL_LINE3.

There are two ways we could repeat this measurement: With 25 minutes of quiet time of the interferometer, we can monitor the changes of KAPPA_TST after changing DRIVEALIGN_L2L gain *or* we can measure simulines before and after changing DRIVEALIGN_L2L. The latter method would illustrate better the sability of the UGF, but it would take around 40 minutes.

To recap the data we collected today was not reliable enough because it had a high uncertainty and it was not enough time to clearly see the effect of our changes. We will try again by either monitoring KAPPA_TST or running simuline measurements.

• O4b so far = 110 locklosses / 925 hours =  1 lockloss per 9 hours in Observing
• O4a = 350 locklosses / 5700 hours = 1 lockloss per 16 hours in Observing
• O3b = 247 locklosses / 2790 hours = 1 lockloss per 11 hours in Observing
• O3a = 182 locklosses / 3120 hours = 1 lockloss per 17 hours in Observing

I've also looked at the amount of hours we've lost from unknown locklosses, which is the majority of our locklosses at around 65%, and how many theortircal GW's we coudl ahev missed becasue of these. 

O4b so far 110 LHO locklosses from observing in O4b [1] x 65% unknown [2] x 2 hours down per lockloss = 143 hours downtime from unknown locklosses. 925 observing hours in O4b so far (60% of O4b [3], from April 10th).

O4a 350 LHO locklosses from observing in O4a [1] x 65% unknown [2] x 2 hours down per lockloss = 450 hours downtime from unknown locklosses. 5700 observing hours in O4a (67% of O4a [3]) with 81 candidates  in O4b [4]= 1 candidate / 70 hours so 450 hours / 70 observing hours per candidate ~ 6.5 gw candidates lost due to unknown locklosses The Ops team notes that we had considerably more locklosses during the first ~month of O4a while using 75W input power.

O3b 247 LHO locklosses from observing in O3b [5] x 65% unknown [using same as O4a] x 2 hours down per lockloss = 320 hours downtime from unknown locklosses. 2790 observing hours in O4a (79% of O3b [3]) with 35 events [6]= 1 events / 80 hours 320 hours / 80 observing hours per event ~ 4 events lost due to unknown locklosses.

O3a 182 LHO locklosses from observing in O3a [5] x 65% unknown [using same as O4a] x 2 hours down per lockloss = 235 hours downtime from unknown locklosses. 3120 observing hours in O4a (71% of O3a [3]) with 44 events [6]= 1 events / 70 hours 235 hours / 70 observing hours per event ~ 3.3 events lost due to unknown locklosses.

We currently don't track downtime per lockloss, but we could think about tracking it, 2 hours is a guess. It may be as low as 1 hour. 

TITLE: 06/13 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 148Mpc
INCOMING OPERATOR: Ryan S
SHIFT SUMMARY: Commissioning and calibration this morning, followed by a lock loss. Relocking was not automated for two things: touched ETMX Y by 0.3urads and it caught, then PRM to avoid an initial alignment and get PRMI to lock. Locked for 2 hours now.
LOG:

• 1932 Observing
• 1948 Lock loss
• 2107 Observing
• 2202-2206 Dropped observing, SQZ lost lock

For operators: I put back in the MICH Fringes and PRMI flags that Sheila and company put to False (no auto MICH or PRMI) last weekend (alog78319). So it should try MICH and PRMI on its own again.

TITLE: 06/13 Eve Shift: 2300-0800 UTC (1600-0100 PST), all times posted in UTC
STATE of H1: Observing at 149Mpc
OUTGOING OPERATOR: TJ
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 6mph Gusts, 4mph 5min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.09 μm/s
QUICK SUMMARY: H1 has been locked and observing for 2 hours; things are running smoothly after the commissioning window earlier today.

• Environment is calm
• All other systems nominal

I took the script that we have been using to run our A2L and converted it to run the measurements for all quads and degrees of freedom at the same time, or less, as desired. The new script is (userapps)/isc/h1/scripts/a2l/a2l_min_multi.py. Today Sheila and I tested it for all quads with just Y with the results below. These values were accepted in SDF, updated in lscparams.py, and ISC_LOCK reloaded. More details about the script at the bottom of this log.

Results for ETMX Y
Initial:  4.99
Final:    4.94
Diff:     -0.04999999999999982

Results for ETMY Y
Initial:  0.86
Final:    0.94
Diff:     0.07999999999999996

Results for ITMX Y
Initial:  2.93
Final:    2.89
Diff:     -0.040000000000000036

Results for ITMY Y
Initial:  -2.59
Final:    -2.51
Diff:     0.08000000000000007

The script we used to use was (userapps)/isc/common/scripts/decoup/a2l_min_generic_LHO.py which was, I think, originally written by Vlad B. and then Jenne changed it up to work for us at LHO. I took this and changed a few things around to then call the optimiseDOF function for each desired quad and dof under a ThreadPool class from multiprocess to run all of the measurements simultaneously. We had to move or change filters in the H1:ASC-ADS_{PIT,YAW}{bank#}_DEMOD_{SIG, I, Q} banks so that each optic and dof is associated with a particular frequency and used the ADS banks 6-9. These frequencies needed to be spaced apart enought but still within our area of interest. We also had to engage notches for all of these potential lines in the H1:SUS-{QUAD}_L3_ISCINF_{P,Y} banks (FM6&7). We also accepted the ADS output matrix values in SDF for these new banks with a gain of 1.

This hasn't been tested for all quads and both P&Y, so far only Y.

I ran the DARM offset step code starting at:

2024 Jun 13 16:13:20 UTC (GPS 1402330418)

Before recording this time stamp it records the PCAL current line settings and makes sure notches for 2 PCAL frequencies are set in the DARM2 filter bank.

It then puts all the PCAL power into these lines at 410.3 and 255Hz (giving them both a height of 4000 counts), and measures the current DARM offset value.

It then steps the DARM offset and waits for 120s each time.

The script stopped at 2024 Jun 13 16:27:48 UTC (GPS 1402331286).

In the analysis the PCAL lines can be used to calculate how the optical gain changes at each offset.

See the attached traces, where you can see that H1:OMC-READOUT_X0_OFFSET is stepped and the OMC-DCPD_SUM and ASC-AS_C respond to this change.

Watch this space for analysed data.

The script sets all the PCAL settings back to nominal after the test from the record it ook at the start.

The script lives here:

/ligo/gitcommon/labutils/darm_offset_step/auto_darm_offset_step.py

The data lives here:

/ligo/gitcommon/labutils/darm_offset_step/data/darm_offset_steps_2024_Jun_13_16_13_20_UTC.txt

On Friday 06/07/2024 Dave Barker sent an email to the vacuum group noting 3 spikes on the pressure of the main vacuum envelope, I took a closer look at the 3 different events and noticed that the events correlated to the IFO losing lock.  I contacted Dave, and together we contacted the operator, Corey, who made others aware of our findings.

The pressure "spikes" were noted by different components integral to the vacuum envelope.  Gauges noted the sudden rise on pressure, and almost at the same time ion pumps reacted to the rise on pressure.  The outgassing was noted on all stations, very noticeable a the mid stations, and with less effect at both end stations, and for both with a delay.

The largest spike for all 3 events is noted at HAM6 gauge, we do not have a gauge at HAM5 or HAM4.  The one near HAM6 is the one on the relay tube that joins HAM5/7 (PT152), with the restriction of the relay tube, then the next gauge is at BSC2 (PT120), however the spike is not as "high" as the one noted on HAM6 gauge.

A list of aLOGs made by others related to the pressure anomalies and their findings:
https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=78308
https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=78320
https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=78323
https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=78343

Note: the oscillation visible on the plot of the outer stations (Mids and Ends) is the diurnal cycle, nominal behavior.

Jennie W, Sheila

Sheila wanted me to look at how good our optical gain is doing since the burn on the OFI that roughly happened (we think) on the 22nd April.

Before this happened we made a measurement of the OMC alignment using dithers on the OMC ASC degrees of freedom. We got a set of new alignment offsets for the OMC QPDs that would have increased our optical gain but did not implement these at the time.

After the OFI burn we remeasured these alignment dithers and found a similar set of offsets that would imrpove our optical gain. Thus I have assumed that we would have achieved this optical gain increase before the OFI burn if we had implemented the offset changes then.

Below is the sequence of events then a table stating our actual or inferred optical gain and the date on which it was measured.

Optical gain before vent as tracked by kappa C: 2024/01/10 22:36:26 UTC is 1.0117 +/- 0.0039

Optical gain after vent: 2024/04/14 03:54:38 UTC is 1.0158 +/- 0.0028, optical gain if we had improved OMC alignment = 1.0158 + 0.0189 = 1.0347

SR3 yaw position and SR2 yaw and pitch positions were changed on the 24th April ( starting 17:18:15 UTC time) to gain some of our throughput back.

The OMC QPD offsets were changed on 1st May (18:26:26 UTC time) to improve our optical gain - this improved kappa c by 0.0189.

Optical gain after spot on OFI moved due to OFI damage: 2024-05-23 06:35:25 UTC 1.0051 +/- 0.0035

SR3 pitch and yaw positions and the SR2 pitch and yaw positions were changed on 28th May (starting at 19:14:34 UTC time).

SR3 and SR2 positions moved back to pre-28th values on 30th May (20:51:03 UTC time).

So we still should be able to gain around 0.0296 ~ 3% optical gain increase, provided we stay at the spot on the OFI we had post 24th April:

SR2 Yaw slider = 2068 uradians
SR2 Pitch slider = -3 uradians

SR3 Yaw slider = 120 uradians

SR3 Pitch slider = 438 uradians