Since the lockloss at earlier this morning, high winds up to 40mph have made locking H1 problematic; on occasion we can lock DRMI, but rarely. Unfortunately, the forecast says these winds will stick around for most of today.
Lockloss @ 16:51 UTC - link to lockloss tool
Cause looks to me to be from high winds; gusts just hit up to 40mph and suspensions were moving right before the lockloss in a familiar way.
H1 back to observing at 21:09 UTC. After a few hours, thanks to a lull in the wind, H1 was able to get relocked. Fully automatic acquisition.
Sun Sep 29 08:11:15 2024 INFO: Fill completed in 11min 11secs
TITLE: 09/29 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: Ryan C
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 17mph Gusts, 10mph 3min avg
Primary useism: 0.03 μm/s
Secondary useism: 0.16 μm/s
QUICK SUMMARY: H1 appears to have unlocked around an hour ago and is currently relocking up to INJECT_SQUEEZING. Four locklosses in total overnight; I'll start looking into them shortly.
TITLE: 09/29 Eve Shift: 2330-0500 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Observing at 153Mpc
INCOMING OPERATOR: Ryan C
SHIFT SUMMARY: Observing at 155Mpc and have been Locked for 9 hours. We've been Locked my entire shift so it's been a quiet evening. Didn't get a chance to take today's calibration measurements because Virgo and Livingston stayed nice and locked.
LOG:
23:00 Observing and have been Locked for 3 hours
Closes FAMIS#26447, last checked 79824
Driftmon plots, auxiliary plots
Everything is looking good. BRSX is close-ish to the lower threshold after the latest picomotor move (80269) but looks to be staying level.
First spike on BRSX plot is pico work from September 19th (80183)
Second spike on BRSX plot is pico work from September 24th (80269)
Observing at 150Mpc and have been Locked for 7 hours now. It's been a quiet evening.
Closes FAMIS#26331, last checked 80086
Note: I'm trending them back two weeks since last week the check wasn't able to be done.
Corner Station Fans (attachment1)
- All fans are looking normal and within range.
Outbuilding Fans (attachment2)
- All fans are looking normal and within range.
TITLE: 09/28 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 150Mpc
INCOMING OPERATOR: Oli
SHIFT SUMMARY: Mostly quiet shift with just one lockloss and an easy reacquisition. Because of the lockloss, however, we did not get the usual calibration sweeps done today.
H1 has now been locked and observing for 3 hours.
TITLE: 09/28 Eve Shift: 2330-0500 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Observing at 149Mpc
OUTGOING OPERATOR: Ryan S
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 7mph Gusts, 4mph 3min avg
Primary useism: 0.02 μm/s
Secondary useism: 0.24 μm/s
QUICK SUMMARY:
Currently Observing at 155Mpc and have been Locked for 3 hours. Microseism is trending back down and winds are low.
Lockloss @ 18:58 UTC - link to lockloss tool
No obvious cause, but the lockloss tool gave this the FSS_OSCILLATION tag. This happened two minutes before I was planning on running the scheduled calibration sweeps.
H1 back to observing at 20:04 UTC. Fully automated relock without an initial alignment.
FAMIS 26301
Laser Status:
NPRO output power is 1.828W (nominal ~2W)
AMP1 output power is 64.52W (nominal ~70W)
AMP2 output power is 136.7W (nominal 135-140W)
NPRO watchdog is GREEN
AMP1 watchdog is GREEN
AMP2 watchdog is GREEN
PDWD watchdog is GREEN
PMC:
It has been locked 0 days, 11 hr 34 minutes
Reflected power = 22.33W
Transmitted power = 104.0W
PowerSum = 126.3W
FSS:
It has been locked for 0 days 1 hr and 1 min
TPD[V] = 0.7807V
ISS:
The diffracted power is around 2.4%
Last saturation event was 0 days 1 hours and 2 minutes ago
Possible Issues: (both are known)
AMP1 power is low
PMC reflected power is high
Sat Sep 28 08:13:16 2024 INFO: Fill completed in 13min 12secs
TITLE: 09/28 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: Ryan C
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 7mph Gusts, 6mph 3min avg
Primary useism: 0.02 μm/s
Secondary useism: 0.27 μm/s
QUICK SUMMARY: H1 just lost lock at 14:16 UTC, 8:34 into the lock stretch. I don't notice anything obvious as to the cause except maybe some ETMX motion immediately prior, but this doesn't look like one of the ETMX "glitches" like we've seen before. Have been trying to lock green arms for 30 minutes, so I'll start touching them up by hand.
DIAG_MAIN is reporting "TCS_LASER: Y power too low," and that seems to be the case since the CO2Y laser relocked at 10:13 UTC this morning, when the laser output dropped from 30.20W to 29.94W (screenshot attached, tagging TCS).
This caused H1 to drop observing for only about a minute and a half.
H1 back to observing at 16:05 UTC. Fully automatic relock after running an also automatic initial alignment.
TITLE: 09/28 Eve Shift: 2330-0500 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Lock Acquisition
INCOMING OPERATOR: Ryan C
SHIFT SUMMARY:
Currently relocking and at ACQUIRE_DRMI_1F. We were cleared to go into Observing by the CAL team 45 minutes ago when we were still Locked and got in 10 minutes of Observation time before losing lock from an FSS oscillation.
Before that, troubleshooting was being done by the calibration team for the entirety of my shift, so I had just been trying to keep us locked and by running broadband measurements when they needed them. Current status of the calibration work is that they tried multiple things with ultimately no success. The things they tried have been reverted, and they will be writing up their own alog and continuing work on Monday.
While trying to relock after the first lockloss of my shift and this last lockloss, we got stuck in READY due to IMC_LOCK being in FAULT with the DIAG_MAIN messages: "PSL_FSS: PZT MON is high, may be oscillating" and "PSL_FSS_TPD: RefCav transmission low, fix alignment". I tried taking the IMC to DOWN and then back to LOCKED and tried toggling the FSS loop automation OFF and then back ON, but neither seemed to help. But both times the IMC was eventually able to lock, but I'm not sure that it had anything to do with my toggling. Once the IMC was locked, relocking went smoothly for that first relock and so far is going well this relock.
So the FSS oscillations are not only causing locklosses, but now are also starting to delay relocking. Tagging ISC
LOG:
22:16UTC Lockloss
22:19 Lockloss from LOCKING_ARMS_GREEN
22:20 Lockloss from LOCKING_ARMS_GREEN
22:21 Lockloss from LOCKING_ARMS_GREEN
IMC in FAULT with the messages "PSL_FSS: PZT MON is high, may be oscillating" and "PSL_FSS_TPD: RefCav transmission low, fix alignment"
- Taking IMC_LOCK to DOWN+INIT and then back to LOCKED didn't work
- Toggled FSS loop automation autolock OFF/ON, eventually IMC locked a couple minutes after doing this so not sure if it helped
23:29 NOMINAL_LOW_NOISE
23:29 NLN_CAL_MEAS
23:30 Started broadband calibration measurement
23:36 measurement done (PCALY2DARM_BB_20240927T233009Z.xml), back to NOMINAL_LOW_NOISE
23:41 NLN_CAL_MEAS
23:42 Started broadband calibratoin measurement #2
23:48 Measurement done, (PCALY2DARM_BB_20240927T234226Z.xml) back to NOMINAL_LOW_NOISE
00:11 Lockloss
00:34 Started an initial alignment
00:52 Initial alignment done, relocking
01:37 NOMINAL_LOW_NOISE
01:43 NLN_CAL_MEAS
01:44 Started broadband calibration measurement #3
01:50 measurements done (PCALY2DARM_BB_20240928T014428Z.xml)
01:50 NOMINAL_LOW_NOISE
03:00 NLN_CAL_MEAS
03:01 Started broadband calibration measurement
03:08 Measurement done, back to NOMINAL_LOW_NOISE
03:08 Lockloss due to earthquake https://ldas-jobs.ligo-wa.caltech.edu/~lockloss/index.cgi/event/1411528136
04:05 NOMINAL_LOW_NOISE
04:16 Observing
04:22 Lockloss due to FSS oscillation starting 30 before LL
FSS oscillating and won't lock - same DIAG_MAIN messages as earlier
04:39 Got it locked, starting an initial alignment
04:56 Initial alignment done, relocking
| Start Time | System | Name | Location | Lazer_Haz | Task | Time End |
|---|---|---|---|---|---|---|
| 22:44 | Keita | CER | n | Putting laptop away | 22:46 |
Currently relocking and at MOVE_SPOTS - we lost lock about 30 minutes ago from a nearby earthquake.
The CAL team has been spending the last several hours troubleshooting the issues with the H1 calibration. We've had two other locklosses during my shift besides this latest earthquake one, both were easy to relock from, but have slightly slowed down the calibration troubleshooting.
Note: There is an issue with the flag that decides whether or not we have clean data for H1:CDS-SENSEMON_CAL_SNSC_EFFECTIVE_RANGE_MPC. Since 09/27 20:25UTC we have not been Observing (so the past 5 hours), but diaggui and ndscope are both showing data for the CLEAN channel.
Lockloss during Post-Event Standdown. Unknown cause not enviuronmental. Relocking now buyt having difficulties with both ALSY and ALSX.
