H1 called for help after "Not ready for 15 minutes" at 11:38 UTC. IMC_LOCK and LASER_PWR are in fault. My medms aren't opening on my no machine unless I do guardmedm in the console. IMC lock is saying the PMC and FSS can't lock, LASER_PWR is in fault, unable to get to 2Ws "IMC has no power". Following the IMC locking issues wiki page I've tried DOWN initing ISC_LOCK, IMCLOCK, and LASER_PWR, the FSS noise eater is green.
TITLE: 10/21 Eve Shift: 2330-0500 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Observing at 157Mpc
INCOMING OPERATOR: Ryan C
SHIFT SUMMARY: Two locklosses this shift, one from the PSL glitching, each followed by fully automatic relocks. Otherwise, it's been a fairly quiet evening.
Lockloss @ 03:25 UTC - link to lockloss tool
The NPRO/FSS had been glitching with high frequency for about a minute prior to the lockloss, so I feel confident saying it's the cause here.
H1 back to observing at 04:23 UTC. Fully automatic relock.
Lockloss @ 01:09 UTC - link to lockloss tool
No obvious cause; doesn't look like the FSS, and I don't see evidence of an ETM glitch or DARM wiggle.
H1 returned to observing at 02:18 UTC after a fully automatic relock.
TITLE: 10/20 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 156Mpc
INCOMING OPERATOR: Ryan S
SHIFT SUMMARY:
IFO is in NLN and OBSERVING as of 21:31 UTC
Mostly quiet shift with one fussy lock acquisition.
Lock Acquisition had no special problems but rather, a slew of small annoyances (unknown cause Lockloss alog 80773):
Other:
LOG:
None
TITLE: 10/20 Eve Shift: 2330-0500 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Observing at 157Mpc
OUTGOING OPERATOR: Ibrahim
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 21mph Gusts, 16mph 3min avg
Primary useism: 0.03 μm/s
Secondary useism: 0.15 μm/s
QUICK SUMMARY: H1 has been observing for about 1.5 hours.
Closes FAMIS 26307. Last checked in alog 80649. Service mode error, see SYSSTAT.adl is the only new difference between last week's check.
Laser Status:
NPRO output power is 1.827W (nominal ~2W)
AMP1 output power is 64.44W (nominal ~70W)
AMP2 output power is 137.4W (nominal 135-140W)
NPRO watchdog is GREEN
AMP1 watchdog is GREEN
AMP2 watchdog is GREEN
PDWD watchdog is GREEN
PMC:
It has been locked 1 days, 22 hr 40 minutes
Reflected power = 24.82W
Transmitted power = 100.8W
PowerSum = 125.7W
FSS:
It has been locked for 0 days 0 hr and 4 min
TPD[V] = 0.8048V
ISS:
The diffracted power is around 2.4%
Last saturation event was 0 days 0 hours and 5 minutes ago
Possible Issues:
AMP1 power is low
PMC reflected power is high
Service mode error, see SYSSTAT.adl
Looks like I had left the PSL computer in service mode after work on Tuesday; fortunately this doesn't actually effect anything operationally, so no harm done. I've just taken the computer out of service mode.
I also added 100mL of water into the chiller since Ibrahim had gotten the warning from Verbal today.
Lockloss due to unknown cause. Not PSL this time. Still investigating.
Sun Oct 20 10:13:53 2024 INFO: Fill completed in 13min 49secs
TITLE: 10/20 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 149Mpc
OUTGOING OPERATOR: Ryan C
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 6mph Gusts, 4mph 3min avg
Primary useism: 0.02 μm/s
Secondary useism: 0.15 μm/s
QUICK SUMMARY:
IFO is in NLN and OBSERVING as of 09:46 UTC (5 hr lock).
Range is a bit low so will see what that's about.
H1 called for assistance at 09:42 UTC from the NLN timer expiring, by the time I logged in 4 minutes later we were Observing 09:46 UTC. There was a high state lockloss, at LASERNOISE_SUPRESSION.
TITLE: 10/20 Eve Shift: 2330-0500 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Observing at 152Mpc
INCOMING OPERATOR: Ryan C
SHIFT SUMMARY: Fairly quiet shift with one lockloss followed by an automatic relock. H1 has now been observing for over 2 hours.
Lockloss @ 01:32 UTC - link to lockloss tool
No obvious cause; looks like a sizeable ETMX glitch about a half second before the lockloss.
H1 back to observing at 02:47 UTC. Fully automatic relock.
The MCs and IMs alignment all looks fine. The AMPs and NPRO all went to disabled about an hour ago.
The NPRO and AMPs are all disabled as of 11:15 UTC
Running PSL weekly scripy yielded the following:
Laser Status:
NPRO output power is 0.1373W (nominal ~2W)
AMP1 output power is -0.4512W (nominal ~70W)
AMP2 output power is 0.1128W (nominal 135-140W)
NPRO watchdog is RED
AMP1 watchdog is RED
AMP2 watchdog is RED
PDWD watchdog is GREEN
PMC:
It has been locked 0 days, 0 hr 1 minutes
Reflected power = -0.1761W
Transmitted power = -0.02753W
PowerSum = -0.2036W
FSS:
It has been locked for 0 days 0 hr and 0 min
TPD[V] = -0.01628V
ISS:
The diffracted power is around 3.0%
Last saturation event was 0 days 1 hours and 26 minutes ago
Possible Issues:
NPRO power is low
AMP1 power is low
AMP2 power is low
NPRO watchdog is inactive
AMP1 watchdog is inactive
AMP2 watchdog is inactive
FSS TPD is low
NPRO error, see SYSSTAT.adl
After talking to Jason, there's not much to do about this right now. More investigation needs to be done to see what happened to the laser, potentially needs to be swapped.
There power issues on the Hanford site that could have easily glitched our power and caused the npro to trip.
Fil re-enabled the PMC high voltage after arriving on site, then I brought the PSL back up with little issue. One hiccup I ran into was with the new PDWD for the amplifiers; when the NPRO shut off this morning, the regular power watchdogs tripped the system off, but the PDWD did not. This meant I had to disable the PDWD before I was able to reset the system and turn the NPRO back on. After that, there were no issues recovering the PSL.