Reports until 16:25, Friday 05 December 2025
H1 PSL (OpsInfo)
jason.oberling@LIGO.ORG - posted 16:25, Friday 05 December 2025 (88389)
PSL Enclosure Environmental Controls in Weird State After Power Outage

Jenne sent me a Mattermost message this afternoon pointing out an odd "oscillation" in Amp2 output power, so I took a look.  Sure enough, it was doing something weird.  Ever since the power outage, on a roughly 2-hour period, the output power would drop slightly and come back up.  I looked at items that directly impact the amplifier: Water/amp/pump diode temperatures, pump diode operating currents, and pump diode output.  Everything looked good with the temperatures and operating currents, but the pump diode output for 3 of the 4 pump diodes (1, 2, and 4) showed the same periodic behavior as the amp output; every ~2 hours, the pump diode output would spike and come back down, causing the "oscillation" in Amp2 output power.  See first attachment.  But what was causing this behavior?

At first, I couldn't think of anything beyond, "Maybe the pump diodes are finally starting to fail..."  Gerardo, who was nearby at the time, reminded me that during the last power outage the H2 enclosure had an odd sound coming from its environmental control system, and that it was not showing up on the control panel for the system; the control panel showed everything as OFF, but when Randy climbed on top of the enclosure he found one of the anteroom fans moving very slowly and haltingly, and making a noise as it did so.  Turning the fans off at the control panel fixed the issue at the time.  So, I took a look at the signal for the PEM microphone in the PSL enclosure, which Gerardo also reminded me of, and sure enough, the mic was picking up more noise than it was before the power outage (see 2nd attachment).  Around the same time, from the front of the Control Room Sheila noted that Diag_Main was throwing an alarm about the PSL air conditioner being ON.  It had been doing this throughout the day, but everytime I checked the AC temperature it was reading 70 °F, which was a little lower than normal but not as low as it reads when the AC is actually on (which is down around 67 °F or so).  This time, however, when I checked the AC temperature it was reading 68 °F.  Huh.  So I pulled up a trend of the PSL enclosure temps and sure enough, every 2 hours it looks like the AC comes on, drops the temperature a little, then turns off, and this behavior lines up with the "oscillation" in Amp2 output (see 3rd attachment; not much data for the enclosure temps since those come in through Beckhoff, which was recovered earlier this afternoon, but enough).

I went out and turned every PSL environmental item (HEPA fans, ACs, and make-up air) ON then OFF again and placed the enclosure back in Science Mode (HEPA fans and AC off, make-up air at 20%).  Won't know for sure if this cured the issue, as it's been happening on a 2-hour period, but looking at the PEM microphone in the enclosure shows promise.  The PEM mic is not picking up the extra noise, it's back to where it was before the power outage (see final 2 attachments).  Also encouraging, at the time of writing the AC temperatures are above the temperature where the ACs would kick on before I cycled the environmental controls.  I'll continue to monitor this over the weekend.

Images attached to this report