Data is flowing to the data concentrator again, but h1sush56 is locked up.  Restarting will almost certainly take down the Dolphin IPC network, which will disrupt most of the computers in the corner station.  So don't get too excited about taking data this morning until the system is stable.

I was looking to upload the source code tar ball for the record in LHO aLOG 8325, but after selecting the file in the browser, I hit the "upload file" button and the file just disappears from the browse field without uploading anything.

We've requested that *any* file type should be upload-able, as long as it's under the 10 Mb limit (see LHO aLOG 3798 and subsequent comments) but I guess the infrastructure doesn't allow for it?

P.S. looking if we've requested this before ('cause I thought ), I rediscovered a report of a bug that's still present:
.

J. Kissel

I've now updated the HLTS front-end simulink models as per ECR E1300578, similar to the QUADs, TMTS, and BSFM, as described in G1301192. After successful compilation of both H1SUSPR3 and H1SUSSR3, Fabrice informed me that Arnaud was gathering some data looking for long-term drift on PR3, so I only installed and restarted SR3. Of course, up successful compilation and install, I went to restart the front end with the new process and it hung halfway through, completely taking down the data concentrator / frame builder / DAQ, and took down the entire h1sush56 front end. I attach a screenshot of the CDS overview screen. *sigh*. The reigning king of finding crazy obscure bugs in CDS and exercising them wins again. The only debugging I've done is trying to reboot the data concentrator once, by doing the following

controls@opsws3:models 0$ telnet h1dc0 8087
Trying 10.101.0.20...
Connected to h1dc0.cds.ligo-wa.caltech.edu.
Escape character is '^]'.
daqd> shutdown
OK
Connection closed by foreign host.
controls@opsws3:models 1$

This brought back *some* of the front ends back up and to green status (The SEI and SUS computers at the end stations), but the corner is cooked.

Sorry Arnaud, and anyone else who was gather data overnight.

Giving up for the night and will continue fighting the good fight tomorrow morning.

I tried uploading the source from the target area, but the aLOG doesn't like tar.gz's at all.

------
Here's the status of the sus corner of the SVN repo that's a result of my work:

MM      common/models/HAUX_MASTER.mdl                  # haven't started on the HAUX yet
MM      common/models/HLTS_MASTER.mdl                  # changes complete, but don't want to commit until I can successfully start the front end process
M       common/models/SIXOSEM_T_STAGE_MASTER.mdl       # same as above
MM      common/models/MC_MASTER.mdl                    # haven't started on the HSTS yet
M       common/models/OMCS_MASTER.mdl                  # haven't started on the OMCS yet
M       common/models/SIXOSEM_T_WD_AC_MASTER.mdl       # changes complete, but don't want to commit until I can successfully start the front end process
M       common/models/SIXOSEM_T_WD_DC_MASTER.mdl       # changes complete, but don't want to commit until I can successfully start the front end process
MM      common/models/HSTS_MASTER.mdl                  # haven't started on the HSTS yet

M       h1/filterfiles/H1SUSTMSX.txt                   # Haven't committed since new code has been installed, these still need a hand clean up of now-vestigial filter banks
M       h1/filterfiles/H1SUSTMSY.txt                   #     | 
M       h1/filterfiles/H1SUSBS.txt                     #     | 
M       h1/filterfiles/H1SUSSR3.txt                    #     | 
M       h1/filterfiles/H1SUSETMX.txt                   #     | 
M       h1/filterfiles/H1SUSETMY.txt                   #     | 
M       h1/filterfiles/H1SUSITMX.txt                   #     | 
M       h1/filterfiles/H1SUSITMY.txt                   #     V
 
M       h1/models/h1susprm.mdl                         # haven't started on the HSTS yet
M       h1/models/h1sussrm.mdl                         #     |
M       h1/models/h1suspr2.mdl                         #     V
M       h1/models/h1suspr3.mdl                         # changes complete, but don't want to commit until I can successfully start the front end process
M       h1/models/h1sussr2.mdl                         # haven't started on the HSTS yet
M       h1/models/h1sussr3.mdl                         # changes complete, but don't want to commit until I can successfully start the front end process
M       h1/models/h1susomc.mdl                         # haven't started on the OMCS yet
M       h1/models/h1susmc1.mdl                         # haven't started on the HSTS yet
M       h1/models/h1susmc2.mdl                         #     |
M       h1/models/h1susmc3.mdl                         #     V

I noticed that Conlog was not reconnecting to some PEM channels after their IOCs were restarted.

One hypothesis is that it may need to have the caRepeater running, which it has not been.

I stopped Conlog around 18:12. I set the environment variable $PATH to include the path to the caRepeater binary. I restarted Conlog. I got an error not finding a library needed by caRepeater. I stopped Conlog and set the environment variable $LD_LIBRARY_PATH to include the path to the library. I restarted Conlog and it ran without any further errors. These environment variables are set in /home/controls/bashrc_import which is sourced by /home/controls/.bashrc on h1conlog.

It remains to be seen if this fixes the problem.

Should get the remaining Actuators attached tomorrow morning.

Today's activities:
- Jim W, to LVEA, lock HEPI HAM02.
- Sheila, RefCav locking lesson to operator.
- Richard M, to LVEA, cable work under HAM02.
- Apollo crew, to LVEA, move IOT2L.
- Apollo crew, door prep work, HAM02.
- Betsy, LVEA, SR2 work.
- Jim W, to LVEA, HEPI lock HAM03.
- Apollo crew, to LVEA, door prep work, HAM03
- Filiberto, LVEA and X-end, ESD measurements.
- Mitchel & Thomas, MCB assembly work, West bay area.
- Hugh and Greg, X-end, HEPI work.
- Jim B and Dave B, to Y-End, troubleshooting.

Vendors:
- Porta potty service.

(Sheila, Gerardo)

Sheila showed me how to lock the reference cavity.
One change occurred to get the system to behave, Sheila lowered the resonant threshold down to 0.5 V from 0.9 V.
The reference cavity was able to lock manually, but now it appears misaligned when locked.

Jim, Cyrus, Dave

Rolf added a new feature to RCG2.8 to permit a front end to run without an IRIGB card (GPS time is obtained via EPICS Channel Access from a remote IOC). We are in the process of testing this on h1pemmx. 

To prepare for the test, I added the line "remoteGPS=1" to the CDS block on h1ioppemmx. I added a cdsEzCaRead part, reading the GPS time from the DAQ data concentrator on channel H1:DAQ-DC0_GPS. I svn updated the trunk area, and compiled h1ioppemmx and h1pemmx against the latest trunk.

Test 1: keep the IRIG-B card in the computer, restart the IOP model several times. We noticed that the sync of the GPS time from IOPPEMMX and its reference DC0 does change from restart to restart but keeps synchronized to within a second.

We are in the process of test 2, removing the IRIGB card from h1pemmx. At the same time, Cyrus is reconfiguring the X-ARM switching sysems for the FrontEnd and DAQ switches, which will permit replacement of two netgear switches at MX with media converters. The use of full switches to support a single front end computer is obviously wasteful.

On completion of today's IRIGB tests, we will re-install the IRIGB card and reload the 2.7.2 version of the IOP and PEM code. While this test is progressing the DAQ status from MX is 0x2000 and its data is bad.

Jim, Dave.

Eagle eyed Kyle noticed that the medm screen snapshots stopped working at 6am this morning. script0 was pingable, but we could not ssh onto it. Its console was frozen, and it had to be rebooted. We restarted the medm screen snapshot program.

Jim and Dave

We restarted the user and iop models on h1susey several times investigating the DAC status bits (follow on from yesterday's ITMX,Y issue). We did not find any problems at EY, the status bits are consistent with the AI units being powered down. We wanted to try powering them up, but they are missing the +15V supply.

Jason and I cut away the Ameristat from around the legs of tripods and realigned the instruments to have them ready in the AM.

I added new scribe lines to the ACB targets to represent the new horizontal centerlines. 

This morning at ~9:30 I locked HAM2 HEPI. At ~ 1:30pm, HAM3 HEPI got a similar treatment. Offsets from floating position for both were about 100 cts (=100cts / [(7.87V/mm)*(1638cts/V)] ~.0003"), which is what Hugh reported he shot for when locking.

Started around 9:00 AM on October 29. Plot attached.

R. McCarthy, P. King

The in-air cables (D1300464) used for the outer loop power stabilisation photodetector array were installed (see
attached pictures).  Looking at the flange, from left to right.  On the left hand side subflange the cables S1301012
and S1301013 were installed.  On the right hand side subflange the cables S1301014 and S1301015 were installed.  These
were attached to the black coloured mating pieces and are face to face flush as shown in the second attached picture.

Results of main and reaction chain of ITMX (in chamber, in air, suspension undamped, ISI locked) are showing a very good match with the model and phase 2b (test stand) measurements, so the suspension is free of rubbing for alignment, cf first attached pdf for the main chain and second one for the reaction chain. The two last pdf are showing the tf against the model (third pdf for main chain and fourth for reaction chain). As usual, the reaction chain's pitch is off due to the stiffness added by the cables.

At about 9:03 PDT, there was a timing error detected by the DACs on h1sush2b, which resulted in all outputs being set to 0.  The only remedy is to restart the IOP model, so I killed h1susim, restarted h1iopsush2b, then started h1susim again.  This has cleared the DAC error.  I burt restored to 8:00 PDT.