We have been seeing a low rate of data drops with the frame writer serving as both the frame writer and the data concentrator for the DAQD 0 systems. This work is to try an alternate configuration. Moving the gds broadcaster to be a data concentrator and making the frame writer single purpose again.
The data drops are due to messages from teh front end arriving too late and being discarded. We are not seeing them on the DAQD 1 leg, only the 0. This leads us to tend towards the issue be the load (and thus responsivness to input messages) of the FW machine.
Today I adjusted the system such that GDS0 became the data concentrator and the broadcaster.
I've attached a diagram of the current layout.
The main changes:
- Link TW0 to GDS0, this was done first to ensure the data link worked well and TW0 keeps functioning.
- Add a 10G card into gds0 and two
- Moved TW0 to read data from gds0
- Move the FE input into the gds machine and reconfigure the FW machine to just be a receiver
The final migration followed this rough order
- stop cps_xmit, cps_recv, dix_xmit on fw0
- stop dix_recv on gds0
- move the fiber to gds0
- configure the interfaces on gds0 and start
- cps_recv
- cps_xmit
- dix_xmit
- configure fw0 to use dix_recv
The control room monitoring and medm screens still expect a DC, FW, TW, NDS, and GDS. So gds0 as the DC is exporting DAQ-DC0 variables, and we are running a epics proxy ioc which maps DAQ-GDS0 channels to DAQ-DC0 for now. This will change in the future. Most of the DC0 variables will probably be taken over by the cps_recv process (this is in testing on the large test stand in LLO).
There were a few daqd restarts to make sure everything was working. Data is flowing and fw0 and fw1 are producing identical frames.
The plan is to let this run though til January and evaluate the error rate.
That is, we compute the ASD using 60s FT and check the amplitude of the ASD at the frequency of the first harmonic of the largest of the near-30 Hz combs, the fundamental at 29.9695 Hz. Then, we compute the median of the +- 5 surrounding Hz and save the ASD value at 29.9695 Hz "peak amplitude" and the ratio of the peak against the median to have a sort of "SNR" or "Peak to Noise ratio".
Note that we also check the permanent magnetometer channels. However, in order to compare them to the rest, we multiplied the ASD of the magnetometers that Robert gave us times a hundred so that all of them had units of Tesla.
After saving the data for all the positions, we have produced the following two plots. The first one shows the peak to noise ratio of all positions we have checked around the LVEA and the electronics room:
Where the X and Y axis are simply the image pixels. The color scale indicates the peak to noise ratio of the magnetometer in each position. The background LVEA has been taken from 
Note that in this figure, the color scale is logarithmic. It can be seen how, looking at the peak amplitudes, there is one particular position in the H1-PSL-R2 rack whose amplitude is around 2 orders of magnitude larger than the other positions. Note that this position also had the largest peak to noise ratio.
This position, that we have tagged as "Coil", is putting the magnetometer into a coil of white cables behind the H1-PSL-R2 rack, as shown in this image:
The reason that led us to put the magnetometer there is that we also found the peak amplitude to be around 1 order of magnitude larger than on any other magnetometer on top of one set of white cables that go from inside the room towards the rack and up towards we are not sure where:
This image shows the magnetometer on top of the cables on the ground behind the H1-PSL-R2 rack, the white ones on the top of the image appear to show the peak at its highest. It could be that the peak is louder in the coil because there being so many cables in a coil distribution will generate a stronger magnetic field.
This is the actual status of the hunt. These white cables might indicate that the source of these combs is the different interlocking system in L1 and H1, which has a chassis in the H1-PSL-R2 rack. However, we still need to track down exactly these white cables and try turning things on and off based on what we find in order to see if the combs dissapear.
So it may be that these lines may be transmitted elsewhere through this power supply.
We connected a voltage divider and connected it to the same channel we were using for the magnetometer (H1:PEM-CS_ADC_5_23_2K_OUT_DQ):
Out of this power supply, two dark green cables come out, the first one goes to the H1-PSL-R1 rack:
However, the comb did not appear as strong when we put the magnetometer besides the chassis where the cable leads.
On the other hand, the comb does appear strong if we follow the other dark green cable, that goes to this object
Which Jason told us it may be related to the interlock system.
Following the white cables that go from this object, it would appear that they go into the coil, where we saw that the comb was very strong.
We think it would be interesting to see what here can be turned off and see if the comb does disappear.
