GNURadio@theATA: Difference between revisions

From GNU Radio
Jump to navigation Jump to search
No edit summary
(Add new section "GNU Radio Testbed usage" with current usage instructions for gnuradio1)
Line 29: Line 29:
* [https://www.astro.uni-bonn.de/hisurvey/profile/index.php LAB Survey HI Profile Search] -- here you can search for HI data given a source's RA, Dec or galactic coordinates. Good if you want to compare the velocity of the HI line that you measure with some preexisting accurate data to ensure your system is working properly.  
* [https://www.astro.uni-bonn.de/hisurvey/profile/index.php LAB Survey HI Profile Search] -- here you can search for HI data given a source's RA, Dec or galactic coordinates. Good if you want to compare the velocity of the HI line that you measure with some preexisting accurate data to ensure your system is working properly.  


== GNU Radio Testbed usage ==
=== GNU Radio installations ===
Each user of <code>gnuradio1</code> manages their own installation of GNU Radio and out-of-tree modules, either by building from source and installing to a prefix insider their <code>$HOME</code> directory, or by using Docker.
For general usage there is an installation of GNU Radio 3.10, UHD 4, and some out-of-tree modules that includes the patches required to enable LO sharing between the two USRP N32x's. The prefix for this installation can be activated with
<pre>
source ~destevez/activate-gr310_uhd4
</pre>
=== USRP tips and tricks ===
There is a known problem that causes a high noise level (and probably other signal distortions) in some of the channels of the two USRPs N320 (often the last channels). It is believed that this problem is related to locking the USRPs to the external 10 MHz reference. The problem can typically reappear after rebooting the USRPs, and it is solved as indicated here. There is a test that can be run to check if the problem is present:
<pre>
~destevez/usrp_test/usrp_test.py
</pre>
After running for a minute, this prints the noise levels present on each input when they are connected to an unused port.
<pre>
[INFO] [MPM.Rhodium-1.init.LMK04828] LMK initialized and locked!
[INFO] [MULTI_USRP]    1) catch time transition at pps edge
[INFO] [MPM.Rhodium-1.DAC37J82] DAC PLL Locked!
[INFO] [MPM.Rhodium-1.AD9695] ADC PLL Locked!
[INFO] [MPM.Rhodium-1.init] JESD204B Link Initialization & Training Complete
[INFO] [MULTI_USRP]    2) set times next pps (synchronously)
>>> Setting LO sharing
vmcircbuf_sysconfig :info: Using gr::vmcircbuf_sysv_shm_factory
[WARNING] [0/Radio#0] Attempting to set tick rate to 0. Skipping.
[WARNING] [0/Radio#1] Attempting to set tick rate to 0. Skipping.
[WARNING] [1/Radio#0] Attempting to set tick rate to 0. Skipping.
[WARNING] [1/Radio#1] Attempting to set tick rate to 0. Skipping.
Channel 0 level -16.23 dB
Channel 1 level -16.18 dB
Channel 2 level -16.05 dB
Channel 3 level -15.98 dB
Note: channel levels should be around -16 dB in this test
</pre>
As indicated by the test, the level in each channel should be around -16 dB. If it is significantly higher in any of the channels, the workaround is to run:
<pre>
uhd_cal_rx_iq_balance --args "addr0=10.11.1.20,addr1=10.11.1.22,master_clock_rate=250e6"
</pre>
This does not calibrate the USRPs. In fact it will give the following error:
<pre>
[INFO] [MPM.Rhodium-0.init] JESD204B Link Initialization & Training Complete
Running calibration for RhodiumDaughterboard serial: 31C82FBError: self-calibration is not supported for this device: n320
</pre>
However, typically the problem is solved after running this. Probably the reason is the change in master clock rate (the test runs at a 245.76 MHz master clock rate). The test should be run again to verify that the problem has been solved. Once the test gives good values, the problem usually does not appear again if the USRPs are not rebooted and no master clock rate changes are done.
=== Some useful tools ===
To view the spectrum of the two USRPs in real time the following GNU Radio flowgraph can be used:
<pre>
~destevez/view/view_two_usrps.py -g 45
</pre>
This runs FFTs and averaging and sends the spectrum data over ZMQ. A <code>view_client_two.grc</code> script in the same folder can be used to receive the spectrum data over ZMQ and plot it on a GUI (this second script is typically run on a local machine with a graphical interface).
To record IQ data the following can be used
<pre>
~destevez/record/record_4x_lo_share.py -g 45 --src-name "Source name" --lo-freq 1500 -b 3.84e6 --antennas 1a,1f
</pre>
This records to 16-bit complex IQ using a [[File Meta Sink]]. The name of the source, LO frequency and antennas are added to the metadata. By default the recording is written to <code>/mnt/buf0</code>, but this can be changed with the <code>--directory</code> argument.
To print the status for each antenna run:
<pre>
~sec/ata_status.py
</pre>
=== Controlling the telescope ===
To observe with the ATA it is necessary to:
* Reserve the antennas in the system to prevent accidental conflicts with other users
* Point the dishes
* Set the tuning of the LO frequency to select the observing frequency and focusing the dish feeds to that frequency (LO D is used for the GNU Radio Testbed)
* Select the two antennas that shall be connected to the two USRPs in the IF switch
* Run autotuning to set the attenuation of the PAMs
This tasks are done with the [https://github.com/SETIatHCRO/ATA-Utils/blob/master/pythonLibs/ATATools/ata_control.py ata_control Python library]. The [https://github.com/daniestevez/reu-2023/blob/main/Hydrogen_line/ATA%20control.ipynb following Jupyter notebook] can serve as a guideline of all the process. Using a Jupyter notebook or IPython terminal allows interactive control of the telescope while a GNU Radio flowgraph is running. Calls to ata_control can also be included in a Python script based around a GNU Radio flowgraph for more complex use cases (for instance, an observation where the antennas are pointed to different sources taken from a list and some data is recorded using the flowgraph for each source).
Additionally, the [https://github.com/SETIatHCRO/gr-ata gr-ata] out-of-tree module can be used to control the telescope in a simple way using GNU Radio Companion blocks. This out-of-tree module is unmaintained, so currently it may not work due to updates done in the ATA control system. See the [[GR-ATA Tutorial]] for usage instructions.


== GNU Radio / SETI Hackathon ==   
== GNU Radio / SETI Hackathon ==   

Revision as of 17:27, 21 November 2023

The Allen Telescope Array (ATA) is a 42-element radio telescope array located in Hat Creek, California, operated by SETI Institute. In an effort to increase accessibility to the array among potential users outside the astronomy community, we are working to integrate the array into GNU Radio software. We plan to start out by connecting USRPs to two ATA antennas and developing control, backend, and beamformer software. In addition to this, we are using a discone antenna connected to a USRP and OmniSig software to analyze radio frequency interference (RFI) at the ATA site.


General Info

GNU Radio Testbed

GR-ATA Testbed

Resources on the ATA and friends

Useful Radio Astronomy Tools

  • Radial Velocity / VLSR Calculator -- good for determining the V_LSR of spectral lines in your data
  • LAB Survey HI Profile Search -- here you can search for HI data given a source's RA, Dec or galactic coordinates. Good if you want to compare the velocity of the HI line that you measure with some preexisting accurate data to ensure your system is working properly.

GNU Radio Testbed usage

GNU Radio installations

Each user of gnuradio1 manages their own installation of GNU Radio and out-of-tree modules, either by building from source and installing to a prefix insider their $HOME directory, or by using Docker.

For general usage there is an installation of GNU Radio 3.10, UHD 4, and some out-of-tree modules that includes the patches required to enable LO sharing between the two USRP N32x's. The prefix for this installation can be activated with

source ~destevez/activate-gr310_uhd4

USRP tips and tricks

There is a known problem that causes a high noise level (and probably other signal distortions) in some of the channels of the two USRPs N320 (often the last channels). It is believed that this problem is related to locking the USRPs to the external 10 MHz reference. The problem can typically reappear after rebooting the USRPs, and it is solved as indicated here. There is a test that can be run to check if the problem is present:

~destevez/usrp_test/usrp_test.py

After running for a minute, this prints the noise levels present on each input when they are connected to an unused port.

[INFO] [MPM.Rhodium-1.init.LMK04828] LMK initialized and locked!
[INFO] [MULTI_USRP]     1) catch time transition at pps edge
[INFO] [MPM.Rhodium-1.DAC37J82] DAC PLL Locked!
[INFO] [MPM.Rhodium-1.AD9695] ADC PLL Locked!
[INFO] [MPM.Rhodium-1.init] JESD204B Link Initialization & Training Complete
[INFO] [MULTI_USRP]     2) set times next pps (synchronously)
>>> Setting LO sharing
vmcircbuf_sysconfig :info: Using gr::vmcircbuf_sysv_shm_factory
[WARNING] [0/Radio#0] Attempting to set tick rate to 0. Skipping.
[WARNING] [0/Radio#1] Attempting to set tick rate to 0. Skipping.
[WARNING] [1/Radio#0] Attempting to set tick rate to 0. Skipping.
[WARNING] [1/Radio#1] Attempting to set tick rate to 0. Skipping.
Channel 0 level -16.23 dB
Channel 1 level -16.18 dB
Channel 2 level -16.05 dB
Channel 3 level -15.98 dB
Note: channel levels should be around -16 dB in this test

As indicated by the test, the level in each channel should be around -16 dB. If it is significantly higher in any of the channels, the workaround is to run:

uhd_cal_rx_iq_balance --args "addr0=10.11.1.20,addr1=10.11.1.22,master_clock_rate=250e6"

This does not calibrate the USRPs. In fact it will give the following error:

[INFO] [MPM.Rhodium-0.init] JESD204B Link Initialization & Training Complete
Running calibration for RhodiumDaughterboard serial: 31C82FBError: self-calibration is not supported for this device: n320

However, typically the problem is solved after running this. Probably the reason is the change in master clock rate (the test runs at a 245.76 MHz master clock rate). The test should be run again to verify that the problem has been solved. Once the test gives good values, the problem usually does not appear again if the USRPs are not rebooted and no master clock rate changes are done.

Some useful tools

To view the spectrum of the two USRPs in real time the following GNU Radio flowgraph can be used:

~destevez/view/view_two_usrps.py -g 45

This runs FFTs and averaging and sends the spectrum data over ZMQ. A view_client_two.grc script in the same folder can be used to receive the spectrum data over ZMQ and plot it on a GUI (this second script is typically run on a local machine with a graphical interface).

To record IQ data the following can be used

~destevez/record/record_4x_lo_share.py -g 45 --src-name "Source name" --lo-freq 1500 -b 3.84e6 --antennas 1a,1f

This records to 16-bit complex IQ using a File Meta Sink. The name of the source, LO frequency and antennas are added to the metadata. By default the recording is written to /mnt/buf0, but this can be changed with the --directory argument.

To print the status for each antenna run:

~sec/ata_status.py

Controlling the telescope

To observe with the ATA it is necessary to:

  • Reserve the antennas in the system to prevent accidental conflicts with other users
  • Point the dishes
  • Set the tuning of the LO frequency to select the observing frequency and focusing the dish feeds to that frequency (LO D is used for the GNU Radio Testbed)
  • Select the two antennas that shall be connected to the two USRPs in the IF switch
  • Run autotuning to set the attenuation of the PAMs

This tasks are done with the ata_control Python library. The following Jupyter notebook can serve as a guideline of all the process. Using a Jupyter notebook or IPython terminal allows interactive control of the telescope while a GNU Radio flowgraph is running. Calls to ata_control can also be included in a Python script based around a GNU Radio flowgraph for more complex use cases (for instance, an observation where the antennas are pointed to different sources taken from a list and some data is recorded using the flowgraph for each source).

Additionally, the gr-ata out-of-tree module can be used to control the telescope in a simple way using GNU Radio Companion blocks. This out-of-tree module is unmaintained, so currently it may not work due to updates done in the ATA control system. See the GR-ATA Tutorial for usage instructions.

GNU Radio / SETI Hackathon

In May 2019, the first step toward collaboration between the GNU Radio community, the ATA, SETI Institute, Breakthrough Listen, and Berkeley SETI communities consisted of a hackathon hosted at the Hat Creek Radio Observatory (home of the ATA). This culminated in, among other things, the beginnings of a GNU Radio module called gr-ata.


Connecting to VNC on ATA gnuradio machines

Below are instructions for connecting to the ATA machines' VNC servers on Windows (note that gnuradio0 machine is connected to the discone, gnuradio1 will be connected to the ATA antennas).

Assuming you are connected to the ATA gnuradio vpn server, you can use PUTTY to connect to gnuradio1 or gnuradio0 machine:

  • Hostname (IP address): 10.1.50.10 for gnuradio0, or 10.1.50.11 for gnuradio1.
  • SSH -> Enable compression
  • SSH -> Tunnels -> Source port: 590x, Destination: localhost:590x

VNC Viewer:

  • Remote Host: localhost:590x
  • Enter password


Setting up the software

The first step in implementing GNU Radio with the ATA is ensuring that we can effectively play back previously-recorded SigMF files taken by the ATA in a GNU Radio flowgraph. We can do this using example code from the gr-ata module discussed above. To get the flowgraphs included in the "examples/original-examples" folder of gr-ata working, here's a recipe of what you'll need (along with some installation tips discovered through trial-and-error). Note that the instructions below are written for Ubuntu users:

  • GNU Radio version 3.8 (ppa for "releases" -- install instructions here).
  • Install liborc-0.4-dev (sudo apt install liborc-0.4-dev)
  • cmake version >= 3.13 (will have to remove the standard cmake version using $ sudo apt remove --autoremove cmake, then follow the instructions here: https://apt.kitware.com/ to install a more recent version. This is necessary for installing gr-osmosdr)


Below is a list of external GNU Radio modules that you can install following the instructions on the linked GitHub pages. One thing to keep in mind -- for the cmake step, in some cases cmake automatically installs the modules to the wrong location, resulting in a ModuleNotFound error. To prevent this, instead of running just cmake .. run the following command: $ cmake -DCMAKE_INSTALL_PREFIX:PATH={prefix} .. where the {prefix} can be found using the command $ gnuradio-config-info --prefix.

  • gr-ata (of course!)
  • gr-osmosdr
  • gr-correctiq -- make sure to clone the branch maint-3.8
  • gr-lfast -- make sure to clone the branch maint-3.8
  • gr-grnet -- branch maint-3.8. Before attempting to install gr-grnet, make sure you've run $ sudo apt-get install zlib1g-dev libpcap-dev
  • gr-filerepeater -- branch maint-3.8
  • gr-gpredict-doppler
  • gr-mesa -- branch maint-3.8
  • gr-fosphor -- before attempting to install gr-fosphor, run the command: $ sudo apt install libglu1-mesa-dev freeglut3-dev mesa-common-dev ocl-icd-opencl-dev freetype2-demos libfreetype6 libfreetype6-dev swig qt5-default
  • gr-compress --branch maint-3.8; follow instructions in README on github for installation prerequisites

All of these modules (excepting gr-fosphor, which requires a GPU) have been installed along with GNU Radio version 3.8 on the gnuradio1 machine at the ATA.

For ATA observing, you will need to install code from the ATA-Utils repository (which I have done on gnuradio1 in my account); run the commands:

  • git clone https://github.com/SETIatHCRO/ATA-Utils
  • cd ATA-Utils/pythonLibs
  • If pip isn't installed, run sudo apt-get install python3-pip
  • Still in the pythonLibs folder, run sudo pip3 install . --user
  • sudo python3 setup.py install

You should be good to go!


Observing with the ATA -- Tutorials linked here

The gr-ata module consists of four blocks:

  • Control Block
  • Track Scan Block
  • On-Off Block
  • IF Switch Block

which can be combined to conduct observations with the ATA. Refer to the GR-ATA Tutorial page for detailed tips on how to use the gr-ata blocks to conduct an ATA observation, and how to collect data with the USRPs onsite.

Public Data

At this link, you can find data from a sample observing run conducted with the USRPs at the ATA: http://blpd0.ssl.berkeley.edu/gnu_ata_workshop/

Some notes on the data:

12aug2020-sourcename2350+646-1420MHz.dat

  • The target source was quasar 2350+646
  • RA = 23.848588 (decimal hours) and Dec. = 64.671619 (decimal degrees)
  • RA = 23 50 54 (hhmmss), Dec. = 64 40 17 (ddmmss)
  • Recorded as spectral -- not raw -- data (float32)
  • The center frequency was 1420 MHz
  • The bandwidth = 50 MHz (complex) / 2 = 25 MHz
  • It was integrated over every 1k samples
  • The FFT size was 16384
  • Antenna = 2h

goes-16_2020-09-15_22_03_37.725663.dat

  • Target = GOES-16, a geosynchronous satellite
  • Az. = 121.956, El. = 23.605
  • Spectral data (float32)
  • Center frequency = 1691.4 MHz
  • Bandwidth = 50 MHz (complex) / 2 = 25 MHz
  • Integration = 10,000
  • FFT size = 4096
  • Total recording time = 1 minute
  • Antenna = 2h

goes-16_raw_2020-09-15_220116.481975.dat

  • Target = GOES-16, a geosynchronous satellite
  • Az. = 121.956, El. = 23.605
  • Raw (complex32) data
  • Center frequency = 1691.4 MHz
  • Bandwidth = 50 MHz (complex) / 2 = 25 MHz
  • Total recording time = 1 minute
  • Antenna = 2h

Sag A -- spectral

  • Target source = Sagittarius A (galactic center region)
  • RA = 17.7611 (decimal hours) and Dec. = -29.0028 (decimal degrees)
  • RA = 17 45 40 (hhmmss), Dec. = -29 00 28 (ddmmss)
  • Recorded as spectral -- not raw -- data (float32)
  • Center Freq = 1420 MHz
  • Bandwidth = 50 MHz (complex) / 2 = 25 MHz
  • Integration = 5000
  • FFT size = 16384
  • Total recording time = ~5 minutes
  • Antenna = 2h

Sag A -- raw

  • Target source = Sagittarius A (galactic center region)
  • RA = 17.7611 (decimal hours) and Dec. = -29.0028 (decimal degrees)
  • RA = 17 45 40 (hhmmss), Dec. = -29 00 28 (ddmmss)
  • Raw (complex32) data
  • Center Freq = 1420 MHz
  • Bandwidth = 50 MHz (complex) / 2 = 25 MHz
  • Total recording time = 1 minute
  • Antenna = 2h