Understanding XMLRPC Blocks

This tutorial presents the GNU Radio XMLRPC blocks. There are two blocks in this collection: XMLRPC Client and XMLRPC Server. Both blocks have IP address/port fields while the Client block also has callback and variable fields. The blocks use the Python XMLRPC module and use a subset of the full XMLRPC specification.

XMLRPC is an XML-based Remote Protocol Control mechanism that does just that. It uses HTTP transport and allows a client to use SET commands to change parameters on a server or use GET commands to obtain the value of parameters on the server.

To understand better how GNURadio implements XMLRPC, look at the block documentation linked in the paragraph above.

To understand the XMLRPC protocol and Python implementation in detail, the reference links below are a good starting point.

Reference Links

http://xmlrpc.com

https://docs.python.org/3.8/library/xmlrpc.html

Prerequisites

• Intro to GR usage: GRC and flowgraphs

• Understanding ZMQ Blocks

Introduction: What does XMLRPC do in GNURadio and Why Should I Care?

WHAT: In GNURadio placing the XMLRPC Server block in a flowgraph will create an XMLRPC server that exposes all the variables in the flowgraph on the specified IP Address and Port. This allows a client to change them at runtime. The XMLRPC Client block allows the user to create a flowgraph that can control any of the parameters in a flowgraph that has the XMLRPC Server block.

WHY: In many cases, we want to adjust the variables of a GNURadio flowgraph at runtime. We usually do this by adding QT GUI Widget block (range, push button, etc). But what if you want to have the same control interface, but in a different flowgraph? XMLRPC will allow you to control all of a flowgraph's variables from a second flowgraph using HTTP. In fact, the second flowgraph doesn't even have to be on the same computer! Therefore, XMLRPC can be used to add runtime control of any flowgraph running on a remote system like a Raspberry PI, headless server or otherwise.

Overview of Tutorial

GNURadio XMLRPC Examples

• We will start with running the basic client/server example flowgraphs from the GNU Radio source tree (gr-blocks/examples/xmlrpc).

GNURadio XMLRPC Examples with added ZMQ Streaming Data Visualization

• Here we will modify the examples to include ZMQ streaming so we can visualize the remote flowgraph's datastream in our XMLRPC Client flowgraph.

GNURadio XMLRPC Remote Control Over IP Network

• Next, we will run the server/client pair over an IP network using 2 separate computers. (Requires 2 hosts on the same network with GNURadio 3.9+ installed)

GNURadio XMLRPC Server Automation using Standalone Python Code

• This section demonstrates the ability to automate our server flowgraph using a standalone python application.

GNURadio XMLRPC Advanced Usage Example (OPTIONAL)

• As a bonus step we will run a project that implements a remote, headless wideband RF receiver and stream the data to a Remote Controller for control and visualization.

GNURadio XMLRPC Examples

In GNURadio Companion:

Open the siggen_xmlrpc_server.grc flowgraph.

Shown here:

In GNURadio Companion:

Open the siggen_controller_xmlrpc_client.grc flowgraph

Shown here:

First, click on the tab in GRC that has the Siggen Server Flowgraph and start it by clicking the 'RUN' button.

Your GUI window should look like this:

Next, click on the tab in GRC that has the tab for the the Siggen Controller flowgraph and start it by clicking the 'RUN' button.

Your GUI window should look like this:

Arrange both GUI windows so you can see them at the same time.

In the Siggen Controller Flowgraph Move the Frequency Slider slightly to the left and observe the Siggen Server time/frequency display. You should see the frequency changing as shown here:

That's it, you've just remote controlled a flowgraph over a network socket!

Close both GUI windows and continue to the next section.

GNURadio XMLRPC Examples with ZMQ Streaming Data Visualization

Make sure you are familiar with ZMQ streaming from the ZMQ tutorials.

• Create a copy ('save as') of both XMLRPC example flowgraphs and name them:

   "siggen_xmlrpc_server_streaming.grc"
   "siggen_controller_xmlrpc_client_streaming.grc"

(don't forget to change the names in the 'ID' of the 'Options' block to match the new .grc filenames)

• In the Streaming Server Flowgraph, copy the throttle and GUI display blocks, then paste them into the Streaming Controller flowgraph.

• Now delete the GUI blocks from the Server Flowgraph and add a ZMQ Pub Sink block. Connect the output of the 'Throttle' block to the input of the ZMQ Pub Sink where the GUI blocks used to be connected.

   ZMQ PUB SINK PARAMETERS:
       'Address': tcp://127.0.0.1:5000
       'Pass Tags': 'Yes'
       All others default

• Change the 'Generate Options' in the Server flowgraph to 'No GUI'. This will allow us to run the flowgraph on our localhost or a remote host as a 'headless' flowgraph.

• In the Streaming Client flowgraph.

Change the sample rate to match the Streaming Server Flowgraph (32ksps)

Add a ZMQ Sub Source block to the flowgraph and connect it to the input port of both the 'QT GUI Time Sink' and 'QT GUI Frequency Sink' GUI blocks.

   ZMQ SUB SOURCE PARAMETERS:
       'Address': tcp://127.0.0.1:5000
       'Pass Tags': 'Yes'
       All others default

• RUN IT!

From GRC: Click the run button on the 'siggen_xmlrpc_server_streaming' flowgraph. A blank terminal window will open (don't worry its doing it's job).

Click the run button on the 'siggen_xmlrpc_controller_client_streaming' flowgraph. A GUI window will open with the same time/frequency display as before, but in the same graph with the GUI Range 'frequency' slider.

In the Controller GUI: Adjust the slider as before and observe the frequency changing.

You are now both remote controlling and viewing streaming data over a network connection!

GNURadio XMLRPC Remote Control Over IP Network

• Extending this example to two hosts using one of two methods.

Assume our Streaming Server machine is Host A and our Streaming Client machine is Host B.

NOTE: These tutorials are focused on functionality not security. Understanding of the IP mechanisms used in this tutorial fall on the user. Know your network!

• METHOD 1

SSH Port Forwarding (more information can be found here Using SSH Port Forwarding, a user can keep everything the same in the flowgraphs. We are using port 8000 for XMLRPC and Port 5000 for our ZMQ Streaming Data. Simply run the following command from Host B:

   ssh -L 8000:localhost:8000 -L 5000:localhost:5000 <username>@<IP Address of Host A>

This SSH command effectively shares any traffic on ports 8000 and 5000 between the two hosts.

• METHOD 2

Changing the IP Addresses in both flowgraphs:

• Streaming Server Flowgraph (Host A)

ZMQ Pub Sink Address <IP Address of Host B -OR- '0.0.0.0'>

XMLRPC Server <IP Address of Host A -OR- '0.0.0.0'>

• Streaming Client Flowgraph (Host B)

ZMQ Sub Source Address <IP Address of host A>

XMLRPC Client Block (frequency) Address <IP Address of host A>

GNURadio XMLRPC Server Automation using Standalone Python Code

XMLRPC is extremely simple to use in Python. The following section will give a simple example of this by changing the frequency of the server flowgraph from a list of frequencies in a python script.

If we run our original siggen_xmlrpc_server.grc flowgraph. We should see the same gui window pop up again as in the first part of this tutorial.

Now instead of running our client flowgraph to manually control the frequency, we will automate the frequency change with a python script.

This script will change the frequency of the signal generator block every 2 seconds. The frequency is determined by a list which our script will iterate through.

With the server running, simply paste this code into a script called 'flowgraph_automator.py'

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
from xmlrpc.client import ServerProxy
import time
xmlrpc_control_client = ServerProxy('http://'+'localhost'+':8000')
freq_steps = [6e3, 11e3, 2e3, 14e3, 4e3, 3.5e3]
while True:
    for freq in freq_steps:
        print("retuning to:",freq/1000,"kHz")
        xmlrpc_control_client.set_rmt_freq(freq)
        time.sleep(2)

Open a terminal and run the script with $ python3 flowgraph_automator.py As soon as you run the python script, you will see terminal output displaying the frequency change. If you look in the GUI window for the server, you will see the frequency shifting every 2 seconds!

GNURadio XMLRPC ZMQ Advanced Usage Example

Remote Spectrum Monitoring Example using Hardware

This portion of the tutorial uses two open source projects that implement the concepts shown above to provide wideband remote spectrum monitoring over a low data rate IP connection.

This tutorial assumes you have one of the following Software Defined Radios:

• Ettus Research USRP
• Lime Micro LimeSDR
• RTL-SDR

This tutorial can be run from the same computer, but is intended to be used on two separate machines. Both machines require GNURadio 3.9+ and the proper driver for the radio. In the case of the USRP, UHD is required. For either the LimeSDR or the RTLSDR, GNURadio 3.9+ includes SoapySDR drivers for both.

Clone the following two repositories on the remote host:

• https://github.com/muaddib1984/stillsuit

• https://github.com/muaddib1984/arrakis

Clone only the arrakis repository on the local host:

• https://github.com/muaddib1984/arrakis

REMOTE HOST

• Attach one of the SDR's listed above to your remote host. Ensure your radio is connected to the host by running either:

For USRP

uhd_find_devices

For LimeSDR/RTLSDR

SoapySDRUtil --find

• From your local host, open an SSH tunnel to the remote host with Local Port Forwarding:

ssh -L 5001:localhost:5001 -L 8001:localhost:8001 -L 8000:localhost:8000 <username>@<hostname(or IP Address)>

EXAMPLE: If my username on the remote host is 'user' and the remote host's ip address is 192.168.1.100, from my local host I would run: ssh -L 5001:localhost:5001 -L 8001:localhost:8001 -L 8000:localhost:8000 user@192.168.1.100

• Now you are connected via SSH to the remote terminal.

• If you prefer to have only one terminal window open, from the SSH terminal, run:

cd path/to/stillsuit ./<uhd|lime|rtl>_stillsuit.py & (this will allow you to run the second application from the same terminal)

• If you prefer to have two terminals, you can open a second SSH session, but leave off the '-L' arguments since we already have the ports forwarded in the first shell.
• With two terminals, you can also omit the "&" in the previous command.

• Whether in one or two terminal windows run:

cd path/to/arakkis
space_folder.py note: If you are using an RTLSDR, pass the -s argument to space_folder.py with a value of 2.5e6 like this
space_folder.py -s 2.5e6

LOCAL HOST

• Now open a terminal window on your local machine and run:

cd path/to/arrakis
./guild_navigator.py note: If you are using an RTLSDR, pass the -s argument to guild_navigator.py with a value of 2.5e6 like this
guild_navigator.py -s 2.5e6

• You should see a frequency plot updating at a fairly slow update rate.
• To make the frequency plot update faster, change the vectors per second dropdown menu to 8.
• To make the spectrum more stable visually, change the 'Average/Raw' dropdown to 'AVERAGED'.
• If using a USRP/Lime, you can adjust the sampling rate to 40MHz (performance will be dependent on your remote host's processing power.)
• Finally, tune the frequency to 98000000.0 Hz in the frequency window (FM Radio).

Your plot should look something like this:

To measure the data usage from one host to another install an application like 'iftop'
sudo apt install iftop and run it like this:
sudo iftop -i lo -P Port 5000 will have the throughput usage and look similar to this with the above settings:

             191Mb         381Mb         572Mb         763Mb    954Mb
└────────────┴─────────────┴─────────────┴─────────────┴─────────────
localhost:5001       => localhost:59814       2.01Mb  2.06Mb  2.01Mb

This is a powerful capability because we are getting a very human readable frequency plot of live RF Spectrum, while only using a small amount of network data to get it from the remote host! Furthermore, XMLRPC is allowing us to remotely adjust all of the parameters of the SDR at runtime!