GRAndApps

Available GNU Radio for Android Apps

Android Programming Concept

The main focus of our work on GNU Radio is twofold. First, Android devices provide a powerful user experience, both with multi-touch input, high resolution graphics, and packaged in nice, mobile, easy-to-use, and battery-powered units. Secondly, these devices continue to improve in speed and capabilities.

The first set of properties makes these devices powerful tools for interacting with running radio systems. The radio itself might be running on the device itself, or we can employ them as the front-end control systems for remotely deployed and/or headless radios. Building on the use of GNU Radio’s ControlPort, we can add Java support for ControlPort clients to allow us to build applications that interact with GNU Radio systems.

Starting from this concept, we focus our efforts on building GNU Radio apps in Android on separating the user experience side from the radio application side. The UX part focuses on the development of standard Android applications in Java. The radio portion in GNU Radio is done in C++ and compiled with the Android’s NDK. Connections between the two are done almost exclusively through ControlPort aside from simple setup and teardown routines. Through this standard interface concept, the radio portion may exist locally on the same Android device as the Java application or even remotely on another system with a network connection between the two systems.

The following applications cover the standard models we talk about here and provide examples for how to build both the ControlPort clients in the Java app as well as full-fledged GNU Radio applications on Android. None of these are fully-structured projects. There is quite a lot left to do to improve support, error checking, and other life-cycle management issues in the Android app world. The hope here is that we have provided enough of the basic elements of the concepts that make it possible to build new apps and extend the features and capabilities.

GrControlPort

https://github.com/trondeau/GrControlPort

GrControlPort is an Android Library designed to be used by other Android apps to provide the standard interfaces to connect to a radio and get and set parameters over ControlPort. New GNU Radio for Android apps should pull this library into their projects and link against it. This then provides simple functions to manage the ControlPort connection, which is currently done using Apache Thrift.

The main drawback to this library as it stands is the lack of full support for the GNU Radio PMT (or Polymorphic Types) library. GNU Radio uses PMTs to handle all message passing between blocks in a flowgraph. Any block can add their message handles to ControlPort, but that means that the message being passed from the ControlPort client must be formatted as a PMT already. While this makes for convenient, opaque interfaces in GNU Radio, it requires PMT support in the client.

We use SWIG to generate wrappers from PMT in C++ to Python. We would like to have a similar capability in Java as well. The problem with using SWIG here is that the interface files it generates link to a shared object library (a .so file) to get to the underlying functionality of the original C++ code. Android tends to have problems with using shared libraries, and so it is uncertain if this method will work for the PMT library. There is likely a way in which this would work, and so a near-term project should be to explore how to make this happen. Doing this would make all of the PMT functionality available in Java and greatly simplify the work currently done in GrControlPort, which only provides a small portion of PMT capabilities.

To handle PMTs in GrControlPort when sending ControlPort commands via message handlers, we make a few dedicated Java Native Interface (JNI) calls to a set of C++ functions that can talk directly to the static PMT library. These JNI functions provide a direct interface to Java, but this requires us to program a new JNI function for every PMT case we have. Right now, we only use the ControlPort message handling scenario for sending commands to UHD devices. In general, these commands are a pair of “string” key values (e.g., ‘freq’ or ‘gain’) and a value as a double precision number. The main exception is the key ‘antenna’ to set the antenna port being used, which takes a string as its value. We can easily create these two cases in the JNI interface. It is here that a full and auto-generated interface from Java to the PMT library would come in useful.

The GrControlPort library also handles other set and get functions over ControlPort. Most ControlPort interfaces as described by either getting or setting a single value of the standard data types like char, int, float, complex, and vectors of the same (this is not a complete list). These are handled easily in Java, which just makes the calls to ControlPort with native Java types that ControlPort can translate between languages. So any block that describes a standard setter or getting in ControlPort is easily accessed through this interface in Java. However, those setters and getters must only handle a single value. That is, a getter must return a single value and take no parameters and a setter can only receive a single parameter and return nothing.

Library Interface

Most of these capabilities and API calls are similar or exact copies of how we handle ControlPort clients through Python.

Connecting to a Radio

The first thing that we need to do with ControlPort is connect the client to the radio. This, of course, requires the radio to be running and have started its ControlPort server. In the GrControlPort library, we call:

RPCConnection(method, host, port)

The method is the backend type, which right now is only “thrift” but with possible alternatives in the future. An alternative way is to use the Thrift child class directly: RPCConnectionThrift(host, port).

The host is the IP address or hostname of the machine running the GNU Radio app. The ControlPort server will run on a specific port, which is either preset by the developer in the Thrift configuration file or set to grab an ephemeral port. The client needs to know this information.

This call creates a new RPCConnection object, which we use to perform the rest of the ControlPort interface calls.

Getting a knob

One function of ControlPort is to allow us to query the properties of a radio and the blocks, which we call knobs. Any knob that a block has exposed to ControlPort can then be accessed through this call to the RPCConnection object:

Map getKnobs(List args)

It takes a list of strings that specify which properties that we are interested in retrieving, so we can easily create a filter for just those properties we want. The String format is “alias::knob” where alias is the block’s alias name, which is auto-assigned by the scheduler when a block is created but can also be set by the user. Remotely, the ControlPort client must be aware of these names specifically to know which block and what knob property to query. Alternatively, we can set args in this call to an empty string, which returns all possible knobs exposed through ControlPort.

The returned Java object is a map where the key is a string that matches the String we used as the filter list in the call to getKnobs. The KnobInfo is a data structure that contains the information in the knob, which holds the name of the knob, the value of the knob, and the data type of the knob, which is one of the BaseTypes enumerated by ControlPort itself.

An alternative method for getting knobs from a ControlPort server is to use the regular expression-based get function:

Map  getRe(List args)

In this method, we create a list of strings that contain regexs, which means we can pull out all knobs from a particular class of blocks or the same knob from every block with that knob. An example of using this method is in GrPerfMon where we plot a bar graph of the “work time” performance counter (PC). To get this PC from all blocks, we make the call:

knobs = new ArrayList<>();
knobs.add(".*::work time");
Map pc = conn.getRe(knobs)

This sets up a list with 1 value as a regular expression. All knobs from any block (“.*”) that contains the name “work time” is retrieved.

Setting a knob

The other side of ControlPort is setting and updating knobs of a radio. Setting the knobs is done through either of these two function calls:

void setKnobs(Map args)
void setKnobs(List args)

We provide both functions to make it easier for developers to use whichever is more appropriate for their needs. Since the KnobInfo data structure contains the knob name directly, the Map’s key string is redundant information. However, it is often easier to work with a map data structure. Also, since the getKnobs functions return the same type of Map object, providing this model here means we can get and set knobs with the same data structure. However, for other applications, having just a list of knobs might make more sense, and creating them can be simpler than creating the map.

Sending a Message

The setKnobs concept is the original way we would update values in GNU Radio blocks. This requires a block to have a setter function with the very simple prototype of “void set_<knob>(<type> knob_value)”. In other words, it can only take a single value and must not return anything. The value itself must be one of a known set of data types (see the BASE_TYPES concept mentioned above).

We are more heavily promoting the use of GNU Radio’s message passing interface for controlling blocks and setting new parameters. Often, these messages are more complicated dictionaries and objects and will not work with the standard set knob model above. For example, the UHD USRP Sink and Source blocks in GNU Radio have setters for changing gains, frequencies, and other parameters, but the setter functions have two inputs: the value itself and the channel to assign the value to. This does not work as a setter in ControlPort. Likewise, it has a “command” message port for setting values based on a message, but the message is a “key: value” dictionary that specifies what command and what the command’s value is.

ControlPort therefore supports a postMessage function call in the form:

void postMessage(String blk_alias, String blk_port, String cmd_name, Double cmd_val)

Here, we explicitly state the block’s alias that we are sending the message, give the name of the message port, which is also the name of the ControlPort’s endpoint (e.g., “command” for a UHD source or sink object). The postMessage function right now only works with a single construction of the messages: it assumes they are “key:value” pairs where the key is a string (e.g., “freq” or “gain”) and the value is a double. This limitation is related to our discussion at the top of this document about the PMT library. If we can open up that PMT library for general use in Java, we can construct our messages directly and create a new postMessage function call:

void postMessage(String blk_alias, String blk_port, PMT msg)

For current usage, see GrUsrpController as an Android app that can change the settings of a running USRP device.

GrUsrpController

https://github.com/trondeau/GrUsrpController

This project is a simple, but useful, example of using the GrControlPort library to control a remotely running radio. A radio running on a desktop, laptop, or even a headless system like a USRP E310 can now be easily manipulated through an Android app, making for a more useful interface for doing radio updates.

The USRP controller app is visually very trivial, using standard Android edit and combo boxes to create a form. The user inserts the hostname and port number of the device running the GNU Radio flowgraph, selects if the commands are for a USRP source (receiver) or sink (transmitter), and then sets the message properties. Through a drop-down box, we select which radio knob to adjust, like frequency or gain, and then enter in the value to set. The user then presses the “Send Command” button to actual transmit the ControlPort command to the device.

We have to have knowledge about the radio host’s name and port it is using for the task. Also, we would need to know that the port is not blocked by any firewalls between the host and the client device.

As an application, this is designed to look and behave as simply as possible to expose how we use the tools, specifically GrControlPort. This application makes use of a separate network thread, since Android will not let us run network commands in the main thread of an Activity. I have tried to keep this behavior simple and just what is required, though this once agains exposes my novice knowledge of programming Android apps. There is hopefully cleaner and/or more robust methods for handling this behavior.

Here is an image of the GrUsrpController app running on a Nexus 7 and controlling the GrRxFM app running remotely on an nVidia shield. I was using this app to change FM channels of the FM receiver app.

GrPerfMon

https://github.com/trondeau/GrPerfMon

This application is also fairly simple, though a bit more of a full Android app than GrUsrpController. This example is both designed to be a useful tool to monitor applications as well as another example of how to use the GrControlPort library, specifically to show how to get knob information from the running GNU Radio flowgraph.

The GrPerfMon app is set up similarly to GrUsrpController in that we have to insert the hostname, port number, and an update time in milliseconds into a form field and then hit go. Once the connection occurs, the app makes the ControlPort connection and starts asking for the “work time” Performance Counter. It then draws a vertical bar graph for every block in the flowgraph that represents the amount of time proportionally spent in each block’s work function. The bar graph is updated periodically based on the update time provided on the initial setup screen.

Here is a screenshot of the GrPerfMon tool in action. This is running on a Nexus 7 and connecte to an nVidia Shield device running the GrRxFM App (same app as above in GrUsrpController). It shows all of the blocks in the (very simple) FM receiver and how much time is spent inside each of the blocks’ work functions.

GrHardwareService

https://github.com/trondeau/GrHardwareService

This app is really a wrapper around and Android service whose job is to monitor the USB ports, detect when a UHD device is plugged in, and then load the firmware and FPGA images onto it. When a new device is detected, the app is automatically launched, which then kicks off the service. The app is shut down and the service continues to run in the background. The service communicates with the user through the Notification Center, where it displays the GNU Radio icon and provides some information about what the app is doing. Another indicator about the state of the app is a simple file, /sdcard/uhd_usb_info.txt. This file just contains the file descriptor of the UHD USB device and the device path. When GrHardwareService starts up, it immediately deletes this file and only recreates it when the FPGA is done loading. Other apps could poll for this file’s existence before trying to talk to the UHD device -- although I would bet that there are better ways to communicate this information between apps.

This service has a simple state machine of behavior:

The GrHardwareService uses a device_filter.xml file to determine for which devices it is monitoring based on the vendor ID (VID) and product ID (PID) numbers. The file looks like this:

Adding new devices is as simple as adding a new line to this file. The values given for the VID and PID are given in decimal.

We have also tested a handful of devices. The Android page on the GNU Radio wiki has this information as well. To date, we have the following information:

Working:

• Nexus 7 (5.1.1)
• Nexus 7 (6.0.1)
• Samsung Galaxy S6 (5.1.1)
• Samsung Galaxy S6 (6.0.1)
• nVidia Shield (6.0.1)
• Moto X Pure (5.1.1)
• Moto X (5.1.0)

Not Working:

• Nexus 7 (5.0.1) - Gave /sdcard permission errors (Android version problem)
• Moto G (3rd gen) (5.1.1) - Did not power USRP B200mini
• Moto G (3rd gen) (6.0.1) - Did not power USRP B200mini
• OnePlus Two (5.1.1) - Did not power USRP B200mini

Because the this service installs the firmware and FPGA images, we bundle the binaries into the APK. This simplifies a couple of major issues. First, we always know that the version of the binary files matches the version of the UHD library we are using. Second, we do not have to ask users to find and sideload these images themselves somehow. The images are unpacked when the app is loaded and written to the /sdcard directory.

This application requires permission to read and write to the external storage system (i.e., /sdcard) as well as talk to the USB system. These permissions are specified in the AndroidManifest.xml file. However, we cannot just give an app USB permission -- we indicate that we will use this feature, but the app itself must request the permission. That means that when the app is first launched, a pop-up box opens asking for permission to use the devices with this app. First, we want to click the box to indicate that we always want to use this app, and then click OK. However, because we load the firmware image first, that actually ends up changing the device as far as Android is concerned. So it will ask a second time after the firmware loads. Again, click the “always use” check box and OK. Then it will load the FPGA. After that, whenever it sees a device in the device list, it will run this app and not have to ask permission any more.

However, for other GNU Radio for Android apps, they must also request the same permission. In this case, we definitely want to OK the permission, but do NOT click the “always use” check box. This will overrule the GrHardwareService setting, and now when we connect a UHD device, this new app will be launched instead, which is not at all what we want.

The other permission issue is reading and writing to the /sdcard directory. Because we have explicitly set this permission in the AndroidManifest.xml file, we should have access. However, I have noticed that, specifically in Android Marshmallow, the permission is turned off by default. I have had to go to the app’s permission settings and enable it by hand. If the app fails to run the first time when you plug in hardware, take a look at this setting and turn it on if necessary. Then unplug and replug the device in and it should work.

Because of the way the UHD driver works, it always checks the firmware and tries to install the FPGA image onto a board whenever you open a connection to it. That means that every GNU Radio application, when it instantiates a UHD USRP source or sink block needs to have the firmware and FPGA images available to test and potentially load. In other words, all GNU Radio for Android apps must also contain all of the logic to open USB permissions to the devices, find the necessary information, and send that to the UHD driver. This GrHardwareService app manages the loading automatically, which is nice, but also provides the code necessary for every other GNU Radio app to use in order to access UHD systems. It is possible that we could turn some of this into an Android library to simplify apps even more.

Todos

• Better method of communicating with other apps when this is done loading the FW/FPGA.
• Handling of unplugging events.
• Handling / user messages when something goes wrong.
• Progress bar when installing Firmware and (especially) FPGA image.

GrRxFM

https://github.com/trondeau/GrRxFM

Simple application that pulls in samples from a USRP B2xx, FM demodulates them, and sends them to the Android audio sink (via gr-grand).

The main point of this app is to showcase a full-fledged GNU Radio on Android app. It relies on GrHardwareService to be present in order to program the hardware for use and a UHD-compatible device. It has been tested so far with a B200 and B200mini. A B210 should work, but I have seen power issues once the FPGA image is loaded, though connecting to wall-power should be good.

This app also shows off the use of gr-grand, the OOT project that provides access to Android-specific hardware such as the audio system and any sensors on the device. In this case, we are just making use of the opensl_sink to output the audio.

As an app, it shows how we connect the Java user interface with the underlying GNU Radio flowgraph. The flowgraph is in the fg.cpp file in the jni directory. We have added our own GrAndroid.mk and Android.mk file to handle the compilation of this with the Android NDK. Also, the app’s build.gradle file has some added functionality to tell gradle how to incorporate the NDK build directly into the project build such that whenever either the Java or C++ files are modified, Android Studio knows how to properly rebuild the entire project.

The actual flowgraph itself is very simple and was kept simple on purpose. As a simple example to test and showcase the layerings between the Java and C++ domains, I did not want this app to be particularly complex, which could be problematic running on underpowered hardware difficult. However, at this point, it’s now a “simple matter of programming” to add GNU Radio blocks and features to this flowgraph to improve its performance.

We have to code the GNU Radio flowgraphs in C++ to work inside the apps like this. We should start to look into developing capabilities in GRC to output C++ instead of just Python. We would then have a much easier model to develop the Java user interface portion of the app in Android Studio and the GNU Radio flowgraph in GRC. Then we can save the GRC output directly to the jni directory, rebuild the APK, and test it.

Todos

• gr-grand audio sink needs double buffering to prevent skipping.
• Extend receiver and demodulator chain to improve audio quality.
  • Use GrPerfMon to monitor block performance.
• GRC output C++ files for the flowgraphs.