OutOfTreeModulesConfig

Configuring GNU Radio and Out-of-tree (OOT) Modules

{{>toc}}

This tutorial follows on from OutOfTreeModules as a way to help understand how to configure your OOT module for other people to use. Please read the OutOfTreeModules tutorial before this to familiarize yourself with the concepts of OOT projects and because we will be extending the gr-howto module here.

Out-of-tree (OOT) Modules that are released and designed for other people to use them will often have to be built from scratch or through PyBOMBS. Building OOTs means finding the installed GNU Radio libraries and header files as well as other dependencies. Often times, the OOT just needs the GNU Radio runtime library and headers, like libgnuradio-runtime.so and gnuradio/block.h. More complicated blocks might use other parts of the GNU Radio system, like FIR filter kernels or FFT blocks, and so we'll need to know where to find these libraries and headers as well. We will also be linking against a specific version of GNU Radio, so we'll need to specify this version to make sure that the installed GNU Radio will meet the requirements of the project.

We have a CMake configuration script, called GnuradioConfig.cmake, that is installed with the GNU Radio development system that is designed to help with all of the above issues. The file is found in $prefix/lib/cmake/gnuradio along with a number of other helper cmake files that we use in GNU Radio to find and build against our dependencies like Boost and FFTW (among many others).

First, we have to make sure that GNU Radio is installed somewhere. We refer to this as $prefix, but it would actually be specified as CMAKE_INSTALL_PREFIX when passed as an option to cmake. By default, $prefix is /usr/local. A common alternative is to put it somewhere in the /opt directory, which makes it easier to separate from the rest of the system. This is often /opt/gr or /opt/gnuradio. We'll use the former, /opt/gr, in this tutorial.

If you are using PyBOMBS, then the concept of a prefix as well as setting the environment variables correctly (see below) should be familiar to you, and PyBOMBS has tools to help you set these up appropriately.

When located in /usr or /usr/local, the OS is usually set up to find the libraries, include files, Python modules, and Package Config scripts properly. However, if it doesn't or if we've installed GNU Radio in another location like /opt/gr, we will have to adjust some of our environmental variables in order to both use and locate GNU Radio. To run GNU Radio, we need to have PATH set to find the binaries, LD_LIBRARY_PATH set to find the libraries, PYTHONPATH for Python to know where to find the installed modules. If we would like to build against GNU Radio, we also need to know where to find it, which we do using the Package Config (pkg-config) tool. We therefore set PKG_CONFIG_PATH to locate the Package Config .pc files. Given our stated $prefix as /opt/gr, these settings look like:

  $ export PATH=/opt/gr/bin:$PATH
  $ export PYTHONPATH=/opt/gr/lib/python2.7/dist-packages:$PYTHONPATH
  $ export LD_LIBRARY_PATH=/opt/gr/lib:$LD_LIBRARY_CONFIG
  $ export PKG_CONFIG_PATH=/opt/gr/lib/pkgconfig:$PKG_CONFIG_PATH

With these set, we should be able to run gnuradio-companion and also use pkg-config to find out information about our installed GNU Radio. For example, to get the location of the GNU Radio version installed, we would see:

  $ gnuradio-config-info --prefix
  /opt/gr

Of course you can also use $pkg-config$ as you would with any other library.

Quick Start Guide

• Create your new project

' gr_modtool newmod --> new_oot'

• Edit the project to add your blocks and code.

' See OutOfTreeModules

• Edit the root CMakeLists.txt file

' Add the correct version information

' Add any other GNU Radio components (FILTER, DIGITAL, FFT, etc.) for OTHER_COMPONENTS'

set(GR_REQUIRED_COMPONENTS RUNTIME OTHER_COMPONENTS)
find_package(Gnuradio "3.7.0" REQUIRED)

• Edit MANIFEST.md
  • Plug in the correct information for the project to nicely display info on CGRAN
• If putting into gr-etcetera, fork this project from Github to your own account
  • If not using gr-etcetera, just create your repo yourself, wherever you want it to exist
• Clone the PyBOMBS recipe repository you want to use
  • gr-etcetera for putting this into the general, public cgran.org
  • Or the recipe repo of your choice, modeled after gr-etcetera
• Create a light weight recipe (LWR) file.
  • Make sure to list dependencies, description, git, and inheritance information
• Add your new recipe file to the repo
  • git add new_oot.lwr
  • git commit new_oot.lwr
  • git push <your repository> master
• If adding to gr-etcetera, create pull request from Github
• Once this has been approved and added to the repo, it will appear soon in CGRAN once it scans and updates the recipes

Configuring Our Project to Use GNU Radio Libraries

If we look in the top-level source directory of our OOT module, gr-howto/CMakeLists.txt, we'll find a very simplified interface to being with. We have lines that read:

set(GR_REQUIRED_COMPONENTS RUNTIME)
find_package(Gnuradio "3.7.0" REQUIRED)

It looks for libgnuradio-runtime, but we will be extending our howto module to use a FIR filter from our gr-filter component, so we need to add "FILTER" to the GR_REQUIRED_COMPONENTS to find libgnuradio-filter.

The CMake structure of all GNU Radio OOT projects created by gr_modtool know how to link against any GNU Radio library and find all GNU Radio header files based on what components are listed in GR_REQUIRED_COMPONENTS. No other setup is required.

For using other external dependencies, please check out the CMake website and documentation.

Tutorial for Configuring Project

We have our gr-howto module from the OutOfTreeModules tutorial that we'll work with here. We want to make a new block that takes advantage of the FIR filter kernels. For a simple example, we'll just create a block that calculates the derivative of a signal. So we'll go in the gr-howto directory and use gr_modtool to build another block:

gr-howto  $ gr_modtool add -t sync derivative_ff
GNU Radio module name identified: howto
Language (python/cpp): cpp
Language: C++
Block/code identifier: derivative_ff
Enter valid argument list, including default arguments: 
Add Python QA code? [Y/n] 
Add C++ QA code? [y/N] 
Adding file 'lib/derivative_ff_impl.h'...
Adding file 'lib/derivative_ff_impl.cc'...
Adding file 'include/howto/derivative_ff.h'...
Editing swig/howto_swig.i...
Adding file 'python/qa_derivative_ff.py'...
Editing python/CMakeLists.txt...
Adding file 'grc/howto_derivative_ff.xml'...
Editing grc/CMakeLists.txt...

To continue using the test-driven method, what do we expect out of a derivative? Let's pass it a vector of samples with an increasing slope of 1, a constant value for a few samples (slope of 0), and then a slope of -2. The results will start with a 0 because of the filter delay.

from gnuradio import gr, gr_unittest
from gnuradio import blocks
import howto_swig as howto

class qa_derivative_ff (gr_unittest.TestCase):

    def setUp (self):
        self.tb = gr.top_block ()

    def tearDown (self):
        self.tb = None

    def test_001_t (self):
        src_data = (0, 1, 2, 3, 4, 4, 4, 4, 2, 0)
        expected_result = (0, 1, 1, 1, 1, 0, 0, 0, -2, -2)
        src = blocks.vector_source_f(src_data)
        sqr = howto.derivative_ff()
        dst = blocks.vector_sink_f()
        self.tb.connect(src, sqr)
        self.tb.connect(sqr, dst)
        self.tb.run()
        result_data = dst.data()
        self.assertFloatTuplesAlmostEqual(expected_result, result_data, 6)

if __name__ == '__main__':
    gr_unittest.run(qa_derivative_ff, "qa_derivative_ff.xml")

Since we'll be using a FIR filter, we need to have a data member of our block for this object. First edit the header file gr-howto/lib/derivative_ff_impl.h. We'll include the fir_filter.h header file and make a private member of the class, d_fir:

#ifndef INCLUDED_HOWTO_DERIVATIVE_FF_IMPL_H
#define INCLUDED_HOWTO_DERIVATIVE_FF_IMPL_H

#include 
#include 

namespace gr {
  namespace howto {

    class derivative_ff_impl : public derivative_ff
    {
     private:
      gr::filter::kernel::fir_filter_fff *d_fir;

     public:
      derivative_ff_impl();
      ~derivative_ff_impl();

      // Where all the action really happens
      int work(int noutput_items,
           gr_vector_const_void_star &input_items,
           gr_vector_void_star &output_items);
    };

  } // namespace howto
} // namespace gr

#endif /* INCLUDED_HOWTO_DERIVATIVE_FF_IMPL_H */

We now open the C++ source code file, derivative_ff_impl.cc. Like before, we take 1 float input stream and produce 1 float output stream. But we need to build the FIR filter block in our constructor. To calculate a derivative, we use the taps [1, -1]. So first, we construct a vector to hold these taps, then we will build the FIR filter by creating a new gr::filter::kernel::fir_filter_fff. Note that we're using pointers here, so we also need to clean up this memory in the destructor. But we also have to remember that for a FIR filter to work, it needs to process all N taps of input data to make 1 output. We set history, as described in the OutOfTreeModules, to 2 in order to look 1 sample ahead in the work function. The constructor and destructor should look like:

    derivative_ff_impl::derivative_ff_impl()
      : gr::sync_block("derivative_ff",
                       gr::io_signature::make(1, 1, sizeof(float)),
                       gr::io_signature::make(1, 1, sizeof(float)))
    {
      std::vector taps;
      taps.push_back(1);
      taps.push_back(-1);
      d_fir = new gr::filter::kernel::fir_filter_fff(1, taps);
      set_history(2);
    }

    derivative_ff_impl::~derivative_ff_impl()
    {
      delete d_fir;
    }

We can now use the FIR filter in our work function. We will simply pass all incoming data to the FIR filter. We could run a for-loop and call d_fir->filter(in[i]) for all inputs, but the GNU Radio FIR filter kernel also has a filterN function that process N samples at once. This makes our work function very simple:

    int
    derivative_ff_impl::work(int noutput_items,
                             gr_vector_const_void_star &input_items,
                             gr_vector_void_star &output_items)
    {
        const float *in = (const float *) input_items[0];
        float *out = (float *) output_items[0];

        d_fir->filterN(out, in, noutput_items);

        // Tell runtime system how many output items we produced.
        return noutput_items;
    }

But! Before we compile and test this code, we have to remember that we are now using libgnuradio-filter. So we have to tell cmake about this. Using what we learned above, edit gr-howto/CMakeLists.txt and add "FILTER" as a "GR_REQUIRED_COMPONENTS" component. Also set the minimum version number for GNU Radio. Generally, its best to just use the version of GNU Radio you are currently working with. For extra vigilance, we could go back through to the earliest version that supports the API and behavior we need in any of our blocks.

set(GR_REQUIRED_COMPONENTS RUNTIME FILTER)
find_package(Gnuradio "3.7.3" REQUIRED)

We're ready to build the project with new block in it. So go into the build directory and type make. Because we edited the CMakeFiles when adding the new block and tweaking things, cmake knows to rerun itself to update the Makefiles. Now run "make test" or just "ctest" to run the new QA code we've built to make sure it works.

Editing the MANIFEST File

All projects built with gr_modtool contain a MANIFEST.md file that describes the project. It contains information about the project, licensing, and authors. Standalone, people can just read this file to find out more about the project. Practically, it is used by CGRAN to format the project page on the website. By default, the MANIFEST.md file looks like:

title: The HOWTO OOT Module
brief: Short description of gr-howto
tags: # Tags are arbitrary, but look at CGRAN what other authors are using
  - sdr
author:
  - Author Name 
copyright_owner:
  - Copyright Owner 1
license:
#repo: # Put the URL of the repository here, or leave blank for default
#website:  # If you have a separate project website, put it here
#icon:  # Put a URL to a square image here that will be used as an icon on CGRAN
---
A longer, multi-line description of gr-howto.
You may use some *basic* Markdown here.
If left empty, it will try to find a README file instead.

The MANIFEST files use YAML for formatting.

See the gr-nacl project as an example.

Since this is a file that is part of your project, don't forget to commit any edits you make to it.

Building the PyBOMBS Recipe File

Finally, we need to make the PyBOMBS recipe file to publish the module. In this case, let's just assume we will be checking this into the gr-etcetera recipe project. You are welcome to use your own git repository to store your recipes, but we won't focus on what else you'll have to do to support this.

First, if you haven't already done so, fork the gr-etcetera repo through Github into your own account, which we'll call "grhowto". We can now clone this repo:

$ git clone git@github.com:grhowto/gr-etcetera.git
$ cd gr-etcetera

Now add your LWR:

$ touch gr-howto.lwr

And open it for editing. Like the MANIFEST.md file, a good example is gr-nacl, shown below.

#
# This file is part of PyBOMBS
#
# PyBOMBS is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 3, or (at your option)
# any later version.
#
# PyBOMBS is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with PyBOMBS; see the file COPYING.  If not, write to
# the Free Software Foundation, Inc., 51 Franklin Street,
# Boston, MA 02110-1301, USA.
#

category: common
depends:
- gnuradio
- libsodium
description: GNU Radio module for data encryption using NaCl
gitbranch: master
inherit: cmake
source: git+https://github.com/stwunsch/gr-nacl.git

Change that to the appropriate information for gr-howto. That is, update the depends list, the description, and source. The gitbranch is likely the same, and we probably only need to inherit from cmake liek this file shows. Also in our howto example, we only depend on gnuradio, so remove libsodium. However, this example is instructive in showing us that if depending on another project, we need to add that to our list here. And then to support that dependency, we need to make sure that there is an LWR file to support building this with PyBOMBS, like libsodium.lwr.