Editing OutOfTreeModules

Jump to navigation Jump to search

Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.

The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then publish the changes below to finish undoing the edit.

Latest revision Your text
Line 1: Line 1:
[[Category:Guide]]
[[Category:Guide]]
'''Extending GNU Radio with own functionality and blocks'''
'''Extending GNU Radio with own functionality and blocks'''


'''Deprecation Warning''':  This article has been superseded by newer tutorials for C++ ( [[Creating c++ OOT with gr-modtool]] ) and Python ( [[Creating Python OOT with gr-modtool]] ).  This article is being kept only as a reference.  Please use the newer tutorials instead of this aricle.  In turn, this article superseded the original "How to write a block?" article written by Eric Blossom.
This article borrows heavily from the original (but very outdated) "How to write a block?" written by Eric Blossom.


== What is an out-of-tree module? ==
== What is an out-of-tree module? ==
Line 8: Line 9:
An out-of-tree module is a GNU Radio component that does not live within the GNU Radio source tree. Typically, if you want to extend GNU Radio with your own functions and blocks, such a module is what you create (i.e. you wouldn't usually add stuff to the actual GNU Radio source tree unless you're planning to submit it to the devs for upstream integration). This allows you to maintain the code yourself and have additional functionality alongside the main code.
An out-of-tree module is a GNU Radio component that does not live within the GNU Radio source tree. Typically, if you want to extend GNU Radio with your own functions and blocks, such a module is what you create (i.e. you wouldn't usually add stuff to the actual GNU Radio source tree unless you're planning to submit it to the devs for upstream integration). This allows you to maintain the code yourself and have additional functionality alongside the main code.


A lot of OOT projects are hosted at [http://cgran.org CGRAN] -- the Comprehensive GNU Radio Archive Network. CGRAN projects are all available through our tool [https://www.gnuradio.org/blog/2016-06-19-pybombs-the-what-the-how-and-the-why/ PyBOMBS]. In fact, when you add your project to the [https://github.com/gnuradio/gr-etcetera PyBOMBS recipe repo], it will automatically update the CGRAN website.
A lot of OOT projects are hosted at [http://cgran.org CGRAN] -- the Comprehensive GNU Radio Archive Network. CGRAN projects are all available through our tool [http://gnuradio.org/pybombs PyBOMBS]. In fact, when you add your project to the [https://github.com/gnuradio/gr-etcetera PyBOMBS recipe repo], it will automatically update the CGRAN website.


The example of such a module is the [https://www.cgran.org/14680/ GNU Radio Digital Audio Broadcasting module], which extends GNU Radio with everything needed to get audio from DAB and DAB+. When installed, you have more blocks available (i.e. in the [[GNURadioCompanion|GNU Radio companion]]) which behave just like the rest of GNU Radio; however, the developers are different people.
For example of such a module is the [https://www.cgran.org/14680/ GNU Radio Digital Audio Broadcasting module], which extends GNU Radio with everything needed to get audio from DAB and DAB+. When installed, you have more blocks available (e.g. in the [[GNURadioCompanion|GNU Radio companion]]) which behave just like the rest of GNU Radio; however, the developers are different people.


== Tools and resources at my disposal ==
== Tools and resources at my disposal ==
Line 38: Line 39:
With gr_modtool, this is dead easy. Just point your command line wherever you want your new module directory (this should be outside the GNU Radio source tree!), and go:
With gr_modtool, this is dead easy. Just point your command line wherever you want your new module directory (this should be outside the GNU Radio source tree!), and go:


<pre>$ gr_modtool newmod howto
<pre>% gr_modtool newmod howto
Creating out-of-tree module in ./gr-howto... Done.
Creating out-of-tree module in ./gr-howto... Done.
Use 'gr_modtool add' to add a new block to this currently empty module.</pre>
Use 'gr_modtool add' to add a new block to this currently empty module.</pre>
Line 46: Line 47:


Let's jump straight into the gr-howto module and see what it's made up of:
Let's jump straight into the gr-howto module and see what it's made up of:
<pre>$ cd gr-howto
 
$ ls
<pre>gr-howto % ls
apps  cmake  CMakeLists.txt  docs  examples  grc  include  lib  python  swig</pre>
apps  cmake  CMakeLists.txt  docs  examples  grc  include  lib  python  swig</pre>
It consists of several subdirectories. Anything that will be written in C++ (or C, or any language that is not Python) is put into <code>lib/</code>. For C++ files, we usually have headers which are put into <code>include/</code> (if they are to be exported) or also in <code>lib/</code> (if they're only relevant during compile time, but are not installed later, such as <code>_impl.h</code> files. You'll see what that is in the next tutorial).
It consists of several subdirectories. Anything that will be written in C++ (or C, or any language that is not Python) is put into <code>lib/</code>. For C++ files, we usually have headers which are put into <code>include/</code> (if they are to be exported) or also in <code>lib/</code> (if they're only relevant during compile time, but are not installed later, such as <code>_impl.h</code> files. You'll see what that is in the next tutorial).
Line 53: Line 54:
Of course, Python stuff goes into the <code>python/</code> directory. This includes unit tests (which are not installed) and parts of the Python module which are installed.
Of course, Python stuff goes into the <code>python/</code> directory. This includes unit tests (which are not installed) and parts of the Python module which are installed.


You probably know already that GNU Radio blocks are available in Python even if they were written in C++. This is done by the help of the Simplified Wrapper and Interface Generator (SWIG), which automatically creates glue code to make this possible. SWIG needs some instructions on how to do this, which are put into the <code>swig/</code> subdirectory. Unless doing something extra clever with your block, you will not need to go into the <code>swig/</code> directory; gr_modtool handles all of that for us.
You probably know already that GNU Radio blocks are available in Python even if they were written in C++. This is done by the help of SWIG, the simplified wrapper and interface generator, which automatically creates glue code to make this possible. SWIG needs some instructions on how to do this, which are put into the <code>swig/</code> subdirectory. Unless doing something extra clever with your block, you will not need to go into the <code>swig/</code> directory; gr_modtool handles all of that for us.


If you want your blocks to be available in the [[GNURadioCompanion|GNU Radio companion]], the graphical UI for GNU Radio, you need to add descriptions of the blocks and put them into <code>grc/</code>. Prior to version 3.8 these descriptions were XML files, but from 3.8 onward they use [[YAML_GRC|YAML instead]].
If you want your blocks to be available in the [[GNURadioCompanion|GNU Radio companion]], the graphical UI for GNU Radio, you need to add descriptions of the blocks and put them into <code>grc/</code>. Prior to version 3.8 these descriptions were XML files, but from 3.8 onward they use [[YAML_GRC|YAML instead]].
Line 65: Line 66:
The build system brings some baggage along, as well: the <code>CMakeLists.txt</code> file (one of which is present in every subdirectory) and the <code>cmake/</code> folder. You can ignore the latter for now, as it brings along mainly instructions for CMake on how to find GNU Radio libraries etc. The CMakeLists.txt files need to be edited a lot in order to make sure your module builds correctly.
The build system brings some baggage along, as well: the <code>CMakeLists.txt</code> file (one of which is present in every subdirectory) and the <code>cmake/</code> folder. You can ignore the latter for now, as it brings along mainly instructions for CMake on how to find GNU Radio libraries etc. The CMakeLists.txt files need to be edited a lot in order to make sure your module builds correctly.


But one step at a time! Now, let's move on to the next step.
But one step at a time! Now, let's move on to our next tutorial.


== Tutorial 2: Writing a block (square_ff) in C++ ==
== Tutorial 2: Writing a block (square_ff) in C++ ==
Line 82: Line 83:
Again, <code>gr_modtool</code> does the job. On the command line, go to the <code>gr-howto</code> directory and enter:
Again, <code>gr_modtool</code> does the job. On the command line, go to the <code>gr-howto</code> directory and enter:


<pre>gr-howto$ gr_modtool add -t general -l cpp square_ff
<pre>gr-howto % gr_modtool add -t general -l cpp square_ff
GNU Radio module name identified: howto
GNU Radio module name identified: howto
Block/code identifier: square_ff
Language: C++
Language: C++
Block/code identifier: square_ff
Please specify the copyright holder:
Enter valid argument list, including default arguments:  
Enter valid argument list, including default arguments:  
Add Python QA code? [Y/n] Y
Add Python QA code? [Y/n]  
Add C++ QA code? [y/N] N
Add C++ QA code? [y/N]  
Adding file 'lib/square_ff_impl.h'...
Adding file 'square_ff_impl.h'...
Adding file 'lib/square_ff_impl.cc'...
Adding file 'square_ff_impl.cc'...
Adding file 'include/howto/square_ff.h'...
Adding file 'square_ff.h'...
Editing swig/howto_swig.i...
Editing swig/howto_swig.i...
Adding file 'python/qa_square_ff.py'...
Adding file 'qa_square_ff.py'...
Editing python/CMakeLists.txt...
Editing python/CMakeLists.txt...
Adding file 'grc/howto_square_ff.xml'...
Adding file 'howto_square_ff.xml'...
Editing grc/CMakeLists.txt...</pre>
Editing grc/CMakeLists.txt...</pre>
On the command line, we specify that we're adding a block, its type is 'general' (because we don't know what block types are, yet) and it is called <code>square_ff</code>. The block should be created in C++ and it currently has no specified copyright holder (by default, gr-module author is the copyright holder).  <code>gr_modtool</code> then asks you if your block takes any arguments (it doesn't, so we leave that empty), whether or not we want QA code for Python (yes, we do) and for C++ (no, we don't right now).
On the command line, we specify that we're adding a block, its type is 'general' (because we don't know what block types are, yet) and it is called <code>square_ff</code>. The block should be created in C++ and it currently has no specified copyright holder (by default, gr-module author is the copyright holder).  <code>gr_modtool</code> then asks you if your block takes any arguments (it doesn't, so we leave that empty), whether or not we want QA code for Python (yes, we do) and for C++ (no, we don't right now).
Line 107: Line 109:
How hard could this be? Turns out that this is easy! So, we open <code>python/qa_square_ff.py</code>, which we edit to look like this:
How hard could this be? Turns out that this is easy! So, we open <code>python/qa_square_ff.py</code>, which we edit to look like this:


<syntaxhighlight lang="python">
<pre>from gnuradio import gr, gr_unittest
from gnuradio import gr, gr_unittest
from gnuradio import blocks
from gnuradio import blocks
import howto_swig as howto
import howto_swig as howto
Line 134: Line 135:
if __name__ == '__main__':
if __name__ == '__main__':
     gr_unittest.run(qa_square_ff, &quot;qa_square_ff.xml&quot;)</pre>
     gr_unittest.run(qa_square_ff, &quot;qa_square_ff.xml&quot;)</pre>
</syntaxhighlight>


For GNU Radio v3.9 and up, use the following:
For Gnuradio v 3.9 and up, use the following.
 
<pre>  
<syntaxhighlight lang="python">
from gnuradio import gr, gr_unittest
from gnuradio import gr, gr_unittest
from gnuradio import blocks
from gnuradio import blocks
Line 176: Line 175:
         # check data
         # check data


if __name__ == '__main__':
if __name__ == '__main__':
     gr_unittest.run(qa_square_ff, "qa_square_ff.yaml")
     gr_unittest.run(qa_square_ff, "qa_square_ff.yaml")
</syntaxhighlight>
</pre>


gr_unittest is an extension to the standard Python module unittest. gr_unittest adds support for checking approximate equality of tuples of float and complex numbers. Unittest uses Python's reflection mechanism to find all methods that start with test_ and runs them. Unittest wraps each call to test_* with matching calls to setUp and tearDown. See the [http://docs.python.org/2/library/unittest.html Python unittest documentation] for details.
gr_unittest is an extension to the standard Python module unittest. gr_unittest adds support for checking approximate equality of tuples of float and complex numbers. Unittest uses Python's reflection mechanism to find all methods that start with test_ and runs them. Unittest wraps each call to test_* with matching calls to setUp and tearDown. See the [http://docs.python.org/2/library/unittest.html Python unittest documentation] for details.
Line 191: Line 191:


In order for CMake to actually know this test exists, <code>gr_modtool</code> modified <code>python/CMakeLists.txt</code> with these lines:
In order for CMake to actually know this test exists, <code>gr_modtool</code> modified <code>python/CMakeLists.txt</code> with these lines:
For GR v3.9+, remove the following <code> set(GR_TEST_PYTHON_DIRS ${CMAKE_BINARY_DIR}/swig) </code>


<pre>########################################################################
<pre>########################################################################
Line 200: Line 202:
set(GR_TEST_PYTHON_DIRS ${CMAKE_BINARY_DIR}/swig)
set(GR_TEST_PYTHON_DIRS ${CMAKE_BINARY_DIR}/swig)
GR_ADD_TEST(qa_square_ff ${PYTHON_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/qa_square_ff.py)</pre>
GR_ADD_TEST(qa_square_ff ${PYTHON_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/qa_square_ff.py)</pre>
For GR v3.9+, remove the following line from <code>python/CMakeLists.txt</code>:
<pre>set(GR_TEST_PYTHON_DIRS ${CMAKE_BINARY_DIR}/swig) </pre>


=== The C++ code (part 1) ===
=== The C++ code (part 1) ===
Line 213: Line 212:


<code>gr_modtool</code> hints at where you have to change code by adding <code>&lt;++&gt;</code> symbols.<br />
<code>gr_modtool</code> hints at where you have to change code by adding <code>&lt;++&gt;</code> symbols.<br />
 
Let's go through these one at a time:
The following is the constructor of the module:
 
<pre>    square_ff_impl::square_ff_impl()
      : gr::block("square_ff",
              gr::io_signature::make(<+MIN_IN+>, <+MAX_IN+>, sizeof(<+ITYPE+>)),
              gr::io_signature::make(<+MIN_OUT+>, <+MAX_OUT+>, sizeof(<+OTYPE+>)))
    {}
    {
        // empty constructor
    }</pre>
For this tutorial, we replace <+MIN_IN+>, <+MAX_IN+>, <+MIN_OUT+>, <+MAX_OUT+> by 1 and set input type (<+ITYPE+>) and output type (<+OTYPE+>) as float.
Replace the above constructor with the following one:


<pre>    square_ff_impl::square_ff_impl()
<pre>    square_ff_impl::square_ff_impl()
Line 234: Line 221:
         // empty constructor
         // empty constructor
     }</pre>
     }</pre>
The constructor itself is empty, as the squaring block has no need to set up anything.
The constructor itself is empty, as the squaring block has no need to set up anything.


The only interesting portion is the definition of the input and output signatures: At the input, we have 1 port that allows float inputs. The output port is the same.
The only interesting portion is the definition of the input and output signatures: At the input, we have 1 port that allows float inputs. The output port is the same.


Replace the forecast function with the following one:
<pre>    void
<pre>    void
     square_ff_impl::forecast (int noutput_items, gr_vector_int &amp;ninput_items_required)
     square_ff_impl::forecast (int noutput_items, gr_vector_int &amp;ninput_items_required)
Line 245: Line 230:
       ninput_items_required[0] = noutput_items;
       ninput_items_required[0] = noutput_items;
     }</pre>
     }</pre>
<code>forecast()</code> is a function which tells the scheduler how many input items are required to produce <code>noutput_items</code> output items. In this case, they're the same. The index 0 indicates that this is for the first port, but we only have one anyway. This is generally the case for <code>forecast</code> in a lot of blocks. For examples, you can look at how <code>gr::block</code>, <code>gr::sync_block</code>, <code>gr::sync_decimator</code>, and <code>gr::sync_interpolator</code> define the default forecast functions to account for things like rate changes and history.
<code>forecast()</code> is a function which tells the scheduler how many input items are required to produce <code>noutput_items</code> output items. In this case, they're the same. The index 0 indicates that this is for the first port, but we only have one any way. This is generally the case for <code>forecast</code> in a lot of blocks. For examples, you can look at how <code>gr::block</code>, <code>gr::sync_block</code>, <code>gr::sync_decimator</code>, and <code>gr::sync_interpolator</code> define the default forecast functions to account for things like rate changes and history.


Finally, there's <code>general_work()</code>, which is pure virtual in <code>gr::block</code>, so we definitely need to override that. <code>general_work()</code> is the method that does the actual signal processing:
Finally, there's <code>general_work()</code>, which is pure virtual in <code>gr::block</code>, so we definitely need to override that. <code>general_work()</code> is the method that does the actual signal processing:
Line 275: Line 260:
If you've never used CMake before, this is a good time to give it a try. The typical workflow of a CMake-based project as seen from the command line is this (if using PyBOMBS, first read '''Build Tree vs. Install Tree'''):
If you've never used CMake before, this is a good time to give it a try. The typical workflow of a CMake-based project as seen from the command line is this (if using PyBOMBS, first read '''Build Tree vs. Install Tree'''):


<pre>gr-howto$ mkdir build     
<pre>$ mkdir build    # We're currently in the module's top directory
gr-howto$ cd build/
$ cd build/
gr-howto/build$ cmake ../      # Tell CMake that all its config files are one dir up
$ cmake ../      # Tell CMake that all its config files are one dir up
gr-howto/build$ make          # And start building (should work after the previous section)</pre>
$ make          # And start building (should work after the previous section)</pre>


If using GR v3.9 you will need to run:
If using GR v3.9 you will need to run:
<pre>gr-howto/build$ sudo make install </pre>
<pre>$ sudo make install </pre>


===== Build Tree vs. Install Tree =====
===== Build Tree vs. Install Tree =====
Line 402: Line 387:
Then, we use <code>gr_modtool</code> to add the block files, skipping the QA code (because we already have that):
Then, we use <code>gr_modtool</code> to add the block files, skipping the QA code (because we already have that):


<pre>gr-howto$ gr_modtool add -t sync -l cpp square2_ff
<pre>gr-howto % gr_modtool add -t sync -l cpp square2_ff
GNU Radio module name identified: howto
GNU Radio module name identified: howto
Block/code identifier: square2_ff
Block/code identifier: square2_ff
Line 418: Line 403:


If running GR v3.9+ you will need to bind the new <code>square2_ff</code> files to the existing files.
If running GR v3.9+ you will need to bind the new <code>square2_ff</code> files to the existing files.
<pre>gr-howto$ gr_modtool bind square2_ff
<pre>gr-howto % gr_modtool bind square2_ff
GNU Radio module name identifiedL howto
GNU Radio module name identifiedL howto
Writing binding code to ./python/bindings/square2_ff_python.cc
Writing binding code to ./python/bindings/square2_ff_python.cc
Line 467: Line 452:


Now, go back into our build directory and run <code>make</code>. Because <code>gr_modtool</code> added the <code>square2_ff</code> block to the necessary CMakeLists.txt files, <code>cmake</code> is automatically rerun for us and followed by <code>make</code>.
Now, go back into our build directory and run <code>make</code>. Because <code>gr_modtool</code> added the <code>square2_ff</code> block to the necessary CMakeLists.txt files, <code>cmake</code> is automatically rerun for us and followed by <code>make</code>.
<pre>gr-howto$ cd build
gr-howto/build$ make
gr-howto/build$ make test</pre>


Again, running <code>make test</code> will spawn a test run with of <code>qa_square_ff.py</code> which should not fail.
Again, running <code>make test</code> will spawn a test run with of <code>qa_square_ff.py</code> which should not fail.
Line 520: Line 501:


Once you've finished writing the block, <code>gr_modtool</code> has a function to help you create the XML code for you. For the howto example, you can invoke it on the <code>square2_ff</code> block by calling
Once you've finished writing the block, <code>gr_modtool</code> has a function to help you create the XML code for you. For the howto example, you can invoke it on the <code>square2_ff</code> block by calling
For GNU Radio 3.8 and up:
For GNURadio ver >= 3.8:
<pre>gr-howto$ gr_modtool makeyaml square2_ff
<pre>gr-howto % gr_modtool makeyaml square2_ff
GNU Radio module name identified: howto
GNU Radio module name identified: howto
Warning: This is an experimental feature. Don't expect any magic.
Warning: This is an experimental feature. Don't expect any magic.
Line 528: Line 509:
Overwrite existing GRC file? [y/N] y</pre>
Overwrite existing GRC file? [y/N] y</pre>


For GNU Radio 3.7:
For GNURadio ver. < 3.8:
<pre>gr-howto$ gr_modtool makexml square2_ff
<pre>gr-howto % gr_modtool makexml square2_ff
GNU Radio module name identified: howto
GNU Radio module name identified: howto
Warning: This is an experimental feature. Don't expect any magic.
Warning: This is an experimental feature. Don't expect any magic.
Line 537: Line 518:
Note that <code>gr_modtool add</code> creates an invalid GRC file, so we can overwrite that.
Note that <code>gr_modtool add</code> creates an invalid GRC file, so we can overwrite that.


In most cases, <code>gr_modtool</code> can't figure out all the parameters by itself and you will have to edit the appropriate XML or YML file by hand. See [[XML_GRC|Writing the XML file for a block (for 3.7)]] or [[YAML_GRC|Writing the YAML file for a block (for 3.8+)]] for details.
In most cases, <code>gr_modtool</code> can't figure out all the parameters by itself and you will have to edit the appropriate XML file by hand. The [[GNURadioCompanion|GRC]] wiki site has a description available.


In this case, because the block is so simple, the XML is actually valid. Have a look at <code>grc/howto_square2_ff.xml</code>:
In this case, because the block is so simple, the XML is actually valid. Have a look at <code>grc/howto_square2_ff.xml</code>:
Line 654: Line 635:
# Install (only when everything works and no tests fail): <code>sudo make install</code>
# Install (only when everything works and no tests fail): <code>sudo make install</code>
# Ubuntu users: reload the libs: <code>sudo ldconfig</code>
# Ubuntu users: reload the libs: <code>sudo ldconfig</code>
# Rename a block in the module: <code>gr_modtool rename BLOCKNAME</code>. The command will then prompt the user to enter the new name for the block.
# Delete blocks from the source tree: <code>gr_modtool rm REGEX</code>
# Delete blocks from the source tree: <code>gr_modtool rm REGEX</code>
# Disable blocks by removing them from the CMake files: <code>gr_modtool disable REGEX</code>
# Disable blocks by removing them from the CMake files: <code>gr_modtool disable REGEX</code>
Line 742: Line 722:
* <code>input_items</code> and <code>output_items</code> are numpy arrays, which is why we can do the very simple element-wise multiplication the way it's done here (instead of a list comprehension)
* <code>input_items</code> and <code>output_items</code> are numpy arrays, which is why we can do the very simple element-wise multiplication the way it's done here (instead of a list comprehension)
* No recompiling is necessary for the <code>make test</code> (faster development cycles, yay!)
* No recompiling is necessary for the <code>make test</code> (faster development cycles, yay!)
* For Python code, you must modify the XML or YAML file yourself. See [[XML_GRC|Writing the XML file for a block (for 3.7)]] or [[YAML_GRC|Writing the YAML file for a block (for 3.8+)]] for details.


=== Other types of Python blocks ===
=== Other types of Python blocks ===

Please note that all contributions to GNU Radio are considered to be released under the Creative Commons Attribution-ShareAlike (see GNU Radio:Copyrights for details). If you do not want your writing to be edited mercilessly and redistributed at will, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource. Do not submit copyrighted work without permission!

To edit this page, please answer the question that appears below (more info):

Cancel Editing help (opens in new window)