ModtoolArchitecture

This is documentation of the Modtool Architecture back from the GSoC project. It doesn't represent current architecture (May 2024)

This article explains the current architecture of gr_modtool in the python3 branch.

For this tutorial, one must have the dependencies click, click-plugins installed.
pip3 install click
pip3 install click-plugins

The click-plugins dependency has been removed from main (May 2024)

What is gr_modtool?

While developing an Out Of Tree module, there's a lot of boring, monotonous work involved: boilerplate code, makefile editing, etc. gr_modtool is a script which aims to help with all these things by automatically editing makefiles, using templates, and doing as much work as possible for the developer such that one can jump straight into the DSP coding.

Note that gr_modtool makes a lot of assumptions on how the code looks like. The more the module is custom and has specific changes, the less useful gr_modtool will be, but it is probably the best place to start with any new module or block. One can go through the block coding guide for getting an insight of coding the block.

gr_modtool is available in the GNU Radio source tree and is installed by default.

Plug-in Architecture

This section is obsolete. The click-plugins dependency was removed from GNU Radio's main branch; no known external plugins have materialized, and the dependency grew stale with the deprecation of importlib.

gr-modtool uses the Click Python package for its command line interface. Some of the advantages of using Click for CLI are:-

• Click is fully nestable and composable.
• Click supports prompting of custom values.
• Click works the same in Python 2 as well as in Pythonn3.
• Click comes with useful helpers like ANSI colors.
• Click is very simple to code and visualize.

Click is based on decorators and therefore enables simple syntax for calling high order internal functions.

gr_modtool uses the class click.Group as the parent class for its base module's CommandCLI class. The parent class functions list_commands and get_command have been overridden in the child class to enable a plug-in architecture for gr_modtool.

The function list_commands lists all the commands for the gr_modtool. It first searches for the in-tree commands, the commands in its own directory, and then searches for other registered commands through external plug-ins or otherwise (in case of the modtool, they are only registered through external plug-ins).
The function get_command returns a command object by the user input command. It first tries to import the module for the command by searching the in-tree modules, then checks for the registration of command via other sources. If the module is found in-tree, then the cli() function of the respective module is called.

CLI functions in Modtool

Initially all commands with gr_modtool in the command-line interface entered by the user pass through the cli() function of modtool_base.py. If no further command is provided in the command-line, the help page is displayed with all the in-tree plug-ins commands as well as the commands from the external plug-ins, if any.

If a command (after gr_modtool) is provided in the interface which has a corresponding module modtool_command.py, where the command is the user input command, the cli() function of the corresponding module is invoked with the context of the group instance in the base module.

The function is decorated by the decorator, @click.command(), which adds the functionalities of the class click.Command and automatically attaches the decorated options and arguments to it. The user input values of all these parameters are then passed to the run() function of the respective module defined in its core class.

Note: The basic difference between arguments and options is that arguments are positional and mandatory by default unlike options, although arguments can also be made optional as in the case with the modtool scripts. Moreover, documentation of the argument is not generated by click and it has to be done manually.

Key Notes

• A decorator function common_params is present in the base module to remove the redundancy of adding the same options in every module. These options are not provided as the options to the base command group intentionally because they actually make sense with the command itself, otherwise the command gr_modtool add -t general --skip-lib will look like gr_modtool add --skip-lib -t general which is not user-friendly. Therefore, these parameters will show up in the help page of the respective command rather than the command group.
• A decorator block_name is present in the base module which adds an argument (non-mandatory) block_name to the particular command that uses it.
• The help page for a particular command can be shown with --help in the CLI. For eg:- gr_modtool add --help.
• For commands, a short help snippet is generated. By default, it is the first sentence of the help message of the command, unless it is too long. This can also be overridden by short_help provided with command. For example, @click.command('newmod', short_help=ModToolNewModule().description) shows the help of the command newmod generated in the help page (gr_modtool --help or simply gr_modtool) as the description variable of the class ModToolNewModule. The main help-text ,when you request the command's help page with commands like gr_modtool makexml --help, is the document string of the function attached to the decorator @click.command.
• For options, the help documentation can be added by help parameter in the decorator @click.option(). For example, @click.option('--copyright', help="Name of the copyright holder (you or your company) MUST be a quoted string.").

The help page of gr_modtool looks like:-

% gr_modtool
Usage: gr_modtool [OPTIONS] COMMAND [ARGS]...

  A tool for editing GNU Radio out-of-tree modules.

Options:
  --help  Show this message and exit.

Commands:
  add      Adds a block to the out-of-tree module.
  disable  Disable selected block in module.
  info     Return information about a given module
  makexml  Generate XML files for GRC block bindings.
  newmod   Create new empty module, use add to add blocks.
  rename   Rename a block inside a module.
  rm       Remove a block from a module.

  Manipulate with GNU Radio modules source code tree. Call it without
  options to run specified command interactively

The help page of the command add of gr_modtool looks like:-

 % gr_modtool add --help
Usage: gr_modtool add [OPTIONS] BLOCK_NAME

  Adds a block to the out-of-tree module.

Options:
  -t, --block-type [sink|source|sync|decimator|interpolator|general|tagged_stream|hier|noblock]
                                  One of sink, source, sync, decimator,
                                  interpolator, general, tagged_stream, hier,
                                  noblock.
  --license-file TEXT             File containing the license header for every
                                  source code file.
  --copyright TEXT                Name of the copyright holder (you or your
                                  company) MUST be a quoted string.
  --argument-list TEXT            The argument list for the constructor and
                                  make functions.
  --add-python-qa                 If given, Python QA code is automatically
                                  added if possible.
  --add-cpp-qa                    If given, C++ QA code is automatically added
                                  if possible.
  --skip-cmakefiles               If given, only source files are written, but
                                  CMakeLists.txt files are left unchanged.
  -l, --lang [cpp|c++|python]     Programming Language
  -d, --directory TEXT            Base directory of the module. Defaults to
                                  the cwd.
  --skip-lib                      Don't do anything in the lib/ subdirectory.
  --skip-swig                     Don't do anything in the swig/ subdirectory.
  --skip-python                   Don't do anything in the python/
                                  subdirectory.
  --skip-grc                      Don't do anything in the grc/ subdirectory.
  --scm-mode [yes|no|auto]        Use source control management [ yes | no |
                                  auto ]).
  -y, --yes                       Answer all questions with 'yes'. This can
                                  overwrite and delete your files, so be
                                  careful.
  --help                          Show this message and exit.

Note:- For more information regarding help-texts : click-documentation.

Adding a module in-tree

Here is an example of coding a click command:-

@click.command('newmod', short_help=ModToolNewModule().description)
@click.option('--srcdir',
              help="Source directory for the module template.")
@ModTool.common_params
@click.argument('module_name', metavar="MODULE-NAME", nargs=1, required=False)
def cli(**kwargs):
    """
    \b
    Create a new out-of-tree module

    The argument MODULE-NAME overrides the current module's name (normally is autodetected).
    """
    args = DictToObject(kwargs)
    try:
        ModToolNewModule().run(args)
    except ModToolException as err:
        print(err, file=sys.stderr)
        exit(1)

Here, the available options with the command are srcdir and all the options that the function of the class ModTool, common_params, adds to it.

So, to add a module in-tree, all that one has to do is to create module as module_command.py with a function cli() in the modtool directory, add options and arguments to it with @click.option() and @click.argument() decorators and specify the functional details with the obtained value of parameters.
Note: Instead of **kwargs, one can specify the particular options.

External Plug-ins

gr_modtool supports the addition of external plug-ins, if required by the user. To achieve this functionality, it uses pkg_resources.iter_entry_points to load an external plug-in and a python package click-plugins to register commands to the base group and show Broken Plug-in if the plug-in is not be loaded properly.

Adding a plug-in

To add an external plug-in to the modtool, all that one has to do is to create a package directory with a file setup.py and specify the entry_point as gnuradio.modtool.plugins through the functional definition of our choice.
For example:-
If package directory ext_plug looks like

ext_plug
....setup.py
....core.py

the setup.py should look like:-

from setuptools import setup


setup(
    py_modules=['core'],
    entry_points='''
        [gnuradio.modtool.cli.plugins]
        cmd=core:cmd
    '''
)

Here core is the module name and cmd is the function inside core.py that is executed.
Note: Any other specifications to setup.py can be added like name, version, etc.

Writing a plug-in

Writing an external plug-in is quite simple. All that one has to do is create a function with the name specified in the entry_point and decorate it with @click.command(), if just an external command is to be added, or @click.group(), if a command group is to be added.
For eg:- For the previous package, core.py can be like

import click


@click.command()
def cmd():
    """
    The external command functionality
    """ 
    pass

Now, one just needs to install this package by running pip3 install ext_plug/ and the external plugin is ready to use.
Now, on running gr_modtool --help, we get

% gr_modtool --help
Usage: gr_modtool [OPTIONS] COMMAND [ARGS]...

  A tool for editing GNU Radio out-of-tree modules.

Options:
  --help  Show this message and exit.

Commands:
  add      Adds a block to the out-of-tree module.
  cmd      The external command functionality
  disable  Disable selected block in module.
  info     Return information about a given module
  makexml  Generate XML files for GRC block bindings.
  newmod   Create new empty module, use add to add blocks.
  rename   Rename a block inside a module.
  rm       Remove a block from a module.

  Manipulate with GNU Radio modules source code tree. Call it without
  options to run specified command interactively

If the plug-in is not loaded due to reasons like an incorrect entry_point setup, the help page will duly indicate that.
For example:-
If the setup.py file for the above-mentioned package looks like

from setuptools import setup


setup(
    py_modules=['core'],
    entry_points='''
        [gnuradio.modtool.plugins]
        cmd=core:cmdddd
    '''
)

the plug-in will not load to incorrect functional definition in the entry_points.
Now, on running gr_modtool --help, we get

% gr_modtool --help
Usage: gr_modtool [OPTIONS] COMMAND [ARGS]...

  A tool for editing GNU Radio out-of-tree modules.

Options:
  --help  Show this message and exit.

Commands:
  add      Adds a block to the out-of-tree module.
  cmd      † Warning: could not load plugin. See `gr_modtool cmd --help`.
  disable  Disable selected block in module.
  info     Return information about a given module
  makexml  Generate XML files for GRC block bindings.
  newmod   Create new empty module, use add to add blocks.
  rename   Rename a block inside a module.
  rm       Remove a block from a module.

  Manipulate with GNU Radio modules source code tree. Call it without
  options to run specified command interactively

So, gr_modtool makes even the adding an external plug-in highly interactive.

Information regarding creating an out of tree module and in-tree commands is available here.