https://wiki.gnuradio.org/api.php?action=feedcontributions&user=Solomonbstoner&feedformat=atomGNU Radio - User contributions [en]2024-03-29T11:02:12ZUser contributionsMediaWiki 1.39.5https://wiki.gnuradio.org/index.php?title=GFSK_Mod&diff=8769GFSK Mod2021-08-23T23:46:07Z<p>Solomonbstoner: Make it clearer how to replicate the results on the flowgraph</p>
<hr />
<div>[[Category:Block Docs]]<br />
Hierarchical block for Gaussian Frequency Shift Key (GFSK) modulation.<br />
<br />
The input is a byte stream (unsigned char) and the output is the complex modulated signal at baseband.<br />
<br />
== Parameters ==<br />
<br />
; Samples/Symbol<br />
: Samples per baud >= 2 (integer)<br />
<br />
; Sensitivity<br />
: Given to the [[Frequency Mod]]<br />
<br />
; BT<br />
: Gaussian filter bandwidth * symbol time (float)<br />
<br />
; Verbose<br />
: Print information about modulator?<br />
<br />
; Log<br />
: Print modulation data to files?<br />
<br />
; Unpack<br />
: Unpack input byte stream?<br />
<br />
== Example Flowgraph ==<br />
<br />
The flowgraph below shows an example of the GFSK Mod and GFSK Demod blocks in action. We GFSK modulate 9-bit long bit stream '000111011', and then GFSK demodulate it. Then we compare the two bit streams to make sure that they are the same.<br />
<br />
[[File:Gfsk_mod_and_demod.png|800px]]<br />
<br />
When the flowgraph runs, we see that the red and blue signals are the same, meaning that the bit streams before and after the GFSK modulation and demodulation are the same.<br />
<br />
In order to replicate the results shown below, make sure you do the following:<br />
# Make sure you have a vector of 9 bits in your vector source. For example, I used (0,0,0,1,1,1,0,1,1) in this example.<br />
# Make sure you turn off the unpack function in the GFSK Mod block. You will need to double click the block to bring up the Properties pop-up in order to turn it off. Turn it off. Then you will see "Unpack: Off" show up on the block after you close its Properties pop-up.<br />
<br />
[[File:Gfsk_mod_and_demod_in_action.png|800px]]<br />
<br />
== Source Files ==<br />
<br />
; Python files<br />
: [https://github.com/gnuradio/gnuradio/blob/master/gr-digital/python/digital/gfsk.py]<br />
<br />
; Block definition<br />
: [https://github.com/gnuradio/gnuradio/blob/master/gr-digital/grc/digital_gfsk_mod.block.yml]</div>Solomonbstonerhttps://wiki.gnuradio.org/index.php?title=GFSK_Demod&diff=8768GFSK Demod2021-08-23T23:45:01Z<p>Solomonbstoner: Make it clearer how users can replicate the results of the example flowgraph</p>
<hr />
<div>[[Category:Block Docs]]<br />
Hierarchical block for Gaussian Minimum Shift Key (GFSK) demodulation.<br />
<br />
The input is the complex modulated signal at baseband.<br />
<br />
The output is a stream of bits packed 1 bit per byte (the LSB)<br />
<br />
== Parameters ==<br />
<br />
; Samples/Symbol<br />
: Samples per baud >= 2 (integer)<br />
<br />
; Sensitivity<br />
: Given to the [[Quadrature Demod]]<br />
<br />
; Gain Mu<br />
: Controls rate of mu adjustment<br />
<br />
; Mu<br />
: Fractional delay [0.0, 1.0]<br />
<br />
; Omega Relative Limit<br />
: Sets max variation in omega (float, typically 0.000200 (200 ppm))<br />
<br />
; Freq Error<br />
: Bit rate error as a fraction<br />
<br />
; Verbose<br />
: Print information about modulator?<br />
<br />
; Log<br />
: Print modualtion data to files? (bool)<br />
<br />
== Example Flowgraph ==<br />
<br />
<br />
The flowgraph below shows an example of the GFSK Mod and GFSK Demod blocks in action. We GFSK modulate 9-bit long bit stream '000111011', and then GFSK demodulate it. Then we compare the two bit streams to make sure that they are the same.<br />
<br />
[[File:Gfsk_mod_and_demod.png|800px]]<br />
<br />
When the flowgraph runs, we see that the red and blue signals are the same, meaning that the bit streams before and after the GFSK modulation and demodulation are the same.<br />
<br />
In order to replicate the results shown below, make sure you do the following:<br />
# Make sure you have a vector of 9 bits in your vector source. For example, I used (0,0,0,1,1,1,0,1,1) in this example.<br />
# Make sure you turn off the unpack function in the GFSK Mod block. You will need to double click the block to bring up the Properties pop-up in order to turn it off. Turn it off. Then you will see "Unpack: Off" show up on the block after you close its Properties pop-up.<br />
<br />
[[File:Gfsk_mod_and_demod_in_action.png|800px]]<br />
<br />
== Source Files ==<br />
<br />
; Python files<br />
: [https://github.com/gnuradio/gnuradio/blob/master/gr-digital/python/digital/gfsk.py]<br />
<br />
; Block definition<br />
: [https://github.com/gnuradio/gnuradio/blob/master/gr-digital/grc/digital_gfsk_demod.block.yml]</div>Solomonbstonerhttps://wiki.gnuradio.org/index.php?title=GMSK_Mod&diff=8738GMSK Mod2021-08-02T14:54:18Z<p>Solomonbstoner: To reflect new parameter introduced with PR #4970</p>
<hr />
<div>[[Category:Block Docs]]<br />
Hierarchical block for Gaussian Minimum Shift Key (GMSK) modulation.<br />
<br />
The input is a byte stream (unsigned char with packed bits)<br />
<br />
The output is the complex modulated signal at baseband.<br />
<br />
== Parameters ==<br />
<br />
; Samples/Symbol<br />
: samples per baud >= 2<br />
<br />
; BT<br />
: Gaussian filter bandwidth * symbol time<br />
<br />
; Verbose<br />
: Print information about modulator?<br />
<br />
; Log<br />
: Print modulation data to files?<br />
<br />
; Unpack<br />
: Unpack input byte stream?<br />
<br />
== Example Flowgraph ==<br />
<br />
This flowgraph can be found at [https://github.com/gnuradio/gnuradio/blob/master/gr-channels/examples/demo_gmsk.grc]<br />
<br />
[[File:Demo_gmsk_fg.png|799px]]<br />
<br />
== Source Files ==<br />
<br />
; Python files<br />
: [https://github.com/gnuradio/gnuradio/blob/master/gr-digital/python/digital/gmsk.py]<br />
<br />
; Block definition<br />
: [https://github.com/gnuradio/gnuradio/blob/master/gr-digital/grc/digital_gmsk_mod.block.yml]</div>Solomonbstonerhttps://wiki.gnuradio.org/index.php?title=GFSK_Demod&diff=8732GFSK Demod2021-07-21T14:51:35Z<p>Solomonbstoner: Add example flowgraph for the GFSK Demod block</p>
<hr />
<div>[[Category:Block Docs]]<br />
Hierarchical block for Gaussian Minimum Shift Key (GFSK) demodulation.<br />
<br />
The input is the complex modulated signal at baseband.<br />
<br />
The output is a stream of bits packed 1 bit per byte (the LSB)<br />
<br />
== Parameters ==<br />
<br />
; Samples/Symbol<br />
: Samples per baud >= 2 (integer)<br />
<br />
; Sensitivity<br />
: Given to the [[Quadrature Demod]]<br />
<br />
; Gain Mu<br />
: Controls rate of mu adjustment<br />
<br />
; Mu<br />
: Fractional delay [0.0, 1.0]<br />
<br />
; Omega Relative Limit<br />
: Sets max variation in omega (float, typically 0.000200 (200 ppm))<br />
<br />
; Freq Error<br />
: Bit rate error as a fraction<br />
<br />
; Verbose<br />
: Print information about modulator?<br />
<br />
; Log<br />
: Print modualtion data to files? (bool)<br />
<br />
== Example Flowgraph ==<br />
<br />
<br />
The flowgraph below shows an example of the GFSK Mod and GFSK Demod blocks in action. We GFSK modulate 9-bit long bit stream '000111011', and then GFSK demodulate it. Then we compare the two bit streams to make sure that they are the same.<br />
<br />
[[File:Gfsk_mod_and_demod.png|800px]]<br />
<br />
When the flowgraph runs, we see that the red and blue signals are the same, meaning that the bit streams before and after the GFSK modulation and demodulation are the same.<br />
<br />
[[File:Gfsk_mod_and_demod_in_action.png|800px]]<br />
<br />
== Source Files ==<br />
<br />
; Python files<br />
: [https://github.com/gnuradio/gnuradio/blob/master/gr-digital/python/digital/gfsk.py]<br />
<br />
; Block definition<br />
: [https://github.com/gnuradio/gnuradio/blob/master/gr-digital/grc/digital_gfsk_demod.block.yml]</div>Solomonbstonerhttps://wiki.gnuradio.org/index.php?title=GFSK_Mod&diff=8731GFSK Mod2021-07-21T14:46:26Z<p>Solomonbstoner: Add example flowgraph for GFSK Mod block</p>
<hr />
<div>[[Category:Block Docs]]<br />
Hierarchical block for Gaussian Frequency Shift Key (GFSK) modulation.<br />
<br />
The input is a byte stream (unsigned char) and the output is the complex modulated signal at baseband.<br />
<br />
== Parameters ==<br />
<br />
; Samples/Symbol<br />
: Samples per baud >= 2 (integer)<br />
<br />
; Sensitivity<br />
: Given to the [[Frequency Mod]]<br />
<br />
; BT<br />
: Gaussian filter bandwidth * symbol time (float)<br />
<br />
; Verbose<br />
: Print information about modulator?<br />
<br />
; Log<br />
: Print modulation data to files?<br />
<br />
; Unpack<br />
: Unpack input byte stream?<br />
<br />
== Example Flowgraph ==<br />
<br />
The flowgraph below shows an example of the GFSK Mod and GFSK Demod blocks in action. We GFSK modulate 9-bit long bit stream '000111011', and then GFSK demodulate it. Then we compare the two bit streams to make sure that they are the same.<br />
<br />
[[File:Gfsk_mod_and_demod.png|800px]]<br />
<br />
When the flowgraph runs, we see that the red and blue signals are the same, meaning that the bit streams before and after the GFSK modulation and demodulation are the same.<br />
<br />
[[File:Gfsk_mod_and_demod_in_action.png|800px]]<br />
<br />
<br />
== Source Files ==<br />
<br />
; Python files<br />
: [https://github.com/gnuradio/gnuradio/blob/master/gr-digital/python/digital/gfsk.py]<br />
<br />
; Block definition<br />
: [https://github.com/gnuradio/gnuradio/blob/master/gr-digital/grc/digital_gfsk_mod.block.yml]</div>Solomonbstonerhttps://wiki.gnuradio.org/index.php?title=File:Gfsk_mod_and_demod_in_action.png&diff=8730File:Gfsk mod and demod in action.png2021-07-21T14:31:52Z<p>Solomonbstoner: The gfsk mod example flowgraph in action.</p>
<hr />
<div>The gfsk mod example flowgraph in action.</div>Solomonbstonerhttps://wiki.gnuradio.org/index.php?title=File:Gfsk_mod_and_demod.png&diff=8729File:Gfsk mod and demod.png2021-07-21T14:28:54Z<p>Solomonbstoner: Example flowgraph of the GFSK Mod block</p>
<hr />
<div>Example flowgraph of the GFSK Mod block</div>Solomonbstonerhttps://wiki.gnuradio.org/index.php?title=GFSK_Mod&diff=8728GFSK Mod2021-07-21T14:27:27Z<p>Solomonbstoner: Add new parameter to the GFSK Mod block</p>
<hr />
<div>[[Category:Block Docs]]<br />
Hierarchical block for Gaussian Frequency Shift Key (GFSK) modulation.<br />
<br />
The input is a byte stream (unsigned char) and the output is the complex modulated signal at baseband.<br />
<br />
== Parameters ==<br />
<br />
; Samples/Symbol<br />
: Samples per baud >= 2 (integer)<br />
<br />
; Sensitivity<br />
: Given to the [[Frequency Mod]]<br />
<br />
; BT<br />
: Gaussian filter bandwidth * symbol time (float)<br />
<br />
; Verbose<br />
: Print information about modulator?<br />
<br />
; Log<br />
: Print modulation data to files?<br />
<br />
; Unpack<br />
: Unpack input byte stream?<br />
<br />
== Example Flowgraph ==<br />
<br />
Insert description of flowgraph here, then show a screenshot of the flowgraph and the output if there is an interesting GUI. Currently we have no standard method of uploading the actual flowgraph to the wiki or git repo, unfortunately. The plan is to have an example flowgraph showing how the block might be used, for every block, and the flowgraphs will live in the git repo.<br />
<br />
== Source Files ==<br />
<br />
; Python files<br />
: [https://github.com/gnuradio/gnuradio/blob/master/gr-digital/python/digital/gfsk.py]<br />
<br />
; Block definition<br />
: [https://github.com/gnuradio/gnuradio/blob/master/gr-digital/grc/digital_gfsk_mod.block.yml]</div>Solomonbstonerhttps://wiki.gnuradio.org/index.php?title=Message_Passing&diff=8615Message Passing2021-05-26T10:43:33Z<p>Solomonbstoner: Add picture that describes a PDU for the visually inclined</p>
<hr />
<div>[[Category:Usage Manual]]<br />
== Introduction ==<br />
<br />
GNU Radio was originally a streaming system with no other mechanism to<br />
pass data between blocks. Streams of data are a model that work well<br />
for samples, bits, etc., but are not really the right mechanism for<br />
control data, metadata, or packet structures (at least at<br />
some point in the processing chain).<br />
<br />
We solved part of this problem by introducing the tag stream (see [[Stream Tags]]). This is a parallel stream to the data<br />
streaming. The difference is that tags are designed to hold metadata<br />
and control information. Tags are specifically associated with a<br />
particular sample in the data stream and flow downstream alongside the<br />
data. This model allows other blocks to identify that an event or<br />
action has occurred or should occur on a particular item. The major<br />
limitation is that the tag stream is really only accessible inside a<br />
work function and only flows in one direction. Its benefit is that it<br />
is isosynchronous with the data.<br />
<br />
We want a more general message passing system for a couple of<br />
reasons. The first is to allow blocks downstream to communicate back<br />
to blocks upstream. The second is to allow an easier way for us to<br />
communicate back and forth between external applications and GNU<br />
Radio. GNU Radio's message passing interface handles these cases, although<br />
it does so on an asynchronous basis.<br />
<br />
The message passing interface heavily relies on Polymorphic Types<br />
(PMTs) in GNU Radio. For further information about these data<br />
structures, see the page [[Polymorphic Types (PMTs)]].<br />
<br />
[[File:Gnuradio_pdu_illustration.png|550px|Gnuradio_pdu_illustration.png]]<br />
<br />
== Message Passing API ==<br />
<br />
The message passing interface is designed into the gr::basic_block,<br />
which is the parent class for all blocks in GNU Radio. Each block has<br />
a set of message queues to hold incoming messages and can post<br />
messages to the message queues of other blocks. The blocks also<br />
distinguish between input and output ports.<br />
<br />
A block has to declare its input and output message ports in its<br />
constructor. The message ports are described by a name, which is in<br />
practice a PMT symbol (i.e., an interned string). The API calls<br />
to register a new port are:<br />
<br />
void message_port_register_in(pmt::pmt_t port_id)<br />
void message_port_register_out(pmt::pmt_t port_id)<br />
<br />
In Python:<br />
<br />
self.message_port_register_in(pmt.intern("port name"))<br />
self.message_port_register_out(pmt.intern("port name"))<br />
<br />
The ports are now identifiable by that port name. Other blocks who may<br />
want to post or receive messages on a port must subscribe to it. When<br />
a block has a message to send, they are published on a particular<br />
port using the following API:<br />
<br />
void message_port_pub(pmt::pmt_t port_id, pmt::pmt_t msg);<br />
<br />
In Python:<br />
<br />
self.message_port_pub(pmt.intern("port name"), <pmt message>)<br />
<br />
Subscribing is usually done in the form of connecting message ports <br />
as part of the flowgraph, as discussed later. Internally, when message<br />
ports are connected, the gr::basic_block::message_port_sub method is <br />
called.<br />
<br />
Any block that has a subscription to another block's output message<br />
port will receive the message when it is published. Internally, when a<br />
block publishes a message, it simply iterates through all blocks that<br />
have subscribed and uses the gr::basic_block::_post method to send the<br />
message to that block's message queue.<br />
<br />
=== Message Handler Functions ===<br />
<br />
A subscriber block must declare a message handler function to process<br />
the messages that are posted to it. After using the<br />
gr::basic_block::message_port_register_in to declare a subscriber port, we<br />
must then bind this port to the message handler. <br />
<br />
Starting in GNU Radio 3.8¹ using C++11 we do that using a lambda function:<br />
<br />
set_msg_handler(pmt::pmt_t port_id, <br />
[this](const pmt::pmt_t& msg) { message_handler_function(msg); });<br />
<br />
In Python:<br />
<br />
self.set_msg_handler(pmt.intern("port name"), <msg handler function>)<br />
<br />
When a new message is pushed onto a port's message queue,<br />
it is this function that is used to process the message.<br />
The 'port_id' is the same PMT as used when registering the input<br />
port. The 'block_class::message_handler_function' is the member<br />
function of the class designated to handle messages to this port. <br />
<br />
The prototype for all message handling functions<br />
is:<br />
<br />
void block_class::message_handler_function(const pmt::pmt_t& msg);<br />
<br />
In Python the equivalent function would be:<br />
<br />
def handle_msg(self, msg):<br />
<br />
We give examples of using this below.<br />
<br />
=== Connecting Messages through the Flowgraph ===<br />
<br />
From the flowgraph level, we have instrumented a gr::hier_block2::msg_connect<br />
method to make it easy to subscribe blocks to other blocks'<br />
messages. Assume that the block '''src''' has an output message port named<br />
''pdus'' and the block '''dbg''' has an input port named ''print''. The message <br />
connection in the flowgraph (in Python) looks like the following:<br />
<br />
self.tb.msg_connect(src, "pdus", dbg, "print")<br />
<br />
All messages published by the '''src''' block on port ''pdus'' will be<br />
received by '''dbg''' on port ''print''. Note here how we are just using<br />
strings to define the ports, not PMT symbols. This is a convenience to<br />
the user to be able to more easily type in the port names (for<br />
reference, you can create a PMT symbol in Python using the<br />
pmt::intern function as pmt.intern("string")).<br />
<br />
Users can also query blocks for the names of their input and output<br />
ports using the following API calls:<br />
<br />
pmt::pmt_t message_ports_in();<br />
pmt::pmt_t message_ports_out();<br />
<br />
The return value for these are a PMT vector filled with PMT symbols,<br />
so PMT operators must be used to manipulate them.<br />
<br />
Each block has internal methods to handle posting and receiving of<br />
messages. The gr::basic_block::_post method takes in a message and<br />
places it into its queue. The publishing model uses the<br />
gr::basic_block::_post method of the blocks as the way to access the<br />
message queue. So the message queue of the right name will have a new<br />
message. Posting messages also has the benefit of waking up the<br />
block's thread if it is in a wait state. So if idle, as soon as a<br />
message is posted, it will wake up and call the message handler.<br />
<br />
== Posting from External Sources ==<br />
<br />
An important feature of the message passing architecture<br />
is how it can be used to take in messages from an external source. We<br />
can call a block's gr::basic_block::_post method directly and pass it a<br />
message. So any block with an input message port can receive messages<br />
from the outside in this way.<br />
<br />
The following example uses a gr::blocks::pdu_to_tagged_stream block<br />
as the source block to a flowgraph. Its purpose is to wait for<br />
messages as PDUs posted to it and convert them to a normal stream. The<br />
payload will be sent on as a normal stream while the meta data will be<br />
decoded into tags and sent on the tagged stream.<br />
<br />
So if we have created a '''src''' block as a PDU to stream, it has a<br />
''pdus'' input port, which is how we will inject PDU messages into the<br />
flowgraph. These PDUs could come from another block or flowgraph, but<br />
here, we will create and insert them by hand.<br />
<br />
port = pmt.intern("pdus")<br />
msg = pmt.cons(pmt.PMT_NIL, pmt.make_u8vector(16, 0xFF))<br />
src.to_basic_block()._post(port, msg)<br />
<br />
The PDU's metadata section is empty, hence the pmt::PMT_NIL<br />
object. The payload is now just a simple vector of 16 bytes of all<br />
1's. To post the message, we have to access the block's gr::basic_block<br />
class, which we do using the gr::basic_block::to_basic_block method and<br />
then call the gr::basic_block::_post method to pass the PDU to the<br />
right port.<br />
<br />
All of these mechanisms are explored and tested in the QA code of the<br />
file qa_pdu.py.<br />
<br />
There are some examples of using the message passing infrastructure<br />
through GRC in:<br />
<br />
gr-blocks/examples/msg_passing<br />
<br />
== Using Messages as Commands ==<br />
<br />
One important use of messages is to send commands to blocks. Examples for this include:<br />
<br />
* gr::qtgui::freq_sink_c: The scaling of the frequency axis can be changed by messages<br />
* gr::uhd::usrp_source and gr::uhd::usrp_sink: Many transceiver-related settings can be manipulated through command messages, such as frequency, gain and LO offset<br />
* gr::digital::header_payload_demux, which receives an acknowledgement from a header parser block on how many payload items there are to process<br />
<br />
There is no special PMT type to encode commands, however, it is strongly recommended<br />
to use one of the following formats:<br />
<br />
* pmt::cons(KEY, VALUE): This format is useful for commands that take a single value. Think of KEY and VALUE as the argument name and value, respectively. For the case of the QT GUI Frequency Sink, KEY would be "freq" and VALUE would be the new center frequency in Hz.<br />
* pmt::dict((KEY1: VALUE1), (KEY2: VALUE2), ...): This is basically the same as the previous format, but you can provide multiple key/value pairs. This is particularly useful when a single command takes multiple arguments which can't be broken into multiple command messages (e.g., the USRP blocks might have both a timestamp and a center frequency in a command message, which are closely associated).<br />
<br />
In both cases, all KEYs should be pmt::symbols (i.e. strings). VALUEs can be<br />
whatever the block requires.<br />
<br />
It might be tempting to deviate from this format, e.g. the QT Frequency sink could<br />
simply take a float value as a command message, and it would still work fine.<br />
However, there are some very good reasons to stick to this format:<br />
<br />
* Interoperability: The more people use the standard format, the more likely it is that blocks from different sources can work together<br />
* Inspectability: A message debug block will display more useful information about a message if it's containing both a value and a key<br />
* Intuition: This format is pretty versatile and unlikely to create situations where it is not sufficient (especially considering that values are PMTs themselves). As a counterexample, using positional arguments (something like "the first argument is the frequency, the second the gain") is easily forgotten, or changed in one place and not another, etc.<br />
<br />
== Code Examples ==<br />
<br />
=== C++ ===<br />
<br />
The following is snippets of code from blocks currently in GNU Radio<br />
that take advantage of message passing. We will be using<br />
gr::blocks::message_debug and gr::blocks::tagged_stream_to_pdu below<br />
to show setting up both input and output message passing capabilities.<br />
<br />
The gr::blocks::message_debug block is used for debugging the message<br />
passing system. It describes two input message ports: ''print'' and<br />
''store''. The ''print'' port simply prints out all<br />
messages to standard out while the ''store'' port keeps a list of all<br />
messages posted to it. The ''store'' port works in<br />
conjunction with a gr::blocks::message_debug::get_message(size_t i) call<br />
that allows us to retrieve message i afterward.<br />
<br />
The constructor of this block looks like this:<br />
<br />
<syntaxhighlight lang="cpp"><br />
{<br />
message_port_register_in(pmt::mp("print"));<br />
set_msg_handler(pmt::mp("print"),<br />
[this](const pmt::pmt_t& msg) { print(msg); });<br />
<br />
message_port_register_in(pmt::mp("store"));<br />
set_msg_handler(pmt::mp("store"),<br />
[this](const pmt::pmt_t& msg) { store(msg); });<br />
}<br />
</syntaxhighlight><br />
<br />
The three message input ports are registered by their respective<br />
names. We then use the gr::basic_block::set_msg_handler function to <br />
identify this particular port name with a callback function.<br />
<br />
So now<br />
the functions in the block's private implementation class,<br />
gr::blocks::message_debug_impl::print and<br />
gr::blocks::message_debug_impl::store are assigned to handle<br />
messages passed to them. Below is the ''print'' function for reference.<br />
<br />
<syntaxhighlight lang="cpp"><br />
void<br />
message_debug_impl::print(const pmt::pmt_t& msg)<br />
{<br />
std::cout << "***** MESSAGE DEBUG PRINT ********\n";<br />
pmt::print(msg);<br />
std::cout << "**********************************\n";<br />
}<br />
</syntaxhighlight><br />
<br />
The function simply takes in the PMT message and prints it. The method<br />
pmt::print is a function in the PMT library to print the<br />
PMT in a friendly and (mostly) pretty manner.<br />
<br />
The gr::blocks::tagged_stream_to_pdu block only defines a single<br />
output message port. In this case, its constructor contains the line:<br />
<br />
<syntaxhighlight lang="cpp"><br />
{<br />
message_port_register_out(pdu_port_id);<br />
}<br />
</syntaxhighlight><br />
<br />
So we are only creating a single output port where ''pdu_port_id''<br />
is defined in the file pdu.h as ''pdus''.<br />
<br />
This block's purpose is to take in a stream of samples along with<br />
stream tags and construct a predefined PDU message from it. In GNU<br />
Radio, we define a PDU as a PMT pair of (metadata, data). The metadata<br />
describes the samples found in the data portion of the<br />
pair. Specifically, the metadata can contain the length of the data<br />
segment and any other information (sample rate, etc.). The PMT vectors<br />
know their own length, so the length value is not actually necessary<br />
unless useful for purposes down the line. The metadata is a PMT<br />
dictionary while the data segment is a PMT uniform vector of either<br />
bytes, floats, or complex values.<br />
<br />
In the end, when a PDU message is ready, the block calls its<br />
gr::blocks::tagged_stream_to_pdu_impl::send_message function that is<br />
shown below.<br />
<br />
<syntaxhighlight lang="cpp"><br />
void<br />
tagged_stream_to_pdu_impl::send_message()<br />
{<br />
if(pmt::length(d_pdu_vector) != d_pdu_length) {<br />
throw std::runtime_error("msg length not correct");<br />
}<br />
<br />
pmt::pmt_t msg = pmt::cons(d_pdu_meta,<br />
d_pdu_vector);<br />
message_port_pub(pdu_port_id, msg);<br />
<br />
d_pdu_meta = pmt::PMT_NIL;<br />
d_pdu_vector = pmt::PMT_NIL;<br />
d_pdu_length = 0;<br />
d_pdu_remain = 0;<br />
d_inpdu = false;<br />
}<br />
</syntaxhighlight><br />
<br />
This function does a bit of checking to make sure the PDU is OK as<br />
well as some cleanup in the end. But it is the line where the message<br />
is published that is important to this discussion. Here, the block<br />
posts the PDU message to any subscribers by calling<br />
gr::basic_block::message_port_pub publishing method.<br />
<br />
There is similarly a gr::blocks::pdu_to_tagged_stream block that essentially<br />
does the opposite. It acts as a source to a flowgraph and waits for<br />
PDU messages to be posted to it on its input port ''pdus''. It extracts<br />
the metadata and data and processes them. The metadata dictionary is<br />
split up into key:value pairs and stream tags are created out of<br />
them. The data is then converted into an output stream of items and<br />
passed along. The next section describes how PDUs can be passed into a<br />
flowgraph using the gr::blocks::pdu_to_tagged_stream block.<br />
<br />
=== Python ===<br />
<br />
A Python Block example:<br />
<br />
<syntaxhighlight lang="python"><br />
class msg_block(gr.basic_block):<br />
def __init__(self):<br />
gr.basic_block.__init__(<br />
self,<br />
name="msg_block",<br />
in_sig=None,<br />
out_sig=None)<br />
<br />
self.message_port_register_out(pmt.intern('msg_out'))<br />
self.message_port_register_in(pmt.intern('msg_in'))<br />
self.set_msg_handler(pmt.intern('msg_in'), self.handle_msg)<br />
<br />
def handle_msg(self, msg):<br />
self.message_port_pub(pmt.intern('msg_out'), pmt.intern('message received!'))<br />
</syntaxhighlight><br />
<br />
== Flowgraph Example ==<br />
<br />
Here's a simple example of a flow graph using both streaming and messages:<br />
<br />
[[File:strobe.grc.png|550px|strobe.grc.png]]<br />
<br />
There are several interesting things to point out. First, there are two source blocks, which both output items at regular intervals, one every 1000 and one every 750 milliseconds. Dotted lines denote connected message ports, as opposed to solid lines, which denote connected streaming ports. In the top half of the flow graph, we can see that it is, in fact, possible to switch between message passing and streaming ports, but only if the type of the PMTs matches the type of the streaming ports (in this example, the pink color of the streaming ports denotes bytes, which means the PMT should be a u8vector if we want to stream the same data we sent as PMT).<br />
<br />
Another interesting fact is that we can connect more than one message output port to a single message input port, which is not possible with streaming ports. This is due to the '''asynchronous''' nature of messages: The receiving block will process all messages whenever it has a chance to do so, and not necessarily in any specific order. Receiving messages from multiple blocks simply means that there might be more messages to process.<br />
<br />
What happens to a message once it was posted to a block? This depends on the actual block implementation, but there are two possibilities:<br />
<br />
1) A message handler is called, which processes the message immediately.<br /><br />
2) The message is written to a FIFO buffer, and the block can make use of it whenever it likes, usually in the work function.<br />
<br />
For a block that has both message ports and streaming ports, any of these two options is OK, depending on the application. However, we strongly discourage the processing of messages inside of a work function and instead recommend the use of message handlers. Using messages in the work function encourages us to block in work waiting for a message to arrive. This is bad behavior for a work function, which should never block. If a block depends upon a message to operate, use the message handler concept to receive the message, which may then be used to inform the block's actions when the work function is called. Only on specially, well-identified occasions should we use method 2 above in a block.<br />
<br />
With a message passing interface, we can write blocks that don't have streaming ports, and then the work function becomes useless, since it's a function that is designed to work on streaming items. In fact, blocks that don't have streaming ports usually don't even have a work function.<br />
<br />
=== PDUs ===<br />
<br />
In the previous flow graph, we have a block called ''PDU to Tagged Stream''. A PDU (protocol data unit) in GNU Radio has a special PMT type, it is a pair of a dictionary (on CAR) and a uniform vector type. So, this would yield a valid PDU, with no metadata and 10 zeros as stream data:<br />
<br />
<pre>pdu = pmt.cons(pmt.make_dict(), pmt.make_u8vector(10, 0))</pre><br />
The key/value pairs in the dictionary are then interpreted as key/value pairs of stream tags.<br />
<br />
== Flowgraph Example: Chat Application ==<br />
<br />
Let's build an application that uses message passing. A chat program is an ideal use case, since it waits for the user to type a message, and then sends it. Because of that, no [[Throttle]] block is needed.<br />
<br />
Create the following flowgraph and save it as 'chat_app2.grc':<br />
<br />
[[File:Chat_app2_fg.png]]<br />
<br />
The ZMQ Message blocks have an Address of 'tcp://127.0.0.1:50261'. Typing in the [[QT GUI Message Edit Box]] will send the text once the Enter key is pressed. Output is on the terminal screen where gnuradio-companion was started.<br />
<br />
If you want to talk to another user (instead of just yourself), you can create an additional flowgraph with a different name such as 'chat_app3.grc'. Then change the ZMQ port numbers as follows:<br />
* chat_app2<br />
** ZMQ PUSH Sink: tcp://127.0.0.1:50261<br />
** ZMQ PULL Source: tcp://127.0.0.1:50262<br />
<br />
* chat_app3<br />
** ZMQ PUSH Sink: tcp://127.0.0.1:50262<br />
** ZMQ PULL Source: tcp://127.0.0.1:50261<br />
<br />
When using GRC, doing a Generate and/or Run creates a Python file with the same name as the .grc file. You can execute the Python file without running GRC again.<br />
<br />
For testing this system we will use two processes, so we will need two terminal windows.<br />
<br />
Terminal 1:<br />
* since you just finished building the chat_app3 flowgraph, you can just do a Run.<br />
<br />
Terminal 2:<br />
Open another terminal window.<br />
* change to whatever directory you used to generate the flowgraph for chat_app2.<br />
* execute the following command:<br />
python3 -u chat_app2.py<br />
<br />
Typing in the Message Edit Box for chat_app2 should be displayed on the Terminal 1 screen (chat_app3) and vice versa.<br />
<br />
To terminate each of the processes cleanly, click on the 'X' in the upper corner of the GUI rather than using Control-C.<br />
<br />
<br />
----<br />
<br />
¹ In old GNU Radio 3.7, we used Boost's 'bind' function:<br />
<br />
set_msg_handler(pmt::pmt_t port_id,<br />
boost::bind(&block_class::message_handler_function, this, _1));}}<br />
<br />
The 'this' and '_1' are standard ways of using the Boost bind function to<br />
pass the 'this' pointer as the first argument to the class (standard<br />
OOP practice) and the _1 is an indicator that the function expects 1<br />
additional argument. <br />
<br />
[[Category:Guided Tutorials]]</div>Solomonbstonerhttps://wiki.gnuradio.org/index.php?title=File:Gnuradio_pdu_illustration.png&diff=8614File:Gnuradio pdu illustration.png2021-05-26T10:33:12Z<p>Solomonbstoner: Illustration of how PDU works</p>
<hr />
<div>Illustration of how PDU works</div>Solomonbstonerhttps://wiki.gnuradio.org/index.php?title=File:Gnuradio_stream_illustration.png&diff=8613File:Gnuradio stream illustration.png2021-05-26T10:32:38Z<p>Solomonbstoner: Illustration of how stream tags work.</p>
<hr />
<div>Illustration of how stream tags work.</div>Solomonbstonerhttps://wiki.gnuradio.org/index.php?title=Tagged_Stream_Blocks&diff=8603Tagged Stream Blocks2021-05-23T04:56:26Z<p>Solomonbstoner: Troubleshooting should not be classified under examples</p>
<hr />
<div>[[Category:Usage Manual]]<br />
== Introduction ==<br />
<br />
A tagged stream block is a block that works on streamed, but packetized input data.<br />
Think of packet data transmission: A data packet consists of N bytes. However, in<br />
traditional GNU Radio blocks, if we stream N bytes into a block, there's no way of<br />
knowing the packet boundary. This might be relevant: Perhaps the modulator has to<br />
prepend a synchronization word before every packet or append a CRC. So while some<br />
blocks don't care about packet boundaries, other blocks do: These are tagged<br />
stream blocks.<br />
<br />
These blocks are different from all the other GNU Radio block types (gr::block,<br />
gr::sync_block etc.) in that they are driven by the input: The PDU length tag tells the block how to operate, whereas other blocks are output-driven (the scheduler tries to fill up the output buffer as much as possible).<br />
<br />
=== How do they work? ===<br />
<br />
As the name implies, tagged stream blocks use tags to identify PDU boundaries.<br />
On the first item of a streamed PDU, there ''must'' be a tag with a specific<br />
key, which stores the length of the PDU as a PMT integer. If anything else, or no tag,<br />
is on this first item, this will cause the flow graph to crash!<br />
<br />
The scheduler then takes care of everything. When the work function is called, it<br />
is guaranteed to contain exactly one complete PDU and enough space in the output buffer for<br />
the output.<br />
<br />
=== How do they relate to other block types (e.g. sync blocks)? ===<br />
<br />
Tagged stream blocks and sync blocks are really orthogonal concepts, and a block could be<br />
both (gr::digital::ofdm_frame_equalizer_vcvc is such a block). However, because the work<br />
function is defined differently in these block types, there is no way to derive a block<br />
from both gr::tagged_stream_block and gr::sync_block.<br />
<br />
If a block needs the tagged stream mechanism (i.e. knowing about PDU boundaries), it must be<br />
derived from gr::tagged_stream_block. If it's also a sync block, it is still possible to<br />
set gr::block::set_relative_rate(1.0) and/or the fixed rate functions.<br />
<br />
The way gr::tagged_stream_block works, it is still beneficial to specify a relative rate,<br />
if possible.<br />
<br />
== Creating a tagged stream block ==<br />
<br />
To create a tagged stream block, the block must be derived from gr::tagged_stream_block.<br />
Here's a minimal example of how the header file could look:<br />
<br />
<syntaxhighlight lang="cpp"><br />
#include <gnuradio/digital/api.h><br />
#include <gnuradio/tagged_stream_block.h> <br />
<br />
namespace gr {<br />
namespace digital { <br />
<br />
class DIGITAL_API crc32_bb : virtual public tagged_stream_block<br />
{<br />
public:<br />
typedef boost::shared_ptr<crc32_bb> sptr;<br />
<br />
static sptr make(bool check=false, const std::string& len_tag_key="packet_len");<br />
}; <br />
<br />
} // namespace digital<br />
} // namespace gr<br />
</syntaxhighlight><br />
<br />
It is very similar to any other block definition. Two things are stand out: First,<br />
gnuradio/tagged_stream_block.h is included to allow deriving from gr::tagged_stream_block.<br />
<br />
The other thing is in the make function: the second argument is a string containing<br />
the key of the length tags. This is not necessary (the block could get this information<br />
hard-coded), but the usual way is to allow the user to change this tag, but give a<br />
default value (in this case, packet_len).<br />
<br />
The implementation header (*_impl.h) also looks a bit different (again this is cropped to the relevant parts):<br />
<br />
<br />
<syntaxhighlight lang="cpp"><br />
#include <digital/crc32_bb.h><br />
namespace gr {<br />
namespace digital {<br />
<br />
class crc32_bb_impl : public crc32_bb<br />
{<br />
public:<br />
crc32_bb_impl(bool check, const std::string& len_tag_key);<br />
~crc32_bb_impl();<br />
<br />
int calculate_output_stream_length(const gr_vector_int &ninput_items);<br />
int work(int noutput_items,<br />
gr_vector_int &ninput_items,<br />
gr_vector_const_void_star &input_items,<br />
gr_vector_void_star &output_items);<br />
};<br />
<br />
} // namespace digital<br />
} // namespace gr<br />
</syntaxhighlight><br />
<br />
First, the work function signature is new. The argument list looks like that from<br />
gr::block::general_work() (note: the arguments mean the same, too), but it's called<br />
work like with the other derived block types (such as gr::sync_block). Also, there's a new<br />
function: calculate_output_stream_length() is, in a sense, the opposite function to<br />
gr::block::forecast(). Given a number of input items, it calculates the required number of<br />
output items. Note how this relates to the fact that these blocks are input-driven.<br />
<br />
These two overrides (work() and calculate_output_stream_length()) are what you need<br />
for most tagged stream blocks. There are cases when you don't need to override the latter<br />
because the default behaviour is enough, and other cases where you have to override more<br />
than these two functions. These are discussed in [[Tagged Stream Blocks#Advanced Usage|Advanced Usage]].<br />
<br />
Finally, this is part of the actual block implementation (heavily cropped again, to highlight the relevant parts):<br />
<br />
<br />
<syntaxhighlight lang="cpp"><br />
#include <gnuradio/io_signature.h><br />
#include "crc32_bb_impl.h"<br />
<br />
namespace gr {<br />
namespace digital {<br />
<br />
crc32_bb::sptr crc32_bb::make(bool check, const std::string& len_tag_key)<br />
{<br />
return gnuradio::get_initial_sptr (new crc32_bb_impl(check, len_tag_key));<br />
}<br />
<br />
crc32_bb_impl::crc32_bb_impl(bool check, const std::string& len_tag_key)<br />
: tagged_stream_block("crc32_bb",<br />
io_signature::make(2, 1, sizeof (char)),<br />
io_signature::make(1, 1, sizeof (char)),<br />
len_tag_key),<br />
d_check(check)<br />
{<br />
}<br />
<br />
int<br />
crc32_bb_impl::calculate_output_stream_length(const gr_vector_int &ninput_items)<br />
{<br />
if (d_check) {<br />
return ninput_items[0] - 4;<br />
} else {<br />
return ninput_items[0] + 4;<br />
}<br />
}<br />
<br />
int<br />
crc32_bb_impl::work (int noutput_items,<br />
gr_vector_int &ninput_items,<br />
gr_vector_const_void_star &input_items,<br />
gr_vector_void_star &output_items)<br />
{<br />
const unsigned char *in = (const unsigned char *) input_items[0];<br />
unsigned char *out = (unsigned char *) output_items[0];<br />
<br />
// Do all the signal processing...<br />
// Don't call consume!<br />
<br />
return new_packet_length;<br />
}<br />
<br />
} // namespace digital<br />
} // namespace gr<br />
</syntaxhighlight><br />
<br />
The make function is not different to any other block. The constructor calls<br />
gr::tagged_stream_block::tagged_stream_block() as expected, but note that it passes the<br />
key of the length tag to the parent constructor.<br />
<br />
The block in question is a CRC block, and it has two modes: It can check the CRC (which is<br />
then removed), or it can append a CRC to a sequence of bytes. The calculate_output_stream_length() function is thus very simple: depending on how the block is configured, the output is either 4 bytes longer or shorter than the input stream.<br />
<br />
The work() function looks very similar to any other work function. When writing the<br />
signal processing code, the following things must be kept in mind:<br />
- The work function is called for exactly one PDU, and no more (or less) may be processed<br />
- ninput_items contains the exact number of items in this PDU (at every port). These items ''will'' be consumed after work() exits.<br />
- Don't call consume() or consume_each() yourself! gr::tagged_stream_block will do that for you.<br />
- You can call produce() or produce_each(), if you're doing something complicated. Don't forget to return WORK_CALLED_PRODUCE in that case.<br />
<br />
=== A note on tag propagation ===<br />
<br />
Despite using tags for a special purpose, all tags that are not the length tag are treated<br />
exactly as before: use gr::block::set_tag_propagation_policy() in the constructor.<br />
<br />
In a lot of the cases, though, you will need to specify set_tag_propagation_policy(TPP_DONT)<br />
and manually handle the tag propagation in work(). This is because the unknown length of<br />
the PDUs at compile time prohibits us from setting a precise relative rate of the block,<br />
which is a requirement for automatic tag propagation. Only if the tagged stream block is also a sync block (including interpolators and decimators, i.e. blocks with an integer rate change), can automatic tag propagation be reliably used.<br />
<br />
It is a general rule of GNU Radio blocks that they "properly" propagate tags, whatever this<br />
means for a specific application.<br />
<br />
The CRC block seems to a very simple block, but it's already complicated enough to confuse<br />
the automatic tag propagation. For example, what happens to tags which are on the CRC? Do<br />
they get removed, or do they get moved to the last item before the CRC? Also, the input to<br />
output rate is different for every PDU length.<br />
<br />
In this case, it is necessary for the developer to define a tag propagation policy and<br />
implement it in work(). Also, it is good practice to specify that tag propagation policy<br />
in the blocks documentation.<br />
<br />
The actual length tags ''are'' treated differently, though. Most importantly, you don't have<br />
to write the new length tag yourself. The key for the output length tag is the same as that<br />
on the input, if you don't want this, you must override gr::tagged_stream_block::update_length_tags().<br />
<br />
== Connecting regular streaming blocks and tagged stream blocks ==<br />
<br />
From the scheduler's point of view, all blocks are equivalent, and as long as the I/O<br />
signatures are compatible, all of these blocks can be connected.<br />
<br />
However, it is important to note that tagged stream blocks expect correctly tagged<br />
streams, i.e. a length tag with a number of items at the beginning of every packet.<br />
If this is not the case, the '''flow graph will crash'''.<br />
The most common cases are discussed separately:<br />
<br />
Connecting a tagged stream block to a regular stream block: This is never a problem,<br />
since regular blocks don't care about the values of the tags.<br />
<br />
Connecting regular stream blocks to tagged stream blocks: This will usually not work.<br />
One solution is to use the gr::blocks:stream_to_tagged_stream adapter block. It will<br />
periodically add tags at regular intervals, making the input valid for the tagged stream<br />
block. Make sure to directly connect this block to the tagged stream block, or the<br />
packet size might be changed to a different value from the tag value.<br />
<br />
Mixing block types: This is possible if none of the regular stream blocks change<br />
the rate. The ofdm_tx and ofdm_rx hierarchical blocks do this.<br />
<br />
== Advanced Usage ==<br />
<br />
It is generally recommended to read the block documentation of gr::tagged_stream_block.<br />
<br />
A few special cases are described here:<br />
<br />
=== Multiple length tags ===<br />
<br />
In some cases, a single tag is not enough. One example is the OFDM receiver: one OFDM frame contains a certain number of OFDM symbols, and another number of bytes--these numbers are only<br />
very loosely related, and one cannot be calculated from the other.<br />
<br />
gr::digital::ofdm_serializer_vcc is such a block. It is driven by the number of OFDM frames,<br />
but the output is determined by the number of complex symbols. In order to use multiple length<br />
tag keys, it overrides gr::tagged_stream_block::update_length_tags().<br />
<br />
=== Falling back to gr::block behavior ===<br />
<br />
If, at compile-time, it is uncertain whether or not a block should be a<br />
gr::tagged_stream_block, there is the possibility of falling back to gr::block behaviour.<br />
<br />
To do this, simple don't pass an empty string as length tag key. Instead of crashing,<br />
a tagged stream block will behave like a gr::block.<br />
<br />
This has some consequences: The work function must have all the elements of a<br />
gr::block::general_work() function, including calls to consume(). Because such a block must<br />
allow both modes of operation (PDUs with tags, and infinite-stream), the work function<br />
must check which mode is currently relevant. Checking if<br />
gr::tagged_stream_block::d_length_tag_key_str is empty is a good choice.<br />
<br />
gr::digital::ofdm_cyclic_prefixer implements this.<br />
<br />
=== Other ways to determine the number of input items ===<br />
<br />
If the number of input items is not stored as a pmt::pmt_integer, but there is a way to determine<br />
it, gr::tagged_stream_block::parse_length_tags() can be overridden to figure out the length<br />
of the PDU.<br />
<br />
== Examples ==<br />
<br />
=== CRC32 ===<br />
<br />
Block: [https://github.com/gnuradio/gnuradio/blob/master/gr-digital/lib/crc32_bb_impl.cc crc32_bb]<br />
<br />
This is a very simple block, and a good example to start with.<br />
<br />
=== OFDM Frame Equalizer ===<br />
<br />
Block: [https://github.com/gnuradio/gnuradio/blob/master/gr-digital/lib/ofdm_frame_equalizer_vcvc_impl.cc ofdm_frame_equalizer_vcvc]<br />
<br />
This block would be a sync block if tagged stream blocks didn't exist. It also uses more<br />
than one tag to determine the output.<br />
<br />
=== Tagged Stream Muxer ===<br />
<br />
Block: [https://github.com/gnuradio/gnuradio/blob/master/gr-blocks/lib/tagged_stream_mux_impl.cc tagged_stream_mux]<br />
<br />
Use this to multiplex any number of tagged streams.<br />
<br />
=== Cyclic Prefixer (OFDM) ===<br />
<br />
Block: [https://github.com/gnuradio/gnuradio/blob/master/gr-digital/lib/ofdm_cyclic_prefixer_impl.cc ofdm_cyclic_prefixer]<br />
<br />
This block uses the gr::block behavior fallback.<br />
<br />
== Troubleshooting ==<br />
<br />
'''My flow graph crashes with the error message "Missing length tag".'''<br />
<br />
This means the input of a tagged stream block was not correctly tagged.<br />
The most common cause is when connecting a regular streaming block to a tagged<br />
stream block. You can check the log output for the item number and port where<br />
this happened.</div>Solomonbstonerhttps://wiki.gnuradio.org/index.php?title=Stream_Tags&diff=8602Stream Tags2021-05-23T00:24:10Z<p>Solomonbstoner: Add comments to code example to make it easier to comprehend</p>
<hr />
<div>[[Category:Usage Manual]]<br />
== Introduction ==<br />
<br />
GNU Radio was originally a streaming system with no other mechanism to<br />
pass data between blocks. Streams of data are a model that work well<br />
for samples, bits, etc., but can lack for control and meta data.<br />
<br />
Part of this is solved using the existing message passing interface, which<br />
allows blocks to subscribe to messages published by any other block in<br />
the flowgraph (see [[Message Passing]]). The main drawback to the<br />
message passing system is that it works asynchronously, meaning that<br />
there is no guarantee when a message may arrive relative to the data<br />
stream.<br />
<br />
Stream tags are an isosynchronous data stream that runs parallel to<br />
the main data stream. A stream ''tag'' is generated by a block's work<br />
function and from there on flows downstream alongside a particular sample,<br />
until it reaches a sink or is forced to stop propagating by another<br />
block.<br />
<br />
Stream tags are defined for a specific item in the data stream and are<br />
formed as a key:value pair. The ''key'' identifies what the ''value'' represents<br />
while the value holds the data that the tag contains. Both ''key'' and<br />
''value'' are [[Polymorphic Types (PMTs)]] where the ''key'' is a PMT symbol while<br />
the ''value'' is any type of PMT and can therefore handle any data we wish<br />
to pass. An additional part of the tag is the ''srcid'', which is a PMT<br />
symbol and is used to identify the block that created the tag (which<br />
is usually the block's alias).<br />
<br />
== API Extensions to the gr::block ==<br />
<br />
To enable the stream tags, we have extended the API of gr::block to<br />
understand ''absolute'' item numbers. In the data stream model, each<br />
block's work function is given a buffer in the data stream that is<br />
referenced from 0 to N-1. This is a ''relative'' offset into the data<br />
stream. The absolute reference starts from the beginning of the<br />
flowgraph and continues to count up with every item. Each input stream<br />
is associated with a concept of the 'number of items read' and each<br />
output stream has a 'number of items written'. These are retrieved during <br />
runtime using the two API calls:<br />
<br />
unsigned long int nitems_read(unsigned int which_input);<br />
unsigned long int nitems_written(unsigned int which_output);<br />
<br />
Each tag is associated with some item in this absolute time scale that<br />
is calculated using these functions.<br />
<br />
Like the rest of the data stream, the number of items read/written are<br />
only updated once during the call to work. So in a work function,<br />
nitems_read/written will refer to the state of the data stream at the<br />
start of the work function. We must therefore add to this value the<br />
current relative offset in the data stream. So if we are iterating ''i'' over all output items, we would write the stream tag to output ports<br />
at nitems_written(0)+i for the 0th output port.<br />
<br />
== Stream Tags API ==<br />
<br />
The stream tags API is split into two parts: adding tags to a stream, <br />
and getting tags from a stream. <br />
Note that the functions described below are only meant to be accessed<br />
within a call to general_work/work. While they can be called at other points<br />
in time by a block, the behavior outside of work is undefined without<br />
exact knowledge of the item counts in the buffers.<br />
<br />
=== Adding a Tag to a Stream ===<br />
<br />
We add a tag to a particular output stream of the block using:<br />
<br />
* gr::block::add_item_tag: Adds an item tag to a particular output port using a gr::tag_t data type or by specifying the tag values.<br />
<br />
We can output them to multiple output streams if we want, but to do so<br />
means calling this function once for each port. This function can be <br />
provided with a gr::tag_t data type, or each value of the tag can be<br />
explicitly given.<br />
<br />
Again, a tag is defined as:<br />
<br />
# offset: The offset, in absolute item time, of the tag in the data stream.<br />
# key: the PMT symbol identifying the type of tag.<br />
# value: the PMT holding the data of the tag.<br />
# srcid: (optional) the PMT symbol identifying the block which created the tag.<br />
<br />
We can create a gr::tag_t structure to hold all of the above<br />
information of a tag, which is probably the easiest/best way to do<br />
it. The gr::tag_t struct is defined as having the same members as in<br />
the above list. To add a gr::tag_t tag to a stream, use the function:<br />
<br />
void add_item_tag(unsigned int which_output, const tag_t &tag);<br />
<br />
The secondary API allows us to create a tag by explicitly listing all<br />
of the tag information in the function call:<br />
<br />
void add_item_tag(unsigned int which_output,<br />
uint64_t abs_offset,<br />
const pmt::pmt_t &key,<br />
const pmt::pmt_t &value,<br />
const pmt::pmt_t &srcid=pmt::PMT_F);<br />
<br />
In Python, we can add a tag to a stream using one of the following:<br />
<br />
add_item_tag(which_output, abs_offset, key, value)<br />
add_item_tag(which_output, abs_offset, key, value, srcid)<br />
<br />
Note that the key and value are both PMTs. To create a string type PMT you can use pmt.intern("example_key").<br />
<br />
Consider the following flowgraph as an example. We will have an embedded python block insert stream tags at random intervals, and view these tags on the QT GUI Time sink. <br />
<br />
[[File:Epy_stream_tags_example_flowchart.png|800px]]<br />
<br />
Add the following code into the embedded python block. This code outputs the same signal as in the input, except with tags on randomly selected input samples:<br />
<br />
<syntaxhighlight lang="python"><br />
import numpy as np<br />
from gnuradio import gr<br />
import pmt<br />
<br />
class blk(gr.sync_block):<br />
def __init__(self):<br />
gr.sync_block.__init__(<br />
self,<br />
name='Embedded Python Block',<br />
in_sig=[np.complex64],<br />
out_sig=[np.complex64]<br />
)<br />
<br />
def work(self, input_items, output_items):<br />
for indx, sample in enumerate(input_items[0]): # Enumerate through the input samples in port in0<br />
if np.random.rand() > 0.95: # 5% chance this sample is chosen<br />
key = pmt.intern("example_key")<br />
value = pmt.intern("example_value")<br />
self.add_item_tag(0, # Write to output port 0<br />
self.nitems_written(0) + indx, # Index of the tag in absolute terms<br />
key, # Key of the tag<br />
value # Value of the tag<br />
)<br />
# note: (self.nitems_written(0) + indx) is our current sample, in absolute time<br />
output_items[0][:] = input_items[0] # copy input to output<br />
return len(output_items[0])<br />
</syntaxhighlight><br />
<br />
You may expect an output similar to the screenshot of the time sink below.<br />
<br />
[[File:Epy_stream_tags_example_timesink.png|800px]]<br />
<br />
=== Getting tags from a Stream ===<br />
<br />
To get tags from a particular input stream, we have two<br />
functions we can use:<br />
<br />
# gr::block::get_tags_in_range: Gets all tags from a particular input port between a certain range of items (in absolute item time).<br />
<br />
# gr::block::get_tags_in_window: Gets all tags from a particular input port between a certain range of items (in relative item time within the work function).<br />
<br />
The difference between these functions is working in absolute item<br />
time versus relative item time. Both of these pass back vectors of<br />
gr::tag_t, and they both allow<br />
specifying a particular key (as a PMT symbol) to filter against <br />
(or the fifth argument can be left out to search for all keys). <br />
Filtering for a certain key reduces the effort inside the work function<br />
for getting the right tag's data.<br />
<br />
For example, this call just returns any tags between the given range of items:<br />
<br />
void get_tags_in_range(std::vector<tag_t> &v,<br />
unsigned int which_input,<br />
uint64_t abs_start,<br />
uint64_t abs_end);<br />
<br />
Adding a fifth argument to this function allows us to filter on the<br />
key ''key''.<br />
<br />
void get_tags_in_range(std::vector<tag_t> &v,<br />
unsigned int which_input,<br />
uint64_t abs_start,<br />
uint64_t abs_end,<br />
const pmt::pmt_t &key);<br />
<br />
In Python, the main difference from the C++ function is that instead of having the first<br />
argument be a vector where the tags are stored, the Python version<br />
just returns a list of tags. We would use it like this:<br />
<br />
<syntaxhighlight lang="python"><br />
def work(self, input_items, output_items):<br />
....<br />
tags = self.get_tags_in_window(which_input, rel_start, rel_end)<br />
....<br />
</syntaxhighlight><br />
<br />
If you want to grab all tags on the samples currently being processed by work(), on input port 0, here's a minimal example of doing that:<br />
<br />
<syntaxhighlight lang="python"><br />
import numpy as np<br />
from gnuradio import gr<br />
import pmt<br />
<br />
class blk(gr.sync_block):<br />
def __init__(self):<br />
gr.sync_block.__init__(self,name='Read Tags', in_sig=[np.float32], out_sig=None)<br />
<br />
def work(self, input_items, output_items):<br />
tags = self.get_tags_in_window(0, 0, len(input_items[0]))<br />
for tag in tags:<br />
key = pmt.to_python(tag.key) # convert from PMT to python string<br />
value = pmt.to_python(tag.value) # Note that the type(value) can be several things, it depends what PMT type it was<br />
print 'key:', key<br />
print 'value:', value, type(value)<br />
print ''<br />
return len(input_items[0])<br />
</syntaxhighlight><br />
<br />
== Tag Propagation ==<br />
<br />
We now know how to add tags to streams, and how to read them. But what happens to tags after they were read? And what happens to tags that aren't used? After all, there are many blocks that don't care about tags at all.<br />
<br />
The answer is: It depends on the ''tag propagation policy'' of a block what happens to tags that enter it.<br /><br />
There are [http://gnuradio.org/doc/doxygen/classgr_1_1block.html#abc40fd4d514724a5446a2b34b2352b4e three policies] to choose from:<br />
<br />
# All-to-All: all tags from any input port are replicated to all output ports<br />
# One-to-One: tags from input port ''i'' are only copied to output port ''i'' (depends on num inputs = num outputs).<br />
# Dont: Does not propagate tags. Tags are either stopped here or the work function recreates them in some manner.<br />
<br />
The default behavior of a block is the 'All-to-All' method of<br />
propagation.<br />
<br />
We generally set the tag propagation policy in the block's constructor using [http://gnuradio.org/doc/doxygen/classgr_1_1block.html#a476e218927e426ac88c26431cbf086cd @set_tag_propagation_policy]<br />
<br />
void set_tag_propagation_policy(tag_propagation_policy_t p);<br />
<br />
See the gr::block::tag_propagation_policy_t documentation for details<br />
on this enum type.<br />
<br />
When the tag propagation policy is set to TPP_ALL_TO_ALL or TPP_ONE_TO_ONE, the GNU Radio scheduler uses any information available to figure out which output item corresponds to which input item. The block may read them and add new tags, but existing tags are automatically moved downstream in a manner deemed appropriate.<br />
<br />
When a tag is propagated through a block that has a rate change, the<br />
item's offset in the data stream will change. The scheduler uses the<br />
block's gr::block::relative_rate concept to perform the update on the<br />
tag's offset value. The relative rate of a block determines the<br />
relationship between the input rate and output rate. Decimators that<br />
decimate by a factor of D have a relative rate of 1/D.<br />
<br />
Synchronous blocks (gr::sync_block), decimators (gr::sync_decimator),<br />
and interpolators (gr::sync_interpolator) all have pre-defined and<br />
well-understood relative rates. A standard gr::block has a default<br />
relative rate of 1.0, but this must be set if it does not work this<br />
way. Often, we use a gr::block because we have no pre-conceived notion<br />
of the number of input to output items. If it is important to pass<br />
tags through these blocks that respect the change in item value, we<br />
would have to use the TPP_DONT tag propagation policy and handle the<br />
propagation internally.<br />
<br />
In no case is the value of the tag modified when propagating through a<br />
block. This becomes relevant when using [[Tagged Stream Blocks]].<br />
<br />
As an example, consider an interpolating block. See the following flow graph:<br />
<br />
[[File:tags_interp.grc.png|700px|tags_interp.grc.png]]<br />
<br />
[[File:tags_interp.png|500px|tags_interp.png]]<br /><br />
<br />
As you can tell, we produce tags on every 10th sample, and then pass them through a block that repeats every sample 100 times. Tags do not get repeated with the standard tag propagation policies (after all, they're not tag ''manipulation'' policies), so the scheduler takes care that every tag is put on the output item that corresponds to the input item it was on before. Here, the scheduler makes an educated guess and puts the tag on the '''first''' of 100 items.<br />
<br />
Note: We can't use one QT GUI Time Sink for both signals here, because they are running at a different rate. Note the time difference on the x-axis!<br />
<br />
On decimating blocks, the behavior is similar. Consider this very simple flow graph, and the position of the samples:<br />
<br />
[[File:tags_decim.grc.png|400px|tags_decim.grc.png]]<br />
<br />
[[File:tags_decim.png|500px|tags_decim.png]]<br /><br />
<br />
<br />
We can see that no tags are destroyed, and tags are indeed spaced at one-tenth of the original spacing of 100 items. Of course, the actual item that was passed through the block might be destroyed, or modified (think of a decimating FIR filter).<br />
<br />
In fact, this works with any rate-changing block. Note that there are cases where the relation of tag positions of in- and output are ambiguous, the GNU Radio scheduler will then try and get as close as possible.<br />
<br />
Here's another interesting example: Consider this flow graph, which has a delay block, and the position of the tags after it:<br />
<br />
[[File:tags_ramp_delay.grc.png|500px|tags_ramp_delay.grc.png]]<br />
<br />
[[File:tags_ramp_delay.png|500px|tags_ramp_delay.png]]<br /><br />
<br />
<br />
Before the delay block, tags were positioned at the beginning of the ramp. After the delay, they're still in the same position! Would we inspect the source code of the delay block, we'd find that there is absolutely no tag handling code. Instead, the block [http://gnuradio.org/doc/doxygen/classgr_1_1block.html#acad5d6e62ea885cb77d19f72451581c2 declares a delay] to the scheduler, which then propagates tags with this delay.<br />
<br />
Using these mechanisms, we can let GNU Radio handle tag propagation for a large set of cases. For specialized or corner cases, there is no option than to set the tag propagation policy to <code>TPP_DONT</code> and manually propagate tags (actually, there's another way: Say we want most tags to propagate normally, but a select few should be treated differently. We can use [http://gnuradio.org/doc/doxygen/classgr_1_1block.html#a461f6cc92174e83b10c3ec5336036768 <code>remove_item_tag()</code>] (DEPRECATED. Will be removed in 3.8.) to remove these tags from the input; they will then not be propagated any more even if the tag propagation policy is set to something other than <code>TPP_DONT</code>. But that's more advanced use and will not be elaborated on here).<br />
<br />
== Notes on How to Use Tags ==<br />
<br />
Tags can be very useful to an application, and their use is<br />
spreading. USRP sources generate tag information on the time, sample<br />
rate, and frequency of the board if anything changes. We have a meta<br />
data file source/sink (see [[Metadata_Information]]) that use tags to store<br />
information about the data stream. But there are things to think about<br />
when using tags in a block.<br />
<br />
First, when tags are not being used, there is almost no effect on the<br />
scheduler. However, when we use tags, we add overhead by getting and<br />
extracting tags from a data stream. We also use overhead in<br />
propagating the tags. For each tag, each block must copy a vector of<br />
tags from the output port(s) of one block to the input port(s) of the<br />
next block(s). These copy operations can add up.<br />
<br />
The key is to minimize the use of tags. Use them when and only when<br />
necessary and try to provide some control over how tags are generated<br />
to control their frequency. A good example is the USRP source, which<br />
generates a time tag. If it generated a tag with every sample, we<br />
would have thousands of tags per second, which would add a significant<br />
amount of overhead. This is because if we started at time t0 at<br />
sample rate sr, then after N samples, we know that<br />
we are now at time t0 + N/sr. So continuously producing new<br />
tags adds no information.<br />
<br />
The main issue we need to deal with in the above situation is when<br />
there is a discontinuity in the packets received from the USRP. Since<br />
we have no way of knowing in the flowgraph how many samples were<br />
potentially lost, we have lost track of the timing information. The<br />
USRP driver recognizes when packets have been dropped and uses this to<br />
queue another tag, which allows us to resync. Likewise, any point the<br />
sample rate or frequency changes, a new tag is issued.<br />
<br />
== Example Flowgraph ==<br />
<br />
Let's have a look at a simple example:<br />
<br />
[[File:tut5_tagstest_fg.png|600px|tut5_tagstest_fg.png]]<br />
<br />
In this flow graph, we have two sources: A sinusoid and a tag strobe. A tag strobe is a block that will output a constant tag, in this case, on every 1000th item (the actual value of the items is always zero). Those sources get added up. The signal after the adder is identical to the sine wave we produced, because we are always adding zeros. However, the tags stay attached to the same position as they were coming from the tag strobe! This means every 1000th sample of the sinusoid now has a tag. The QT scope can display tags, and even trigger on them.<br />
<br />
[[File:tut5_tagstest_scope.png|500px|tut5_tagstest_scope.png]]<br /><br />
<br />
We now have a mechanism to randomly attach any metadata to specific items. There are several blocks that use tags. One of them is the UHD Sink block, the driver used for transmitting with USRP devices. It will react to tags with certain keys, one of them being <code>tx_freq</code>, which can be used to set the transmit frequency of a USRP while streaming.<br />
<br />
=== Adding tags to the QPSK demodulator ===<br />
<br />
Going back to our QPSK demodulation example, we might want to add a feature to tell downstream blocks that the demodulation is not going well. Remember the output of our block is always hard-decision, and we have to output something. So we could use tags to notify that the input is not well formed, and that the output is not reliable.<br />
<br />
As a failure criterion, we discuss the case where the input amplitude is too small, say smaller than 0.01. When the amplitude drops below this value, we output one tag. Another tag is only sent when the amplitude has recovered, and falls back below the threshold. We extend our work function like this:<br />
<br />
<pre>if (std::abs(in[i]) &lt; 0.01 and not d_low_ampl_state) {<br />
add_item_tag(0, // Port number<br />
nitems_written(0) + i, // Offset<br />
pmt::mp(&quot;amplitude_warning&quot;), // Key<br />
pmt::from_double(std::abs(in[i])) // Value<br />
);<br />
d_low_ampl_state = true;<br />
}<br />
else if (std::abs(in[i]) &gt;= 0.01 and d_low_ampl_state) {<br />
add_item_tag(0, // Port number<br />
nitems_written(0) + i, // Offset<br />
pmt::mp(&quot;amplitude_recovered&quot;), // Key<br />
pmt::PMT_T // Value<br />
);<br />
d_low_ampl_state = false; // Reset state<br />
}</pre><br />
In Python, the code would look like this (assuming we have a member of our block class called <code>d_low_ampl_state</code>):<br />
<br />
<pre># The vector 'in' is called 'in0' here because 'in' is a Python keyword<br />
if abs(in0[i]) &lt; 0.01 and not d_low_ampl_state:<br />
self.add_item_tag(0, # Port number<br />
self.nitems_written(0) + i, # Offset<br />
pmt.intern(&quot;amplitude_warning&quot;), # Key<br />
pmt.from_double(numpy.double(abs(in0[i]))) # Value<br />
# Note: We need to explicitly create a 'double' here,<br />
# because in0[i] is an explicit 32-bit float here<br />
)<br />
self.d_low_ampl_state = True<br />
elif abs(in0[i]) &gt;= 0.01 and d_low_ampl_state:<br />
self.add_item_tag(0, # Port number<br />
self.nitems_written(0) + i, # Offset<br />
pmt.intern(&quot;amplitude_recovered&quot;), # Key<br />
pmt.PMT_T # Value<br />
)<br />
self.d_low_ampl_state = False; // Reset state</pre><br />
We can also create a tag data type [http://gnuradio.org/doc/doxygen/structgr_1_1tag__t.html tag_t] and directly pass this along:<br />
<br />
<pre>if (std::abs(in[i]) &lt; 0.01 and not d_low_ampl_state) {<br />
tag_t tag;<br />
tag.offset = nitems_written(0) + i;<br />
tag.key = pmt::mp(&quot;amplitude_warning&quot;);<br />
tag.value = pmt::from_double(std::abs(in[i]));<br />
add_item_tag(0, tag);<br />
d_low_ampl_state = true;<br />
}</pre><br />
Here's a flow graph that uses the tagged version of the demodulator. We input 20 valid QPSK symbols, then 10 zeros. Since the output of this block is always either 0, 1, 2 or 3, we normally have no way to see if the input was not clearly one of these values.<br />
<br />
[[File:Demod_with_tags.grc.png|750px|Demod_with_tags.grc.png]]<br />
<br />
Here's the output. You can see we have tags on those values which were not from a valid QPSK symbol, but from something unreliable.<br />
<br />
[[File:Demod_with_tags.png|500px|Demod_with_tags.png]]<br /><br />
<br />
== Use case: FIR filters ==<br />
<br />
Assume we have a block that is actually an FIR filter. We want to let GNU Radio handle the tag propagation. How do we configure the block?<br />
<br />
Now, an FIR filter has one input and one output. So, it doesn't matter if we set the propagation policy to <code>TPP_ALL_TO_ALL</code> or <code>TPP_ONE_TO_ONE</code>, and we can leave it as the default. The items going in and those coming out are different, so how do we match input to output? Since we want to preserve the timing of the tag position, we need to use the filter's '''group delay''' as a delay for tags (which for this symmetric FIR filter is <code>(N-1)/2</code>, where <code>N</code> is the number of filter taps). Finally, we might be interpolating, decimating or both (say, for sample rate changes) and we need to tell the scheduler about this as well.</div>Solomonbstonerhttps://wiki.gnuradio.org/index.php?title=Stream_Tags&diff=8601Stream Tags2021-05-23T00:16:54Z<p>Solomonbstoner: Make the first stream tag tutorial easier to follow</p>
<hr />
<div>[[Category:Usage Manual]]<br />
== Introduction ==<br />
<br />
GNU Radio was originally a streaming system with no other mechanism to<br />
pass data between blocks. Streams of data are a model that work well<br />
for samples, bits, etc., but can lack for control and meta data.<br />
<br />
Part of this is solved using the existing message passing interface, which<br />
allows blocks to subscribe to messages published by any other block in<br />
the flowgraph (see [[Message Passing]]). The main drawback to the<br />
message passing system is that it works asynchronously, meaning that<br />
there is no guarantee when a message may arrive relative to the data<br />
stream.<br />
<br />
Stream tags are an isosynchronous data stream that runs parallel to<br />
the main data stream. A stream ''tag'' is generated by a block's work<br />
function and from there on flows downstream alongside a particular sample,<br />
until it reaches a sink or is forced to stop propagating by another<br />
block.<br />
<br />
Stream tags are defined for a specific item in the data stream and are<br />
formed as a key:value pair. The ''key'' identifies what the ''value'' represents<br />
while the value holds the data that the tag contains. Both ''key'' and<br />
''value'' are [[Polymorphic Types (PMTs)]] where the ''key'' is a PMT symbol while<br />
the ''value'' is any type of PMT and can therefore handle any data we wish<br />
to pass. An additional part of the tag is the ''srcid'', which is a PMT<br />
symbol and is used to identify the block that created the tag (which<br />
is usually the block's alias).<br />
<br />
== API Extensions to the gr::block ==<br />
<br />
To enable the stream tags, we have extended the API of gr::block to<br />
understand ''absolute'' item numbers. In the data stream model, each<br />
block's work function is given a buffer in the data stream that is<br />
referenced from 0 to N-1. This is a ''relative'' offset into the data<br />
stream. The absolute reference starts from the beginning of the<br />
flowgraph and continues to count up with every item. Each input stream<br />
is associated with a concept of the 'number of items read' and each<br />
output stream has a 'number of items written'. These are retrieved during <br />
runtime using the two API calls:<br />
<br />
unsigned long int nitems_read(unsigned int which_input);<br />
unsigned long int nitems_written(unsigned int which_output);<br />
<br />
Each tag is associated with some item in this absolute time scale that<br />
is calculated using these functions.<br />
<br />
Like the rest of the data stream, the number of items read/written are<br />
only updated once during the call to work. So in a work function,<br />
nitems_read/written will refer to the state of the data stream at the<br />
start of the work function. We must therefore add to this value the<br />
current relative offset in the data stream. So if we are iterating ''i'' over all output items, we would write the stream tag to output ports<br />
at nitems_written(0)+i for the 0th output port.<br />
<br />
== Stream Tags API ==<br />
<br />
The stream tags API is split into two parts: adding tags to a stream, <br />
and getting tags from a stream. <br />
Note that the functions described below are only meant to be accessed<br />
within a call to general_work/work. While they can be called at other points<br />
in time by a block, the behavior outside of work is undefined without<br />
exact knowledge of the item counts in the buffers.<br />
<br />
=== Adding a Tag to a Stream ===<br />
<br />
We add a tag to a particular output stream of the block using:<br />
<br />
* gr::block::add_item_tag: Adds an item tag to a particular output port using a gr::tag_t data type or by specifying the tag values.<br />
<br />
We can output them to multiple output streams if we want, but to do so<br />
means calling this function once for each port. This function can be <br />
provided with a gr::tag_t data type, or each value of the tag can be<br />
explicitly given.<br />
<br />
Again, a tag is defined as:<br />
<br />
# offset: The offset, in absolute item time, of the tag in the data stream.<br />
# key: the PMT symbol identifying the type of tag.<br />
# value: the PMT holding the data of the tag.<br />
# srcid: (optional) the PMT symbol identifying the block which created the tag.<br />
<br />
We can create a gr::tag_t structure to hold all of the above<br />
information of a tag, which is probably the easiest/best way to do<br />
it. The gr::tag_t struct is defined as having the same members as in<br />
the above list. To add a gr::tag_t tag to a stream, use the function:<br />
<br />
void add_item_tag(unsigned int which_output, const tag_t &tag);<br />
<br />
The secondary API allows us to create a tag by explicitly listing all<br />
of the tag information in the function call:<br />
<br />
void add_item_tag(unsigned int which_output,<br />
uint64_t abs_offset,<br />
const pmt::pmt_t &key,<br />
const pmt::pmt_t &value,<br />
const pmt::pmt_t &srcid=pmt::PMT_F);<br />
<br />
In Python, we can add a tag to a stream using one of the following:<br />
<br />
add_item_tag(which_output, abs_offset, key, value)<br />
add_item_tag(which_output, abs_offset, key, value, srcid)<br />
<br />
Note that the key and value are both PMTs. To create a string type PMT you can use pmt.intern("example_key").<br />
<br />
Consider the following flowgraph as an example. We will have an embedded python block insert stream tags at random intervals, and view these tags on the QT GUI Time sink. <br />
<br />
[[File:Epy_stream_tags_example_flowchart.png|800px]]<br />
<br />
Add the following code into the embedded python block. This code outputs the same signal as in the input, except with tags on randomly selected input samples:<br />
<br />
<syntaxhighlight lang="python"><br />
import numpy as np<br />
from gnuradio import gr<br />
import pmt<br />
<br />
class blk(gr.sync_block):<br />
def __init__(self):<br />
gr.sync_block.__init__(<br />
self,<br />
name='Embedded Python Block',<br />
in_sig=[np.complex64],<br />
out_sig=[np.complex64]<br />
)<br />
<br />
def work(self, input_items, output_items):<br />
for indx, sample in enumerate(input_items[0]):<br />
if np.random.rand() > 0.95: # 5% chance this sample is chosen<br />
key = pmt.intern("example_key")<br />
value = pmt.intern("example_value")<br />
self.add_item_tag(0, self.nitems_written(0) + indx, key, value)<br />
# note: (self.nitems_written(0) + indx) is our current sample, in absolute time<br />
output_items[0][:] = input_items[0] # copy input to output<br />
return len(output_items[0])<br />
</syntaxhighlight><br />
<br />
You may expect an output similar to the screenshot of the time sink below.<br />
<br />
[[File:Epy_stream_tags_example_timesink.png|800px]]<br />
<br />
=== Getting tags from a Stream ===<br />
<br />
To get tags from a particular input stream, we have two<br />
functions we can use:<br />
<br />
# gr::block::get_tags_in_range: Gets all tags from a particular input port between a certain range of items (in absolute item time).<br />
<br />
# gr::block::get_tags_in_window: Gets all tags from a particular input port between a certain range of items (in relative item time within the work function).<br />
<br />
The difference between these functions is working in absolute item<br />
time versus relative item time. Both of these pass back vectors of<br />
gr::tag_t, and they both allow<br />
specifying a particular key (as a PMT symbol) to filter against <br />
(or the fifth argument can be left out to search for all keys). <br />
Filtering for a certain key reduces the effort inside the work function<br />
for getting the right tag's data.<br />
<br />
For example, this call just returns any tags between the given range of items:<br />
<br />
void get_tags_in_range(std::vector<tag_t> &v,<br />
unsigned int which_input,<br />
uint64_t abs_start,<br />
uint64_t abs_end);<br />
<br />
Adding a fifth argument to this function allows us to filter on the<br />
key ''key''.<br />
<br />
void get_tags_in_range(std::vector<tag_t> &v,<br />
unsigned int which_input,<br />
uint64_t abs_start,<br />
uint64_t abs_end,<br />
const pmt::pmt_t &key);<br />
<br />
In Python, the main difference from the C++ function is that instead of having the first<br />
argument be a vector where the tags are stored, the Python version<br />
just returns a list of tags. We would use it like this:<br />
<br />
<syntaxhighlight lang="python"><br />
def work(self, input_items, output_items):<br />
....<br />
tags = self.get_tags_in_window(which_input, rel_start, rel_end)<br />
....<br />
</syntaxhighlight><br />
<br />
If you want to grab all tags on the samples currently being processed by work(), on input port 0, here's a minimal example of doing that:<br />
<br />
<syntaxhighlight lang="python"><br />
import numpy as np<br />
from gnuradio import gr<br />
import pmt<br />
<br />
class blk(gr.sync_block):<br />
def __init__(self):<br />
gr.sync_block.__init__(self,name='Read Tags', in_sig=[np.float32], out_sig=None)<br />
<br />
def work(self, input_items, output_items):<br />
tags = self.get_tags_in_window(0, 0, len(input_items[0]))<br />
for tag in tags:<br />
key = pmt.to_python(tag.key) # convert from PMT to python string<br />
value = pmt.to_python(tag.value) # Note that the type(value) can be several things, it depends what PMT type it was<br />
print 'key:', key<br />
print 'value:', value, type(value)<br />
print ''<br />
return len(input_items[0])<br />
</syntaxhighlight><br />
<br />
== Tag Propagation ==<br />
<br />
We now know how to add tags to streams, and how to read them. But what happens to tags after they were read? And what happens to tags that aren't used? After all, there are many blocks that don't care about tags at all.<br />
<br />
The answer is: It depends on the ''tag propagation policy'' of a block what happens to tags that enter it.<br /><br />
There are [http://gnuradio.org/doc/doxygen/classgr_1_1block.html#abc40fd4d514724a5446a2b34b2352b4e three policies] to choose from:<br />
<br />
# All-to-All: all tags from any input port are replicated to all output ports<br />
# One-to-One: tags from input port ''i'' are only copied to output port ''i'' (depends on num inputs = num outputs).<br />
# Dont: Does not propagate tags. Tags are either stopped here or the work function recreates them in some manner.<br />
<br />
The default behavior of a block is the 'All-to-All' method of<br />
propagation.<br />
<br />
We generally set the tag propagation policy in the block's constructor using [http://gnuradio.org/doc/doxygen/classgr_1_1block.html#a476e218927e426ac88c26431cbf086cd @set_tag_propagation_policy]<br />
<br />
void set_tag_propagation_policy(tag_propagation_policy_t p);<br />
<br />
See the gr::block::tag_propagation_policy_t documentation for details<br />
on this enum type.<br />
<br />
When the tag propagation policy is set to TPP_ALL_TO_ALL or TPP_ONE_TO_ONE, the GNU Radio scheduler uses any information available to figure out which output item corresponds to which input item. The block may read them and add new tags, but existing tags are automatically moved downstream in a manner deemed appropriate.<br />
<br />
When a tag is propagated through a block that has a rate change, the<br />
item's offset in the data stream will change. The scheduler uses the<br />
block's gr::block::relative_rate concept to perform the update on the<br />
tag's offset value. The relative rate of a block determines the<br />
relationship between the input rate and output rate. Decimators that<br />
decimate by a factor of D have a relative rate of 1/D.<br />
<br />
Synchronous blocks (gr::sync_block), decimators (gr::sync_decimator),<br />
and interpolators (gr::sync_interpolator) all have pre-defined and<br />
well-understood relative rates. A standard gr::block has a default<br />
relative rate of 1.0, but this must be set if it does not work this<br />
way. Often, we use a gr::block because we have no pre-conceived notion<br />
of the number of input to output items. If it is important to pass<br />
tags through these blocks that respect the change in item value, we<br />
would have to use the TPP_DONT tag propagation policy and handle the<br />
propagation internally.<br />
<br />
In no case is the value of the tag modified when propagating through a<br />
block. This becomes relevant when using [[Tagged Stream Blocks]].<br />
<br />
As an example, consider an interpolating block. See the following flow graph:<br />
<br />
[[File:tags_interp.grc.png|700px|tags_interp.grc.png]]<br />
<br />
[[File:tags_interp.png|500px|tags_interp.png]]<br /><br />
<br />
As you can tell, we produce tags on every 10th sample, and then pass them through a block that repeats every sample 100 times. Tags do not get repeated with the standard tag propagation policies (after all, they're not tag ''manipulation'' policies), so the scheduler takes care that every tag is put on the output item that corresponds to the input item it was on before. Here, the scheduler makes an educated guess and puts the tag on the '''first''' of 100 items.<br />
<br />
Note: We can't use one QT GUI Time Sink for both signals here, because they are running at a different rate. Note the time difference on the x-axis!<br />
<br />
On decimating blocks, the behavior is similar. Consider this very simple flow graph, and the position of the samples:<br />
<br />
[[File:tags_decim.grc.png|400px|tags_decim.grc.png]]<br />
<br />
[[File:tags_decim.png|500px|tags_decim.png]]<br /><br />
<br />
<br />
We can see that no tags are destroyed, and tags are indeed spaced at one-tenth of the original spacing of 100 items. Of course, the actual item that was passed through the block might be destroyed, or modified (think of a decimating FIR filter).<br />
<br />
In fact, this works with any rate-changing block. Note that there are cases where the relation of tag positions of in- and output are ambiguous, the GNU Radio scheduler will then try and get as close as possible.<br />
<br />
Here's another interesting example: Consider this flow graph, which has a delay block, and the position of the tags after it:<br />
<br />
[[File:tags_ramp_delay.grc.png|500px|tags_ramp_delay.grc.png]]<br />
<br />
[[File:tags_ramp_delay.png|500px|tags_ramp_delay.png]]<br /><br />
<br />
<br />
Before the delay block, tags were positioned at the beginning of the ramp. After the delay, they're still in the same position! Would we inspect the source code of the delay block, we'd find that there is absolutely no tag handling code. Instead, the block [http://gnuradio.org/doc/doxygen/classgr_1_1block.html#acad5d6e62ea885cb77d19f72451581c2 declares a delay] to the scheduler, which then propagates tags with this delay.<br />
<br />
Using these mechanisms, we can let GNU Radio handle tag propagation for a large set of cases. For specialized or corner cases, there is no option than to set the tag propagation policy to <code>TPP_DONT</code> and manually propagate tags (actually, there's another way: Say we want most tags to propagate normally, but a select few should be treated differently. We can use [http://gnuradio.org/doc/doxygen/classgr_1_1block.html#a461f6cc92174e83b10c3ec5336036768 <code>remove_item_tag()</code>] (DEPRECATED. Will be removed in 3.8.) to remove these tags from the input; they will then not be propagated any more even if the tag propagation policy is set to something other than <code>TPP_DONT</code>. But that's more advanced use and will not be elaborated on here).<br />
<br />
== Notes on How to Use Tags ==<br />
<br />
Tags can be very useful to an application, and their use is<br />
spreading. USRP sources generate tag information on the time, sample<br />
rate, and frequency of the board if anything changes. We have a meta<br />
data file source/sink (see [[Metadata_Information]]) that use tags to store<br />
information about the data stream. But there are things to think about<br />
when using tags in a block.<br />
<br />
First, when tags are not being used, there is almost no effect on the<br />
scheduler. However, when we use tags, we add overhead by getting and<br />
extracting tags from a data stream. We also use overhead in<br />
propagating the tags. For each tag, each block must copy a vector of<br />
tags from the output port(s) of one block to the input port(s) of the<br />
next block(s). These copy operations can add up.<br />
<br />
The key is to minimize the use of tags. Use them when and only when<br />
necessary and try to provide some control over how tags are generated<br />
to control their frequency. A good example is the USRP source, which<br />
generates a time tag. If it generated a tag with every sample, we<br />
would have thousands of tags per second, which would add a significant<br />
amount of overhead. This is because if we started at time t0 at<br />
sample rate sr, then after N samples, we know that<br />
we are now at time t0 + N/sr. So continuously producing new<br />
tags adds no information.<br />
<br />
The main issue we need to deal with in the above situation is when<br />
there is a discontinuity in the packets received from the USRP. Since<br />
we have no way of knowing in the flowgraph how many samples were<br />
potentially lost, we have lost track of the timing information. The<br />
USRP driver recognizes when packets have been dropped and uses this to<br />
queue another tag, which allows us to resync. Likewise, any point the<br />
sample rate or frequency changes, a new tag is issued.<br />
<br />
== Example Flowgraph ==<br />
<br />
Let's have a look at a simple example:<br />
<br />
[[File:tut5_tagstest_fg.png|600px|tut5_tagstest_fg.png]]<br />
<br />
In this flow graph, we have two sources: A sinusoid and a tag strobe. A tag strobe is a block that will output a constant tag, in this case, on every 1000th item (the actual value of the items is always zero). Those sources get added up. The signal after the adder is identical to the sine wave we produced, because we are always adding zeros. However, the tags stay attached to the same position as they were coming from the tag strobe! This means every 1000th sample of the sinusoid now has a tag. The QT scope can display tags, and even trigger on them.<br />
<br />
[[File:tut5_tagstest_scope.png|500px|tut5_tagstest_scope.png]]<br /><br />
<br />
We now have a mechanism to randomly attach any metadata to specific items. There are several blocks that use tags. One of them is the UHD Sink block, the driver used for transmitting with USRP devices. It will react to tags with certain keys, one of them being <code>tx_freq</code>, which can be used to set the transmit frequency of a USRP while streaming.<br />
<br />
=== Adding tags to the QPSK demodulator ===<br />
<br />
Going back to our QPSK demodulation example, we might want to add a feature to tell downstream blocks that the demodulation is not going well. Remember the output of our block is always hard-decision, and we have to output something. So we could use tags to notify that the input is not well formed, and that the output is not reliable.<br />
<br />
As a failure criterion, we discuss the case where the input amplitude is too small, say smaller than 0.01. When the amplitude drops below this value, we output one tag. Another tag is only sent when the amplitude has recovered, and falls back below the threshold. We extend our work function like this:<br />
<br />
<pre>if (std::abs(in[i]) &lt; 0.01 and not d_low_ampl_state) {<br />
add_item_tag(0, // Port number<br />
nitems_written(0) + i, // Offset<br />
pmt::mp(&quot;amplitude_warning&quot;), // Key<br />
pmt::from_double(std::abs(in[i])) // Value<br />
);<br />
d_low_ampl_state = true;<br />
}<br />
else if (std::abs(in[i]) &gt;= 0.01 and d_low_ampl_state) {<br />
add_item_tag(0, // Port number<br />
nitems_written(0) + i, // Offset<br />
pmt::mp(&quot;amplitude_recovered&quot;), // Key<br />
pmt::PMT_T // Value<br />
);<br />
d_low_ampl_state = false; // Reset state<br />
}</pre><br />
In Python, the code would look like this (assuming we have a member of our block class called <code>d_low_ampl_state</code>):<br />
<br />
<pre># The vector 'in' is called 'in0' here because 'in' is a Python keyword<br />
if abs(in0[i]) &lt; 0.01 and not d_low_ampl_state:<br />
self.add_item_tag(0, # Port number<br />
self.nitems_written(0) + i, # Offset<br />
pmt.intern(&quot;amplitude_warning&quot;), # Key<br />
pmt.from_double(numpy.double(abs(in0[i]))) # Value<br />
# Note: We need to explicitly create a 'double' here,<br />
# because in0[i] is an explicit 32-bit float here<br />
)<br />
self.d_low_ampl_state = True<br />
elif abs(in0[i]) &gt;= 0.01 and d_low_ampl_state:<br />
self.add_item_tag(0, # Port number<br />
self.nitems_written(0) + i, # Offset<br />
pmt.intern(&quot;amplitude_recovered&quot;), # Key<br />
pmt.PMT_T # Value<br />
)<br />
self.d_low_ampl_state = False; // Reset state</pre><br />
We can also create a tag data type [http://gnuradio.org/doc/doxygen/structgr_1_1tag__t.html tag_t] and directly pass this along:<br />
<br />
<pre>if (std::abs(in[i]) &lt; 0.01 and not d_low_ampl_state) {<br />
tag_t tag;<br />
tag.offset = nitems_written(0) + i;<br />
tag.key = pmt::mp(&quot;amplitude_warning&quot;);<br />
tag.value = pmt::from_double(std::abs(in[i]));<br />
add_item_tag(0, tag);<br />
d_low_ampl_state = true;<br />
}</pre><br />
Here's a flow graph that uses the tagged version of the demodulator. We input 20 valid QPSK symbols, then 10 zeros. Since the output of this block is always either 0, 1, 2 or 3, we normally have no way to see if the input was not clearly one of these values.<br />
<br />
[[File:Demod_with_tags.grc.png|750px|Demod_with_tags.grc.png]]<br />
<br />
Here's the output. You can see we have tags on those values which were not from a valid QPSK symbol, but from something unreliable.<br />
<br />
[[File:Demod_with_tags.png|500px|Demod_with_tags.png]]<br /><br />
<br />
== Use case: FIR filters ==<br />
<br />
Assume we have a block that is actually an FIR filter. We want to let GNU Radio handle the tag propagation. How do we configure the block?<br />
<br />
Now, an FIR filter has one input and one output. So, it doesn't matter if we set the propagation policy to <code>TPP_ALL_TO_ALL</code> or <code>TPP_ONE_TO_ONE</code>, and we can leave it as the default. The items going in and those coming out are different, so how do we match input to output? Since we want to preserve the timing of the tag position, we need to use the filter's '''group delay''' as a delay for tags (which for this symmetric FIR filter is <code>(N-1)/2</code>, where <code>N</code> is the number of filter taps). Finally, we might be interpolating, decimating or both (say, for sample rate changes) and we need to tell the scheduler about this as well.</div>Solomonbstonerhttps://wiki.gnuradio.org/index.php?title=File:Epy_stream_tags_example_timesink.png&diff=8600File:Epy stream tags example timesink.png2021-05-23T00:08:33Z<p>Solomonbstoner: This flowchart gives the reader a simple example of what to expect from the stream tags epy example so he can follow the stream tags tutorial</p>
<hr />
<div>This flowchart gives the reader a simple example of what to expect from the stream tags epy example so he can follow the stream tags tutorial</div>Solomonbstonerhttps://wiki.gnuradio.org/index.php?title=File:Epy_stream_tags_example_flowchart.png&diff=8599File:Epy stream tags example flowchart.png2021-05-23T00:04:23Z<p>Solomonbstoner: This flowchart gives the reader a simple example of how to connect an epy so he can follow the stream tags tutorial</p>
<hr />
<div>This flowchart gives the reader a simple example of how to connect an epy so he can follow the stream tags tutorial</div>Solomonbstonerhttps://wiki.gnuradio.org/index.php?title=Deinterleave&diff=8584Deinterleave2021-05-18T15:30:49Z<p>Solomonbstoner: Update Deinterleave parameters</p>
<hr />
<div>[[Category:Block Docs]]<br />
Deinterleave an input block of samples into N outputs.<br />
<br />
This block deinterleaves blocks of samples. For each output connection, the input stream will be deinterleaved successively to the output connections. <br />
By default, the block deinterleaves a single input to each output.<br />
<br />
blocksize = 1<br />
connections = 2<br />
input = [a, b, c, d, e, f, g, h]<br />
output[0] = [a, c, e, g]<br />
output[1] = [b, d, f, h]<br />
<br />
blocksize = 2<br />
connections = 2<br />
input = [a, b, c, d, e, f, g, h]<br />
output[0] = [a, b, e, f]<br />
output[1] = [c, d, g, h]<br />
<br />
See also [[Interleave]].<br />
<br />
== Parameters ==<br />
<br />
; Num Streams : Total number of output ports<br />
<br />
; Block size: Number of items to output before switching to the next output<br />
<br />
; Vector length : Number of samples in a vector item<br />
<br />
== Example Flowgraph ==<br />
<br />
Insert description of flowgraph here, then show a screenshot of the flowgraph and the output if there is an interesting GUI. Currently we have no standard method of uploading the actual flowgraph to the wiki or git repo, unfortunately. The plan is to have an example flowgraph showing how the block might be used, for every block, and the flowgraphs will live in the git repo.<br />
<br />
== Source Files ==<br />
<br />
; C++ files<br />
: [https://github.com/gnuradio/gnuradio/blob/master/gr-blocks/lib/deinterleave_impl.cc]<br />
<br />
; Header files<br />
: [https://github.com/gnuradio/gnuradio/blob/master/gr-blocks/lib/deinterleave_impl.h]<br />
<br />
; Public header files<br />
: [https://github.com/gnuradio/gnuradio/blob/master/gr-blocks/include/gnuradio/blocks/deinterleave.h]<br />
<br />
; Block definition<br />
: [https://github.com/gnuradio/gnuradio/blob/master/gr-blocks/grc/blocks_deinterleave.block.yml]</div>Solomonbstonerhttps://wiki.gnuradio.org/index.php?title=Interleave&diff=8578Interleave2021-05-18T15:20:18Z<p>Solomonbstoner: Add Vector Length and Num Streams parameters</p>
<hr />
<div>[[Category:Block Docs]]<br />
Interleave N inputs into a single output<br />
<br />
This block interleaves blocks of samples. For each input connection, the samples are interleaved successively to the output connection. <br />
<br />
== Parameters ==<br />
<br />
; Num Streams : Total number of input ports<br />
<br />
; Block size : Number of items from one input to output before switching to the next input<br />
<br />
; Vector length : Number of samples in a vector item<br />
<br />
== Example Flowgraph ==<br />
<br />
Insert description of flowgraph here, then show a screenshot of the flowgraph and the output if there is an interesting GUI. Currently we have no standard method of uploading the actual flowgraph to the wiki or git repo, unfortunately. The plan is to have an example flowgraph showing how the block might be used, for every block, and the flowgraphs will live in the git repo.<br />
<br />
== Source Files ==<br />
<br />
; C++ files<br />
: [https://github.com/gnuradio/gnuradio/blob/master/gr-blocks/lib/interleave_impl.cc]<br />
<br />
; Header files<br />
: [https://github.com/gnuradio/gnuradio/blob/master/gr-blocks/lib/interleave_impl.h]<br />
<br />
; Public header files<br />
: [https://github.com/gnuradio/gnuradio/blob/master/gr-blocks/include/gnuradio/blocks/interleave.h]<br />
<br />
; Block definition<br />
: [https://github.com/gnuradio/gnuradio/blob/master/gr-blocks/grc/blocks_interleave.block.yml]</div>Solomonbstonerhttps://wiki.gnuradio.org/index.php?title=ModuleNotFoundError&diff=8571ModuleNotFoundError2021-05-17T00:33:46Z<p>Solomonbstoner: Edit wrong advice. `source` doesnt work</p>
<hr />
<div>__NOTOC__<br />
When you start gnuradio-companion or execute grcc, if the system isn't configured properly for GRC to find the GNU Radio Python scripts and/or libraries, then you will see an error message similar to this one:<br />
<br><br><br />
[[File:ModNotFound.png|300px]]<br />
<br><br><br />
<br />
If you get this error message and you're running Linux, try the instructions on this page to see if any of them fix the issue. For issues and settings for OSX, see the [https://wiki.gnuradio.org/index.php/MacInstall#Typical_Errors_and_Warnings MacInstall guide].<br />
<br />
== A. Determine the GNU Radio install prefix ==<br />
<br />
If you don't know or remember your installation prefix, perform the following step:<br />
* on a terminal screen, enter <code>gnuradio-config-info --prefix</code><br />
then use that prefix <b>in place of</b> <code>{your-prefix}</code> in the following commands.<br />
<br />
== B. Finding the Python library ==<br />
<br />
Using a terminal, enter the following command, substituting the prefix you found above <b>in place of</b> <code>{your-prefix}</code>:<br />
<pre><br />
find {your-prefix} -name gnuradio | grep "packages"<br />
</pre><br />
Put the appropriate paths it found into the export commands below. Note that the paths are separated by colons <code>: </code><br />
<br />
== C. Setting PYTHONPATH ==<br />
<br />
- For almost all Debian / Ubuntu (and derivative) systems, and most other 32-bit Unix/Linux systems, the paths will look like this:<br />
<pre><br />
export PYTHONPATH={your-prefix}/lib/{Py-version}/dist-packages:{your-prefix}/lib/{Py-version}/site-packages:$PYTHONPATH<br />
</pre><br />
<br />
- For other 64-bit systems, the paths will look like this:<br />
<pre><br />
export PYTHONPATH={your-prefix}/lib64/{Py-version}/site-packages:$PYTHONPATH<br />
</pre><br />
<br />
== D. Setting LD_LIBRARY_PATH ==<br />
<br />
- For almost all Debian / Ubuntu (and derivative) systems, and most other 32-bit Unix/Linux systems, use:<br />
<pre><br />
export LD_LIBRARY_PATH={your-prefix}/lib:$LD_LIBRARY_PATH<br />
</pre><br />
<br />
- For other 64-bit systems, use:<br />
<pre><br />
export LD_LIBRARY_PATH={your-prefix}/lib64:$LD_LIBRARY_PATH<br />
</pre><br />
<br />
== E. Store the commands in a Bash start-up file ==<br />
<br />
Once you have determined the correct two export commands to use, open your text editor and put them in your <code>~/.bash_aliases</code> or <code>~/.bashrc</code> or <code>~/.profile</code> file. Save the file. There are three ways for for these changes to take effect.<br><br />
1. On your terminal enter <code>exit</code>. Then start a new terminal.<br><br />
2. Type <code>source NAME_OF_FILE_EDITED</code> (e.g. <code>source ~/.profile</code> ).<br><br />
3. Reboot your computer.<br><br />
<br />
As an example, your entries might be:<br />
<pre><br />
export PYTHONPATH=/usr/local/lib/python3/dist-packages:/usr/local/lib/python3.6/dist-packages:$PYTHONPATH<br />
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH<br />
</pre><br />
<br />
== F. Starting GNU Radio from GUI ==<br />
<br />
If you get the ModuleNotFoundError error while starting GNU Radio from GUI, but can start GNU Radio using the terminal, it is because the environment variables are set for the terminal, but not for your graphical environment. Try one of the listed solutions below.<br />
<br />
1. Store the export commands stated in Part E in <code>~/.profile</code>. Log out and back in for the changes to take effect.<br><br />
2. Store the export commands stated in Part E in a new shell script (with any name of your choice) in <code>/etc/profile.d/</code>. Restart your computer thereafter.<br></div>Solomonbstonerhttps://wiki.gnuradio.org/index.php?title=ModuleNotFoundError&diff=8564ModuleNotFoundError2021-05-16T12:08:03Z<p>Solomonbstoner: Add instructions for when gnuradio can be started from the terminal but not from the gui</p>
<hr />
<div>__NOTOC__<br />
When you start gnuradio-companion or execute grcc, if the system isn't configured properly for GRC to find the GNU Radio Python scripts and/or libraries, then you will see an error message similar to this one:<br />
<br><br><br />
[[File:ModNotFound.png|300px]]<br />
<br><br><br />
<br />
If you get this error message and you're running Linux, try the instructions on this page to see if any of them fix the issue. For issues and settings for OSX, see the [https://wiki.gnuradio.org/index.php/MacInstall#Typical_Errors_and_Warnings MacInstall guide].<br />
<br />
== A. Determine the GNU Radio install prefix ==<br />
<br />
If you don't know or remember your installation prefix, perform the following step:<br />
* on a terminal screen, enter <code>gnuradio-config-info --prefix</code><br />
then use that prefix <b>in place of</b> <code>{your-prefix}</code> in the following commands.<br />
<br />
== B. Finding the Python library ==<br />
<br />
Using a terminal, enter the following command, substituting the prefix you found above <b>in place of</b> <code>{your-prefix}</code>:<br />
<pre><br />
find {your-prefix} -name gnuradio | grep "packages"<br />
</pre><br />
Put the appropriate paths it found into the export commands below. Note that the paths are separated by colons <code>: </code><br />
<br />
== C. Setting PYTHONPATH ==<br />
<br />
- For almost all Debian / Ubuntu (and derivative) systems, and most other 32-bit Unix/Linux systems, the paths will look like this:<br />
<pre><br />
export PYTHONPATH={your-prefix}/lib/{Py-version}/dist-packages:{your-prefix}/lib/{Py-version}/site-packages:$PYTHONPATH<br />
</pre><br />
<br />
- For other 64-bit systems, the paths will look like this:<br />
<pre><br />
export PYTHONPATH={your-prefix}/lib64/{Py-version}/site-packages:$PYTHONPATH<br />
</pre><br />
<br />
== D. Setting LD_LIBRARY_PATH ==<br />
<br />
- For almost all Debian / Ubuntu (and derivative) systems, and most other 32-bit Unix/Linux systems, use:<br />
<pre><br />
export LD_LIBRARY_PATH={your-prefix}/lib:$LD_LIBRARY_PATH<br />
</pre><br />
<br />
- For other 64-bit systems, use:<br />
<pre><br />
export LD_LIBRARY_PATH={your-prefix}/lib64:$LD_LIBRARY_PATH<br />
</pre><br />
<br />
== E. Store the commands in a Bash start-up file ==<br />
<br />
Once you have determined the correct two export commands to use, open your text editor and put them in your <code>~/.bash_aliases</code> or <code>~/.bashrc</code> or <code>~/.profile</code> file. Save the file. There are three ways for for these changes to take effect.<br><br />
1. On your terminal enter <code>exit</code>. Then start a new terminal.<br><br />
2. Type <code>source NAME_OF_FILE_EDITED</code> (e.g. <code>source ~/.profile</code> ).<br><br />
3. Reboot your computer.<br><br />
<br />
As an example, your entries might be:<br />
<pre><br />
export PYTHONPATH=/usr/local/lib/python3/dist-packages:/usr/local/lib/python3.6/dist-packages:$PYTHONPATH<br />
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH<br />
</pre><br />
<br />
== F. Starting GNU Radio from GUI ==<br />
<br />
If you get the ModuleNotFoundError error while starting GNU Radio from GUI, but can start GNU Radio using the terminal, it is because the environment variables are set for the terminal, but not for your graphical environment. Try one of the listed solutions below.<br />
<br />
1. Store the export commands stated in Part E in <code>~/.profile</code>. Then run <code>source ~/.profile</code>.<br><br />
2. Store the export commands stated in Part E in a new shell script in <code>/etc/profile.d/</code>. Restart your computer thereafter.<br></div>Solomonbstonerhttps://wiki.gnuradio.org/index.php?title=Stream_Tags&diff=8536Stream Tags2021-05-09T00:45:28Z<p>Solomonbstoner: Fix broken link that directs to the Metadata page</p>
<hr />
<div>[[Category:Usage Manual]]<br />
== Introduction ==<br />
<br />
GNU Radio was originally a streaming system with no other mechanism to<br />
pass data between blocks. Streams of data are a model that work well<br />
for samples, bits, etc., but can lack for control and meta data.<br />
<br />
Part of this is solved using the existing message passing interface, which<br />
allows blocks to subscribe to messages published by any other block in<br />
the flowgraph (see [[Message Passing]]). The main drawback to the<br />
message passing system is that it works asynchronously, meaning that<br />
there is no guarantee when a message may arrive relative to the data<br />
stream.<br />
<br />
Stream tags are an isosynchronous data stream that runs parallel to<br />
the main data stream. A stream ''tag'' is generated by a block's work<br />
function and from there on flows downstream alongside a particular sample,<br />
until it reaches a sink or is forced to stop propagating by another<br />
block.<br />
<br />
Stream tags are defined for a specific item in the data stream and are<br />
formed as a key:value pair. The ''key'' identifies what the ''value'' represents<br />
while the value holds the data that the tag contains. Both ''key'' and<br />
''value'' are [[Polymorphic Types (PMTs)]] where the ''key'' is a PMT symbol while<br />
the ''value'' is any type of PMT and can therefore handle any data we wish<br />
to pass. An additional part of the tag is the ''srcid'', which is a PMT<br />
symbol and is used to identify the block that created the tag (which<br />
is usually the block's alias).<br />
<br />
== API Extensions to the gr::block ==<br />
<br />
To enable the stream tags, we have extended the API of gr::block to<br />
understand ''absolute'' item numbers. In the data stream model, each<br />
block's work function is given a buffer in the data stream that is<br />
referenced from 0 to N-1. This is a ''relative'' offset into the data<br />
stream. The absolute reference starts from the beginning of the<br />
flowgraph and continues to count up with every item. Each input stream<br />
is associated with a concept of the 'number of items read' and each<br />
output stream has a 'number of items written'. These are retrieved during <br />
runtime using the two API calls:<br />
<br />
unsigned long int nitems_read(unsigned int which_input);<br />
unsigned long int nitems_written(unsigned int which_output);<br />
<br />
Each tag is associated with some item in this absolute time scale that<br />
is calculated using these functions.<br />
<br />
Like the rest of the data stream, the number of items read/written are<br />
only updated once during the call to work. So in a work function,<br />
nitems_read/written will refer to the state of the data stream at the<br />
start of the work function. We must therefore add to this value the<br />
current relative offset in the data stream. So if we are iterating ''i'' over all output items, we would write the stream tag to output ports<br />
at nitems_written(0)+i for the 0th output port.<br />
<br />
== Stream Tags API ==<br />
<br />
The stream tags API is split into two parts: adding tags to a stream, <br />
and getting tags from a stream. <br />
Note that the functions described below are only meant to be accessed<br />
within a call to general_work/work. While they can be called at other points<br />
in time by a block, the behavior outside of work is undefined without<br />
exact knowledge of the item counts in the buffers.<br />
<br />
=== Adding a Tag to a Stream ===<br />
<br />
We add a tag to a particular output stream of the block using:<br />
<br />
* gr::block::add_item_tag: Adds an item tag to a particular output port using a gr::tag_t data type or by specifying the tag values.<br />
<br />
We can output them to multiple output streams if we want, but to do so<br />
means calling this function once for each port. This function can be <br />
provided with a gr::tag_t data type, or each value of the tag can be<br />
explicitly given.<br />
<br />
Again, a tag is defined as:<br />
<br />
# offset: The offset, in absolute item time, of the tag in the data stream.<br />
# key: the PMT symbol identifying the type of tag.<br />
# value: the PMT holding the data of the tag.<br />
# srcid: (optional) the PMT symbol identifying the block which created the tag.<br />
<br />
We can create a gr::tag_t structure to hold all of the above<br />
information of a tag, which is probably the easiest/best way to do<br />
it. The gr::tag_t struct is defined as having the same members as in<br />
the above list. To add a gr::tag_t tag to a stream, use the function:<br />
<br />
void add_item_tag(unsigned int which_output, const tag_t &tag);<br />
<br />
The secondary API allows us to create a tag by explicitly listing all<br />
of the tag information in the function call:<br />
<br />
void add_item_tag(unsigned int which_output,<br />
uint64_t abs_offset,<br />
const pmt::pmt_t &key,<br />
const pmt::pmt_t &value,<br />
const pmt::pmt_t &srcid=pmt::PMT_F);<br />
<br />
In Python, we can add a tag to a stream using one of the following:<br />
<br />
add_item_tag(which_output, abs_offset, key, value)<br />
add_item_tag(which_output, abs_offset, key, value, srcid)<br />
<br />
Note that the key and value are both PMTs. To create a string type PMT you can use pmt.intern("example_key")<br />
<br />
Here is an example of a Python block that loops through samples and puts tags on random ones, and outputs the same signal:<br />
<syntaxhighlight lang="python"><br />
import numpy as np<br />
from gnuradio import gr<br />
import pmt<br />
<br />
class blk(gr.sync_block):<br />
def __init__(self):<br />
gr.sync_block.__init__(<br />
self,<br />
name='Embedded Python Block',<br />
in_sig=[np.complex64],<br />
out_sig=[np.complex64]<br />
)<br />
<br />
def work(self, input_items, output_items):<br />
for indx, sample in enumerate(input_items[0]):<br />
if np.random.rand() > 0.95: # 5% chance this sample is chosen<br />
key = pmt.intern("example_key")<br />
value = pmt.intern("example_value")<br />
self.add_item_tag(0, self.nitems_written(0) + indx, key, value)<br />
# note: (self.nitems_written(0) + indx) is our current sample, in absolute time<br />
output_items[0][:] = input_items[0] # copy input to output<br />
return len(output_items[0])<br />
</syntaxhighlight><br />
<br />
=== Getting tags from a Stream ===<br />
<br />
To get tags from a particular input stream, we have two<br />
functions we can use:<br />
<br />
# gr::block::get_tags_in_range: Gets all tags from a particular input port between a certain range of items (in absolute item time).<br />
<br />
# gr::block::get_tags_in_window: Gets all tags from a particular input port between a certain range of items (in relative item time within the work function).<br />
<br />
The difference between these functions is working in absolute item<br />
time versus relative item time. Both of these pass back vectors of<br />
gr::tag_t, and they both allow<br />
specifying a particular key (as a PMT symbol) to filter against <br />
(or the fifth argument can be left out to search for all keys). <br />
Filtering for a certain key reduces the effort inside the work function<br />
for getting the right tag's data.<br />
<br />
For example, this call just returns any tags between the given range of items:<br />
<br />
void get_tags_in_range(std::vector<tag_t> &v,<br />
unsigned int which_input,<br />
uint64_t abs_start,<br />
uint64_t abs_end);<br />
<br />
Adding a fifth argument to this function allows us to filter on the<br />
key ''key''.<br />
<br />
void get_tags_in_range(std::vector<tag_t> &v,<br />
unsigned int which_input,<br />
uint64_t abs_start,<br />
uint64_t abs_end,<br />
const pmt::pmt_t &key);<br />
<br />
In Python, the main difference from the C++ function is that instead of having the first<br />
argument be a vector where the tags are stored, the Python version<br />
just returns a list of tags. We would use it like this:<br />
<br />
<syntaxhighlight lang="python"><br />
def work(self, input_items, output_items):<br />
....<br />
tags = self.get_tags_in_window(which_input, rel_start, rel_end)<br />
....<br />
</syntaxhighlight><br />
<br />
If you want to grab all tags on the samples currently being processed by work(), on input port 0, here's a minimal example of doing that:<br />
<br />
<syntaxhighlight lang="python"><br />
import numpy as np<br />
from gnuradio import gr<br />
import pmt<br />
<br />
class blk(gr.sync_block):<br />
def __init__(self):<br />
gr.sync_block.__init__(self,name='Read Tags', in_sig=[np.float32], out_sig=None)<br />
<br />
def work(self, input_items, output_items):<br />
tags = self.get_tags_in_window(0, 0, len(input_items[0]))<br />
for tag in tags:<br />
key = pmt.to_python(tag.key) # convert from PMT to python string<br />
value = pmt.to_python(tag.value) # Note that the type(value) can be several things, it depends what PMT type it was<br />
print 'key:', key<br />
print 'value:', value, type(value)<br />
print ''<br />
return len(input_items[0])<br />
</syntaxhighlight><br />
<br />
== Tag Propagation ==<br />
<br />
We now know how to add tags to streams, and how to read them. But what happens to tags after they were read? And what happens to tags that aren't used? After all, there are many blocks that don't care about tags at all.<br />
<br />
The answer is: It depends on the ''tag propagation policy'' of a block what happens to tags that enter it.<br /><br />
There are [http://gnuradio.org/doc/doxygen/classgr_1_1block.html#abc40fd4d514724a5446a2b34b2352b4e three policies] to choose from:<br />
<br />
# All-to-All: all tags from any input port are replicated to all output ports<br />
# One-to-One: tags from input port ''i'' are only copied to output port ''i'' (depends on num inputs = num outputs).<br />
# Dont: Does not propagate tags. Tags are either stopped here or the work function recreates them in some manner.<br />
<br />
The default behavior of a block is the 'All-to-All' method of<br />
propagation.<br />
<br />
We generally set the tag propagation policy in the block's constructor using [http://gnuradio.org/doc/doxygen/classgr_1_1block.html#a476e218927e426ac88c26431cbf086cd @set_tag_propagation_policy]<br />
<br />
void set_tag_propagation_policy(tag_propagation_policy_t p);<br />
<br />
See the gr::block::tag_propagation_policy_t documentation for details<br />
on this enum type.<br />
<br />
When the tag propagation policy is set to TPP_ALL_TO_ALL or TPP_ONE_TO_ONE, the GNU Radio scheduler uses any information available to figure out which output item corresponds to which input item. The block may read them and add new tags, but existing tags are automatically moved downstream in a manner deemed appropriate.<br />
<br />
When a tag is propagated through a block that has a rate change, the<br />
item's offset in the data stream will change. The scheduler uses the<br />
block's gr::block::relative_rate concept to perform the update on the<br />
tag's offset value. The relative rate of a block determines the<br />
relationship between the input rate and output rate. Decimators that<br />
decimate by a factor of D have a relative rate of 1/D.<br />
<br />
Synchronous blocks (gr::sync_block), decimators (gr::sync_decimator),<br />
and interpolators (gr::sync_interpolator) all have pre-defined and<br />
well-understood relative rates. A standard gr::block has a default<br />
relative rate of 1.0, but this must be set if it does not work this<br />
way. Often, we use a gr::block because we have no pre-conceived notion<br />
of the number of input to output items. If it is important to pass<br />
tags through these blocks that respect the change in item value, we<br />
would have to use the TPP_DONT tag propagation policy and handle the<br />
propagation internally.<br />
<br />
In no case is the value of the tag modified when propagating through a<br />
block. This becomes relevant when using [[Tagged Stream Blocks]].<br />
<br />
As an example, consider an interpolating block. See the following flow graph:<br />
<br />
[[File:tags_interp.grc.png|700px|tags_interp.grc.png]]<br />
<br />
[[File:tags_interp.png|500px|tags_interp.png]]<br /><br />
<br />
As you can tell, we produce tags on every 10th sample, and then pass them through a block that repeats every sample 100 times. Tags do not get repeated with the standard tag propagation policies (after all, they're not tag ''manipulation'' policies), so the scheduler takes care that every tag is put on the output item that corresponds to the input item it was on before. Here, the scheduler makes an educated guess and puts the tag on the '''first''' of 100 items.<br />
<br />
Note: We can't use one QT GUI Time Sink for both signals here, because they are running at a different rate. Note the time difference on the x-axis!<br />
<br />
On decimating blocks, the behavior is similar. Consider this very simple flow graph, and the position of the samples:<br />
<br />
[[File:tags_decim.grc.png|400px|tags_decim.grc.png]]<br />
<br />
[[File:tags_decim.png|500px|tags_decim.png]]<br /><br />
<br />
<br />
We can see that no tags are destroyed, and tags are indeed spaced at one-tenth of the original spacing of 100 items. Of course, the actual item that was passed through the block might be destroyed, or modified (think of a decimating FIR filter).<br />
<br />
In fact, this works with any rate-changing block. Note that there are cases where the relation of tag positions of in- and output are ambiguous, the GNU Radio scheduler will then try and get as close as possible.<br />
<br />
Here's another interesting example: Consider this flow graph, which has a delay block, and the position of the tags after it:<br />
<br />
[[File:tags_ramp_delay.grc.png|500px|tags_ramp_delay.grc.png]]<br />
<br />
[[File:tags_ramp_delay.png|500px|tags_ramp_delay.png]]<br /><br />
<br />
<br />
Before the delay block, tags were positioned at the beginning of the ramp. After the delay, they're still in the same position! Would we inspect the source code of the delay block, we'd find that there is absolutely no tag handling code. Instead, the block [http://gnuradio.org/doc/doxygen/classgr_1_1block.html#acad5d6e62ea885cb77d19f72451581c2 declares a delay] to the scheduler, which then propagates tags with this delay.<br />
<br />
Using these mechanisms, we can let GNU Radio handle tag propagation for a large set of cases. For specialized or corner cases, there is no option than to set the tag propagation policy to <code>TPP_DONT</code> and manually propagate tags (actually, there's another way: Say we want most tags to propagate normally, but a select few should be treated differently. We can use [http://gnuradio.org/doc/doxygen/classgr_1_1block.html#a461f6cc92174e83b10c3ec5336036768 <code>remove_item_tag()</code>] (DEPRECATED. Will be removed in 3.8.) to remove these tags from the input; they will then not be propagated any more even if the tag propagation policy is set to something other than <code>TPP_DONT</code>. But that's more advanced use and will not be elaborated on here).<br />
<br />
== Notes on How to Use Tags ==<br />
<br />
Tags can be very useful to an application, and their use is<br />
spreading. USRP sources generate tag information on the time, sample<br />
rate, and frequency of the board if anything changes. We have a meta<br />
data file source/sink (see [[Metadata_Information]]) that use tags to store<br />
information about the data stream. But there are things to think about<br />
when using tags in a block.<br />
<br />
First, when tags are not being used, there is almost no effect on the<br />
scheduler. However, when we use tags, we add overhead by getting and<br />
extracting tags from a data stream. We also use overhead in<br />
propagating the tags. For each tag, each block must copy a vector of<br />
tags from the output port(s) of one block to the input port(s) of the<br />
next block(s). These copy operations can add up.<br />
<br />
The key is to minimize the use of tags. Use them when and only when<br />
necessary and try to provide some control over how tags are generated<br />
to control their frequency. A good example is the USRP source, which<br />
generates a time tag. If it generated a tag with every sample, we<br />
would have thousands of tags per second, which would add a significant<br />
amount of overhead. This is because if we started at time t0 at<br />
sample rate sr, then after N samples, we know that<br />
we are now at time t0 + N/sr. So continuously producing new<br />
tags adds no information.<br />
<br />
The main issue we need to deal with in the above situation is when<br />
there is a discontinuity in the packets received from the USRP. Since<br />
we have no way of knowing in the flowgraph how many samples were<br />
potentially lost, we have lost track of the timing information. The<br />
USRP driver recognizes when packets have been dropped and uses this to<br />
queue another tag, which allows us to resync. Likewise, any point the<br />
sample rate or frequency changes, a new tag is issued.<br />
<br />
== Example Flowgraph ==<br />
<br />
Let's have a look at a simple example:<br />
<br />
[[File:tut5_tagstest_fg.png|600px|tut5_tagstest_fg.png]]<br />
<br />
In this flow graph, we have two sources: A sinusoid and a tag strobe. A tag strobe is a block that will output a constant tag, in this case, on every 1000th item (the actual value of the items is always zero). Those sources get added up. The signal after the adder is identical to the sine wave we produced, because we are always adding zeros. However, the tags stay attached to the same position as they were coming from the tag strobe! This means every 1000th sample of the sinusoid now has a tag. The QT scope can display tags, and even trigger on them.<br />
<br />
[[File:tut5_tagstest_scope.png|500px|tut5_tagstest_scope.png]]<br /><br />
<br />
We now have a mechanism to randomly attach any metadata to specific items. There are several blocks that use tags. One of them is the UHD Sink block, the driver used for transmitting with USRP devices. It will react to tags with certain keys, one of them being <code>tx_freq</code>, which can be used to set the transmit frequency of a USRP while streaming.<br />
<br />
=== Adding tags to the QPSK demodulator ===<br />
<br />
Going back to our QPSK demodulation example, we might want to add a feature to tell downstream blocks that the demodulation is not going well. Remember the output of our block is always hard-decision, and we have to output something. So we could use tags to notify that the input is not well formed, and that the output is not reliable.<br />
<br />
As a failure criterion, we discuss the case where the input amplitude is too small, say smaller than 0.01. When the amplitude drops below this value, we output one tag. Another tag is only sent when the amplitude has recovered, and falls back below the threshold. We extend our work function like this:<br />
<br />
<pre>if (std::abs(in[i]) &lt; 0.01 and not d_low_ampl_state) {<br />
add_item_tag(0, // Port number<br />
nitems_written(0) + i, // Offset<br />
pmt::mp(&quot;amplitude_warning&quot;), // Key<br />
pmt::from_double(std::abs(in[i])) // Value<br />
);<br />
d_low_ampl_state = true;<br />
}<br />
else if (std::abs(in[i]) &gt;= 0.01 and d_low_ampl_state) {<br />
add_item_tag(0, // Port number<br />
nitems_written(0) + i, // Offset<br />
pmt::mp(&quot;amplitude_recovered&quot;), // Key<br />
pmt::PMT_T // Value<br />
);<br />
d_low_ampl_state = false; // Reset state<br />
}</pre><br />
In Python, the code would look like this (assuming we have a member of our block class called <code>d_low_ampl_state</code>):<br />
<br />
<pre># The vector 'in' is called 'in0' here because 'in' is a Python keyword<br />
if abs(in0[i]) &lt; 0.01 and not d_low_ampl_state:<br />
self.add_item_tag(0, # Port number<br />
self.nitems_written(0) + i, # Offset<br />
pmt.intern(&quot;amplitude_warning&quot;), # Key<br />
pmt.from_double(numpy.double(abs(in0[i]))) # Value<br />
# Note: We need to explicitly create a 'double' here,<br />
# because in0[i] is an explicit 32-bit float here<br />
)<br />
self.d_low_ampl_state = True<br />
elif abs(in0[i]) &gt;= 0.01 and d_low_ampl_state:<br />
self.add_item_tag(0, # Port number<br />
self.nitems_written(0) + i, # Offset<br />
pmt.intern(&quot;amplitude_recovered&quot;), # Key<br />
pmt.PMT_T # Value<br />
)<br />
self.d_low_ampl_state = False; // Reset state</pre><br />
We can also create a tag data type [http://gnuradio.org/doc/doxygen/structgr_1_1tag__t.html tag_t] and directly pass this along:<br />
<br />
<pre>if (std::abs(in[i]) &lt; 0.01 and not d_low_ampl_state) {<br />
tag_t tag;<br />
tag.offset = nitems_written(0) + i;<br />
tag.key = pmt::mp(&quot;amplitude_warning&quot;);<br />
tag.value = pmt::from_double(std::abs(in[i]));<br />
add_item_tag(0, tag);<br />
d_low_ampl_state = true;<br />
}</pre><br />
Here's a flow graph that uses the tagged version of the demodulator. We input 20 valid QPSK symbols, then 10 zeros. Since the output of this block is always either 0, 1, 2 or 3, we normally have no way to see if the input was not clearly one of these values.<br />
<br />
[[File:Demod_with_tags.grc.png|750px|Demod_with_tags.grc.png]]<br />
<br />
Here's the output. You can see we have tags on those values which were not from a valid QPSK symbol, but from something unreliable.<br />
<br />
[[File:Demod_with_tags.png|500px|Demod_with_tags.png]]<br /><br />
<br />
== Use case: FIR filters ==<br />
<br />
Assume we have a block that is actually an FIR filter. We want to let GNU Radio handle the tag propagation. How do we configure the block?<br />
<br />
Now, an FIR filter has one input and one output. So, it doesn't matter if we set the propagation policy to <code>TPP_ALL_TO_ALL</code> or <code>TPP_ONE_TO_ONE</code>, and we can leave it as the default. The items going in and those coming out are different, so how do we match input to output? Since we want to preserve the timing of the tag position, we need to use the filter's '''group delay''' as a delay for tags (which for this symmetric FIR filter is <code>(N-1)/2</code>, where <code>N</code> is the number of filter taps). Finally, we might be interpolating, decimating or both (say, for sample rate changes) and we need to tell the scheduler about this as well.</div>Solomonbstonerhttps://wiki.gnuradio.org/index.php?title=Polymorphic_Types_(PMTs)&diff=8534Polymorphic Types (PMTs)2021-05-08T15:28:22Z<p>Solomonbstoner: Fixed example code</p>
<hr />
<div>[[Category:Usage Manual]]<br />
== Introduction ==<br />
<br />
Polymorphic Types are used as the carrier of data from one block/thread to another such as stream tags and message passing interfaces. <br />
PMT data types can represent a variety of data ranging from the Booleans to dictionaries. <br />
They are heavily used in the stream tags and message passing interfaces. <br />
In a sense, PMTs are a way to extend C++' strict typing with something more flexible. <br />
The most complete list of PMT function is, of course, the source code, specifically the header file [https://gnuradio.org/doc/doxygen/pmt_8h.html pmt.h]. <br />
This page summarizes the most important features and points of PMTs.<br />
<br />
Let's dive straight into some Python code and see how we can use PMTs:<br />
<br />
<syntaxhighlight lang="python"><br />
>>> import pmt<br />
>>> P = pmt.from_long(23)<br />
>>> type(P)<br />
<class 'pmt.pmt_swig.swig_int_ptr'><br />
>>> print P<br />
23<br />
>>> P2 = pmt.from_complex(1j)<br />
>>> type(P2)<br />
<class 'pmt.pmt_swig.swig_int_ptr'><br />
>>> print P2<br />
0+1i<br />
>>> pmt.is_complex(P2)<br />
True<br />
</syntaxhighlight><br />
First, the <code>pmt</code> module is imported. We assign two values (<code>P</code> and <code>P2</code>) with PMTs using the <code>from_long()</code> and <code>from_complex()</code> calls, respectively. As we can see, they are both of the same type! This means we can pass these variables to C++ through SWIG, and C++ can handle this type accordingly.<br />
<br />
The same code as above in C++ would look like this:<br />
<syntaxhighlight lang="c++"><br />
#include <pmt/pmt.h><br />
// [...]<br />
pmt::pmt_t P = pmt::from_long(23);<br />
std::cout << P << std::endl;<br />
pmt::pmt_t P2 = pmt::from_complex(gr_complex(0, 1)); <br />
// Alternatively: pmt::from_complex(0, 1)<br />
std::cout << P2 << std::endl;<br />
std::cout << pmt::is_complex(P2) << std::endl;<br />
</syntaxhighlight><br />
Two things stand out in both Python and C++: First, we can simply print the contents of a PMT. How is this possible? Well, the PMTs have in-built capability to cast their value to a string (this is not possible with all types, though). Second, PMTs must obviously know their type, so we can query that, e.g. by calling the <code>is_complex()</code> method.<br />
<br />
Note: When running the above as a standalone, the compiler command will look something like <code>g++ pmt_tutorial.cpp -I$(gnuradio-config-info --prefix)/include -lgnuradio-pmt -o pmt_tutorial</code><br />
<br />
When assigning a non-PMT value to a PMT, we can use the <code>from_*</code> methods, and use the <code>to_*</code> methods to convert back:<br />
<syntaxhighlight lang="c++"><br />
pmt::pmt_t P_int = pmt::from_long(42);<br />
int i = pmt::to_long(P_int);<br />
pmt::pmt_t P_double = pmt::from_double(0.2);<br />
double d = pmt::to_double(P_double);<br />
pmt::pmt_t P_double = pmt::mp(0.2);<br />
</syntaxhighlight><br />
<br />
The last row shows the [http://gnuradio.org/doc/doxygen/namespacepmt.html#a90faad6086ac00280e0cfd8bb541bd64 pmt::mp()] shorthand function. It basically saves some typing, as it infers the correct <code>from_</code> function from the given type.<br />
<br />
String types play a bit of a special role in PMTs, as we will see<br />
later, and have their own converter:<br />
<br />
<syntaxhighlight lang="c++"><br />
pmt::pmt_t P_str = pmt::string_to_symbol("spam");<br />
pmt::pmt_t P_str2 = pmt::intern("spam");<br />
std::string str = pmt::symbol_to_string(P_str);<br />
</syntaxhighlight><br />
The pmt::intern is another way of saying pmt::string_to_symbol.<br />
<br />
See the [https://www.gnuradio.org/doc/doxygen/namespacepmt.html PMT docs] and the header file [http://gnuradio.org/doc/doxygen/pmt_8h.html pmt.h] for a full list of conversion functions.<br />
<br />
In Python, we can make use of the dynamic typing, and there's actually a<br />
helper function to do these conversions (C++ also has a helper<br />
function for converting to PMTs called pmt::mp(), but it's less<br />
powerful, and not quite as useful, because types are always strictly<br />
known in C++):<br />
<br />
<syntaxhighlight lang="python"><br />
P_int = pmt.to_pmt(42)<br />
i = pmt.to_python(P_int)<br />
P_double = pmt.to_pmt(0.2)<br />
d = pmt.to_double(P_double)<br />
</syntaxhighlight><br />
<br />
On a side note, there are three useful PMT constants, which can be<br />
used in both Python and C++ domains. In C++, these can be used as<br />
such:<br />
<br />
<syntaxhighlight lang="c++"><br />
pmt::pmt_t P_true = pmt::PMT_T;<br />
pmt::pmt_t P_false = pmt::PMT_F;<br />
pmt::pmt_t P_nil = pmt::PMT_NIL;<br />
</syntaxhighlight><br />
<br />
In Python:<br />
<br />
<syntaxhighlight lang="python"><br />
P_true = pmt.PMT_T<br />
P_false = pmt.PMT_F<br />
P_nil = pmt.PMT_NIL<br />
</syntaxhighlight><br />
<br />
<code>pmt.PMT_T</code> and <code>pmt.PMT_F</code> are boolean PMT types representing True and False, respectively. The PMT_NIL is like a NULL or None and can be used for default arguments or return values, often indicating an error has occurred.<br />
<br />
To be able to go back to C++ data types, we need to be able to find<br />
out the type from a PMT. The family of <code>is_*</code> methods helps us do that:<br />
<br />
<syntaxhighlight lang="c++"><br />
double d;<br />
if (pmt::is_integer(P)) {<br />
d = (double) pmt::to_long(P);<br />
} else if (pmt::is_real(P)) {<br />
d = pmt::to_double(P);<br />
} else {<br />
// We really expected an integer or a double here, so we don't know what to do<br />
throw std::runtime_error("expected an integer!");<br />
}<br />
</syntaxhighlight><br />
<br />
It is important to do type checking since we cannot unpack a PMT of<br />
the wrong data type.<br />
<br />
We can compare PMTs without knowing their type by using the <code>pmt::equal()</code> function:<br />
<br />
<syntaxhighlight lang="c++"><br />
if (pmt::eq(P_int, P_double)) {<br />
std::cout << "Equal!" << std::endl; // This line will never be reached<br />
</syntaxhighlight><br />
<br />
There are more equality functions, which compare different things: [http://gnuradio.org/doc/doxygen/namespacepmt.html#a5c28635e14287cc0e2f762841c11032f <code>pmt::eq()</code>] and [http://gnuradio.org/doc/doxygen/namespacepmt.html#a25467c81e1c5f4619a9cabad7a88eed5 <code>pmt::eqv()</code>]. We won't need these for this tutorial.<br />
<br />
The rest of this page provides more depth into how to handle different<br />
data types with the PMT library.<br />
<br />
=== More Complex Types ===<br />
<br />
PMTs can hold a variety of types. Using the Python method <code>pmt.to_pmt()</code>, we can convert most of Pythons standard types out-of-the-box:<br />
<br />
<pre>P_tuple = pmt.to_pmt((1, 2, 3, 'spam', 'eggs'))<br />
P_dict = pmt.to_pmt({'spam': 42, 'eggs': 23})</pre><br />
But what does this mean in the C++ domain? Well, there are PMT types that define [http://gnuradio.org/doc/doxygen/namespacepmt.html#a32895cc5a614a46b66b869c4a7bd283c tuples] and [http://gnuradio.org/doc/doxygen/namespacepmt.html#aba10563e3ab43b8d52f9cb13132047cf dictionaries], keys and values being PMTs, again.<br /><br />
So, to create the tuple from the Python example, the C++ code would look like this:<br />
<br />
<pre>pmt::pmt_t P_tuple = pmt::make_tuple(pmt::from_long(1), pmt::from_long(2), pmt::from_long(3), pmt::string_to_symbol(&quot;spam&quot;), pmt::string_to_symbol(&quot;eggs&quot;))</pre><br />
For the dictionary, it's a bit more complex:<br />
<br />
<pre>pmt::pmt_t P_dict = pmt::make_dict();<br />
P_dict = pmt::dict_add(P_dict, pmt::string_to_symbol(&quot;spam&quot;), pmt::from_long(42));<br />
P_dict = pmt::dict_add(P_dict, pmt::string_to_symbol(&quot;eggs&quot;), pmt::from_long(23));</pre><br />
As you can see, we first need to create a dictionary, then assign every key/value pair individually.<br />
<br />
A variant of tuples are ''vectors''. Like Python's tuples and lists, PMT vectors are mutable, whereas PMT tuples are not. In fact, PMT vectors are the only PMT data types that are mutable. When changing the value or adding an item to a dictionary, we are actually creating a new PMT.<br />
<br />
To create a vector, we can initialize it to a certain lengths, and fill all elements with an initial value. We can then change items or reference them:<br />
<br />
<pre>pmt::pmt_t P_vector = pmt::make_vector(5, pmt::from_long(23)); // Creates a vector with 5 23's as PMTs<br />
pmt::vector_set(P_vector, 0, pmt::from_long(42)); // Change the first element to a 42<br />
std::cout &lt;&lt; pmt::vector_ref(P_vector, 0); // Will print 42</pre><br />
In Python, we can do all these steps (using <code>pmt.make_vector()</code> etc.), or convert a list:<br />
<br />
<pre>P_vector = pmt.to_pmt([42, 23, 23, 23, 23])</pre><br />
Vectors are also different from tuples in a sense that we can '''directly''' load data types into the elements, which don't have to be PMTs.<br /><br />
Say we want to pass a series of 8 float values to another block (these might be characteristics of a filter, for example). It would be cumbersome to convert every single element to and from PMTs, since all elements of the vector are the same type.<br />
<br />
We can use special vector types for this case:<br />
<br />
<pre>pmt::pmt_t P_f32vector = pmt::make_f32vector(8, 5.0); // Creates a vector with 8 5.0s's as floats<br />
pmt::f32vector_set(P_f32vector, 0, 2.0); // Change the first element to a 2.0<br />
float f = f32vector_ref(P_f32vector, 0);<br />
std::cout &lt;&lt; f &lt;&lt; std::endl; // Prints 2.0<br />
size_t len;<br />
float *fp = pmt::f32vector_elements(P_f32vector, len);<br />
for (size_t i = 0; i &lt; len; i++)<br />
std::cout &lt;&lt; fp[i] &lt;&lt; std::endl; // Prints all elements from P_f32vector, one after another</pre><br />
Python has a similar concept: [http://docs.scipy.org/doc/numpy/reference/generated/numpy.array.html Numpy arrays]. As usual, the PMT library understands this and converts as expected:<br />
<br />
<pre>P_f32vector = pmt.to_pmt(numpy.array([2.0, 5.0, 5.0, 5.0, 5.0], dtype=numpy.float32))<br />
print pmt.is_f32vector(P_f32vector) # Prints 'True'</pre><br />
Here, 'f32' stands for 'float, 32 bits'. PMTs know about most typical fixed-width data types, such as 'u8' (unsigned 8-bit character) or 'c32' (complex with 32-bit floats for each I and Q). Consult the [http://gnuradio.org/doc/doxygen/namespacepmt.html manual] for a full list of types.<br />
<br />
The most generic PMT type is probably the blob (binary large object). Use this with care - it allows us to pass around anything that can be represented in memory.<br />
<br />
== PMT Data Type ==<br />
<br />
All PMTs are of the type pmt::pmt_t. This is an opaque container and<br />
PMT functions must be used to manipulate and even do things like<br />
compare PMTs. PMTs are also ''immutable'' (except PMT vectors). We<br />
never change the data in a PMT; instead, we create a new PMT with the<br />
new data. The main reason for this is thread safety. We can pass PMTs<br />
as tags and messages between blocks and each receives its own copy<br />
that we can read from. However, we can never write to this object, and<br />
so if multiple blocks have a reference to the same PMT, there is no<br />
possibility of thread-safety issues of one reading the PMT data while<br />
another is writing the data. If a block is trying to write new data to<br />
a PMT, it actually creates a new PMT to put the data into. Thus we<br />
allow easy access to data in the PMT format without worrying about<br />
mutex locking and unlocking while manipulating them.<br />
<br />
PMTs can represent the following:<br />
<br />
* Boolean values of true/false<br />
* Strings (as symbols)<br />
* Integers (long and uint64)<br />
* Floats (as doubles)<br />
* Complex (as two doubles)<br />
* Pairs<br />
* Tuples<br />
* Vectors (of PMTs)<br />
* Uniform vectors (of any standard data type)<br />
* Dictionaries (list of key:value pairs)<br />
* Any (contains a boost::any pointer to hold anything)<br />
<br />
The PMT library also defines a set of functions that operate directly<br />
on PMTs such as:<br />
<br />
* Equal/equivalence between PMTs<br />
* Length (of a tuple or vector)<br />
* Map (apply a function to all elements in the PMT)<br />
* Reverse<br />
* Get a PMT at a position in a list<br />
* Serialize and deserialize<br />
* Printing<br />
<br />
The constants in the PMT library are:<br />
<br />
* pmt::PMT_T - a PMT True<br />
* pmt::PMT_F - a PMT False<br />
* pmt::PMT_NIL - an empty PMT (think Python's 'None')<br />
<br />
== Inserting and Extracting Data ==<br />
<br />
Use pmt.h for a complete guide to the list of functions used to create<br />
PMTs and get the data from a PMT. When using these functions, remember<br />
that while PMTs are opaque and designed to hold any data, the data<br />
underneath is still a C++ typed object, and so the right type of<br />
set/get function must be used for the data type.<br />
<br />
Typically, a PMT object can be made from a scalar item using a call<br />
like "pmt::from_<type>". Similarly, when getting data out of a<br />
PMT, we use a call like "pmt::to_<type>". For example:<br />
<syntaxhighlight lang="c++"><br />
double a = 1.2345;<br />
pmt::pmt_t pmt_a = pmt::from_double(a);<br />
double b = pmt::to_double(pmt_a);<br />
<br />
int c = 12345;<br />
pmt::pmt_t pmt_c = pmt::from_long(c);<br />
int d = pmt::to_long(pmt_c);<br />
</syntaxhighlight><br />
As a side-note, making a PMT from a complex number is not obvious:<br />
<syntaxhighlight lang="c++"><br />
std::complex<double> a(1.2, 3.4);<br />
pmt::pmt_t pmt_a = pmt::make_rectangular(a.real(), a.imag());<br />
std::complex<double> b = pmt::to_complex(pmt_a);<br />
</syntaxhighlight><br />
Pairs, dictionaries, and vectors have different constructors and ways<br />
to manipulate them, and these are explained in their own sections.<br />
<br />
== Strings ==<br />
<br />
PMTs have a way of representing short strings. These strings are<br />
actually stored as interned symbols in a hash table, so in other<br />
words, only one PMT object for a given string exists. If creating a<br />
new symbol from a string, if that string already exists in the hash<br />
table, the constructor will return a reference to the existing PMT.<br />
<br />
We create strings with the following functions, where the second<br />
function, pmt::intern, is simply an alias of the first.<br />
<syntaxhighlight lang="c++"><br />
pmt::pmt_t str0 = pmt::string_to_symbol(std::string("some string"));<br />
pmt::pmt_t str1 = pmt::intern(std::string("some string"));<br />
</syntaxhighlight><br />
The string can be retrieved using the inverse function:<br />
<syntaxhighlight lang="c++"><br />
std::string s = pmt::symbol_to_string(str0);<br />
</syntaxhighlight><br />
== Tests and Comparisons ==<br />
<br />
The PMT library comes with a number of functions to test and compare<br />
PMT objects. In general, for any PMT data type, there is an equivalent<br />
"pmt::is_<type>". We can use these to test the PMT before trying<br />
to access the data inside. Expanding our examples above, we have:<br />
<syntaxhighlight lang="c++"><br />
pmt::pmt_t str0 = pmt::string_to_symbol(std::string("some string"));<br />
if(pmt::is_symbol(str0))<br />
std::string s = pmt::symbol_to_string(str0);<br />
<br />
double a = 1.2345;<br />
pmt::pmt_t pmt_a = pmt::from_double(a);<br />
if(pmt::is_double(pmt_a))<br />
double b = pmt::to_double(pmt_a);<br />
<br />
int c = 12345;<br />
pmt::pmt_t pmt_c = pmt::from_long(c);<br />
if(pmt::is_long(pmt_a))<br />
int d = pmt::to_long(pmt_c);<br />
<br />
\\ This will fail the test. Otherwise, trying to coerce \b pmt_c as a<br />
\\ double when internally it is a long will result in an exception.<br />
if(pmt::is_double(pmt_a))<br />
double d = pmt::to_double(pmt_c);<br />
</syntaxhighlight><br />
<br />
== Dictionaries ==<br />
<br />
PMT dictionaries are lists of key:value pairs. They have a<br />
well-defined interface for creating, adding, removing, and accessing<br />
items in the dictionary. Note that every operation that changes the<br />
dictionary both takes a PMT dictionary as an argument and returns a<br />
PMT dictionary. The dictionary used as an input is not changed and the<br />
returned dictionary is a new PMT with the changes made there.<br />
<br />
The following is a list of PMT dictionary functions. View each function in the [https://www.gnuradio.org/doc/doxygen/index.html GNU Radio C++ Manual]<br />
to get more information on what each does.<br />
<br />
* bool pmt::is_dict(const pmt_t &obj)<br />
* pmt_t pmt::make_dict()<br />
* pmt_t pmt::dict_add(const pmt_t &dict, const pmt_t &key, const pmt_t &value)<br />
* pmt_t pmt::dict_delete(const pmt_t &dict, const pmt_t &key)<br />
* bool pmt::dict_has_key(const pmt_t &dict, const pmt_t &key)<br />
* pmt_t pmt::dict_ref(const pmt_t &dict, const pmt_t &key, const pmt_t &not_found)<br />
* pmt_t pmt::dict_items(pmt_t dict)<br />
* pmt_t pmt::dict_keys(pmt_t dict)<br />
* pmt_t pmt::dict_values(pmt_t dict)<br />
<br />
This example does some basic manipulations of PMT dictionaries in<br />
Python. Notice that we pass the dictionary ''a'' and return the results<br />
to ''a''. This still creates a new dictionary and removes the local<br />
reference to the old dictionary. This just keeps our number of<br />
variables small.<br />
<br />
<syntaxhighlight lang="python"><br />
import pmt<br />
<br />
key0 = pmt.intern("int")<br />
val0 = pmt.from_long(123)<br />
val1 = pmt.from_long(234)<br />
<br />
key1 = pmt.intern("double")<br />
val2 = pmt.from_double(5.4321)<br />
<br />
# Make an empty dictionary<br />
a = pmt.make_dict()<br />
<br />
# Add a key:value pair to the dictionary<br />
a = pmt.dict_add(a, key0, val0)<br />
print a<br />
<br />
# Add a new value to the same key;<br />
# new dict will still have one item with new value<br />
a = pmt.dict_add(a, key0, val1)<br />
print a<br />
<br />
# Add a new key:value pair<br />
a = pmt.dict_add(a, key1, val2)<br />
print a<br />
<br />
# Test if we have a key, then delete it<br />
print pmt.dict_has_key(a, key1)<br />
a = pmt.dict_delete(a, key1)<br />
print pmt.dict_has_key(a, key1)<br />
<br />
ref = pmt.dict_ref(a, key0, pmt.PMT_NIL)<br />
print ref<br />
<br />
# The following should never print<br />
if(pmt.dict_has_key(a, key0) and pmt.eq(ref, pmt.PMT_NIL)):<br />
print "Trouble! We have key0, but it returned PMT_NIL"<br />
</syntaxhighlight><br />
<br />
== Vectors ==<br />
<br />
PMT vectors come in two forms: vectors of PMTs and vectors of uniform<br />
data. The standard PMT vector is a vector of PMTs, and each PMT can be<br />
of any internal type. On the other hand, uniform PMTs are of a<br />
specific data type which come in the form:<br />
<br />
* (u)int8<br />
* (u)int16<br />
* (u)int32<br />
* (u)int64<br />
* float32<br />
* float64<br />
* complex 32 (std::complex<float>)<br />
* complex 64 (std::complex<double>)<br />
<br />
That is, the standard sizes of integers, floats, and complex types of<br />
both signed and unsigned.<br />
<br />
Vectors have a well-defined interface that allows us to make, set,<br />
get, and fill them. We can also get the length of a vector with<br />
pmt::length, or in Python:<br />
<br />
pmt_t p1 = pmt_integer(1);<br />
pmt_t p2 = pmt_integer(2);<br />
pmt_t p3 = pmt_integer(3);<br />
<br />
pmt_t p_list = pmt_list3(p1, p2, p3);<br />
<br />
pmt_length(p_list); // Returns 3<br />
<br />
For standard vectors, these functions look like:<br />
<br />
* bool pmt::is_vector(pmt_t x)<br />
* pmt_t pmt::make_vector(size_t k, pmt_t fill)<br />
* pmt_t pmt::vector_ref(pmt_t vector, size_t k)<br />
* void pmt::vector_set(pmt_t vector, size_t k, pmt_t obj)<br />
* void pmt::vector_fill(pmt_t vector, pmt_t fill)<br />
<br />
Uniform vectors have the same types of functions, but they are data<br />
type-dependent. The following list tries to explain them where you<br />
substitute the specific data type prefix for \a dtype (prefixes being:<br />
u8, u16, u32, u64, s8, s16, s32, s64, f32, f64, c32, c64).<br />
<br />
* bool pmt::is_(dtype)vector(pmt_t x)<br />
* pmt_t pmt::make_(dtype)vector(size_t k, (dtype) fill)<br />
* pmt_t pmt::init_(dtype)vector(size_t k, const (dtype*) data)<br />
* pmt_t pmt::init_(dtype)vector(size_t k, const std::vector<dtype> data)<br />
* pmt_t pmt::(dtype)vector_ref(pmt_t vector, size_t k)<br />
* void pmt::(dtype)vector_set(pmt_t vector, size_t k, (dtype) x)<br />
* const dtype* pmt::(dtype)vector_elements(pmt_t vector, size_t &len)<br />
* dtype* pmt::(dtype)vector_writable_elements(pmt_t vector, size_t &len)<br />
<br />
List of available pmt::is_(dtype)vector(pmt_t x) methods from <code>pmt.h</code>:<br><br />
<br />
bool pmt::is_uniform_vector()<br />
bool pmt::is_u8vector()<br />
bool pmt::is_s8vector()<br />
bool pmt::is_u16vector()<br />
bool pmt::is_s16vector()<br />
bool pmt::is_u32vector()<br />
bool pmt::is_s32vector()<br />
bool pmt::is_u64vector()<br />
bool pmt::is_s64vector()<br />
bool pmt::is_f32vector()<br />
bool pmt::is_f64vector()<br />
bool pmt::is_c32vector()<br />
bool pmt::is_c64vector()<br />
<br />
'''Note:''' We break the contract with vectors. The 'set' functions<br />
actually change the data underneath. It is important to keep track of<br />
the implications of setting a new value as well as accessing the<br />
'vector_writable_elements' data. Since these are mostly standard data<br />
types, sets and gets are atomic, so it is unlikely to cause a great<br />
deal of harm. But it's only unlikely, not impossible. Best to use<br />
mutexes whenever manipulating data in a vector.<br />
<br />
=== BLOB ===<br />
<br />
A BLOB is a 'binary large object' type. In PMT's, this is actually<br />
just a thin wrapper around a u8vector.<br />
<br />
== Pairs ==<br />
<br />
A concept that originates in Lisp dialects are [http://en.wikipedia.org/wiki/Cons ''pairs'' and ''cons'']. The simplest explanation is just that: If you combine two PMTs, they form a new PMT, which is a pair (or cons) of those two PMTs (don't worry about the weird name, a lot of things originating in Lisp have weird names. Think of a 'construct').<br />
<br />
Similarly to vectors or tuples, pairs are a useful way of packing several components of a message into a single PMT. Using the PMTs generated in the previous section, we can combine two of these to form a pair, here in Python:<br />
<br />
<pre>P_pair = pmt.cons(pmt.string_to_symbol(&quot;taps&quot;), P_f32vector)<br />
print pmt.is_pair(P_pair) # Prints 'true'</pre><br />
You can combine PMTs as tuples, dictionaries, vectors, or pairs, it's just a matter of taste. This construct is well-established though, and as such used in GNU Radio quite often.<br />
<br />
So how do we deconstruct a pair? That's what the <code>car</code> and <code>cdr</code> functions do. Let's deconstruct that previous pair in C++:<br />
<br />
<pre>pmt::pmt_t P_key = pmt::car(P_pair);<br />
pmt::pmt_t P_f32vector2 = pmt::cdr(P_pair);<br />
std::cout &lt;&lt; P_key &lt;&lt; std::endl; // Will print 'taps' using the PMT automatic conversion to strings</pre><br />
<br />
Here is a summary of the pairs-related functions in C++ and Python:<br />
<br />
* bool pmt::is_pair(const pmt_t &obj): Return true if obj is a pair, else false<br />
* pmt_t pmt::cons(const pmt_t &x, const pmt_t &y): construct new pair<br />
* pmt_t pmt::car(const pmt_t &pair): get the car of the pair (first object)<br />
* pmt_t pmt::cdr(const pmt_t &pair): get the cdr of the pair (second object)<br />
* void pmt::set_car(pmt_t pair, pmt_t value): Stores value in the car field<br />
* void pmt::set_cdr(pmt_t pair, pmt_t value): Stores value in the cdr field<br />
<br />
And in Python we have:<br />
<syntaxhighlight lang="python"><br />
pmt.is_pair(pair_obj) # Return True if is a pair, else False (warning: also returns True for a dict)<br />
pmt.cons(x, y) # Return a newly allocated pair whose car is x and whose cdr is y<br />
pmt.car(pair_obj) # If is a pair, return the car, otherwise raise wrong_type<br />
pmt.cdr(pair_obj) # If is a pair, return the cdr, otherwise raise wrong_type<br />
pmt.set_car(pair_obj, value) # Store value in car field<br />
pmt.set_cdr(pair_obj, value) # Store value in cdr field<br />
</syntaxhighlight><br />
<br />
For more advanced pair manipulation, refer to the [http://gnuradio.org/doc/doxygen/namespacepmt.html#a7ab95721db5cbda1852f13a92eee5362 documentation] and the [https://en.wikipedia.org/wiki/Car_and_cdr Wikipedia page for car and cdr].<br />
<br />
== Serializing and Deserializing ==<br />
<br />
It is often important to hide the fact that we are working with PMTs<br />
to make them easier to transmit, store, write to file, etc. The PMT<br />
library has methods to serialize data into a string buffer or a<br />
string and then methods to deserialize the string buffer or string<br />
back into a PMT. We use this extensively in the metadata files (see<br />
[[Metadata Information]]).<br />
<br />
* bool pmt::serialize(pmt_t obj, std::streambuf &sink)<br />
* std::string pmt::serialize_str(pmt_t obj)<br />
* pmt_t pmt::deserialize(std::streambuf &source)<br />
* pmt_t pmt::deserialize_str(std::string str)<br />
<br />
For example, we will serialize the data above to make it into a string<br />
ready to be written to a file and then deserialize it back to its<br />
original PMT.<br />
<br />
<syntaxhighlight lang="python"><br />
import pmt<br />
<br />
key0 = pmt.intern("int")<br />
val0 = pmt.from_long(123)<br />
<br />
key1 = pmt.intern("double")<br />
val1 = pmt.from_double(5.4321)<br />
<br />
# Make an empty dictionary<br />
a = pmt.make_dict()<br />
<br />
# Add a key:value pair to the dictionary<br />
a = pmt.dict_add(a, key0, val0)<br />
a = pmt.dict_add(a, key1, val1)<br />
<br />
print a<br />
<br />
ser_str = pmt.serialize_str(a)<br />
print ser_str<br />
<br />
b = pmt.deserialize_str(ser_str)<br />
print b<br />
</syntaxhighlight><br />
<br />
The line where we 'print ser_str' will print and parts will be<br />
readable, but the point of serializing is not to make a human-readable<br />
string. This is only done here as a test.<br />
<br />
== Printing ==<br />
<br />
In Python, the __repr__ function of a PMT object is overloaded to call<br />
'pmt::write_string'. This means that any time we call a formatted<br />
printing operation on a PMT object, the PMT library will properly<br />
format the object for display.<br />
<br />
In C++, we can use the 'pmt::print(object)' function or print the<br />
contents is using the overloaded "<<" operator with a stream buffer<br />
object. In C++, we can inline print the contents of a PMT like:<br />
<br />
<syntaxhighlight lang="c++"><br />
pmt::pmt_t a = pmt::from_double(1.0);<br />
std::cout << "The PMT a contains " << a << std::endl;<br />
</syntaxhighlight><br />
<br />
=== Collection Notation ===<br />
PMTs use a different bracket notation from what one might be use to in Python.<br />
<br />
<syntaxhighlight lang="plaintext"><br />
>>> my_dict<br />
((meaning . 42))<br />
>>> my_vector<br />
#[1 2 3 4]<br />
>>> pmt.make_tuple(pmt.from_long(321), pmt.from_float(3.14))<br />
{321 3.14}<br />
>>> pmt.cons(pmt.from_long(1), pmt.from_long(2))<br />
(1 . 2)<br />
>>> my_pdu<br />
(() . #[1 2 3 4])<br />
</syntaxhighlight><br />
<br />
== Conversion between Python Objects and PMTs ==<br />
<br />
Although PMTs can be manipulated in Python using the Python versions<br />
of the C++ interfaces, there are some additional goodies that make it<br />
easier to work with PMTs in python. There are functions to automate<br />
the conversion between PMTs and Python types for booleans, strings,<br />
integers, longs, floats, complex numbers, dictionaries, lists, tuples<br />
and combinations thereof.<br />
<br />
Two functions capture most of this functionality:<br />
<br />
<syntaxhighlight lang="python"><br />
pmt.to_pmt # Converts a python object to a PMT.<br />
pmt.to_python # Converts a PMT into a python object.<br />
</syntaxhighlight></div>Solomonbstonerhttps://wiki.gnuradio.org/index.php?title=Polymorphic_Types_(PMTs)&diff=8533Polymorphic Types (PMTs)2021-05-08T15:20:51Z<p>Solomonbstoner: Added missing equal sign in code</p>
<hr />
<div>[[Category:Usage Manual]]<br />
== Introduction ==<br />
<br />
Polymorphic Types are used as the carrier of data from one block/thread to another such as stream tags and message passing interfaces. <br />
PMT data types can represent a variety of data ranging from the Booleans to dictionaries. <br />
They are heavily used in the stream tags and message passing interfaces. <br />
In a sense, PMTs are a way to extend C++' strict typing with something more flexible. <br />
The most complete list of PMT function is, of course, the source code, specifically the header file [https://gnuradio.org/doc/doxygen/pmt_8h.html pmt.h]. <br />
This page summarizes the most important features and points of PMTs.<br />
<br />
Let's dive straight into some Python code and see how we can use PMTs:<br />
<br />
<syntaxhighlight lang="python"><br />
>>> import pmt<br />
>>> P = pmt.from_long(23)<br />
>>> type(P)<br />
<class 'pmt.pmt_swig.swig_int_ptr'><br />
>>> print P<br />
23<br />
>>> P2 = pmt.from_complex(1j)<br />
>>> type(P2)<br />
<class 'pmt.pmt_swig.swig_int_ptr'><br />
>>> print P2<br />
0+1i<br />
>>> pmt.is_complex(P2)<br />
True<br />
</syntaxhighlight><br />
First, the <code>pmt</code> module is imported. We assign two values (<code>P</code> and <code>P2</code>) with PMTs using the <code>from_long()</code> and <code>from_complex()</code> calls, respectively. As we can see, they are both of the same type! This means we can pass these variables to C++ through SWIG, and C++ can handle this type accordingly.<br />
<br />
The same code as above in C++ would look like this:<br />
<syntaxhighlight lang="c++"><br />
#include <pmt/pmt.h><br />
// [...]<br />
pmt::pmt_t P = pmt::from_long(23);<br />
std::cout << P << std::endl;<br />
pmt::pmt_t P2 = pmt::from_complex(gr_complex(0, 1)); <br />
// Alternatively: pmt::from_complex(0, 1)<br />
std::cout << P2 << std::endl;<br />
std::cout << pmt::is_complex(P2) << std::endl;<br />
</syntaxhighlight><br />
Two things stand out in both Python and C++: First, we can simply print the contents of a PMT. How is this possible? Well, the PMTs have in-built capability to cast their value to a string (this is not possible with all types, though). Second, PMTs must obviously know their type, so we can query that, e.g. by calling the <code>is_complex()</code> method.<br />
<br />
Note: When running the above as a standalone, the compiler command will look something like <code>g++ pmt_tutorial.cpp -I$(gnuradio-config-info --prefix)/include -lgnuradio-pmt -o pmt_tutorial</code><br />
<br />
When assigning a non-PMT value to a PMT, we can use the <code>from_*</code> methods, and use the <code>to_*</code> methods to convert back:<br />
<syntaxhighlight lang="c++"><br />
pmt::pmt_t P_int = pmt::from_long(42);<br />
int i = pmt::to_long(P_int);<br />
pmt::pmt_t P_double = pmt::from_double(0.2);<br />
double d = pmt::to_double(P_double);<br />
pmt::pmt_t P_double = pmt::mp(0.2);<br />
</syntaxhighlight><br />
<br />
The last row shows the [http://gnuradio.org/doc/doxygen/namespacepmt.html#a90faad6086ac00280e0cfd8bb541bd64 pmt::mp()] shorthand function. It basically saves some typing, as it infers the correct <code>from_</code> function from the given type.<br />
<br />
String types play a bit of a special role in PMTs, as we will see<br />
later, and have their own converter:<br />
<br />
<syntaxhighlight lang="c++"><br />
pmt::pmt_t P_str = pmt::string_to_symbol("spam");<br />
pmt::pmt_t P_str2 = pmt::intern("spam");<br />
std::string str = pmt::symbol_to_string(P_str);<br />
</syntaxhighlight><br />
The pmt::intern is another way of saying pmt::string_to_symbol.<br />
<br />
See the [https://www.gnuradio.org/doc/doxygen/namespacepmt.html PMT docs] and the header file [http://gnuradio.org/doc/doxygen/pmt_8h.html pmt.h] for a full list of conversion functions.<br />
<br />
In Python, we can make use of the dynamic typing, and there's actually a<br />
helper function to do these conversions (C++ also has a helper<br />
function for converting to PMTs called pmt::mp(), but it's less<br />
powerful, and not quite as useful, because types are always strictly<br />
known in C++):<br />
<br />
<syntaxhighlight lang="python"><br />
P_int = pmt.to_pmt(42)<br />
i = pmt.to_python(P_int)<br />
P_double = pmt.to_pmt(0.2)<br />
d = pmt.to_double(P_double)<br />
</syntaxhighlight><br />
<br />
On a side note, there are three useful PMT constants, which can be<br />
used in both Python and C++ domains. In C++, these can be used as<br />
such:<br />
<br />
<syntaxhighlight lang="c++"><br />
pmt::pmt_t P_true = pmt::PMT_T;<br />
pmt::pmt_t P_false = pmt::PMT_F;<br />
pmt::pmt_t P_nil = pmt::PMT_NIL;<br />
</syntaxhighlight><br />
<br />
In Python:<br />
<br />
<syntaxhighlight lang="python"><br />
P_true = pmt.PMT_T<br />
P_false = pmt.PMT_F<br />
P_nil = pmt.PMT_NIL<br />
</syntaxhighlight><br />
<br />
<code>pmt.PMT_T</code> and <code>pmt.PMT_F</code> are boolean PMT types representing True and False, respectively. The PMT_NIL is like a NULL or None and can be used for default arguments or return values, often indicating an error has occurred.<br />
<br />
To be able to go back to C++ data types, we need to be able to find<br />
out the type from a PMT. The family of <code>is_*</code> methods helps us do that:<br />
<br />
<syntaxhighlight lang="c++"><br />
double d;<br />
if (pmt::is_integer(P)) {<br />
d = (double) pmt::to_long(P);<br />
} else if (pmt::is_real(P)) {<br />
d = pmt::to_double(P);<br />
} else {<br />
// We really expected an integer or a double here, so we don't know what to do<br />
throw std::runtime_error("expected an integer!");<br />
}<br />
</syntaxhighlight><br />
<br />
It is important to do type checking since we cannot unpack a PMT of<br />
the wrong data type.<br />
<br />
We can compare PMTs without knowing their type by using the <code>pmt::equal()</code> function:<br />
<br />
<syntaxhighlight lang="c++"><br />
if (pmt::eq(P_int, P_double)) {<br />
std::cout << "Equal!" << std::endl; // This line will never be reached<br />
</syntaxhighlight><br />
<br />
There are more equality functions, which compare different things: [http://gnuradio.org/doc/doxygen/namespacepmt.html#a5c28635e14287cc0e2f762841c11032f <code>pmt::eq()</code>] and [http://gnuradio.org/doc/doxygen/namespacepmt.html#a25467c81e1c5f4619a9cabad7a88eed5 <code>pmt::eqv()</code>]. We won't need these for this tutorial.<br />
<br />
The rest of this page provides more depth into how to handle different<br />
data types with the PMT library.<br />
<br />
=== More Complex Types ===<br />
<br />
PMTs can hold a variety of types. Using the Python method <code>pmt.to_pmt()</code>, we can convert most of Pythons standard types out-of-the-box:<br />
<br />
<pre>P_tuple = pmt.to_pmt((1, 2, 3, 'spam', 'eggs'))<br />
P_dict = pmt.to_pmt({'spam': 42, 'eggs': 23})</pre><br />
But what does this mean in the C++ domain? Well, there are PMT types that define [http://gnuradio.org/doc/doxygen/namespacepmt.html#a32895cc5a614a46b66b869c4a7bd283c tuples] and [http://gnuradio.org/doc/doxygen/namespacepmt.html#aba10563e3ab43b8d52f9cb13132047cf dictionaries], keys and values being PMTs, again.<br /><br />
So, to create the tuple from the Python example, the C++ code would look like this:<br />
<br />
<pre>pmt::pmt_t P_tuple = pmt::make_tuple(pmt::from_long(1), pmt::from_long(2), pmt::from_long(3), pmt::string_to_symbol(&quot;spam&quot;), pmt::string_to_symbol(&quot;eggs&quot;))</pre><br />
For the dictionary, it's a bit more complex:<br />
<br />
<pre>pmt::pmt_t P_dict = pmt::make_dict();<br />
P_dict = pmt::dict_add(P_dict, pmt::string_to_symbol(&quot;spam&quot;), pmt::from_long(42));<br />
P_dict = pmt::dict_add(P_dict, pmt::string_to_symbol(&quot;eggs&quot;), pmt::from_long(23));</pre><br />
As you can see, we first need to create a dictionary, then assign every key/value pair individually.<br />
<br />
A variant of tuples are ''vectors''. Like Python's tuples and lists, PMT vectors are mutable, whereas PMT tuples are not. In fact, PMT vectors are the only PMT data types that are mutable. When changing the value or adding an item to a dictionary, we are actually creating a new PMT.<br />
<br />
To create a vector, we can initialize it to a certain lengths, and fill all elements with an initial value. We can then change items or reference them:<br />
<br />
<pre>pmt::pmt_t P_vector = pmt::make_vector(5, pmt::from_long(23)); // Creates a vector with 5 23's as PMTs<br />
pmt::vector_set(P_vector, 0, pmt::from_long(42)); // Change the first element to a 42<br />
std::cout &lt;&lt; pmt::vector_ref(P_vector, 0); // Will print 42</pre><br />
In Python, we can do all these steps (using <code>pmt.make_vector()</code> etc.), or convert a list:<br />
<br />
<pre>P_vector = pmt.to_pmt([42, 23, 23, 23, 23])</pre><br />
Vectors are also different from tuples in a sense that we can '''directly''' load data types into the elements, which don't have to be PMTs.<br /><br />
Say we want to pass a series of 8 float values to another block (these might be characteristics of a filter, for example). It would be cumbersome to convert every single element to and from PMTs, since all elements of the vector are the same type.<br />
<br />
We can use special vector types for this case:<br />
<br />
<pre>pmt::pmt_t P_f32vector = pmt::make_f32vector(8, 5.0); // Creates a vector with 8 5.0s's as floats<br />
pmt::f32vector_set(P_f32vector, 0, 2.0); // Change the first element to a 2.0<br />
float f = f32vector_ref(P_f32vector, 0);<br />
std::cout &lt;&lt; f &lt;&lt; std::endl; // Prints 2.0<br />
size_t len;<br />
float *fp = pmt::f32vector_elements(P_f32vector, len);<br />
for (size_t i = 0; i &lt; len; i++)<br />
std::cout &lt;&lt; fp[i] &lt;&lt; std::endl; // Prints all elements from P_f32vector, one after another</pre><br />
Python has a similar concept: [http://docs.scipy.org/doc/numpy/reference/generated/numpy.array.html Numpy arrays]. As usual, the PMT library understands this and converts as expected:<br />
<br />
<pre>P_f32vector = pmt.to_pmt(numpy.array([2.0, 5.0, 5.0, 5.0, 5.0], dtype=numpy.float32))<br />
print pmt.is_f32vector(P_f32vector) # Prints 'True'</pre><br />
Here, 'f32' stands for 'float, 32 bits'. PMTs know about most typical fixed-width data types, such as 'u8' (unsigned 8-bit character) or 'c32' (complex with 32-bit floats for each I and Q). Consult the [http://gnuradio.org/doc/doxygen/namespacepmt.html manual] for a full list of types.<br />
<br />
The most generic PMT type is probably the blob (binary large object). Use this with care - it allows us to pass around anything that can be represented in memory.<br />
<br />
== PMT Data Type ==<br />
<br />
All PMTs are of the type pmt::pmt_t. This is an opaque container and<br />
PMT functions must be used to manipulate and even do things like<br />
compare PMTs. PMTs are also ''immutable'' (except PMT vectors). We<br />
never change the data in a PMT; instead, we create a new PMT with the<br />
new data. The main reason for this is thread safety. We can pass PMTs<br />
as tags and messages between blocks and each receives its own copy<br />
that we can read from. However, we can never write to this object, and<br />
so if multiple blocks have a reference to the same PMT, there is no<br />
possibility of thread-safety issues of one reading the PMT data while<br />
another is writing the data. If a block is trying to write new data to<br />
a PMT, it actually creates a new PMT to put the data into. Thus we<br />
allow easy access to data in the PMT format without worrying about<br />
mutex locking and unlocking while manipulating them.<br />
<br />
PMTs can represent the following:<br />
<br />
* Boolean values of true/false<br />
* Strings (as symbols)<br />
* Integers (long and uint64)<br />
* Floats (as doubles)<br />
* Complex (as two doubles)<br />
* Pairs<br />
* Tuples<br />
* Vectors (of PMTs)<br />
* Uniform vectors (of any standard data type)<br />
* Dictionaries (list of key:value pairs)<br />
* Any (contains a boost::any pointer to hold anything)<br />
<br />
The PMT library also defines a set of functions that operate directly<br />
on PMTs such as:<br />
<br />
* Equal/equivalence between PMTs<br />
* Length (of a tuple or vector)<br />
* Map (apply a function to all elements in the PMT)<br />
* Reverse<br />
* Get a PMT at a position in a list<br />
* Serialize and deserialize<br />
* Printing<br />
<br />
The constants in the PMT library are:<br />
<br />
* pmt::PMT_T - a PMT True<br />
* pmt::PMT_F - a PMT False<br />
* pmt::PMT_NIL - an empty PMT (think Python's 'None')<br />
<br />
== Inserting and Extracting Data ==<br />
<br />
Use pmt.h for a complete guide to the list of functions used to create<br />
PMTs and get the data from a PMT. When using these functions, remember<br />
that while PMTs are opaque and designed to hold any data, the data<br />
underneath is still a C++ typed object, and so the right type of<br />
set/get function must be used for the data type.<br />
<br />
Typically, a PMT object can be made from a scalar item using a call<br />
like "pmt::from_<type>". Similarly, when getting data out of a<br />
PMT, we use a call like "pmt::to_<type>". For example:<br />
<syntaxhighlight lang="c++"><br />
double a = 1.2345;<br />
pmt::pmt_t pmt_a = pmt::from_double(a);<br />
double b = pmt::to_double(pmt_a);<br />
<br />
int c = 12345;<br />
pmt::pmt_t pmt_c = pmt::from_long(c);<br />
int d = pmt::to_long(pmt_c);<br />
</syntaxhighlight><br />
As a side-note, making a PMT from a complex number is not obvious:<br />
<syntaxhighlight lang="c++"><br />
std::complex<double> a(1.2, 3.4);<br />
pmt::pmt_t pmt_a = pmt::make_rectangular(a.real(), b.imag());<br />
std::complex<double> b = pmt::to_complex(pmt_a);<br />
</syntaxhighlight><br />
Pairs, dictionaries, and vectors have different constructors and ways<br />
to manipulate them, and these are explained in their own sections.<br />
<br />
== Strings ==<br />
<br />
PMTs have a way of representing short strings. These strings are<br />
actually stored as interned symbols in a hash table, so in other<br />
words, only one PMT object for a given string exists. If creating a<br />
new symbol from a string, if that string already exists in the hash<br />
table, the constructor will return a reference to the existing PMT.<br />
<br />
We create strings with the following functions, where the second<br />
function, pmt::intern, is simply an alias of the first.<br />
<syntaxhighlight lang="c++"><br />
pmt::pmt_t str0 = pmt::string_to_symbol(std::string("some string"));<br />
pmt::pmt_t str1 = pmt::intern(std::string("some string"));<br />
</syntaxhighlight><br />
The string can be retrieved using the inverse function:<br />
<syntaxhighlight lang="c++"><br />
std::string s = pmt::symbol_to_string(str0);<br />
</syntaxhighlight><br />
== Tests and Comparisons ==<br />
<br />
The PMT library comes with a number of functions to test and compare<br />
PMT objects. In general, for any PMT data type, there is an equivalent<br />
"pmt::is_<type>". We can use these to test the PMT before trying<br />
to access the data inside. Expanding our examples above, we have:<br />
<syntaxhighlight lang="c++"><br />
pmt::pmt_t str0 = pmt::string_to_symbol(std::string("some string"));<br />
if(pmt::is_symbol(str0))<br />
std::string s = pmt::symbol_to_string(str0);<br />
<br />
double a = 1.2345;<br />
pmt::pmt_t pmt_a = pmt::from_double(a);<br />
if(pmt::is_double(pmt_a))<br />
double b = pmt::to_double(pmt_a);<br />
<br />
int c = 12345;<br />
pmt::pmt_t pmt_c = pmt::from_long(c);<br />
if(pmt::is_long(pmt_a))<br />
int d = pmt::to_long(pmt_c);<br />
<br />
\\ This will fail the test. Otherwise, trying to coerce \b pmt_c as a<br />
\\ double when internally it is a long will result in an exception.<br />
if(pmt::is_double(pmt_a))<br />
double d = pmt::to_double(pmt_c);<br />
</syntaxhighlight><br />
<br />
== Dictionaries ==<br />
<br />
PMT dictionaries are lists of key:value pairs. They have a<br />
well-defined interface for creating, adding, removing, and accessing<br />
items in the dictionary. Note that every operation that changes the<br />
dictionary both takes a PMT dictionary as an argument and returns a<br />
PMT dictionary. The dictionary used as an input is not changed and the<br />
returned dictionary is a new PMT with the changes made there.<br />
<br />
The following is a list of PMT dictionary functions. View each function in the [https://www.gnuradio.org/doc/doxygen/index.html GNU Radio C++ Manual]<br />
to get more information on what each does.<br />
<br />
* bool pmt::is_dict(const pmt_t &obj)<br />
* pmt_t pmt::make_dict()<br />
* pmt_t pmt::dict_add(const pmt_t &dict, const pmt_t &key, const pmt_t &value)<br />
* pmt_t pmt::dict_delete(const pmt_t &dict, const pmt_t &key)<br />
* bool pmt::dict_has_key(const pmt_t &dict, const pmt_t &key)<br />
* pmt_t pmt::dict_ref(const pmt_t &dict, const pmt_t &key, const pmt_t &not_found)<br />
* pmt_t pmt::dict_items(pmt_t dict)<br />
* pmt_t pmt::dict_keys(pmt_t dict)<br />
* pmt_t pmt::dict_values(pmt_t dict)<br />
<br />
This example does some basic manipulations of PMT dictionaries in<br />
Python. Notice that we pass the dictionary ''a'' and return the results<br />
to ''a''. This still creates a new dictionary and removes the local<br />
reference to the old dictionary. This just keeps our number of<br />
variables small.<br />
<br />
<syntaxhighlight lang="python"><br />
import pmt<br />
<br />
key0 = pmt.intern("int")<br />
val0 = pmt.from_long(123)<br />
val1 = pmt.from_long(234)<br />
<br />
key1 = pmt.intern("double")<br />
val2 = pmt.from_double(5.4321)<br />
<br />
# Make an empty dictionary<br />
a = pmt.make_dict()<br />
<br />
# Add a key:value pair to the dictionary<br />
a = pmt.dict_add(a, key0, val0)<br />
print a<br />
<br />
# Add a new value to the same key;<br />
# new dict will still have one item with new value<br />
a = pmt.dict_add(a, key0, val1)<br />
print a<br />
<br />
# Add a new key:value pair<br />
a = pmt.dict_add(a, key1, val2)<br />
print a<br />
<br />
# Test if we have a key, then delete it<br />
print pmt.dict_has_key(a, key1)<br />
a = pmt.dict_delete(a, key1)<br />
print pmt.dict_has_key(a, key1)<br />
<br />
ref = pmt.dict_ref(a, key0, pmt.PMT_NIL)<br />
print ref<br />
<br />
# The following should never print<br />
if(pmt.dict_has_key(a, key0) and pmt.eq(ref, pmt.PMT_NIL)):<br />
print "Trouble! We have key0, but it returned PMT_NIL"<br />
</syntaxhighlight><br />
<br />
== Vectors ==<br />
<br />
PMT vectors come in two forms: vectors of PMTs and vectors of uniform<br />
data. The standard PMT vector is a vector of PMTs, and each PMT can be<br />
of any internal type. On the other hand, uniform PMTs are of a<br />
specific data type which come in the form:<br />
<br />
* (u)int8<br />
* (u)int16<br />
* (u)int32<br />
* (u)int64<br />
* float32<br />
* float64<br />
* complex 32 (std::complex<float>)<br />
* complex 64 (std::complex<double>)<br />
<br />
That is, the standard sizes of integers, floats, and complex types of<br />
both signed and unsigned.<br />
<br />
Vectors have a well-defined interface that allows us to make, set,<br />
get, and fill them. We can also get the length of a vector with<br />
pmt::length, or in Python:<br />
<br />
pmt_t p1 = pmt_integer(1);<br />
pmt_t p2 = pmt_integer(2);<br />
pmt_t p3 = pmt_integer(3);<br />
<br />
pmt_t p_list = pmt_list3(p1, p2, p3);<br />
<br />
pmt_length(p_list); // Returns 3<br />
<br />
For standard vectors, these functions look like:<br />
<br />
* bool pmt::is_vector(pmt_t x)<br />
* pmt_t pmt::make_vector(size_t k, pmt_t fill)<br />
* pmt_t pmt::vector_ref(pmt_t vector, size_t k)<br />
* void pmt::vector_set(pmt_t vector, size_t k, pmt_t obj)<br />
* void pmt::vector_fill(pmt_t vector, pmt_t fill)<br />
<br />
Uniform vectors have the same types of functions, but they are data<br />
type-dependent. The following list tries to explain them where you<br />
substitute the specific data type prefix for \a dtype (prefixes being:<br />
u8, u16, u32, u64, s8, s16, s32, s64, f32, f64, c32, c64).<br />
<br />
* bool pmt::is_(dtype)vector(pmt_t x)<br />
* pmt_t pmt::make_(dtype)vector(size_t k, (dtype) fill)<br />
* pmt_t pmt::init_(dtype)vector(size_t k, const (dtype*) data)<br />
* pmt_t pmt::init_(dtype)vector(size_t k, const std::vector<dtype> data)<br />
* pmt_t pmt::(dtype)vector_ref(pmt_t vector, size_t k)<br />
* void pmt::(dtype)vector_set(pmt_t vector, size_t k, (dtype) x)<br />
* const dtype* pmt::(dtype)vector_elements(pmt_t vector, size_t &len)<br />
* dtype* pmt::(dtype)vector_writable_elements(pmt_t vector, size_t &len)<br />
<br />
List of available pmt::is_(dtype)vector(pmt_t x) methods from <code>pmt.h</code>:<br><br />
<br />
bool pmt::is_uniform_vector()<br />
bool pmt::is_u8vector()<br />
bool pmt::is_s8vector()<br />
bool pmt::is_u16vector()<br />
bool pmt::is_s16vector()<br />
bool pmt::is_u32vector()<br />
bool pmt::is_s32vector()<br />
bool pmt::is_u64vector()<br />
bool pmt::is_s64vector()<br />
bool pmt::is_f32vector()<br />
bool pmt::is_f64vector()<br />
bool pmt::is_c32vector()<br />
bool pmt::is_c64vector()<br />
<br />
'''Note:''' We break the contract with vectors. The 'set' functions<br />
actually change the data underneath. It is important to keep track of<br />
the implications of setting a new value as well as accessing the<br />
'vector_writable_elements' data. Since these are mostly standard data<br />
types, sets and gets are atomic, so it is unlikely to cause a great<br />
deal of harm. But it's only unlikely, not impossible. Best to use<br />
mutexes whenever manipulating data in a vector.<br />
<br />
=== BLOB ===<br />
<br />
A BLOB is a 'binary large object' type. In PMT's, this is actually<br />
just a thin wrapper around a u8vector.<br />
<br />
== Pairs ==<br />
<br />
A concept that originates in Lisp dialects are [http://en.wikipedia.org/wiki/Cons ''pairs'' and ''cons'']. The simplest explanation is just that: If you combine two PMTs, they form a new PMT, which is a pair (or cons) of those two PMTs (don't worry about the weird name, a lot of things originating in Lisp have weird names. Think of a 'construct').<br />
<br />
Similarly to vectors or tuples, pairs are a useful way of packing several components of a message into a single PMT. Using the PMTs generated in the previous section, we can combine two of these to form a pair, here in Python:<br />
<br />
<pre>P_pair = pmt.cons(pmt.string_to_symbol(&quot;taps&quot;), P_f32vector)<br />
print pmt.is_pair(P_pair) # Prints 'true'</pre><br />
You can combine PMTs as tuples, dictionaries, vectors, or pairs, it's just a matter of taste. This construct is well-established though, and as such used in GNU Radio quite often.<br />
<br />
So how do we deconstruct a pair? That's what the <code>car</code> and <code>cdr</code> functions do. Let's deconstruct that previous pair in C++:<br />
<br />
<pre>pmt::pmt_t P_key = pmt::car(P_pair);<br />
pmt::pmt_t P_f32vector2 = pmt::cdr(P_pair);<br />
std::cout &lt;&lt; P_key &lt;&lt; std::endl; // Will print 'taps' using the PMT automatic conversion to strings</pre><br />
<br />
Here is a summary of the pairs-related functions in C++ and Python:<br />
<br />
* bool pmt::is_pair(const pmt_t &obj): Return true if obj is a pair, else false<br />
* pmt_t pmt::cons(const pmt_t &x, const pmt_t &y): construct new pair<br />
* pmt_t pmt::car(const pmt_t &pair): get the car of the pair (first object)<br />
* pmt_t pmt::cdr(const pmt_t &pair): get the cdr of the pair (second object)<br />
* void pmt::set_car(pmt_t pair, pmt_t value): Stores value in the car field<br />
* void pmt::set_cdr(pmt_t pair, pmt_t value): Stores value in the cdr field<br />
<br />
And in Python we have:<br />
<syntaxhighlight lang="python"><br />
pmt.is_pair(pair_obj) # Return True if is a pair, else False (warning: also returns True for a dict)<br />
pmt.cons(x, y) # Return a newly allocated pair whose car is x and whose cdr is y<br />
pmt.car(pair_obj) # If is a pair, return the car, otherwise raise wrong_type<br />
pmt.cdr(pair_obj) # If is a pair, return the cdr, otherwise raise wrong_type<br />
pmt.set_car(pair_obj, value) # Store value in car field<br />
pmt.set_cdr(pair_obj, value) # Store value in cdr field<br />
</syntaxhighlight><br />
<br />
For more advanced pair manipulation, refer to the [http://gnuradio.org/doc/doxygen/namespacepmt.html#a7ab95721db5cbda1852f13a92eee5362 documentation] and the [https://en.wikipedia.org/wiki/Car_and_cdr Wikipedia page for car and cdr].<br />
<br />
== Serializing and Deserializing ==<br />
<br />
It is often important to hide the fact that we are working with PMTs<br />
to make them easier to transmit, store, write to file, etc. The PMT<br />
library has methods to serialize data into a string buffer or a<br />
string and then methods to deserialize the string buffer or string<br />
back into a PMT. We use this extensively in the metadata files (see<br />
[[Metadata Information]]).<br />
<br />
* bool pmt::serialize(pmt_t obj, std::streambuf &sink)<br />
* std::string pmt::serialize_str(pmt_t obj)<br />
* pmt_t pmt::deserialize(std::streambuf &source)<br />
* pmt_t pmt::deserialize_str(std::string str)<br />
<br />
For example, we will serialize the data above to make it into a string<br />
ready to be written to a file and then deserialize it back to its<br />
original PMT.<br />
<br />
<syntaxhighlight lang="python"><br />
import pmt<br />
<br />
key0 = pmt.intern("int")<br />
val0 = pmt.from_long(123)<br />
<br />
key1 = pmt.intern("double")<br />
val1 = pmt.from_double(5.4321)<br />
<br />
# Make an empty dictionary<br />
a = pmt.make_dict()<br />
<br />
# Add a key:value pair to the dictionary<br />
a = pmt.dict_add(a, key0, val0)<br />
a = pmt.dict_add(a, key1, val1)<br />
<br />
print a<br />
<br />
ser_str = pmt.serialize_str(a)<br />
print ser_str<br />
<br />
b = pmt.deserialize_str(ser_str)<br />
print b<br />
</syntaxhighlight><br />
<br />
The line where we 'print ser_str' will print and parts will be<br />
readable, but the point of serializing is not to make a human-readable<br />
string. This is only done here as a test.<br />
<br />
== Printing ==<br />
<br />
In Python, the __repr__ function of a PMT object is overloaded to call<br />
'pmt::write_string'. This means that any time we call a formatted<br />
printing operation on a PMT object, the PMT library will properly<br />
format the object for display.<br />
<br />
In C++, we can use the 'pmt::print(object)' function or print the<br />
contents is using the overloaded "<<" operator with a stream buffer<br />
object. In C++, we can inline print the contents of a PMT like:<br />
<br />
<syntaxhighlight lang="c++"><br />
pmt::pmt_t a = pmt::from_double(1.0);<br />
std::cout << "The PMT a contains " << a << std::endl;<br />
</syntaxhighlight><br />
<br />
=== Collection Notation ===<br />
PMTs use a different bracket notation from what one might be use to in Python.<br />
<br />
<syntaxhighlight lang="plaintext"><br />
>>> my_dict<br />
((meaning . 42))<br />
>>> my_vector<br />
#[1 2 3 4]<br />
>>> pmt.make_tuple(pmt.from_long(321), pmt.from_float(3.14))<br />
{321 3.14}<br />
>>> pmt.cons(pmt.from_long(1), pmt.from_long(2))<br />
(1 . 2)<br />
>>> my_pdu<br />
(() . #[1 2 3 4])<br />
</syntaxhighlight><br />
<br />
== Conversion between Python Objects and PMTs ==<br />
<br />
Although PMTs can be manipulated in Python using the Python versions<br />
of the C++ interfaces, there are some additional goodies that make it<br />
easier to work with PMTs in python. There are functions to automate<br />
the conversion between PMTs and Python types for booleans, strings,<br />
integers, longs, floats, complex numbers, dictionaries, lists, tuples<br />
and combinations thereof.<br />
<br />
Two functions capture most of this functionality:<br />
<br />
<syntaxhighlight lang="python"><br />
pmt.to_pmt # Converts a python object to a PMT.<br />
pmt.to_python # Converts a PMT into a python object.<br />
</syntaxhighlight></div>Solomonbstonerhttps://wiki.gnuradio.org/index.php?title=Polymorphic_Types_(PMTs)&diff=8526Polymorphic Types (PMTs)2021-05-07T10:45:39Z<p>Solomonbstoner: Fixed the comment in the cpp code</p>
<hr />
<div>[[Category:Usage Manual]]<br />
== Introduction ==<br />
<br />
Polymorphic Types are used as the carrier of data from one block/thread to another such as stream tags and message passing interfaces. <br />
PMT data types can represent a variety of data ranging from the Booleans to dictionaries. <br />
They are heavily used in the stream tags and message passing interfaces. <br />
In a sense, PMTs are a way to extend C++' strict typing with something more flexible. <br />
The most complete list of PMT function is, of course, the source code, specifically the header file [https://gnuradio.org/doc/doxygen/pmt_8h.html pmt.h]. <br />
This page summarizes the most important features and points of PMTs.<br />
<br />
Let's dive straight into some Python code and see how we can use PMTs:<br />
<br />
<syntaxhighlight lang="python"><br />
>>> import pmt<br />
>>> P = pmt.from_long(23)<br />
>>> type(P)<br />
<class 'pmt.pmt_swig.swig_int_ptr'><br />
>>> print P<br />
23<br />
>>> P2 = pmt.from_complex(1j)<br />
>>> type(P2)<br />
<class 'pmt.pmt_swig.swig_int_ptr'><br />
>>> print P2<br />
0+1i<br />
>>> pmt.is_complex(P2)<br />
True<br />
</syntaxhighlight><br />
First, the <code>pmt</code> module is imported. We assign two values (<code>P</code> and <code>P2</code>) with PMTs using the <code>from_long()</code> and <code>from_complex()</code> calls, respectively. As we can see, they are both of the same type! This means we can pass these variables to C++ through SWIG, and C++ can handle this type accordingly.<br />
<br />
The same code as above in C++ would look like this:<br />
<syntaxhighlight lang="c++"><br />
#include <pmt/pmt.h><br />
// [...]<br />
pmt::pmt_t P = pmt::from_long(23);<br />
std::cout << P << std::endl;<br />
pmt::pmt_t P2 = pmt::from_complex(gr_complex(0, 1)); <br />
// Alternatively: pmt::from_complex(0, 1)<br />
std::cout << P2 << std::endl;<br />
std::cout << pmt::is_complex(P2) << std::endl;<br />
</syntaxhighlight><br />
Two things stand out in both Python and C++: First, we can simply print the contents of a PMT. How is this possible? Well, the PMTs have in-built capability to cast their value to a string (this is not possible with all types, though). Second, PMTs must obviously know their type, so we can query that, e.g. by calling the <code>is_complex()</code> method.<br />
<br />
Note: When running the above as a standalone, the compiler command will look something like <code>g++ pmt_tutorial.cpp -I$(gnuradio-config-info --prefix)/include -lgnuradio-pmt -o pmt_tutorial</code><br />
<br />
When assigning a non-PMT value to a PMT, we can use the <code>from_*</code> methods, and use the <code>to_*</code> methods to convert back:<br />
<syntaxhighlight lang="c++"><br />
pmt::pmt_t P_int = pmt::from_long(42);<br />
int i = pmt::to_long(P_int);<br />
pmt::pmt_t P_double = pmt::from_double(0.2);<br />
double d = pmt::to_double(P_double);<br />
pmt::pmt_t P_double = pmt::mp(0.2);<br />
</syntaxhighlight><br />
<br />
The last row shows the [http://gnuradio.org/doc/doxygen/namespacepmt.html#a90faad6086ac00280e0cfd8bb541bd64 pmt::mp()] shorthand function. It basically saves some typing, as it infers the correct <code>from_</code> function from the given type.<br />
<br />
String types play a bit of a special role in PMTs, as we will see<br />
later, and have their own converter:<br />
<br />
<syntaxhighlight lang="c++"><br />
pmt::pmt_t P_str = pmt::string_to_symbol("spam");<br />
pmt::pmt_t P_str2 = pmt::intern("spam");<br />
std::string str = pmt::symbol_to_string(P_str);<br />
</syntaxhighlight><br />
The pmt::intern is another way of saying pmt::string_to_symbol.<br />
<br />
See the [https://www.gnuradio.org/doc/doxygen/namespacepmt.html PMT docs] and the header file [http://gnuradio.org/doc/doxygen/pmt_8h.html pmt.h] for a full list of conversion functions.<br />
<br />
In Python, we can make use of the dynamic typing, and there's actually a<br />
helper function to do these conversions (C++ also has a helper<br />
function for converting to PMTs called pmt::mp(), but it's less<br />
powerful, and not quite as useful, because types are always strictly<br />
known in C++):<br />
<br />
<syntaxhighlight lang="python"><br />
P_int = pmt.to_pmt(42)<br />
i = pmt.to_python(P_int)<br />
P_double = pmt.to_pmt(0.2)<br />
d = pmt.to_double(P_double)<br />
</syntaxhighlight><br />
<br />
On a side note, there are three useful PMT constants, which can be<br />
used in both Python and C++ domains. In C++, these can be used as<br />
such:<br />
<br />
<syntaxhighlight lang="c++"><br />
pmt::pmt_t P_true = pmt::PMT_T;<br />
pmt::pmt_t P_false = pmt::PMT_F;<br />
pmt::pmt_t P_nil = pmt::PMT_NIL;<br />
</syntaxhighlight><br />
<br />
In Python:<br />
<br />
<syntaxhighlight lang="python"><br />
P_true = pmt.PMT_T<br />
P_false = pmt.PMT_F<br />
P_nil = pmt.PMT_NIL<br />
</syntaxhighlight><br />
<br />
<code>pmt.PMT_T</code> and <code>pmt.PMT_F</code> are boolean PMT types representing True and False, respectively. The PMT_NIL is like a NULL or None and can be used for default arguments or return values, often indicating an error has occurred.<br />
<br />
To be able to go back to C++ data types, we need to be able to find<br />
out the type from a PMT. The family of <code>is_*</code> methods helps us do that:<br />
<br />
<syntaxhighlight lang="c++"><br />
double d;<br />
if (pmt::is_integer(P)) {<br />
d = (double) pmt::to_long(P);<br />
} else if (pmt::is_real(P)) {<br />
d = pmt::to_double(P);<br />
} else {<br />
// We really expected an integer or a double here, so we don't know what to do<br />
throw std::runtime_error("expected an integer!");<br />
}<br />
</syntaxhighlight><br />
<br />
It is important to do type checking since we cannot unpack a PMT of<br />
the wrong data type.<br />
<br />
We can compare PMTs without knowing their type by using the <code>pmt::equal()</code> function:<br />
<br />
<syntaxhighlight lang="c++"><br />
if (pmt::eq(P_int, P_double)) {<br />
std::cout << "Equal!" << std::endl; // This line will never be reached<br />
</syntaxhighlight><br />
<br />
There are more equality functions, which compare different things: [http://gnuradio.org/doc/doxygen/namespacepmt.html#a5c28635e14287cc0e2f762841c11032f <code>pmt::eq()</code>] and [http://gnuradio.org/doc/doxygen/namespacepmt.html#a25467c81e1c5f4619a9cabad7a88eed5 <code>pmt::eqv()</code>]. We won't need these for this tutorial.<br />
<br />
The rest of this page provides more depth into how to handle different<br />
data types with the PMT library.<br />
<br />
=== More Complex Types ===<br />
<br />
PMTs can hold a variety of types. Using the Python method <code>pmt.to_pmt()</code>, we can convert most of Pythons standard types out-of-the-box:<br />
<br />
<pre>P_tuple = pmt.to_pmt((1, 2, 3, 'spam', 'eggs'))<br />
P_dict = pmt.to_pmt({'spam': 42, 'eggs': 23})</pre><br />
But what does this mean in the C++ domain? Well, there are PMT types that define [http://gnuradio.org/doc/doxygen/namespacepmt.html#a32895cc5a614a46b66b869c4a7bd283c tuples] and [http://gnuradio.org/doc/doxygen/namespacepmt.html#aba10563e3ab43b8d52f9cb13132047cf dictionaries], keys and values being PMTs, again.<br /><br />
So, to create the tuple from the Python example, the C++ code would look like this:<br />
<br />
<pre>pmt::pmt_t P_tuple = pmt::make_tuple(pmt::from_long(1), pmt::from_long(2), pmt::from_long(3), pmt::string_to_symbol(&quot;spam&quot;), pmt::string_to_symbol(&quot;eggs&quot;))</pre><br />
For the dictionary, it's a bit more complex:<br />
<br />
<pre>pmt::pmt_t P_dict = pmt::make_dict();<br />
P_dict = pmt::dict_add(P_dict, pmt::string_to_symbol(&quot;spam&quot;), pmt::from_long(42));<br />
P_dict = pmt::dict_add(P_dict, pmt::string_to_symbol(&quot;eggs&quot;), pmt::from_long(23));</pre><br />
As you can see, we first need to create a dictionary, then assign every key/value pair individually.<br />
<br />
A variant of tuples are ''vectors''. Like Python's tuples and lists, PMT vectors are mutable, whereas PMT tuples are not. In fact, PMT vectors are the only PMT data types that are mutable. When changing the value or adding an item to a dictionary, we are actually creating a new PMT.<br />
<br />
To create a vector, we can initialize it to a certain lengths, and fill all elements with an initial value. We can then change items or reference them:<br />
<br />
<pre>pmt::pmt_t P_vector = pmt::make_vector(5, pmt::from_long(23)); // Creates a vector with 5 23's as PMTs<br />
pmt::vector_set(P_vector, 0, pmt::from_long(42)); // Change the first element to a 42<br />
std::cout &lt;&lt; pmt::vector_ref(P_vector, 0); // Will print 42</pre><br />
In Python, we can do all these steps (using <code>pmt.make_vector()</code> etc.), or convert a list:<br />
<br />
<pre>P_vector = pmt.to_pmt([42, 23, 23, 23, 23])</pre><br />
Vectors are also different from tuples in a sense that we can '''directly''' load data types into the elements, which don't have to be PMTs.<br /><br />
Say we want to pass a series of 8 float values to another block (these might be characteristics of a filter, for example). It would be cumbersome to convert every single element to and from PMTs, since all elements of the vector are the same type.<br />
<br />
We can use special vector types for this case:<br />
<br />
<pre>pmt::pmt_t P_f32vector = pmt::make_f32vector(8, 5.0); // Creates a vector with 8 5.0s's as floats<br />
pmt::f32vector_set(P_f32vector, 0, 2.0); // Change the first element to a 2.0<br />
float f = f32vector_ref(P_f32vector, 0);<br />
std::cout &lt;&lt; f &lt;&lt; std::endl; // Prints 2.0<br />
size_t len;<br />
float *fp = pmt::f32vector_elements(P_f32vector, len);<br />
for (size_t i = 0; i &lt; len; i++)<br />
std::cout &lt;&lt; fp[i] &lt;&lt; std::endl; // Prints all elements from P_f32vector, one after another</pre><br />
Python has a similar concept: [http://docs.scipy.org/doc/numpy/reference/generated/numpy.array.html Numpy arrays]. As usual, the PMT library understands this and converts as expected:<br />
<br />
<pre>P_f32vector = pmt.to_pmt(numpy.array([2.0, 5.0, 5.0, 5.0, 5.0], dtype=numpy.float32))<br />
print pmt.is_f32vector(P_f32vector) # Prints 'True'</pre><br />
Here, 'f32' stands for 'float, 32 bits'. PMTs know about most typical fixed-width data types, such as 'u8' (unsigned 8-bit character) or 'c32' (complex with 32-bit floats for each I and Q). Consult the [http://gnuradio.org/doc/doxygen/namespacepmt.html manual] for a full list of types.<br />
<br />
The most generic PMT type is probably the blob (binary large object). Use this with care - it allows us to pass around anything that can be represented in memory.<br />
<br />
== PMT Data Type ==<br />
<br />
All PMTs are of the type pmt::pmt_t. This is an opaque container and<br />
PMT functions must be used to manipulate and even do things like<br />
compare PMTs. PMTs are also ''immutable'' (except PMT vectors). We<br />
never change the data in a PMT; instead, we create a new PMT with the<br />
new data. The main reason for this is thread safety. We can pass PMTs<br />
as tags and messages between blocks and each receives its own copy<br />
that we can read from. However, we can never write to this object, and<br />
so if multiple blocks have a reference to the same PMT, there is no<br />
possibility of thread-safety issues of one reading the PMT data while<br />
another is writing the data. If a block is trying to write new data to<br />
a PMT, it actually creates a new PMT to put the data into. Thus we<br />
allow easy access to data in the PMT format without worrying about<br />
mutex locking and unlocking while manipulating them.<br />
<br />
PMTs can represent the following:<br />
<br />
* Boolean values of true/false<br />
* Strings (as symbols)<br />
* Integers (long and uint64)<br />
* Floats (as doubles)<br />
* Complex (as two doubles)<br />
* Pairs<br />
* Tuples<br />
* Vectors (of PMTs)<br />
* Uniform vectors (of any standard data type)<br />
* Dictionaries (list of key:value pairs)<br />
* Any (contains a boost::any pointer to hold anything)<br />
<br />
The PMT library also defines a set of functions that operate directly<br />
on PMTs such as:<br />
<br />
* Equal/equivalence between PMTs<br />
* Length (of a tuple or vector)<br />
* Map (apply a function to all elements in the PMT)<br />
* Reverse<br />
* Get a PMT at a position in a list<br />
* Serialize and deserialize<br />
* Printing<br />
<br />
The constants in the PMT library are:<br />
<br />
* pmt::PMT_T - a PMT True<br />
* pmt::PMT_F - a PMT False<br />
* pmt::PMT_NIL - an empty PMT (think Python's 'None')<br />
<br />
== Inserting and Extracting Data ==<br />
<br />
Use pmt.h for a complete guide to the list of functions used to create<br />
PMTs and get the data from a PMT. When using these functions, remember<br />
that while PMTs are opaque and designed to hold any data, the data<br />
underneath is still a C++ typed object, and so the right type of<br />
set/get function must be used for the data type.<br />
<br />
Typically, a PMT object can be made from a scalar item using a call<br />
like "pmt::from_<type>". Similarly, when getting data out of a<br />
PMT, we use a call like "pmt::to_<type>". For example:<br />
<syntaxhighlight lang="c++"><br />
double a = 1.2345;<br />
pmt::pmt_t pmt_a = pmt::from_double(a);<br />
double b = pmt::to_double(pmt_a);<br />
<br />
int c = 12345;<br />
pmt::pmt_t pmt_c = pmt::from_long(c);<br />
int d = pmt::to_long(pmt_c);<br />
</syntaxhighlight><br />
As a side-note, making a PMT from a complex number is not obvious:<br />
<syntaxhighlight lang="c++"><br />
std::complex<double> a(1.2, 3.4);<br />
pmt::pmt_t pmt_a = pmt::make_rectangular(a.real(), b.imag());<br />
std::complex<double> b = pmt::to_complex(pmt_a);<br />
</syntaxhighlight><br />
Pairs, dictionaries, and vectors have different constructors and ways<br />
to manipulate them, and these are explained in their own sections.<br />
<br />
== Strings ==<br />
<br />
PMTs have a way of representing short strings. These strings are<br />
actually stored as interned symbols in a hash table, so in other<br />
words, only one PMT object for a given string exists. If creating a<br />
new symbol from a string, if that string already exists in the hash<br />
table, the constructor will return a reference to the existing PMT.<br />
<br />
We create strings with the following functions, where the second<br />
function, pmt::intern, is simply an alias of the first.<br />
<syntaxhighlight lang="c++"><br />
pmt::pmt_t str0 = pmt::string_to_symbol(std::string("some string"));<br />
pmt::pmt_t str1 = pmt::intern(std::string("some string"));<br />
</syntaxhighlight><br />
The string can be retrieved using the inverse function:<br />
<syntaxhighlight lang="c++"><br />
std::string s = pmt::symbol_to_string(str0);<br />
</syntaxhighlight><br />
== Tests and Comparisons ==<br />
<br />
The PMT library comes with a number of functions to test and compare<br />
PMT objects. In general, for any PMT data type, there is an equivalent<br />
"pmt::is_<type>". We can use these to test the PMT before trying<br />
to access the data inside. Expanding our examples above, we have:<br />
<syntaxhighlight lang="c++"><br />
pmt::pmt_t str0 = pmt::string_to_symbol(std::string("some string"));<br />
if(pmt::is_symbol(str0))<br />
std::string s = pmt::symbol_to_string(str0);<br />
<br />
double a = 1.2345;<br />
pmt::pmt_t pmt_a = pmt::from_double(a);<br />
if(pmt::is_double(pmt_a))<br />
double b = pmt::to_double(pmt_a);<br />
<br />
int c = 12345;<br />
pmt::pmt_t pmt_c = pmt::from_long(c);<br />
if(pmt::is_long(pmt_a))<br />
int d = pmt::to_long(pmt_c);<br />
<br />
\\ This will fail the test. Otherwise, trying to coerce \b pmt_c as a<br />
\\ double when internally it is a long will result in an exception.<br />
if(pmt::is_double(pmt_a))<br />
double d = pmt::to_double(pmt_c);<br />
</syntaxhighlight><br />
<br />
== Dictionaries ==<br />
<br />
PMT dictionaries are lists of key:value pairs. They have a<br />
well-defined interface for creating, adding, removing, and accessing<br />
items in the dictionary. Note that every operation that changes the<br />
dictionary both takes a PMT dictionary as an argument and returns a<br />
PMT dictionary. The dictionary used as an input is not changed and the<br />
returned dictionary is a new PMT with the changes made there.<br />
<br />
The following is a list of PMT dictionary functions. View each function in the [https://www.gnuradio.org/doc/doxygen/index.html GNU Radio C++ Manual]<br />
to get more information on what each does.<br />
<br />
* bool pmt::is_dict(const pmt_t &obj)<br />
* pmt_t pmt::make_dict()<br />
* pmt_t pmt::dict_add(const pmt_t &dict, const pmt_t &key, const pmt_t &value)<br />
* pmt_t pmt::dict_delete(const pmt_t &dict, const pmt_t &key)<br />
* bool pmt::dict_has_key(const pmt_t &dict, const pmt_t &key)<br />
* pmt_t pmt::dict_ref(const pmt_t &dict, const pmt_t &key, const pmt_t &not_found)<br />
* pmt_t pmt::dict_items(pmt_t dict)<br />
* pmt_t pmt::dict_keys(pmt_t dict)<br />
* pmt_t pmt::dict_values(pmt_t dict)<br />
<br />
This example does some basic manipulations of PMT dictionaries in<br />
Python. Notice that we pass the dictionary ''a'' and return the results<br />
to ''a''. This still creates a new dictionary and removes the local<br />
reference to the old dictionary. This just keeps our number of<br />
variables small.<br />
<br />
<syntaxhighlight lang="python"><br />
import pmt<br />
<br />
key0 = pmt.intern("int")<br />
val0 = pmt.from_long(123)<br />
val1 = pmt.from_long(234)<br />
<br />
key1 = pmt.intern("double")<br />
val2 = pmt.from_double(5.4321)<br />
<br />
# Make an empty dictionary<br />
a = pmt.make_dict()<br />
<br />
# Add a key:value pair to the dictionary<br />
a = pmt.dict_add(a, key0, val0)<br />
print a<br />
<br />
# Add a new value to the same key;<br />
# new dict will still have one item with new value<br />
a = pmt.dict_add(a, key0, val1)<br />
print a<br />
<br />
# Add a new key:value pair<br />
a = pmt.dict_add(a, key1, val2)<br />
print a<br />
<br />
# Test if we have a key, then delete it<br />
print pmt.dict_has_key(a, key1)<br />
a = pmt.dict_delete(a, key1)<br />
print pmt.dict_has_key(a, key1)<br />
<br />
ref = pmt.dict_ref(a, key0, pmt.PMT_NIL)<br />
print ref<br />
<br />
# The following should never print<br />
if(pmt.dict_has_key(a, key0) and pmt.eq(ref, pmt.PMT_NIL)):<br />
print "Trouble! We have key0, but it returned PMT_NIL"<br />
</syntaxhighlight><br />
<br />
== Vectors ==<br />
<br />
PMT vectors come in two forms: vectors of PMTs and vectors of uniform<br />
data. The standard PMT vector is a vector of PMTs, and each PMT can be<br />
of any internal type. On the other hand, uniform PMTs are of a<br />
specific data type which come in the form:<br />
<br />
* (u)int8<br />
* (u)int16<br />
* (u)int32<br />
* (u)int64<br />
* float32<br />
* float64<br />
* complex 32 (std::complex<float>)<br />
* complex 64 (std::complex<double>)<br />
<br />
That is, the standard sizes of integers, floats, and complex types of<br />
both signed and unsigned.<br />
<br />
Vectors have a well-defined interface that allows us to make, set,<br />
get, and fill them. We can also get the length of a vector with<br />
pmt::length, or in Python:<br />
<br />
pmt_t p1 = pmt_integer(1);<br />
pmt_t p2 = pmt_integer(2);<br />
pmt_t p3 = pmt_integer(3);<br />
<br />
pmt_t p_list = pmt_list3(p1, p2, p3);<br />
<br />
pmt_length(p_list); // Returns 3<br />
<br />
For standard vectors, these functions look like:<br />
<br />
* bool pmt::is_vector(pmt_t x)<br />
* pmt_t pmt::make_vector(size_t k, pmt_t fill)<br />
* pmt_t pmt::vector_ref(pmt_t vector, size_t k)<br />
* void pmt::vector_set(pmt_t vector, size_t k, pmt_t obj)<br />
* void pmt::vector_fill(pmt_t vector, pmt_t fill)<br />
<br />
Uniform vectors have the same types of functions, but they are data<br />
type-dependent. The following list tries to explain them where you<br />
substitute the specific data type prefix for \a dtype (prefixes being:<br />
u8, u16, u32, u64, s8, s16, s32, s64, f32, f64, c32, c64).<br />
<br />
* bool pmt::is_(dtype)vector(pmt_t x)<br />
* pmt_t pmt::make_(dtype)vector(size_t k, (dtype) fill)<br />
* pmt_t pmt::init_(dtype)vector(size_t k, const (dtype*) data)<br />
* pmt_t pmt::init_(dtype)vector(size_t k, const std::vector<dtype> data)<br />
* pmt_t pmt::(dtype)vector_ref(pmt_t vector, size_t k)<br />
* void pmt::(dtype)vector_set(pmt_t vector, size_t k, (dtype) x)<br />
* const dtype* pmt::(dtype)vector_elements(pmt_t vector, size_t &len)<br />
* dtype* pmt::(dtype)vector_writable_elements(pmt_t vector, size_t &len)<br />
<br />
List of available pmt::is_(dtype)vector(pmt_t x) methods from <code>pmt.h</code>:<br><br />
<br />
bool pmt::is_uniform_vector()<br />
bool pmt::is_u8vector()<br />
bool pmt::is_s8vector()<br />
bool pmt::is_u16vector()<br />
bool pmt::is_s16vector()<br />
bool pmt::is_u32vector()<br />
bool pmt::is_s32vector()<br />
bool pmt::is_u64vector()<br />
bool pmt::is_s64vector()<br />
bool pmt::is_f32vector()<br />
bool pmt::is_f64vector()<br />
bool pmt::is_c32vector()<br />
bool pmt::is_c64vector()<br />
<br />
'''Note:''' We break the contract with vectors. The 'set' functions<br />
actually change the data underneath. It is important to keep track of<br />
the implications of setting a new value as well as accessing the<br />
'vector_writable_elements' data. Since these are mostly standard data<br />
types, sets and gets are atomic, so it is unlikely to cause a great<br />
deal of harm. But it's only unlikely, not impossible. Best to use<br />
mutexes whenever manipulating data in a vector.<br />
<br />
=== BLOB ===<br />
<br />
A BLOB is a 'binary large object' type. In PMT's, this is actually<br />
just a thin wrapper around a u8vector.<br />
<br />
== Pairs ==<br />
<br />
A concept that originates in Lisp dialects are [http://en.wikipedia.org/wiki/Cons ''pairs'' and ''cons'']. The simplest explanation is just that: If you combine two PMTs, they form a new PMT, which is a pair (or cons) of those two PMTs (don't worry about the weird name, a lot of things originating in Lisp have weird names. Think of a 'construct').<br />
<br />
Similarly to vectors or tuples, pairs are a useful way of packing several components of a message into a single PMT. Using the PMTs generated in the previous section, we can combine two of these to form a pair, here in Python:<br />
<br />
<pre>P_pair = pmt.cons(pmt.string_to_symbol(&quot;taps&quot;), P_f32vector)<br />
print pmt.is_pair(P_pair) # Prints 'true'</pre><br />
You can combine PMTs as tuples, dictionaries, vectors, or pairs, it's just a matter of taste. This construct is well-established though, and as such used in GNU Radio quite often.<br />
<br />
So how do we deconstruct a pair? That's what the <code>car</code> and <code>cdr</code> functions do. Let's deconstruct that previous pair in C++:<br />
<br />
<pre>pmt::pmt_t P_key = pmt::car(P_pair);<br />
pmt::pmt_t P_f32vector2 = pmt::cdr(P_pair);<br />
std::cout &lt;&lt; P_key &lt;&lt; std::endl; // Will print 'taps' using the PMT automatic conversion to strings</pre><br />
<br />
Here is a summary of the pairs-related functions in C++ and Python:<br />
<br />
* bool pmt::is_pair(const pmt_t &obj): Return true if obj is a pair, else false<br />
* pmt_t pmt::cons(const pmt_t &x, const pmt_t &y): construct new pair<br />
* pmt_t pmt::car(const pmt_t &pair): get the car of the pair (first object)<br />
* pmt_t pmt::cdr(const pmt_t &pair): get the cdr of the pair (second object)<br />
* void pmt::set_car(pmt_t pair, pmt_t value): Stores value in the car field<br />
* void pmt::set_cdr(pmt_t pair, pmt_t value): Stores value in the cdr field<br />
<br />
And in Python we have:<br />
<syntaxhighlight lang="python"><br />
pmt.is_pair(pair_obj) # Return True if is a pair, else False (warning: also returns True for a dict)<br />
pmt.cons(x, y) # Return a newly allocated pair whose car is x and whose cdr is y<br />
pmt.car(pair_obj) # If is a pair, return the car, otherwise raise wrong_type<br />
pmt.cdr(pair_obj) # If is a pair, return the cdr, otherwise raise wrong_type<br />
pmt.set_car(pair_obj, value) # Store value in car field<br />
pmt.set_cdr(pair_obj, value) # Store value in cdr field<br />
</syntaxhighlight><br />
<br />
For more advanced pair manipulation, refer to the [http://gnuradio.org/doc/doxygen/namespacepmt.html#a7ab95721db5cbda1852f13a92eee5362 documentation] and the [https://en.wikipedia.org/wiki/Car_and_cdr Wikipedia page for car and cdr].<br />
<br />
== Serializing and Deserializing ==<br />
<br />
It is often important to hide the fact that we are working with PMTs<br />
to make them easier to transmit, store, write to file, etc. The PMT<br />
library has methods to serialize data into a string buffer or a<br />
string and then methods to deserialize the string buffer or string<br />
back into a PMT. We use this extensively in the metadata files (see<br />
[[Metadata Information]]).<br />
<br />
* bool pmt::serialize(pmt_t obj, std::streambuf &sink)<br />
* std::string pmt::serialize_str(pmt_t obj)<br />
* pmt_t pmt::deserialize(std::streambuf &source)<br />
* pmt_t pmt::deserialize_str(std::string str)<br />
<br />
For example, we will serialize the data above to make it into a string<br />
ready to be written to a file and then deserialize it back to its<br />
original PMT.<br />
<br />
<syntaxhighlight lang="python"><br />
import pmt<br />
<br />
key0 = pmt.intern("int")<br />
val0 = pmt.from_long(123)<br />
<br />
key1 = pmt.intern("double")<br />
val1 = pmt.from_double(5.4321)<br />
<br />
# Make an empty dictionary<br />
a = pmt.make_dict()<br />
<br />
# Add a key:value pair to the dictionary<br />
a = pmt.dict_add(a, key0, val0)<br />
a = pmt.dict_add(a, key1, val1)<br />
<br />
print a<br />
<br />
ser_str = pmt.serialize_str(a)<br />
print ser_str<br />
<br />
b = pmt.deserialize_str(ser_str)<br />
print b<br />
</syntaxhighlight><br />
<br />
The line where we 'print ser_str' will print and parts will be<br />
readable, but the point of serializing is not to make a human-readable<br />
string. This is only done here as a test.<br />
<br />
== Printing ==<br />
<br />
In Python, the __repr__ function of a PMT object is overloaded to call<br />
'pmt::write_string'. This means that any time we call a formatted<br />
printing operation on a PMT object, the PMT library will properly<br />
format the object for display.<br />
<br />
In C++, we can use the 'pmt::print(object)' function or print the<br />
contents is using the overloaded "<<" operator with a stream buffer<br />
object. In C++, we can inline print the contents of a PMT like:<br />
<br />
<syntaxhighlight lang="c++"><br />
pmt::pmt_t a pmt::from_double(1.0);<br />
std::cout << "The PMT a contains " << a << std::endl;<br />
</syntaxhighlight><br />
<br />
=== Collection Notation ===<br />
PMTs use a different bracket notation from what one might be use to in Python.<br />
<br />
<syntaxhighlight lang="plaintext"><br />
>>> my_dict<br />
((meaning . 42))<br />
>>> my_vector<br />
#[1 2 3 4]<br />
>>> pmt.make_tuple(pmt.from_long(321), pmt.from_float(3.14))<br />
{321 3.14}<br />
>>> pmt.cons(pmt.from_long(1), pmt.from_long(2))<br />
(1 . 2)<br />
>>> my_pdu<br />
(() . #[1 2 3 4])<br />
</syntaxhighlight><br />
<br />
== Conversion between Python Objects and PMTs ==<br />
<br />
Although PMTs can be manipulated in Python using the Python versions<br />
of the C++ interfaces, there are some additional goodies that make it<br />
easier to work with PMTs in python. There are functions to automate<br />
the conversion between PMTs and Python types for booleans, strings,<br />
integers, longs, floats, complex numbers, dictionaries, lists, tuples<br />
and combinations thereof.<br />
<br />
Two functions capture most of this functionality:<br />
<br />
<syntaxhighlight lang="python"><br />
pmt.to_pmt # Converts a python object to a PMT.<br />
pmt.to_python # Converts a PMT into a python object.<br />
</syntaxhighlight></div>Solomonbstonerhttps://wiki.gnuradio.org/index.php?title=OutOfTreeModules&diff=8524OutOfTreeModules2021-05-06T11:27:50Z<p>Solomonbstoner: Add possible error the reader might encounter after generating a yml file from the template implementation cpp file on GRC 3.9</p>
<hr />
<div>[[Category:Guide]]<br />
<br />
'''Extending GNU Radio with own functionality and blocks'''<br />
<br />
This article borrows heavily from the original (but very outdated) &quot;How to write a block?&quot; written by Eric Blossom.<br />
<br />
== What is an out-of-tree module? ==<br />
<br />
An out-of-tree module is a GNU Radio component that does not live within the GNU Radio source tree. Typically, if you want to extend GNU Radio with your own functions and blocks, such a module is what you create (i.e. you wouldn't usually add stuff to the actual GNU Radio source tree unless you're planning to submit it to the devs for upstream integration). This allows you to maintain the code yourself and have additional functionality alongside the main code.<br />
<br />
A lot of OOT projects are hosted at [http://cgran.org CGRAN] -- the Comprehensive GNU Radio Archive Network. CGRAN projects are all available through our tool [http://gnuradio.org/pybombs PyBOMBS]. In fact, when you add your project to the [https://github.com/gnuradio/gr-etcetera PyBOMBS recipe repo], it will automatically update the CGRAN website.<br />
<br />
For example of such a module is the [https://www.cgran.org/14680/ GNU Radio Digital Audio Broadcasting module], which extends GNU Radio with everything needed to get audio from DAB and DAB+. When installed, you have more blocks available (e.g. in the [[GNURadioCompanion|GNU Radio companion]]) which behave just like the rest of GNU Radio; however, the developers are different people.<br />
<br />
== Tools and resources at my disposal ==<br />
<br />
There are a couple of tools, scripts, and documents that are available as 3rd-party programs or as part of GNU Radio.<br />
<br />
=== gr_modtool - The swiss army knife of module editing ===<br />
<br />
When developing a module, there's a lot of boring, monotonous work involved: boilerplate code, makefile editing, etc. gr_modtool is a script which aims to help with all these things by automatically editing makefiles, using templates, and doing as much work as possible for the developer such that you can jump straight into the DSP coding.<br />
<br />
Note that gr_modtool makes a lot of assumptions on what the code looks like. The more your module is custom and has specific changes, the less useful gr_modtool will be, but it is probably the best place to start with any new module or block.<br />
<br />
gr_modtool is now available in the GNU Radio source tree and is installed by default.<br />
<br />
=== Developer resources on the wiki ===<br />
<br />
Most important is definitely the [[BlocksCodingGuide|block coding guide]]. While this is written for the GNU Radio main tree, this should also be applied to all modules. Specifically, have a look at the naming conventions!<br />
<br />
=== CMake, make, etc. ===<br />
<br />
GNU Radio uses CMake as a build system. Building a module, therefore, requires you to have cmake installed, and whatever build manager you prefer (most often this is 'make', but you could also be using Eclipse or MS Visual Studio).<br />
<br />
== Tutorial 1: Creating an out-of-tree module ==<br />
<br />
In the following tutorials, we will use an out-of-tree module called '''howto'''. The first step is to create this module.<br />
<br />
With gr_modtool, this is dead easy. Just point your command line wherever you want your new module directory (this should be outside the GNU Radio source tree!), and go:<br />
<br />
<pre>% gr_modtool newmod howto<br />
Creating out-of-tree module in ./gr-howto... Done.<br />
Use 'gr_modtool add' to add a new block to this currently empty module.</pre><br />
If all went well, you now have a new directory called <code>gr-howto</code> in which we will work for the other tutorials.<br />
<br />
== Structure of a module ==<br />
<br />
Let's jump straight into the gr-howto module and see what it's made up of:<br />
<br />
<pre>gr-howto % ls<br />
apps cmake CMakeLists.txt docs examples grc include lib python swig</pre><br />
It consists of several subdirectories. Anything that will be written in C++ (or C, or any language that is not Python) is put into <code>lib/</code>. For C++ files, we usually have headers which are put into <code>include/</code> (if they are to be exported) or also in <code>lib/</code> (if they're only relevant during compile time, but are not installed later, such as <code>_impl.h</code> files. You'll see what that is in the next tutorial).<br />
<br />
Of course, Python stuff goes into the <code>python/</code> directory. This includes unit tests (which are not installed) and parts of the Python module which are installed.<br />
<br />
You probably know already that GNU Radio blocks are available in Python even if they were written in C++. This is done by the help of SWIG, the simplified wrapper and interface generator, which automatically creates glue code to make this possible. SWIG needs some instructions on how to do this, which are put into the <code>swig/</code> subdirectory. Unless doing something extra clever with your block, you will not need to go into the <code>swig/</code> directory; gr_modtool handles all of that for us.<br />
<br />
If you want your blocks to be available in the [[GNURadioCompanion|GNU Radio companion]], the graphical UI for GNU Radio, you need to add descriptions of the blocks and put them into <code>grc/</code>. Prior to version 3.8 these descriptions were XML files, but from 3.8 onward they use [[YAML_GRC|YAML instead]].<br />
<br />
For documentation, <code>docs/</code> contains some instructions on how to extract documentation from the C++ files and Python files (we use Doxygen and Sphinx for this) and also make sure they're available as docstrings in Python. Of course, you can add custom documentation here as well.<br />
<br />
The <code>apps/</code> subdir contains any complete applications (both for GRC and standalone executables) which are installed to the system alongside with the blocks.<br />
<br />
The directory, <code>examples/</code> can be used to save (guess what) examples, which are a great addendum to documentation because other developers can simply look straight at the code to see how your blocks are used.<br />
<br />
The build system brings some baggage along, as well: the <code>CMakeLists.txt</code> file (one of which is present in every subdirectory) and the <code>cmake/</code> folder. You can ignore the latter for now, as it brings along mainly instructions for CMake on how to find GNU Radio libraries etc. The CMakeLists.txt files need to be edited a lot in order to make sure your module builds correctly.<br />
<br />
But one step at a time! Now, let's move on to our next tutorial.<br />
<br />
== Tutorial 2: Writing a block (square_ff) in C++ ==<br />
<br />
For our first example, we'll create a block that computes the square of its single float input. This block will accept a single float input stream and produce a single float output stream, i.e., for every incoming float item, we output one float item which is the square of that input item.<br />
<br />
Following the naming conventions, the block will be called <code>square_ff</code> because it has float inputs, float outputs.<br />
<br />
We are going to arrange that this block, as well as the others that we write in this article, end up in the <code>howto</code> Python module. This will allow us to access it from Python like this:<br />
<br />
<pre>import howto<br />
sqr = howto.square_ff()</pre><br />
=== Creating the files ===<br />
<br />
First step is to create empty files for the block and edit the CMakeLists.txt files.<br /><br />
Again, <code>gr_modtool</code> does the job. On the command line, go to the <code>gr-howto</code> directory and enter:<br />
<br />
<pre>gr-howto % gr_modtool add -t general -l cpp square_ff<br />
GNU Radio module name identified: howto<br />
Block/code identifier: square_ff<br />
Language: C++<br />
Please specify the copyright holder:<br />
Enter valid argument list, including default arguments: <br />
Add Python QA code? [Y/n] <br />
Add C++ QA code? [y/N] <br />
Adding file 'square_ff_impl.h'...<br />
Adding file 'square_ff_impl.cc'...<br />
Adding file 'square_ff.h'...<br />
Editing swig/howto_swig.i...<br />
Adding file 'qa_square_ff.py'...<br />
Editing python/CMakeLists.txt...<br />
Adding file 'howto_square_ff.xml'...<br />
Editing grc/CMakeLists.txt...</pre><br />
On the command line, we specify that we're adding a block, its type is 'general' (because we don't know what block types are, yet) and it is called <code>square_ff</code>. The block should be created in C++ and it currently has no specified copyright holder (by default, gr-module author is the copyright holder). <code>gr_modtool</code> then asks you if your block takes any arguments (it doesn't, so we leave that empty), whether or not we want QA code for Python (yes, we do) and for C++ (no, we don't right now).<br />
<br />
Now, have another look at the different CMakeLists.txt files and see what <code>gr_modtool</code> did. You can also see a lot of new files, which now have to be edited if we want the block to work.<br />
<br />
=== Test Driven Programming ===<br />
<br />
We could just start banging out the C++ code, but being highly evolved modern programmers, we're going to write the test code first. After all, we do have a good spec for the behavior: take a single stream of floats as the input and produce a single stream of floats as the output. The output should be the square of the input.<br />
<br />
How hard could this be? Turns out that this is easy! So, we open <code>python/qa_square_ff.py</code>, which we edit to look like this:<br />
<br />
<pre>from gnuradio import gr, gr_unittest<br />
from gnuradio import blocks<br />
import howto_swig as howto<br />
<br />
class qa_square_ff (gr_unittest.TestCase):<br />
<br />
def setUp (self):<br />
self.tb = gr.top_block ()<br />
<br />
def tearDown (self):<br />
self.tb = None<br />
<br />
def test_001_square_ff(self):<br />
src_data = (-3, 4, -5.5, 2, 3)<br />
expected_result = (9, 16, 30.25, 4, 9)<br />
src = blocks.vector_source_f(src_data)<br />
sqr = howto.square_ff()<br />
dst = blocks.vector_sink_f()<br />
self.tb.connect(src, sqr)<br />
self.tb.connect(sqr, dst)<br />
self.tb.run()<br />
result_data = dst.data()<br />
self.assertFloatTuplesAlmostEqual(expected_result, result_data, 6)<br />
<br />
if __name__ == '__main__':<br />
gr_unittest.run(qa_square_ff, &quot;qa_square_ff.xml&quot;)</pre><br />
<br />
For Gnuradio v 3.9 and up, use the following.<br />
<pre> <br />
from gnuradio import gr, gr_unittest<br />
from gnuradio import blocks<br />
<br />
try:<br />
from howto import square_ff<br />
except ImportError:<br />
import os<br />
import sys<br />
dirname, filename = os.path.split(os.path.abspath(__file__))<br />
sys.path.append(os.path.join(dirname, "bindings"))<br />
from howto import square_ff<br />
<br />
class qa_square_ff(gr_unittest.TestCase):<br />
<br />
def setUp(self):<br />
self.tb = gr.top_block()<br />
<br />
def tearDown(self):<br />
self.tb = None<br />
<br />
def test_instance(self):<br />
# FIXME: Test will fail until you pass sensible arguments to the constructor<br />
instance = square_ff()<br />
<br />
def test_001_square_ff(self): <br />
src_data = (-3, 4, -5.5, 2, 3)<br />
expected_result = (9, 16, 30.25, 4, 9)<br />
src = blocks.vector_source_f(src_data)<br />
sqr = square_ff()<br />
dst = blocks.vector_sink_f()<br />
self.tb.connect(src, sqr)<br />
self.tb.connect(sqr,dst)<br />
self.tb.run()<br />
result_data = dst.data()<br />
self.assertFloatTuplesAlmostEqual(expected_result, result_data, 6)<br />
# check data<br />
<br />
<br />
if __name__ == '__main__':<br />
gr_unittest.run(qa_square_ff, "qa_square_ff.yaml")<br />
</pre><br />
<br />
gr_unittest is an extension to the standard Python module unittest. gr_unittest adds support for checking approximate equality of tuples of float and complex numbers. Unittest uses Python's reflection mechanism to find all methods that start with test_ and runs them. Unittest wraps each call to test_* with matching calls to setUp and tearDown. See the [http://docs.python.org/2/library/unittest.html Python unittest documentation] for details.<br />
<br />
When we run the test, gr_unittest.main is going to invoke setUp, test_001_square_ff, and tearDown, in that order.<br />
<br />
<code>test_001_square_ff</code> builds a small graph that contains three nodes. <code>blocks.vector_source_f(src_data)</code> will source the elements of <code>src_data</code> and then say that it's finished. <code>howto.square_ff</code> is the block we're testing. <code>blocks.vector_sink_f</code> gathers the output of <code>howto.square_ff</code>.<br />
<br />
The <code>run()</code> method runs the graph until all the blocks indicate they are finished. Finally, we check that the result of executing <code>square_ff</code> on <code>src_data</code> matches what we expect.<br />
<br />
Note that such a test is usually called before installing the module. This means that we need some trickery to be able to load the blocks when testing. CMake takes care of most things by changing PYTHONPATH appropriately. Also, we import <code>howto_swig</code> instead of <code>howto</code> in this file.<br />
<br />
In order for CMake to actually know this test exists, <code>gr_modtool</code> modified <code>python/CMakeLists.txt</code> with these lines:<br />
<br />
For GR v3.9+, remove the following <code> set(GR_TEST_PYTHON_DIRS ${CMAKE_BINARY_DIR}/swig) </code><br />
<br />
<pre>########################################################################<br />
# Handle the unit tests<br />
########################################################################<br />
include(GrTest)<br />
<br />
set(GR_TEST_TARGET_DEPS gnuradio-howto)<br />
set(GR_TEST_PYTHON_DIRS ${CMAKE_BINARY_DIR}/swig)<br />
GR_ADD_TEST(qa_square_ff ${PYTHON_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/qa_square_ff.py)</pre><br />
<br />
=== The C++ code (part 1) ===<br />
<br />
Now that we've got a test case, let's write the C++ code. All signal processing blocks are derived from <code>gr::block</code> or one of its subclasses. Go check out the [http://gnuradio.org/doc/doxygen/classgr_1_1block.html block documentation] on the Doxygen-generated manual.<br />
<br />
<code>gr_modtool</code> already provided us with three files that define the block: <code>lib/square_ff_impl.h</code>, <code>lib/square_ff_impl.cc</code> and <code>include/howto/square_ff.h</code>. All we have to do is modify them to do our bidding. After you've finished with this tutorial ''please'' read and understand the [https://wiki.gnuradio.org/index.php/BlocksCodingGuide Blocks Coding Guide] to find out how these files are structured and why!<br />
<br />
First of all, we have a look at our header files. Because the block we're writing is so simple, we don't have to actually change them (the header file in <code>include/</code> is often quite complete after running <code>gr_modtool</code>, unless we need to add some public methods such as mutator methods, i.e., getters and setters). That leaves us with <code>lib/square_ff_impl.cc</code>.<br />
<br />
<code>gr_modtool</code> hints at where you have to change code by adding <code>&lt;++&gt;</code> symbols.<br /><br />
Let's go through these one at a time:<br />
<br />
<pre> square_ff_impl::square_ff_impl()<br />
: gr::block(&quot;square_ff&quot;,<br />
gr::io_signature::make(1, 1, sizeof (float)), // input signature<br />
gr::io_signature::make(1, 1, sizeof (float))) // output signature<br />
{<br />
// empty constructor<br />
}</pre><br />
The constructor itself is empty, as the squaring block has no need to set up anything.<br />
<br />
The only interesting portion is the definition of the input and output signatures: At the input, we have 1 port that allows float inputs. The output port is the same.<br />
<br />
<pre> void<br />
square_ff_impl::forecast (int noutput_items, gr_vector_int &amp;ninput_items_required)<br />
{<br />
ninput_items_required[0] = noutput_items;<br />
}</pre><br />
<code>forecast()</code> is a function which tells the scheduler how many input items are required to produce <code>noutput_items</code> output items. In this case, they're the same. The index 0 indicates that this is for the first port, but we only have one any way. This is generally the case for <code>forecast</code> in a lot of blocks. For examples, you can look at how <code>gr::block</code>, <code>gr::sync_block</code>, <code>gr::sync_decimator</code>, and <code>gr::sync_interpolator</code> define the default forecast functions to account for things like rate changes and history.<br />
<br />
Finally, there's <code>general_work()</code>, which is pure virtual in <code>gr::block</code>, so we definitely need to override that. <code>general_work()</code> is the method that does the actual signal processing:<br />
<br />
<pre> int<br />
square_ff_impl::general_work (int noutput_items,<br />
gr_vector_int &amp;ninput_items,<br />
gr_vector_const_void_star &amp;input_items,<br />
gr_vector_void_star &amp;output_items)<br />
{<br />
const float *in = (const float *) input_items[0];<br />
float *out = (float *) output_items[0];<br />
<br />
for(int i = 0; i &lt; noutput_items; i++) {<br />
out[i] = in[i] * in[i];<br />
}<br />
<br />
// Tell runtime system how many input items we consumed on<br />
// each input stream.<br />
consume_each (noutput_items);<br />
<br />
// Tell runtime system how many output items we produced.<br />
return noutput_items;<br />
}</pre><br />
There is one pointer to the input- and one pointer to the output buffer, respectively, and a for-loop which copies the square of the input buffer to the output buffer.<br />
<br />
=== Using CMake ===<br />
<br />
If you've never used CMake before, this is a good time to give it a try. The typical workflow of a CMake-based project as seen from the command line is this (if using PyBOMBS, first read '''Build Tree vs. Install Tree'''):<br />
<br />
<pre>$ mkdir build # We're currently in the module's top directory<br />
$ cd build/<br />
$ cmake ../ # Tell CMake that all its config files are one dir up<br />
$ make # And start building (should work after the previous section)</pre><br />
<br />
If using GR v3.9 you will need to run:<br />
<pre>$ sudo make install </pre><br />
<br />
===== Build Tree vs. Install Tree =====<br />
<br />
When you run cmake, you usually run it in a separate directory (e.g. <code>build/</code>). This is the build tree. The path to the install tree is <code>$prefix/lib/$pythonversion/dist-packages</code>, where <code>$prefix</code> is whatever you specified to CMake during configuration (usually <code>/usr/local/</code>) with the <code>-DCMAKE_INSTALL_PREFIX</code> switch. (Note: different versions of Python will either use site-packages or dist-packages; dist-packages is the newer way and most likely for newer OSes and installations.)<br />
<br />
If you installed GNU Radio using PyBOMBS, the install tree is located in the <code>target/</code> directory set during the initial PyBOMBS configuration. Make sure to add the -DCMAKE_INSTALL_PREFIX switch for CMake, so that it will correctly locate your GNU Radio installation. The command should look similar to this:<br />
<br />
<pre>$ cmake -DCMAKE_INSTALL_PREFIX=~/prefix-3.8 ../ # should be the configured PyBOMBS target</pre><br />
Now we have a new directory <code>build/</code> in our module's directory. All the compiling etc. is done in here, so the actual source tree is not littered with temporary files. If we change any CMakeLists.txt files, we should re-run <code>cmake ../</code> (although in truth, cmake detects these changes and reruns automatically when you next run <code>make</code>). During compilation, the libraries are copied into the build tree. Only during installation, files are installed to the install tree, thus making our blocks available to GNU Radio apps.<br />
<br />
We write our applications such that they access the code and libraries in the install tree. On the other hand, we want our test code to run on the build tree, where we can detect problems before installation.<br />
<br />
=== Let's try that -- running <code>make test</code> ===<br />
<br />
Because we wrote the QA code before the C++ code, we can immediately see if what we did was correct.<br />
<br />
We use <code>make test</code> to run our tests (run this from the <code>build/</code> subdirectory, after calling <code>cmake</code> and <code>make</code>). This invokes a shell script which sets up the PYTHONPATH environment variable so that our tests use the build tree versions of our code and libraries. It then runs all files which have names of the form qa_*.py and reports the overall success or failure.<br />
<br />
There is quite a bit of behind-the-scenes action required to use the non-installed versions of our code (look at the <code>cmake/</code> directory for a cheap thrill.)<br />
<br />
If you completed the <code>square_ff</code> block, this should work fine:<br />
<br />
<pre>gr-howto/build % make test<br />
Running tests...<br />
Test project /home/braun/tmp/gr-howto/build<br />
Start 1: test_howto<br />
1/2 Test #1: test_howto ....................... Passed 0.01 sec<br />
Start 2: qa_square_ff<br />
2/2 Test #2: qa_square_ff ..................... Passed 0.38 sec<br />
<br />
100% tests passed, 0 tests failed out of 2<br />
<br />
Total Test time (real) = 0.39 sec</pre><br />
If something fails during the tests, we can dig a little deeper. When we run <code>make test</code>, we're actually invoking the CMake program <code>ctest</code>, which has a number of options we can pass to it for more detailed information. Say we forgot to multiply <code>in[i] * in[i]</code> and so aren't actually squaring the signal. If we just run <code>make test</code> or even just <code>ctest</code>, we would get this:<br />
<br />
<pre>gr-howto/build $ ctest<br />
Test project /home/braun/tmp/gr-howto/build<br />
Start 1: test_howto<br />
1/2 Test #1: test_howto ....................... Passed 0.02 sec<br />
Start 2: qa_square_ff<br />
2/2 Test #2: qa_square_ff .....................***Failed 0.21 sec<br />
<br />
50% tests passed, 1 tests failed out of 2<br />
<br />
Total Test time (real) = 0.23 sec<br />
<br />
The following tests FAILED:<br />
2 - qa_square_ff (Failed)<br />
Errors while running CTest</pre><br />
To find out what happened with our qa_square_ff test, we run <code>ctest -V -R square</code>. The '-V' flag gives us verbose output and the '-R' flag is a regex expression to only run those tests which match.<br />
<br />
<pre>gr-howto/build $ ctest -V -R square<br />
UpdateCTestConfiguration from :/home/braun/tmp/gr-howto/build/DartConfiguration.tcl<br />
UpdateCTestConfiguration from :/home/bruan/tmp/gr-howto/build/DartConfiguration.tcl<br />
Test project /home/braun/tmp/gr-howto/build<br />
Constructing a list of tests<br />
Done constructing a list of tests<br />
Checking test dependency graph...<br />
Checking test dependency graph end<br />
test 2<br />
Start 2: qa_square_ff<br />
<br />
2: Test command: /bin/sh &quot;/home/bruan/tmp/gr-howto/build/python/qa_square_ff_test.sh&quot;<br />
2: Test timeout computed to be: 9.99988e+06<br />
2: F<br />
2: ======================================================================<br />
2: FAIL: test_001_t (__main__.qa_square_ff)<br />
2: ----------------------------------------------------------------------<br />
2: Traceback (most recent call last):<br />
2: File &quot;/home/braun/tmp/gr-howto/python/qa_square_ff.py&quot;, line 44, in test_001_t<br />
2: self.assertFloatTuplesAlmostEqual(expected_result, result_data, 6)<br />
2: File &quot;/opt/gr/lib/python2.7/dist-packages/gnuradio/gr_unittest.py&quot;, line 90, in assertFloatTuplesAlmostEqual<br />
2: self.assertAlmostEqual (a[i], b[i], places, msg)<br />
2: AssertionError: 9 != -3.0 within 6 places<br />
2: <br />
2: ----------------------------------------------------------------------<br />
2: Ran 1 test in 0.002s<br />
2: <br />
2: FAILED (failures=1)<br />
1/1 Test #2: qa_square_ff .....................***Failed 0.21 sec<br />
<br />
0% tests passed, 1 tests failed out of 1<br />
<br />
Total Test time (real) = 0.21 sec<br />
<br />
The following tests FAILED:<br />
2 - qa_square_ff (Failed)<br />
Errors while running CTest</pre><br />
This tells us that &quot;9 != -3.0&quot; because we expected the output to be (-3)^2 = 9 but really got the input of -3. We can use this information to go back and fix our block until the tests pass.<br />
<br />
We can also put in debug print statements into our QA code on failures, like printing out <code>expected_result</code> and <code>result_data</code> to compare them to better understand the problem.<br />
<br />
=== More C++ code (but better) - Subclasses for common patterns ===<br />
<br />
<code>gr::block</code> allows tremendous flexibility with regard to the consumption of input streams and the production of output streams. Adroit use of <code>forecast()</code> and <code>consume()</code> (see below) allows variable rate blocks to be built. It is possible to construct blocks that consume data at different rates on each input and produce output at a rate that is a function of the contents of the input data.<br />
<br />
On the other hand, it is very common for signal processing blocks to have a fixed relationship between the input rate and the output rate. Many are 1:1, while others have 1:N or N:1 relationships. You must have thought the same thing in the <code>general_work()</code> function of the previous block: if the number of items consumed is identical the number of items produced, why do I have to tell GNU Radio the exact same number twice?<br />
<br />
Another common requirement is the need to examine more than one input sample to produce a single output sample. This is orthogonal to the relationship between input and output rate. For example, a non-decimating, non-interpolating FIR filter needs to examine N input samples for each output sample it produces, where N is the number of taps in the filter. However, it only consumes a single input sample to produce a single output. We call this concept &quot;history&quot;, but you could also think of it as &quot;look-ahead&quot;.<br />
<br />
* <code>gr::sync_block</code><br />
<br />
gr::sync_block is derived from gr::block and implements a 1:1 block with optional history. Given that we know the input to output rate, certain simplifications are possible. From the implementor's point-of-view, the primary change is that we define a <code>work()</code> method instead of <code>general_work()</code>. <code>work()</code> has a slightly different calling sequence; it omits the unnecessary <code>ninput_items</code> parameter, and arranges for <code>consume_each()</code> to be called on our behalf.<br />
<br />
Let's add another block which derives from <code>gr::sync_block</code> and call it <code>square2_ff</code>. First, we edit <code>qa_square_ff.py</code> to add another test:<br />
<br />
<pre> def test_002_square2_ff(self):<br />
src_data = (-3, 4, -5.5, 2, 3)<br />
expected_result = (9, 16, 30.25, 4, 9)<br />
src = blocks.vector_source_f(src_data)<br />
sqr = howto.square2_ff()<br />
dst = blocks.vector_sink_f()<br />
self.tb.connect(src, sqr, dst)<br />
self.tb.run()<br />
result_data = dst.data()<br />
self.assertFloatTuplesAlmostEqual(expected_result, result_data, 6)</pre><br />
You can see it's the exact same test as before except for the use of <code>square2_ff</code>.<br />
<br />
Then, we use <code>gr_modtool</code> to add the block files, skipping the QA code (because we already have that):<br />
<br />
<pre>gr-howto % gr_modtool add -t sync -l cpp square2_ff<br />
GNU Radio module name identified: howto<br />
Block/code identifier: square2_ff<br />
Language: C++<br />
Please specify the copyright holder:<br />
Enter valid argument list, including default arguments: <br />
Add Python QA code? [Y/n] n<br />
Add C++ QA code? [Y/n] n<br />
Adding file 'square2_ff_impl.h'...<br />
Adding file 'square2_ff_impl.cc'...<br />
Adding file 'square2_ff.h'...<br />
Editing swig/howto_swig.i...<br />
Adding file 'howto_square2_ff.xml'...<br />
Editing grc/CMakeLists.txt...</pre><br />
<br />
If running GR v3.9+ you will need to bind the new <code>square2_ff</code> files to the existing files.<br />
<pre>gr-howto % gr_modtool bind square2_ff<br />
GNU Radio module name identifiedL howto<br />
Writing binding code to ./python/bindings/square2_ff_python.cc<br />
Writing binding code to ./python/bindings/docstrings/square2_ff_pydoc_template.h</pre><br />
<br />
Whenever howto/square2_ff.h is changed, <code> gr_modtool bind square2_ff </code> needs to be run.<br />
<br />
The constructor in <code>square2_ff_impl.cc</code> is done the same way as before, except for the parent class being <code>gr::sync_block</code>.<br />
<br />
<pre> square2_ff_impl::square2_ff_impl()<br />
: gr::sync_block(&quot;square2_ff&quot;,<br />
gr::io_signature::make(1, 1, sizeof (float)),<br />
gr::io_signature::make(1, 1, sizeof (float)))<br />
{}<br />
<br />
// [...] skip some lines ...<br />
<br />
int<br />
square2_ff_impl::work(int noutput_items,<br />
gr_vector_const_void_star &amp;input_items,<br />
gr_vector_void_star &amp;output_items)<br />
{<br />
const float *in = (const float *) input_items[0];<br />
float *out = (float *) output_items[0];<br />
<br />
for(int i = 0; i &lt; noutput_items; i++) {<br />
out[i] = in[i] * in[i];<br />
}<br />
<br />
// Tell runtime system how many output items we produced.<br />
return noutput_items;<br />
}</pre><br />
The <code>work</code> function is the real difference (also, we don't have a <code>forecast()</code> function any more). We'll look at it in greater detail in the next section.<br />
<br />
This gives us fewer things to worry about and less code to write. If the block requires history greater than 1, call <code>set_history()</code> in the constructor or any time the requirement changes.<br />
<br />
<code>gr::sync_block</code> provides a version of <code>forecast</code> that handles the history requirement.<br />
<br />
* <code>gr::sync_decimator</code><br />
<br />
<code>gr::sync_decimator</code> is derived from <code>gr::sync_block</code> and implements a N:1 block with optional history.<br />
<br />
* <code>gr::sync_interpolator</code><br />
<br />
<code>gr::sync_interpolator</code> is derived from <code>gr::sync_block</code> and implements a 1:N block with optional history.<br />
<br />
With this knowledge it should be clear that <code>howto_square_ff</code> should be a <code>gr::sync_block</code> with no history.<br />
<br />
Now, go back into our build directory and run <code>make</code>. Because <code>gr_modtool</code> added the <code>square2_ff</code> block to the necessary CMakeLists.txt files, <code>cmake</code> is automatically rerun for us and followed by <code>make</code>.<br />
<br />
Again, running <code>make test</code> will spawn a test run with of <code>qa_square_ff.py</code> which should not fail.<br />
<br />
=== Inside the <code>work()</code> function ===<br />
<br />
If you're using a sync block (including decimator and interpolator), this is how the skeleton code looks like produced by gr_modtool:<br />
<br />
<pre> int<br />
my_block_name::work(int noutput_items,<br />
gr_vector_const_void_star &amp;input_items,<br />
gr_vector_void_star &amp;output_items)<br />
{<br />
const float *in = (const float *) input_items[0];<br />
float *out = (float *) output_items[0];<br />
<br />
// Do &lt;+signal processing+&gt;<br />
<br />
// Tell runtime system how many output items we produced.<br />
return noutput_items;<br />
}</pre><br />
So, given history, vectors, multiple input ports etc., is this really all you need? Yes, it is! Because sync blocks have a fixed output to input rate, all you need to know is the number of output items, and you can calculate how many input items are available.<br />
<br />
Example - the adder block: source:gr-blocks/lib/add_XX_impl.cc.t<br />
<br />
This block has an unknown number of inputs and variable vector lengths. The number of connected ports can be checked by <code>input_items.size()</code> and <code>output_items.size()</code>. The outer <code>for</code> loop, which goes over all the available items, goes up to <code>noutput_items*d_vlen</code>. The number of output items is identical to the number of input items because it is a sync block, and you can trust GNU Radio to have this number of items available. In this case, one item is a vector of samples, but we want to add the individual samples, so the for loop considers that.<br />
<br />
Example - interpolation in gr::blocks::unpack_k_bits_bb: source:gr-blocks/lib/unpack_k_bits_bb_impl.cc [https://github.com/gnuradio/gnuradio/blob/master/gr-blocks/lib/unpack_k_bits_bb_impl.cc source]<br />
<br />
This is a block which picks apart bytes and produces the individual bits. Again, it is unknown at compile time how many bits per byte there are. However, there's a fixed number of output items per input item, so we can simply divide <code>noutput_items/d_k</code> to get the correct number of input items. It will always be correct because GNU Radio knows the input to output ratio and will make sure that <code>noutput_items</code> is always a multiple of this integer ratio.<br />
<br />
Example - history in source:gr-digital/lib/diff_phasor_cc_impl.cc<br />
<br />
If you use history of length k, GNU Radio will keep k-1 entries of the input buffer instead of discarding them. This means that if GNU Radio tells you the input buffer has N items, it actually has N+k-1 items you may use.<br />
<br />
Consider the example above. We need one previous item, so history is set to k=2. If you inspect the for loop closely, you'll find that out of <code>noutput_items</code> items, <code>noutput_items+1</code> items are actually read. This is possible because there is an extra item in the input buffer from the history.<br />
<br />
After consuming <code>noutput_items</code> items, the last entry is not discarded and will be available for the next call of <code>work()</code>.<br />
<br />
=== Help! My test fails! ===<br />
<br />
Congratulations! If your test fails, your QA code has already paid for itself. Obviously, you want to fix everything before you continue.<br />
<br />
You can use the command <code>ctest -V</code> (instead of <code>make test</code>, again, all in your <code>build/</code> subdirectory) to get all the output from the tests. You can also use <code>ctest -V -R REGEX</code> to only run tests that match REGEX, if you have many tests and want to narrow it down. If you can't figure out the problem from the output of your QA code, put in <code>print</code> statements and show intermediary results.<br />
<br />
=== Making your blocks available in GRC ===<br />
<br />
You can now install your module, but it will not be available in GRC. That's because <code>gr_modtool</code> can't create valid XML files before you've even written a block. The XML code generated when you call <code>gr_modtool add</code> is just some skeleton code.<br />
<br />
Once you've finished writing the block, <code>gr_modtool</code> has a function to help you create the XML code for you. For the howto example, you can invoke it on the <code>square2_ff</code> block by calling<br />
For GNURadio ver >= 3.8:<br />
<pre>gr-howto % gr_modtool makeyaml square2_ff<br />
GNU Radio module name identified: howto<br />
Warning: This is an experimental feature. Don't expect any magic.<br />
Searching for matching files in lib/:<br />
Making GRC bindings for lib/square2_ff_impl.cc...<br />
Overwrite existing GRC file? [y/N] y</pre><br />
<br />
For GNURadio ver. < 3.8:<br />
<pre>gr-howto % gr_modtool makexml square2_ff<br />
GNU Radio module name identified: howto<br />
Warning: This is an experimental feature. Don't expect any magic.<br />
Searching for matching files in lib/:<br />
Making GRC bindings for lib/square2_ff_impl.cc...<br />
Overwrite existing GRC file? [y/N] y</pre><br />
Note that <code>gr_modtool add</code> creates an invalid GRC file, so we can overwrite that.<br />
<br />
In most cases, <code>gr_modtool</code> can't figure out all the parameters by itself and you will have to edit the appropriate XML file by hand. The [[GNURadioCompanion|GRC]] wiki site has a description available.<br />
<br />
In this case, because the block is so simple, the XML is actually valid. Have a look at <code>grc/howto_square2_ff.xml</code>:<br />
<br />
<pre><br />
<block> <br />
<name>Square ff</name><br />
<key>howto_square_ff</key><br />
<category>[HOWTO]</category><br />
<import>import howto</import><br />
<make>howto.square_ff()</make><br />
<sink><br />
<name>in</name><br />
<type>float</type><br />
</sink><br />
<source><br />
<name>out</name><br />
<type>float</type><br />
</source><br />
</block><br />
</pre><br />
Perhaps you want to change the autogenerated name to something nicer.<br />
<br />
<b>Note:</b> The category name <b>must</b> be enclosed in square brackets to work!<br />
<br />
If you do a <code>make install</code> from the build directory, you can use the block in GRC. If GRC is already running, you can hit the &quot;Reload Blocks&quot; button in the GRC toolbar; it's a blue circular arrow on the right-hand side. You should now see a &quot;HOWTO&quot; category in the block tree.<br />
<br />
=== There's more: additional <code>gr::block</code>-methods ===<br />
<br />
If you've read the [http://gnuradio.org/doc/doxygen/classgr_1_1block.html gr::block documentation] (which you should have), you'll have noticed there are a great number of methods available to configure your block.<br />
<br />
Here's some of the more important ones:<br />
<br />
==== <code>set_history()</code> ====<br />
<br />
If your block needs a history (i.e., something like an FIR filter), call this in the constructor. GNU Radio then makes sure you have the given number of 'old' items available.<br />
<br />
The smallest history you can have is 1, i.e., for every output item, you need 1 input item. If you choose a larger value, N, this means your output item is calculated from the current input item and from the N-1 previous input items.<br />
<br />
The scheduler takes care of this for you. If you set the history to length N, the first N items in the input buffer include the N-1 previous ones (even though you've already consumed them).<br />
<br />
==== <code>forecast()</code> ====<br />
<br />
The system needs to know how much data is required to ensure validity in each of the input arrays. As stated before, the <code>forecast()</code> method provides this information, and you must therefore override it anytime you write a <code>gr::block</code> derivative (for sync blocks, this is implicit).<br />
<br />
The default implementation of <code>forecast()</code> says there is a 1:1 relationship between <code>noutput_items</code> and the requirements for each input stream. The size of the items is defined by <code>gr::io_signature::make</code> in the constructor of <code>gr::block</code>. The sizes of the input and output items can of course differ; this still qualifies as a 1:1 relationship. Of course, if you had this relationship, you wouldn't want to use a <code>gr::block</code>!<br />
<br />
<pre> // default implementation: 1:1<br />
void<br />
gr::block::forecast(int noutput_items,<br />
gr_vector_int &amp;ninput_items_required)<br />
{<br />
unsigned ninputs = ninput_items_required.size ();<br />
for(unsigned i = 0; i &lt; ninputs; i++)<br />
ninput_items_required[i] = noutput_items;<br />
}</pre><br />
Although the 1:1 implementation worked for <code>square_ff</code>, it wouldn't be appropriate for interpolators, decimators, or blocks with a more complicated relationship between <code>noutput_items</code> and the input requirements. That said, by deriving your classes from gr::sync_block, gr::sync_interpolator or gr::sync_decimator instead of gr::block, you can often avoid implementing forecast.<br />
<br />
==== <code>set_output_multiple()</code> ====<br />
<br />
When implementing your <code>general_work()</code> routine, it's occasionally convenient to have the run time system ensure that you are only asked to produce a number of output items that is a multiple of some particular value. This might occur if your algorithm naturally applies to a fixed sized block of data. Call <code>set_output_multiple</code> in your constructor to specify this requirement. The default output multiple is 1.<br />
<br />
=== Finalizing your work and installing ===<br />
<br />
First, go through this checklist:<br />
<br />
* Have you written one or more blocks, including QA codes?<br />
* Does <code>make test</code> pass?<br />
* Are there GRC bindings available (if that's what you want)?<br />
<br />
In that case, you can go ahead and install your module. On a Linux machine, this would mean going back to the build directory and calling <code>make install</code>:<br />
<br />
<pre>$ cd build/<br />
$ make install # or sudo make install</pre><br />
With Ubuntu, you may have to call <code>ldconfig</code> as well:<br />
<br />
<pre>$ sudo ldconfig</pre><br />
Otherwise, you'll get an error message that the library you just installed cannot be found.<br />
<br />
== Other types of blocks ==<br />
<br />
=== Sources and sinks ===<br />
<br />
Sources and sinks are derived from <code>gr::sync_block</code>. The only thing different about them is that sources have no inputs and sinks have no outputs. This is reflected in the <code>gr::io_signature::make</code> that are passed to the <code>gr::sync_block</code> constructor. Take a look at [source:gr-blocks/lib/file_source_impl.cc file_source.{h,cc}] and file_sink_impl.{h,cc} for some very straight-forward examples.<br />
<br />
=== Hierarchical blocks ===<br />
<br />
<code>gr_modtool</code> supports skeleton code for hierarchical blocks both in Python and C''++.<br />
<br />
<pre>~/gr-howto % gr_modtool.py add -t hier -l cpp hierblockcpp_ff<br />
GNU Radio module name identified: howto<br />
Block/code identifier: hierblockcpp_ff<br />
Language: C++<br />
Please specify the copyright holder:<br />
Enter valid argument list, including default arguments:<br />
Add Python QA code? [Y/n]<br />
Add C++ QA code? [y/N]<br />
Adding file 'hierblockcpp_ff_impl.h'...<br />
Adding file 'hierblockcpp_ff_impl.cc'...<br />
Adding file 'hierblockcpp_ff.h'...<br />
Editing swig/howto_swig.i...<br />
Adding file 'howto_hierblockcpp_ff.xml'...<br />
Editing grc/CMakeLists.txt...</pre><br />
Using the <code>-l python</code> switch creates such a block in Python.<br />
<br />
== Everything at one glance: Cheat sheet for editing modules/components: ==<br />
<br />
Here's a quick list for all the steps necessary to build blocks and out-of-tree modules:<br />
<br />
# Create (do this once per module): <code>gr_modtool newmod MODULENAME</code><br />
# Add a block to the module: <code>gr_modtool add BLOCKNAME</code><br />
# Create a build directory: <code>mkdir build/</code><br />
# Invoke the make process: <code>cd build &amp;&amp; cmake &lt;OPTIONS&gt; ../ &amp;&amp; make</code> (Note that you only have to call cmake if you've changed the CMake files)<br />
# Invoke the testing: <code>make test</code> or <code>ctest</code> or <code>ctest -V</code> for more verbosity<br />
# Call <code>gr_modtool makexml BLOCKNAME</code> or <code>gr_modtool makeyaml BLOCKNAME</code> to generate the xml or yaml file for your blocks. Correct manually if needed.<br />
# Install (only when everything works and no tests fail): <code>sudo make install</code><br />
# Ubuntu users: reload the libs: <code>sudo ldconfig</code><br />
# Delete blocks from the source tree: <code>gr_modtool rm REGEX</code><br />
# Disable blocks by removing them from the CMake files: <code>gr_modtool disable REGEX</code><br />
<br />
== Tutorial 3: Writing a signal processing block in Python ==<br />
<br />
'''Note:''' Writing signal processing blocks in Python comes with a performance penalty. The most common cause for using Python to write blocks is because you want to quickly prototype something without having to argue with C++.<br />
<br />
From the previous tutorials, you already know about blocks and how they work. Lets go through things a bit quicker, and code another squaring block in pure Python, which shall be called <code>square3_ff()</code>.<br />
<br />
=== Adding the test case ===<br />
<br />
So, first of all, we add another test case by editing <code>qa_square_ff.py</code>. Leaving out the test cases for the other two blocks, the QA file now looks like this:<br />
<br />
<pre>from gnuradio import gr, gr_unittest<br />
from gnuradio import blocks<br />
import howto_swig<br />
from square3_ff import square3_ff<br />
<br />
class qa_square_ff (gr_unittest.TestCase):<br />
<br />
def setUp (self):<br />
self.tb = gr.top_block ()<br />
<br />
def tearDown (self):<br />
self.tb = None<br />
<br />
# [...] Skipped the other test cases<br />
<br />
def test_003_square3_ff (self):<br />
src_data = (-3, 4, -5.5, 2, 3)<br />
expected_result = (9, 16, 30.25, 4, 9)<br />
src = blocks.vector_source_f (src_data)<br />
sqr = square3_ff ()<br />
dst = blocks.vector_sink_f ()<br />
self.tb.connect (src, sqr)<br />
self.tb.connect (sqr, dst)<br />
self.tb.run ()<br />
result_data = dst.data ()<br />
self.assertFloatTuplesAlmostEqual (expected_result, result_data, 6)<br />
<br />
if __name__ == '__main__':<br />
gr_unittest.main ()</pre><br />
The actual test case looks '''exactly''' like the previous ones did, only replacing the block definition with <code>square3_ff()</code>. The only other difference is in the import statements: We are now importing a module called <code>square3_ff</code> from which we pull the new block.<br />
<br />
=== Adding the block code ===<br />
<br />
Having put the unit test in place, we add a file called <code>square3_ff.py</code> into the <code>python/</code> directory using <code>gr_modtool</code>:<br />
<br />
<pre>gr-howto % gr_modtool add -t sync -l python square3_ff<br />
GNU Radio module name identified: howto<br />
Block/code identifier: square3_ff<br />
Language: Python<br />
Please specify the copyright holder:<br />
Enter valid argument list, including default arguments: <br />
Add Python QA code? [Y/n] n<br />
Adding file 'square3_ff.py'...<br />
Adding file 'howto_square3_ff.xml'...<br />
Editing grc/CMakeLists.txt...</pre><br />
Remember not to add any QA files as we're using the existing one. Next, edit the new file <code>python/square3_ff.py</code>. It should look a bit like this:<br />
<br />
<pre>import numpy<br />
from gnuradio import gr<br />
<br />
class square3_ff(gr.sync_block):<br />
&quot; Squaring block &quot;<br />
def __init__(self):<br />
gr.sync_block.__init__(<br />
self,<br />
name = &quot;square3_ff&quot;,<br />
in_sig = [numpy.float32], # Input signature: 1 float at a time<br />
out_sig = [numpy.float32], # Output signature: 1 float at a time<br />
)<br />
<br />
def work(self, input_items, output_items):<br />
output_items[0][:] = input_items[0] * input_items[0] # Only works because numpy.array<br />
return len(output_items[0])</pre><br />
Some things should immediately stick out:<br />
<br />
* The block class is derived from <code>gr.sync_block</code>, just like the C++ version was derived from gr::sync_block<br />
* It has a constructor where the name and input/output signatures are set and a <code>work()</code> function<br />
<br />
However, there are some major differences to the C++ version:<br />
<br />
* The input and output signatures are simply defined as a list. Every element contains the item size of that port. So in this case, there is one port per input and one port per output and each has an item size of <code>numpy.float32</code> (a single-precision float). If you want a port to operate on vectors, define a tuple, e.g. [(numpy.float32, 4), numpy.float32] means there are two ports: The first one is for vectors of 4 floats, the second is for scalar floats.<br />
* When assigning vectors to <code>output_items</code>, remember to use the <code>[:]</code> operator. This makes sure Python doesn't rebind the variables or does something clever but guarantees that the data is properly copied<br />
* <code>input_items</code> and <code>output_items</code> are numpy arrays, which is why we can do the very simple element-wise multiplication the way it's done here (instead of a list comprehension)<br />
* No recompiling is necessary for the <code>make test</code> (faster development cycles, yay!)<br />
<br />
=== Other types of Python blocks ===<br />
<br />
Just like the C++ variant, there are four types of blocks in Python:<br />
<br />
* <code>gr.sync_block</code><br />
* <code>gr.decim_block</code><br />
* <code>gr.interp_block</code><br />
* <code>gr.basic_block</code> - The Python version of <code>gr::block</code><br />
<br />
Like their C++ versions, these blocks have <code>forecast()</code>, <code>work()</code>, and <code>general_work()</code> methods you can override. The difference is, the argument list for the work functions is always as shown in the previous example:<br />
<br />
<pre> def work(self, input_items, output_items):<br />
# Do stuff<br />
<br />
def general_work(self, input_items, output_items):<br />
# Do stuff</pre><br />
The number of input/output items is obtained through <code>len(input_items[PORT_NUM])</code>.<br />
<br />
=== More examples ===<br />
<br />
Check out the QA code for the Python blocks for some good examples:<br />
<br />
* [https://github.com/gnuradio/gnuradio/blob/master/gr-blocks/python/blocks/qa_block_gateway.py gr-blocks/python/blocks/qa_block_gateway.py]<br />
<br />
=== Troubleshooting ===<br />
<br />
<code>ValueError: invalid literal for int() with base 10: '...'</code> This occurs in GRC when attempting to drag a block into a flowgraph. It usually means the <code>.yml</code> file cooresponding to the block located in the <code>grc/</code> folder is not filled out. ("..." is a placeholder).<br />
<br />
<code>Error: invalid literal for int() with base 10: '1 /* max inputs */'</code> This occurs in GRC when attempting to drag a block into the flowgraph. This is caused by <code>multiplicity: 1 /* max inputs */</code> in your <code>.yml</code> file. Removing <code>/* max inputs */</code> will fix the error.</div>Solomonbstoner