Guided Tutorial Programming Topics: Difference between revisions

From GNU Radio
Jump to navigation Jump to search
No edit summary
(→‎Prerequisites: Fixed links to message passing wiki)
 
(11 intermediate revisions by 2 users not shown)
Line 11: Line 11:


* General familiarity with C++ and Python
* General familiarity with C++ and Python
* Previous tutorials recommended:
* Tutorials:
** [[Guided_Tutorial_Introduction|Tutorials Introductions]]
** [[Guided_Tutorial_Introduction|'''A brief introduction to GNU Radio, SDR, and DSP''']]
** [[Guided_Tutorial_GRC|GRC Tutorial]]
** [[Guided_Tutorial_GRC|'''Intro to GR usage: GRC and flowgraphs''']]
** [[Guided_Tutorial_GNU_Radio_in_C++|Programming GR in C++]]
 


-----
-----
Line 25: Line 23:
In GNU Radio we have two mechanisms to pass these messages:
In GNU Radio we have two mechanisms to pass these messages:


* Synchronously to a data stream, using [http://gnuradio.org/doc/doxygen/page_stream_tags.html '''stream tags''']
* Synchronously to a data stream, using [https://wiki.gnuradio.org/index.php/Stream_Tags '''stream tags''']
* Asynchronously, using the [http://gnuradio.org/doc/doxygen/page_msg_passing.html '''message passing interface''']
* Asynchronously, using the [https://wiki.gnuradio.org/index.php/Message_Passing '''message passing interface''']


Before we discuss these, let's consider what such a message is from a programming perspective. It could be a string, a vector of items, a dictionary... anything, really, that can be represented as a data type. In Python, this would not be a problem, since it is weakly typed, and a message variable could simply be assigned whatever we need. C++ on the other hand is strongly typed, and it is not possible to create a variable without knowing its type. What makes things harder is that we need to be able to share the same data objects between Python and C++. To circumvent this problem, we introduce ''polymorphic types (PMTs)''.
Before we discuss these, let's consider what such a message is from a programming perspective. It could be a string, a vector of items, a dictionary... anything, really, that can be represented as a data type. In Python, this would not be a problem, since it is weakly typed, and a message variable could simply be assigned whatever we need. C++ on the other hand is strongly typed, and it is not possible to create a variable without knowing its type. What makes things harder is that we need to be able to share the same data objects between Python and C++. To circumvent this problem, we introduce ''polymorphic types (PMTs)''.


== 5.1 Polymorphic Types (PMT) ==
== Polymorphic Types (PMT) ==
 
=== 5.1.1 Introduction ===
 
Polymorphic Types are used as the carrier of data from one block/thread to another such as stream tags and message passing interfaces. PMT data types can represent a variety of data ranging from the Booleans to dictionaries. But let's dive straight into some Python code and see how we can use PMTs:
 
<pre>
&gt;&gt;&gt; import pmt
&gt;&gt;&gt; P = pmt.from_long(23)
&gt;&gt;&gt; type(P)
pmt.pmt_swig.swig_int_ptr
&gt;&gt;&gt; print P
23
&gt;&gt;&gt; P2 = pmt.from_complex(1j)
&gt;&gt;&gt; type(P2)
pmt.pmt_swig.swig_int_ptr
&gt;&gt;&gt; print P2
0+1i
&gt;&gt;&gt; pmt.is_complex(P2)
True
</pre>
First, the <code>pmt</code> module is imported. We assign two values (<code>P</code> and <code>P2</code>) with PMTs using the <code>from_long()</code> and <code>from_complex()</code> calls, respectively. As we can see, they are both of the same type! This means we can pass these variables to C++ through SWIG, and C++ can handle this type accordingly.<br />
The same code as above in C++ would look like this:
 
<pre>
#include
// [...]
pmt::pmt_t P = pmt::from_long(23);
std::cout &lt;&lt; P &lt;&lt; std::endl;
pmt::pmt_t P2 = pmt::from_complex(gr_complex(0, 1)); // Alternatively: pmt::from_complex(0, 1)
std::cout &lt;&lt; P2 &lt;&lt; std::endl;
std::cout &lt;&lt; pmt::is_complex(P2) &lt;&lt; std::endl;
</pre>
Two things stand out in both Python and C++: First, we can simply print the contents of a PMT. How is this possible? Well, the PMTs have in-built capability to cast their value to a string (this is not possible with all types, though). Second, PMTs must obviously know their type, so we can query that, e.g. by calling the <code>is_complex()</code> method.
 
When assigning a non-PMT value to a PMT, we can use the <code>from_*</code> methods, and use the <code>to_*</code> methods to convert back:
 
<pre>
pmt::pmt_t P_int = pmt::from_long(42);
int i = pmt::to_long(P_int);
pmt::pmt_t P_double = pmt::from_double(0.2);
double d = pmt::to_double(P_double);
pmt::pmt_t P_double = pmt::mp(0.2);
</pre>
The last row shows the [http://gnuradio.org/doc/doxygen/namespacepmt.html#a90faad6086ac00280e0cfd8bb541bd64 pmt::mp()] shorthand function. It basically saves some typing, as it infers the correct <code>from_</code> function from the given type.
 
String types play a bit of a special role in PMTs (where they're called '''symbols'''), as we will see later, and have their own converter:
 
<pre>
pmt::pmt_t P_str = pmt::string_to_symbol(&quot;spam&quot;);
pmt::pmt_t P_str = pmt::intern(&quot;spam&quot;); // Alias for string_to_symbol()
std::string str = pmt::symbol_to_string(P_str);
</pre>
See the [http://gnuradio.org/doc/doxygen/page_pmt.html PMT docs] and the header file [http://gnuradio.org/doc/doxygen/pmt_8h.html pmt.h] for a full list of conversion functions. In Python, we can make use of the weak typing, and there's actually a helper function to do these conversions (C++'s helper function <code>pmt::mp()</code> only works towards PMT, and is less powerful):
 
<pre>
P_int = pmt.to_pmt(42)
i = pmt.to_python(P_int)
P_double = pmt.to_pmt(0.2)
d = pmt.to_double(P_double)
</pre>
On a side note, there are three useful PMT constants, which can be used in both Python and C++ domains. In C++, these can be used as such:
 
<pre>pmt::pmt_t P_true = pmt::PMT_T;
pmt::pmt_t P_false = pmt::PMT_F;
pmt::pmt_t P_nil = pmt::PMT_NIL;
</pre>
In Python:
 
<pre>
P_true = pmt.PMT_T
P_false = pmt.PMT_F
P_nil = pmt.PMT_NIL
</pre>
<code>pmt.PMT_T</code> and <code>pmt.PMT_F</code> are boolean PMT types representing True and False, respectively. The PMT_NIL is like a NULL or None and can be used for default arguments or return values, often indicating an error has occurred.
 
To be able to go back to C++ data types, we need to be able to find out the type from a PMT. The family of <code>is_*</code> methods helps us do that:
 
<pre>
double d;
if (pmt::is_integer(P)) {
    d = (double) pmt::to_long(P);
}
else if (pmt::is_real(P)) {
    d = pmt::to_double(P);
}
else {
    // We really expected an integer or a double here, so we don't know what to do
    throw std::runtime_error(&quot;expected an integer!&quot;);
}
</pre>
We can compare PMTs without knowing their type by using the <code>pmt::equal()</code> function:
 
<pre>
if (pmt::eq(P_int, P_double)) {
    std::cout &lt;&lt; &quot;Equal!&quot; &lt;&lt; std::endl; // This line will never be reached
}
</pre>
There are more equality functions, which compare different things: [http://gnuradio.org/doc/doxygen/namespacepmt.html#a5c28635e14287cc0e2f762841c11032f <code>pmt::eq()</code>] and [http://gnuradio.org/doc/doxygen/namespacepmt.html#a25467c81e1c5f4619a9cabad7a88eed5 <code>pmt::eqv()</code>]. We won't need these for this tutorial.
 
=== 5.1.2 More Complex Types ===
 
PMTs can hold a variety of types. Using the Python method <code>pmt.to_pmt()</code>, we can convert most of Pythons standard types out-of-the-box:
 
<pre>P_tuple = pmt.to_pmt((1, 2, 3, 'spam', 'eggs'))
P_dict = pmt.to_pmt({'spam': 42, 'eggs': 23})</pre>
But what does this mean in the C++ domain? Well, there are PMT types that define [http://gnuradio.org/doc/doxygen/namespacepmt.html#a32895cc5a614a46b66b869c4a7bd283c tuples] and [http://gnuradio.org/doc/doxygen/namespacepmt.html#aba10563e3ab43b8d52f9cb13132047cf dictionaries], keys and values being PMTs, again.<br />
So, to create the tuple from the Python example, the C++ code would look like this:
 
<pre>pmt::pmt_t P_tuple = pmt::make_tuple(pmt::from_long(1), pmt::from_long(2), pmt::from_long(3), pmt::string_to_symbol(&quot;spam&quot;), pmt::string_to_symbol(&quot;eggs&quot;))</pre>
For the dictionary, it's a bit more complex:
 
<pre>pmt::pmt_t P_dict = pmt::make_dict();
P_dict = pmt::dict_add(P_dict, pmt::string_to_symbol(&quot;spam&quot;), pmt::from_long(42));
P_dict = pmt::dict_add(P_dict, pmt::string_to_symbol(&quot;eggs&quot;), pmt::from_long(23));</pre>
As you can see, we first need to create a dictionary, then assign every key/value pair individually.
 
A variant of tuples are ''vectors''. Like Python's tuples and lists, PMT vectors are mutable, whereas PMT tuples are not. In fact, PMT vectors are the only PMT data types that are mutable. When changing the value or adding an item to a dictionary, we are actually creating a new PMT.
 
To create a vector, we can initialize it to a certain lengths, and fill all elements with an initial value. We can then change items or reference them:
 
<pre>pmt::pmt_t P_vector = pmt::make_vector(5, pmt::from_long(23)); // Creates a vector with 5 23's as PMTs
pmt::vector_set(P_vector, 0, pmt::from_long(42)); // Change the first element to a 42
std::cout &lt;&lt; pmt::vector_ref(P_vector, 0); // Will print 42</pre>
In Python, we can do all these steps (using <code>pmt.make_vector()</code> etc.), or convert a list:
 
<pre>P_vector = pmt.to_pmt([42, 23, 23, 23, 23])</pre>
Vectors are also different from tuples in a sense that we can '''directly''' load data types into the elements, which don't have to be PMTs.<br />
Say we want to pass a series of 8 float values to another block (these might be characteristics of a filter, for example). It would be cumbersome to convert every single element to and from PMTs, since all elements of the vector are the same type.
 
We can use special vector types for this case:
 
<pre>pmt::pmt_t P_f32vector = pmt::make_f32vector(8, 5.0); // Creates a vector with 8 5.0s's as floats
pmt::f32vector_set(P_f32vector, 0, 2.0); // Change the first element to a 2.0
float f = f32vector_ref(P_f32vector, 0);
std::cout &lt;&lt; f &lt;&lt; std::endl; // Prints 2.0
size_t len;
float *fp = pmt::f32vector_elements(P_f32vector, len);
for (size_t i = 0; i &lt; len; i++)
    std::cout &lt;&lt; fp[i] &lt;&lt; std::endl; // Prints all elements from P_f32vector, one after another</pre>
Python has a similar concept: [http://docs.scipy.org/doc/numpy/reference/generated/numpy.array.html Numpy arrays]. As usual, the PMT library understands this and converts as expected:
 
<pre>P_f32vector = pmt.to_pmt(numpy.array([2.0, 5.0, 5.0, 5.0, 5.0], dtype=numpy.float32))
print pmt.is_f32vector(P_f32vector) # Prints 'True'</pre>
Here, 'f32' stands for 'float, 32 bits'. PMTs know about most typical fixed-width data types, such as 'u8' (unsigned 8-bit character) or 'c32' (complex with 32-bit floats for each I and Q). Consult the [http://gnuradio.org/doc/doxygen/namespacepmt.html manual] for a full list of types.
 
The most generic PMT type is probably the blob (binary large object). Use this with care - it allows us to pass around anything that can be represented in memory.
 
=== 5.1.3 Pairs, cons-es and cars ===
 
A concept that originates in Lisp dialects are [http://en.wikipedia.org/wiki/Cons ''pairs'' and ''cons'']. The simplest explanation is just that: If you combine two PMTs, they form a new PMT, which is a pair (or cons) of those two PMTs (don't worry about the weird name, a lot of things originating in Lisp have weird names. Think of a 'construct').
 
Similarly to vectors or tuples, pairs are a useful way of packing several components of a message into a single PMT. Using the PMTs generated in the previous section, we can combine two of these to form a pair, here in Python:
 
<pre>P_pair = pmt.cons(pmt.string_to_symbol(&quot;taps&quot;), P_f32vector)
print pmt.is_pair(P_pair) # Prints 'true'</pre>
You can combine PMTs as tuples, dictionaries, vectors, or pairs, it's just a matter of taste. This construct is well-established though, and as such used in GNU Radio quite often.
 
So how do we deconstruct a pair? That's what the <code>car</code> and <code>cdr</code> functions do. Let's deconstruct that previous pair in C++:


<pre>pmt::pmt_t P_key = pmt::car(P_pair);
<content merged with PMT page in usage manual>
pmt::pmt_t P_f32vector2 = pmt::cdr(P_pair);
std::cout &lt;&lt; P_key &lt;&lt; std::endl; // Will print 'taps' using the PMT automatic conversion to strings</pre>
For more advanced pair manipulation, refer to the [http://gnuradio.org/doc/doxygen/namespacepmt.html#a7ab95721db5cbda1852f13a92eee5362 documentation] and the [https://en.wikipedia.org/wiki/Car_and_cdr Wikipedia page for car and cdr].


OK, so now we know all about creating messages - but how do we send them from block to block?
OK, so now we know all about creating messages - but how do we send them from block to block?


== 5.2 Stream Tags ==
== Stream Tags ==
 
When two blocks are already communicating over a stream connection (e.g., samples are flowing from one block to another), we can use [http://gnuradio.org/doc/doxygen/page_stream_tags.html ''stream tags'']. A stream tag is a message that is connected to a specific item. This way, we can send messages '''synchronously''' to the stream.
 
Apart from the fixed position in the stream, stream tags have three properties:
 
* A '''key''', which can be used to identify a certain tag
* A '''value''', which can be any PMT
* (Optional) A '''source ID''', which helps identify the origin of this specific tag.
 
Whether or not blocks use this information is up to them. Most blocks will propagate tags transparently, which means the tag is attached to the same item (or a corresponding item) on the output. Blocks that actually make use of tags usually search for tags with specific keys and only process those.
 
Let's have a look at a simple example:
 
[[File:tut5_tagstest_fg.png|600px|tut5_tagstest_fg.png]]
 
In this flow graph, we have two sources: A sinusoid and a tag strobe. A tag strobe is a block that will output a constant tag, in this case, on every 1000th item (the actual value of the items is always zero). Those sources get added up. The signal after the adder is identical to the sine wave we produced, because we are always adding zeros. However, the tags stay attached to the same position as they were coming from the tag strobe! This means every 1000th sample of the sinusoid now has a tag. The QT scope can display tags, and even trigger on them.
 
[[File:tut5_tagstest_scope.png|500px|tut5_tagstest_scope.png]]<br />
 
We now have a mechanism to randomly attach any metadata to specific items. There are several blocks that use tags. One of them is the UHD Sink block, the driver used for transmitting with USRP devices. It will react to tags with certain keys, one of them being <code>tx_freq</code>, which can be used to set the transmit frequency of a USRP while streaming.
 
=== 5.2.1 Adding tags to the QPSK demodulator ===
 
Going back to our QPSK demodulation example, we might want to add a feature to tell downstream blocks that the demodulation is not going well. Remember the output of our block is always hard-decision, and we have to output something. So we could use tags to notify that the input is not well formed, and that the output is not reliable.
 
As a failure criterion, we discuss the case where the input amplitude is too small, say smaller than 0.01. When the amplitude drops below this value, we output one tag. Another tag is only sent when the amplitude has recovered, and falls back below the threshold. We extend our work function like this:
 
<pre>if (std::abs(in[i]) &lt; 0.01 and not d_low_ampl_state) {
    add_item_tag(0, // Port number
                nitems_written(0) + i, // Offset
                pmt::mp(&quot;amplitude_warning&quot;), // Key
                pmt::from_double(std::abs(in[i])) // Value
    );
    d_low_ampl_state = true;
}
else if (std::abs(in[i]) &gt;= 0.01 and d_low_ampl_state) {
    add_item_tag(0, // Port number
        nitems_written(0) + i, // Offset
        pmt::mp(&quot;amplitude_recovered&quot;), // Key
        pmt::PMT_T // Value
    );
    d_low_ampl_state = false; // Reset state
}</pre>
In Python, the code would look like this (assuming we have a member of our block class called <code>d_low_ampl_state</code>):
 
<pre># The vector 'in' is called 'in0' here because 'in' is a Python keyword
if abs(in0[i]) &lt; 0.01 and not d_low_ampl_state:
    self.add_item_tag(0, # Port number
                self.nitems_written(0) + i, # Offset
                pmt.intern(&quot;amplitude_warning&quot;), # Key
                pmt.from_double(numpy.double(abs(in0[i]))) # Value
                # Note: We need to explicitly create a 'double' here,
                # because in0[i] is an explicit 32-bit float here
    )
    self.d_low_ampl_state = True
elif abs(in0[i]) &gt;= 0.01 and d_low_ampl_state:
    self.add_item_tag(0, # Port number
                self.nitems_written(0) + i, # Offset
                pmt.intern(&quot;amplitude_recovered&quot;), # Key
                pmt.PMT_T # Value
    )
    self.d_low_ampl_state = False; // Reset state</pre>
We can also create a tag data type [http://gnuradio.org/doc/doxygen/structgr_1_1tag__t.html tag_t] and directly pass this along:
 
<pre>if (std::abs(in[i]) &lt; 0.01 and not d_low_ampl_state) {
    tag_t tag;
    tag.offset = nitems_written(0) + i;
    tag.key = pmt::mp(&quot;amplitude_warning&quot;);
    tag.value = pmt::from_double(std::abs(in[i]));
    add_item_tag(0, tag);
    d_low_ampl_state = true;
}</pre>
Here's a flow graph that uses the tagged version of the demodulator. We input 20 valid QPSK symbols, then 10 zeros. Since the output of this block is always either 0, 1, 2 or 3, we normally have no way to see if the input was not clearly one of these values.
 
[[File:Demod_with_tags.grc.png|750px|Demod_with_tags.grc.png]]
 
Here's the output. You can see we have tags on those values which were not from a valid QPSK symbol, but from something unreliable.
 
[[File:Demod_with_tags.png|500px|Demod_with_tags.png]]<br />
 
=== 5.2.2 Tag offsets and noutput_items ===
 
Before explaining more about tags, it is important to understand offsets. '''The tag offset is an absolute value, and specific to a certain port!''' The first item that passes through a block has the offset 0, and there will only be one sample with this offset (OK, before the nitpickers get out their pitchforks: The offset is an unsigned 64-bit value, so they will wrap around once 2^64 items have gone through. If you do the math, you'll figure out you need to let a flow graph run for a long time until that happens).
 
Most blocks, however, work on ''relative'' sample indices (e.g., if you have a loop of the form <code>for (int i = 0; i &lt; noutput_items; i++)</code>, <code>i</code> would be a relative sample index, because you don't necessarily care about the absolute sample position). If you know how many samples have been consumed or produced before you start this loop, you can easily convert relative sample indices into absolute ones by using the [http://gnuradio.org/doc/doxygen/classgr_1_1block.html#a2279d1eb421203bc5b0f100a6d5dc263 <code>nitems_read(port_num)</code>] and [http://gnuradio.org/doc/doxygen/classgr_1_1block.html#a742128a481fcb9e43a3e0cd535a57f9e <code>nitems_written(port_num)</code>] methods (note for experts and those who don't want to make difficult to debug mistakes: Calling the [http://gnuradio.org/doc/doxygen/classgr_1_1block.html#a47193333ce3fe1536fb1663bf314f63d <code>consume()</code>] and [http://gnuradio.org/doc/doxygen/classgr_1_1block.html#aa5581727d057bdd8113f8b2a3fc5bd66 <code>produce()</code>] functions will change these values).
 
In the following example, we'll stream valid QPSK symbols and zeros in an alternating fashion. See how the stream tags are added onto the items:
 
(Make picture)
 
A downstream block can now make use of these tags. To do so, we need the [http://gnuradio.org/doc/doxygen/classgr_1_1block.html#aa0272555827fe26a1878e53ce4be092c <code>get_tags_in_range()</code>] call, which reads tags within a given range of absolute offsets. To read a tag from one of the first hundred input items, this would be a valid call:
 
<pre>std::vector tags;
get_tags_in_range(
    tags, // Tags will be saved here
    0, // Port 0
    nitems_read(0), // Start of range
    nitems_read(0) + 100, // End of range
    pmt::mp(&quot;my_tag_key&quot;) // Optional: Only find tags with key &quot;my_tag_key&quot;
);</pre>
The tags will all be saved into the vector <code>tags</code>. Any tag that is connected to the first 100 items will be placed into this vector, although not in any specific order. The optional fifth argument lets us filter for tags with a specific key, in case we're only looking for a single type of tag.
 
There is a shortcut to this call, called [http://gnuradio.org/doc/doxygen/classgr_1_1block.html#abf2cc497d68c4650be55765d0fe12291 get_tags_in_window], which searches for tags on relative indices. The following code would find the same tags as before:
 
<pre>std::vector tags;
get_tags_in_window( // Note the different method name
    tags, // Tags will be saved here
    0, // Port 0
    0, // Start of range (relative to nitems_read(0))
    100, // End of relative range
    pmt::mp(&quot;my_tag_key&quot;) // Optional: Only find tags with key &quot;my_tag_key&quot;
);</pre>
 
=== 5.2.3 Tag propagation ===
 
We now know how to add tags to streams, and how to read them. But what happens to tags after they were read? And what happens to tags that aren't used? After all, there are many blocks that don't care about tags at all.
 
The answer is: It depends on the ''tag propagation policy'' of a block what happens to tags that enter it.<br />
There are [http://gnuradio.org/doc/doxygen/classgr_1_1block.html#abc40fd4d514724a5446a2b34b2352b4e three policies] to choose from:
 
* TPP_ALL_TO_ALL: Any tag that enters on any port is propagated automatically to all output ports (this is the default setting)
* TPP_ONE_TO_ONE: Tags entering on port N are propagated to output port N. This only works for blocks with the same number of in- and output ports.
* TPP_DONT: Tags entering the block are not automatically propagated. Only tags created within this block (using <code>add_item_tag()</code>) appear on the output streams.
 
We generally set the tag propagation policy in the block's constructor using [http://gnuradio.org/doc/doxygen/classgr_1_1block.html#a476e218927e426ac88c26431cbf086cd @set_tag_propagation_policy]
 
When the tag propagation policy is set to TPP_ALL_TO_ALL or TPP_ONE_TO_ONE, the GNU Radio scheduler uses any information available to figure out which output item corresponds to which input item. The block may read them and add new tags, but existing tags are automatically moved downstream in a manner deemed appropriate.
 
As an example, consider an interpolating block. See the following flow graph:
 
[[File:tags_interp.grc.png|700px|tags_interp.grc.png]]
 
[[File:tags_interp.png|500px|tags_interp.png]]<br />
 
As you can tell, we produce tags on every 10th sample, and then pass them through a block that repeats every sample 100 times. Tags do not get repeated with the standard tag propagation policies (after all, they're not tag ''manipulation'' policies), so the scheduler takes care that every tag is put on the output item that corresponds to the input item it was on before. Here, the scheduler makes an educated guess and puts the tag on the '''first''' of 100 items.
 
Note: We can't use one QT GUI Time Sink for both signals here, because they are running at a different rate. Note the time difference on the x-axis!
 
On decimating blocks, the behavior is similar. Consider this very simple flow graph, and the position of the samples:
 
[[File:tags_decim.grc.png|400px|tags_decim.grc.png]]
 
[[File:tags_decim.png|500px|tags_decim.png]]<br />
 
 
We can see that no tags are destroyed, and tags are indeed spaced at one-tenth of the original spacing of 100 items. Of course, the actual item that was passed through the block might be destroyed, or modified (think of a decimating FIR filter).
 
In fact, this works with any rate-changing block. Note that there are cases where the relation of tag positions of in- and output are ambiguous, the GNU Radio scheduler will then try and get as close as possible.
 
Here's another interesting example: Consider this flow graph, which has a delay block, and the position of the tags after it:
 
[[File:tags_ramp_delay.grc.png|500px|tags_ramp_delay.grc.png]]
 
[[File:tags_ramp_delay.png|500px|tags_ramp_delay.png]]<br />
 
 
Before the delay block, tags were positioned at the beginning of the ramp. After the delay, they're still in the same position! Would we inspect the source code of the delay block, we'd find that there is absolutely no tag handling code. Instead, the block [http://gnuradio.org/doc/doxygen/classgr_1_1block.html#acad5d6e62ea885cb77d19f72451581c2 declares a delay] to the scheduler, which then propagates tags with this delay.
 
Using these mechanisms, we can let GNU Radio handle tag propagation for a large set of cases. For specialized or corner cases, there is no option than to set the tag propagation policy to <code>TPP_DONT</code> and manually propagate tags (actually, there's another way: Say we want most tags to propagate normally, but a select few should be treated differently. We can use [http://gnuradio.org/doc/doxygen/classgr_1_1block.html#a461f6cc92174e83b10c3ec5336036768 <code>remove_item_tag()</code>] (DEPRECATED. Will be removed in 3.8.) to remove these tags from the input; they will then not be propagated any more even if the tag propagation policy is set to something other than <code>TPP_DONT</code>. But that's more advanced use and will not be elaborated on here).
 
==== Use case: FIR filters ====
 
(Note: this section requires knowledge of digital signal processing, you can skip to the next section if you don't understand some of the expressions).
 
Assume we have a block that is actually an FIR filter. We want to let GNU Radio handle the tag propagation. How do we configure the block?
 
Now, an FIR filter has one input and one output. So, it doesn't matter if we set the propagation policy to <code>TPP_ALL_TO_ALL</code> or <code>TPP_ONE_TO_ONE</code>, and we can leave it as the default. The items going in and those coming out are different, so how do we match input to output? Since we want to preserve the timing of the tag position, we need to use the filter's '''group delay''' as a delay for tags (which for this symmetric FIR filter is <code>(N-1)/2</code>, where <code>N</code> is the number of filter taps). Finally, we might be interpolating, decimating or both (say, for sample rate changes) and we need to tell the scheduler about this as well.
 
== 5.3 Message Passing ==
 
The [http://gnuradio.org/doc/doxygen/page_msg_passing.html message passing interface] works completely differently than the stream tags. Messages are pure PMTs and they have no offset (this would not make any sense anyway, as they are not connected to an item, which in turn has an offset) and no keys (although we can create key/value pairs by making the PMT a pair or a dictionary). Another difference is that we have a different type of port for messages, which can't be sent to regular streaming ports. We call these new types of ports ''message ports'', as opposed to the kind we've used so far, which we shall call ''streaming ports'' if we need to be explicit.
 
Messages usually aren't used for samples, but rather for packets or data that makes sense to be moved about in a packetized fashion. Of course, there are many cases where both message passing and streaming interfaces can be used, and it's up to us to choose which we prefer.
 
Here's a simple example of a flow graph using both streaming and messages:
 
[[File:strobe.grc.png|550px|strobe.grc.png]]
 
There are several interesting things to point out. First, there are two source blocks, which both output items at regular intervals, one every 1000 and one every 750 milliseconds. Dotted lines denote connected message ports, as opposed to solid lines, which denote connected streaming ports. In the top half of the flow graph, we can see that it is, in fact, possible to switch between message passing and streaming ports, but only if the type of the PMTs matches the type of the streaming ports (in this example, the pink color of the streaming ports denotes bytes, which means the PMT should be a u8vector if we want to stream the same data we sent as PMT).
 
Another interesting fact is that we can connect more than one message output port to a single message input port, which is not possible with streaming ports. This is due to the '''asynchronous''' nature of messages: The receiving block will process all messages whenever it has a chance to do so, and not necessarily in any specific order. Receiving messages from multiple blocks simply means that there might be more messages to process.
 
What happens to a message once it was posted to a block? This depends on the actual block implementation, but there are two possibilities:
 
1) A message handler is called, which processes the message immediately.<br />
2) The message is written to a FIFO buffer, and the block can make use of it whenever it likes, usually in the work function.
 
For a block that has both message ports and streaming ports, any of these two options is OK, depending on the application. However, we strongly discourage the processing of messages inside of a work function and instead recommend the use of message handlers. Using messages in the work function encourages us to block in work waiting for a message to arrive. This is bad behavior for a work function, which should never block. If a block depends upon a message to operate, use the message handler concept to receive the message, which may then be used to inform the block's actions when the work function is called. Only on specially, well-identified occasions should we use method 2 above in a block.
 
With a message passing interface, we can write blocks that don't have streaming ports, and then the work function becomes useless, since it's a function that is designed to work on streaming items. In fact, blocks that don't have streaming ports usually don't even have a work function.
 
=== 5.3.1 PDUs ===
 
In the previous flow graph, we have a block called ''PDU to Tagged Stream''. A PDU (protocol data unit) in GNU Radio has a special PMT type, it is a pair of a dictionary (on CAR) and a uniform vector type. So, this would yield a valid PDU, with no metadata and 10 zeros as stream data:
 
<pre>pdu = pmt.cons(pmt.make_dict(), pmt.make_u8vector(10, 0))</pre>
The key/value pairs in the dictionary are then interpreted as key/value pairs of stream tags.
 
=== 5.3.2 Adding Message Passing to the Code ===
 
Unlike streaming ports, message ports are not defined in the I/O signature, but are declared with the [http://gnuradio.org/doc/doxygen/classgr_1_1basic__block.html#a362b6de38292cca9c1d56439a6efad04 <code>message_port_register_*</code>] functions. Message ports also have identifiers instead of port numbers; those identifiers are also PMTs. To add in- or output ports for messages to a block, the constructor must be amended by lines like these:
 
<pre>// Put this into the constructor to create message ports
message_port_register_in(pmt::mp(&quot;in_port_name&quot;));
message_port_register_out(pmt::mp(&quot;out_port_name&quot;));</pre>
To set up a function as the message handler for an input message port that is called every time we receive a message, we add line like this:
 
<pre>// Put this into the constructor after the input port definition:
set_msg_handler(
    pmt::mp(&quot;in_port_name&quot;), // This is the port identifier
    boost::bind(&amp;block_class_name::msg_handler_method, this, _1)  // [FIXME class name] Bind the class method
);</pre>
We use [http://www.boost.org/doc/libs/1_55_0/libs/bind/bind.html Boost's <code>bind</code>] technique to tell GNU Radio about the function we want to call for messages on this input port. If you don't know <code>boost::bind</code>, don't worry, the syntax is always the same: The first argument is a pointer to the method that is called, the second is <code>this</code> so the method is called from the correct class and the third argument is <code>_1</code>, which means the function takes one argument, which is the message PMT.
 
All message handlers, like the function &quot;msg_handler_method&quot; used above, have the same function prototype and are members of the block's implementation class:
 
<pre>    void msg_handler_method(pmt::pmt_t msg);</pre>
In a Python block, these lines would look like this:
 
<pre># Put this into the constructor to create message ports
self.message_port_register_in(pmt.intern(&quot;in_port_name&quot;))
self.message_port_register_out(pmt.intern(&quot;out_port_name&quot;))
self.set_msg_handler(pmt.intern(&quot;in_port_name&quot;), self.msg_handler_method) # No bind necessary, we can pass the function directly</pre>
Message inputs and outputs can be connected similarly to streaming ports only using [http://gnuradio.org/doc/doxygen/classgr_1_1hier__block2.html#a8abff96c559930893d3ffdb22a3c43fd <code>msg_connect</code>] instead of <code>connect</code>:
 
<pre>tb = gr.top_block()
# Send the string &quot;message&quot; once every 1000 ms
src = blocks.message_strobe(pmt.to_pmt(&quot;message&quot;), 1000)
dbg = blocks.message_debug()
tb.msg_connect(src, &quot;pdus&quot;, dbg, &quot;print&quot;)</pre>
Note the <code>msg_connect()</code> call instead of the <code>connect()</code> function we use for streaming ports.
 
=== 5.3.3 Example: Chat Application ===
 
Let's build an application that uses message passing. A chat program is an ideal use case, since it waits for the user to type a message, and then sends it.
 
We will make two blocks: one for sending message, and one for receiving. The latter is pretty simple: when it receives a message, it will print it to the console. The former is a bit more fancy: first it will prepend a string, which we use to identify the user name of the person transmitting. Also, it will strip any non-printable characters from the string (we will do the same in the receiver, too, just to make sure no one screws up our console with funny characters).
 
Finally, let's specify that we will transmit data as u8vectors. This makes our blocks compatible to other blocks in GNU Radio.
 
Let's have a look at the blocks: [https://github.com/gnuradio/gr-tutorial/blob/master/python/chat_blocks.py]
 
The first block, <code>chat_sanitizer</code>, has a function, <code>post_message()</code>, which can be called from outside to initiate a message transfer. It does all the string sanitizing we mentioned earlier, and then converts the string to a u8vector.
 
The second block, <code>chat_receiver</code>, also has a function (<code>handle_msg()</code>) which is always called when a message is posted to this block. This is enabled by registering this function as a message handler.


Note that we don't simply pass the u8vector, but rather pass a pair of the u8vector and another PMT, which we can use for metadata. As an example, a block that converts a stream to PDUs would output the stream data as data, and tags as metadata in this PMT pair.
<merged with Stream Tags usage manual page>


This file contains the two blocks which we can use in flow graphs, but it can also be run directly. In this case, it will directly connect these two blocks, and provide a prompt for the user so we can test these blocks.
== Message Passing ==


A more elaborate example is given in [https://github.com/gnuradio/gr-tutorial/blob/master/examples/tutorial5/chat_app.grc]. Here we can see how these two blocks are used to create a flow graph that actually uses QPSK mapping to transfer the data.
<moved to message passing page of the usage manual>


[[Category:Guided Tutorials]]
[[Category:Guided Tutorials]]

Latest revision as of 11:45, 8 April 2021

Tags, Messages and PMTs.

Objectives

  • Learn about PMTs
  • Understand what tags are / what they do / when to use them
  • Understand difference between streaming and message passing
  • Point to the manual for more advanced block manipulation

Prerequisites

So far, we have only discussed data streaming from one block to another. The data often consists of samples, and a streaming architecture makes a lot of sense for those. For example, a sound card driver block will constantly produce audio samples once active.

In some cases, we don't want to pipe a stream of samples, though, but rather pass individual messages to another block, such as "this is the first sample of a burst", or "change the transmit frequency to 144 MHz". Or consider a MAC layer on top of a PHY: At higher communication levels, data is usually passed around in PDUs (protocol data units) instead of streams of items.

In GNU Radio we have two mechanisms to pass these messages:

Before we discuss these, let's consider what such a message is from a programming perspective. It could be a string, a vector of items, a dictionary... anything, really, that can be represented as a data type. In Python, this would not be a problem, since it is weakly typed, and a message variable could simply be assigned whatever we need. C++ on the other hand is strongly typed, and it is not possible to create a variable without knowing its type. What makes things harder is that we need to be able to share the same data objects between Python and C++. To circumvent this problem, we introduce polymorphic types (PMTs).

Polymorphic Types (PMT)

<content merged with PMT page in usage manual>

OK, so now we know all about creating messages - but how do we send them from block to block?

Stream Tags

<merged with Stream Tags usage manual page>

Message Passing

<moved to message passing page of the usage manual>

Retrieved from "https://wiki.gnuradio.org/index.php?title=Guided_Tutorial_Programming_Topics&oldid=8455"