Packet Communications Test Page: Difference between revisions

From GNU Radio
Jump to navigation Jump to search
No edit summary
(add section 'Using an alternate File Source block')
 
(30 intermediate revisions by the same user not shown)
Line 1: Line 1:
<span data-vad="gr_vad"></span>
<span id="gr_vad"></span>
 
<b>NOTE:</b> Changing the selected GNU Radio version will not change the Table of Contents.
This tutorial presents methods to transmit and receive packet data using burst transmissions. The examples show data originating from a PDU message source. A hardware simulation is shown using BPSK modulation.
<hr>


This tutorial presents methods to transmit and receive packet data. As was shown in the QPSK and BPSK tutorials, the received bit data stream could be verified by comparing it to the (delayed) transmitted data. However, there was no way to recover the byte alignment of those bits. That is where packet processing comes into play.
<br><br>
== Prerequisites ==
== Prerequisites ==


Line 15: Line 17:
== Notes ==
== Notes ==


The current tutorial has been revised and tested with GNU Radio version 3.9.4.0. The 'Polyphase Clock Sync' has been replaced with the [[Symbol_Sync]]; and the 'CMA Equalizer' has been replaced with the [[Linear_Equalizer]] and [[Adaptive_Algorithm]].
See [[Porting_Existing_Flowgraphs_to_a_Newer_Version|Porting Existing Flowgraphs to a Newer Version]] to adapt the flowgraphs presented below to your GNU Radio version.


Links to the flowgraphs have been updated to GNU Radio version 3.9.4.0.
== Header Format Object ==


If using GNURadio 3.8:  
The following blocks:
* [[Protocol_Formatter]],
* [[Protocol_Formatter_(Async)]],
* [[Protocol_Parser]],
* [[Packet_Header_Generator]], and
* [[Packet_Header_Parser]]
use a "Header Format Object", but there is no block with that name. To create such an object, a [[Variable]] block is used. The 'Value' field can be one of several chioces depending on what type of header is needed, such as:
    * digital.header_format_default(access_code, threshold)
    * digital.header_format_crc(len_key, num_key)
    * digital.header_format_counter(access_code, threshold, bps)


-In the "''Simulating Baseband Packet Processing''" Tutorial, "'''NOTE:'''" sections have been added where a small change is needed or a difference occurs.
See specific examples below.


-The "''Using BPSK with Hardware Simulation''" has an entirely separate section for version 3.8.
== Simulating Packet Comms Using Messages ==


== Simulating Baseband Packet Processing ==
=== Basic Packet Processing ===


In order to grasp the basics of packet processing, this section presents a transmitter and receiver simulation using standard GNU Radio blocks without any modulation or channel impairments. As was shown in the QPSK and BPSK tutorials, the received bit data stream could be verified by comparing it to the (delayed) transmitted data. However, there was no way to recover the byte alignment of those bits. That is where packet processing comes into play.
In order to grasp the basics of packet processing, this section presents a transmitter and receiver simulation using standard GNU Radio blocks without any modulation or channel impairments.


=== Building the flowgraph ===
==== Building the flowgraph ====


Build the following flowgraph using the details given below:
Build the following flowgraph using the details given below:
Line 39: Line 50:
This flowgraph can be downloaded from [[Media:Pkt_7_base.grc]].
This flowgraph can be downloaded from [[Media:Pkt_7_base.grc]].


'''NOTE: If using GNURadio 3.8, a small change will need to be made:
<div data-vad="3.8">
Move the connections coming out of the 'header' and 'payload' ports on the async protocol formatter from the 'print' port of the Message Debug block to the 'print_pdu' port.   
<b>NOTE:</b> If using GNURadio 3.8, move the connections coming out of the 'header' and 'payload' ports on the async protocol formatter from the 'print' port of the Message Debug block to the 'print_pdu' port.   
'''
</div>


==== Message Strobe ====
; Message Strobe
For the Message Strobe to generate a PDU, the Message PMT must be of the form
For the Message Strobe to generate a PDU, the Message PMT must be of the form
   pmt.cons(pmt.PMT_NIL,pmt.init_u8vector(9,(71,78,85,32,82,97,100,105,111)))
   pmt.cons(pmt.PMT_NIL,pmt.init_u8vector(9,(71,78,85,32,82,97,100,105,111)))
This specific vector has a length of 9 and the character values of "GNU Radio". It is sent every two seconds.
This specific vector has a length of 9 and the character values of "GNU Radio". It is sent every two seconds.


==== Async CRC32 ====
; Async CRC32
 
The Async CRC32 block computes the CRC of the payload and appends the value to it.
The Async CRC32 block computes the CRC of the payload and appends the value to it.


==== Variable, Id: hdr_format ====
; Variable, Id = hdr_format
The Protocol Formatter uses a Format Object to define its parameters. The one used in this example is
The Protocol Formatter uses a Format Object to define its parameters. The one used in this example is
   hdr_format digital.header_format_crc(len_key, num_key)
   hdr_format digital.header_format_crc(len_key, num_key)
It generates a header with the packet length, packet number, and a 16-bit CRC. No access code is created.
It generates a header with the packet length, packet number, and a 8-bit CRC. No access code is created.


==== Protocol Formatter ====
; Protocol Formatter (Async)
The entry for the Format Obj. is 'hdr_format'
The entry for the Format Obj. is 'hdr_format'


==== Tagged Stream Mux ====
; PDU to Tagged Stream
Converts received PDUs into a tagged stream by adding length tags.


; Tagged Stream Mux
The Tagged Stream Mux combines the header and the payload.
The Tagged Stream Mux combines the header and the payload.


==== Header/Payload Demux ====
; Header/Payload Demux
 
The Header/Payload Demux block synchronizes with the header data and then outputs the payload data.
The Header/Payload Demux block synchronizes with the header data and then outputs the payload data.


==== Protocol Parser ====
; Protocol Parser
The entry for the Format Obj. is 'hdr_format'
The entry for the Format Obj. is 'hdr_format'


=== Testing ===
<hr>
 
==== Testing ====


To test the flowgraph, click the "Execute the flowgraph" icon, or press F6. An example of the output is:
To test the flowgraph, click the "Execute the flowgraph" icon, or press F6. An example of the output is:


<div data-vad="3.9,3.10">
==== Version 3.9+ ====
<pre>
<pre>
***** VERBOSE PDU DEBUG PRINT ******
***** VERBOSE PDU DEBUG PRINT ******
Line 102: Line 117:
************************************
************************************
</pre>
</pre>
</div>


The first PDU DEBUG PRINT is the header data containing the payload length, packet number, and a CRC8 of the header.
<div data-vad="3.8">
 
==== Version 3.8 ====
The second PDU DEBUG PRINT is the payload including a CRC32 of the data.
 
The Tag Debug shows the length tag and the packet number tag.
 
The third PDU DEBUG PRINT is the received payload ("GNU Radio").
 
'''NOTE: If using GNURadio 3.8, the output will differ in the VERBOSE PDU DEBUG PRINT section. Instead of the hexidecimal representation of "GNU Radio", the actual ASCII text "GNU Radio" will print in the termainal as shown below.'''
 
<pre>
<pre>
* MESSAGE DEBUG PRINT PDU VERBOSE *
* MESSAGE DEBUG PRINT PDU VERBOSE *
Line 137: Line 145:
************************************
************************************
</pre>
</pre>
</div>
The first PDU DEBUG PRINT is the header data containing the payload length, packet number, and a CRC8 of the header.


== Using BPSK with Hardware Simulation (version 3.9) ==
The second PDU DEBUG PRINT is the payload including a CRC32 of the data.


The Protocol Formatter supports three header format types: default, count, and crc. The crc style is illustrated in the simulation above.
The Tag Debug shows the length tag and the packet number tag.
 
The third PDU/MESSAGE DEBUG PRINT is the received payload ("GNU Radio").
<hr>
 
<div data-vad="3.9,3.10">
=== Using BPSK with Hardware Simulation (version 3.9+) ===


In order to create a preamble, a default header, and the payload data in one PDU message, an Embedded Python block was created. The code is contained in the GRC file, but also is shown here:
In order to create a preamble, a default header, and the payload data in one PDU message, an Embedded Python block was created. The code is contained in the GRC file, but also is shown here:
Line 183: Line 200:
</pre>
</pre>


The preamble is 32 bytes of 0x55 giving alternating 1's and 0's. The non-cyclical 32-bit access code is 0xe15ae893 (binary '11100001010110101110100010010011'). The length of the payload is repeated (16 bits). Then the payload is appended.
The preamble is 32 bytes of 0x55 giving alternating 1's and 0's. The non-cyclical 32-bit access code is 0xe15ae893 (decimal 225,90,232,147) (binary '11100001010110101110100010010011'). The length of the payload is repeated (16 bits). Then the payload is appended.


=== Transmitting a Signal ===
==== Transmitting a Signal ====


Build the following flowgraph using the details given below:
Build the following flowgraph using the details given below:
Line 195: Line 212:
This flowgraph can be downloaded from [[Media:Pkt_xmt.grc]]. The BPSK modulation is done in the same manner as the [[Simulation_example:_BPSK_Demodulation|BPSK Demodulation]] tutorial. Do a Generate (F5) or Execute (F6) to create the pkt_xmt.py file.
This flowgraph can be downloaded from [[Media:Pkt_xmt.grc]]. The BPSK modulation is done in the same manner as the [[Simulation_example:_BPSK_Demodulation|BPSK Demodulation]] tutorial. Do a Generate (F5) or Execute (F6) to create the pkt_xmt.py file.


=== Receiving a Signal ===
==== Receiving a Signal ====


Build a separate flowgraph using the details below:
Build a separate flowgraph using the details below:
Line 209: Line 226:
The remaining blocks operate as in the simulation example above.
The remaining blocks operate as in the simulation example above.


=== Testing ===
==== Testing ====


When using GRC, doing a Generate and/or Run creates a Python file with the same name as the .grc file. You can execute the Python file without running GRC again.
When using GRC, doing a Generate and/or Run creates a Python file with the same name as the .grc file. You can execute the Python file without running GRC again.
Line 242: Line 259:


To terminate each of the processes cleanly, click on the 'X' in the upper corner of the GUI rather than using Control-C.
To terminate each of the processes cleanly, click on the 'X' in the upper corner of the GUI rather than using Control-C.
</div>
<hr>


== Using BPSK with Hardware Simulation (version 3.8) ==
<div data-vad="3.8">
 
=== Using BPSK with Hardware Simulation (version 3.8) ===
The Protocol Formatter supports three header format types: default, count, and crc. The crc style is illustrated in the simulation above.


In order to create a preamble, a default header, and the payload data in one PDU message, an Embedded Python block was created. The code is contained in the GRC file, but also is shown here:
In order to create a preamble, a default header, and the payload data in one PDU message, an Embedded Python block was created. The code is contained in the GRC file, but also is shown here:
* note that for GNURadio 3.8 a numpy array is used in the handling of the incoming PMT message. It must also be put into a byte array before being placed into the PMT u8vector for publishing to the "PDU_out" port  
* A numpy array is used in the handling of the incoming PMT message. It must also be put into a byte array before being placed into the PMT u8vector for publishing to the "PDU_out" port  
<pre>
<pre>
"""
"""
Line 295: Line 313:
</pre>
</pre>


=== Transmitting a Signal ===
==== Transmitting a Signal ====
Build the following flowgraph using the details given below:
Build the following flowgraph using the details given below:


Line 304: Line 322:
This flowgraph can be downloaded from [[Media:Pkt_xmt_gr38.grc]]. The BPSK modulation is done in the same manner as the [[Simulation_example:_BPSK_Demodulation|BPSK Demodulation]] tutorial. Do a Generate (F5) or Execute (F6) to create the pkt_xmt_gr38.py file.
This flowgraph can be downloaded from [[Media:Pkt_xmt_gr38.grc]]. The BPSK modulation is done in the same manner as the [[Simulation_example:_BPSK_Demodulation|BPSK Demodulation]] tutorial. Do a Generate (F5) or Execute (F6) to create the pkt_xmt_gr38.py file.


=== Receiving a Signal ===
==== Receiving a Signal ====
Build a separate flowgraph using the details below:
Build a separate flowgraph using the details below:
* note for GNURadio 3.8: the Linear Equalizer shown in the GNURadio 3.9 example is not available. The LMS DD Equalizer has been used instead.


[[File:pkt_rcv_gr38.png|800px]]
[[File:pkt_rcv_gr38.png|800px]]
Line 317: Line 333:
The output of the Map block is the received bit stream. The 'Correlate Access Code - Tag Stream' block searches for the given access code by slicing the soft decision symbol inputs. Once found, it expects the following 32 samples to contain a header that includes the frame length (16 bits for the length, repeated). It decodes the header to get the frame length in order to set up the the tagged stream key information.
The output of the Map block is the received bit stream. The 'Correlate Access Code - Tag Stream' block searches for the given access code by slicing the soft decision symbol inputs. Once found, it expects the following 32 samples to contain a header that includes the frame length (16 bits for the length, repeated). It decodes the header to get the frame length in order to set up the the tagged stream key information.


The remaining blocks operate as in the simulation example above.
==== Testing ====
 
=== Testing ===


When using GRC, doing a Generate and/or Run creates a Python file with the same name as the .grc file. You can execute the Python file without running GRC again.
When using GRC, doing a Generate and/or Run creates a Python file with the same name as the .grc file. You can execute the Python file without running GRC again.
Line 354: Line 368:


To terminate each of the processes cleanly, click on the 'X' in the upper corner of the GUI rather than using Control-C.
To terminate each of the processes cleanly, click on the 'X' in the upper corner of the GUI rather than using Control-C.
</div>
== Simulating Packet Comms Using Streams ==
=== Using Header Format CRC and Protocol Parser ===
==== Building the flowgraph ====
Build the following flowgraph using the details given below:
[[File:Str_pkt_1_fg.png|800px]]
<hr>
This flowgraph can be downloaded from [[Media:Str_pkt_1.grc]].
; File Source
The File Source reads a padded text file and outputs to a stream.
; Stream to Tagged Stream


== Adding Channel Impairments ==
Converts a regular stream into a tagged stream by adding length tags in regular intervals.


<b>NOTE:</b> For the testing above, the transmitter data on port 49203 was fed directly into the receiver on port 49203. In this section, we will insert another flowgraph between those two ZMQ ports, so the ZMQ SUB Source port in the receiver needs to be changed to 49201 and the flowgraph regenerated.
; Stream CRC32


Build the following flowgraph:
The Stream CRC32 block computes the CRC of the payload and appends the value to it.


[[File:Chan_loopback_fg.png|810px]]
; Protocol Formatter
The entry for the Format Obj. is 'hdr_format'
 
; Protocol Parser
The entry for the Format Obj. is 'hdr_format'


<hr>
<hr>
==== Testing ====


This flowgraph can be downloaded from [[Media:Chan_loopback.grc]].
The success of this methodology is based on the input file being padded to a multiple of 'packet_len' bytes so that the last few bytes of the source file are not lost in a partial unprocessed packet. To create a padded file for input to the flowgraph, the following Python program can be used:


For testing this arrangement we will use three processes, so we will need three terminal windows.
<pre>
#!/usr/bin/python3
# -*- coding: utf-8 -*-


Terminal 1:
# Padded_File_Source.py
* since you just finished building the chan_loopback flowgraph, you can just do a Run. After a few seconds, a window will open with GUI Range controls for noise level, frequency offset, and timing offset.
 
import os.path
import sys
 
Pkt_len = 252
_debug = 1          # set to zero to turn off diagnostics
 
if (len(sys.argv) < 2):
    print ('Usage: python3 Padded_File_Source.py <input file>')
    print ('Number of arguments=', len(sys.argv))
    print ('Argument List:', str(sys.argv))
    exit (1)
# test if file exists
fn = sys.argv[1]
if (_debug):
    print (fn)
if not(os.path.exists(fn)):
    print('The input file does not exist')
    exit (1)
 
# open input file
f_in = open (fn, 'r')
 
# open output file
f_out = open ("padded.txt", 'w')
 
while True:
    buff = f_in.read (Pkt_len)
    b_len = len(buff)
    if (_debug):
        print (b_len)
    if b_len == 0:
        print ('End of file')
        break
    while (b_len < Pkt_len):
        buff += ' '
        b_len += 1
    # write output
    f_out.write (buff)
 
f_in.close()
f_out.close()
</pre>
 
Note that the 'Pkt_len' in the Python program and the 'packet_len' value in the flowgraph 'Stream to Tagged Stream' block must be the same. The output of the Python program is a file named "padded.txt".
 
To test the flowgraph:
* Run the Python program to produce a padded text file.
* Execute the flowgraph.
* Examine the output file "output.txt". The output file will be the same as the original file, but will have trailing spaces at the end as a result of the padding.
<hr>
 
=== Using Header Format Default and Correlate Access Code ===
 
==== Building the flowgraph ====
 
Build the following flowgraph using the details given below:
 
[[File:Str_pkt_11_fg.png|800px]]
 
<hr>
This flowgraph can be downloaded from [[Media:Str_pkt_11.grc]].
 
; File Source
 
The File Source reads a padded text file and outputs to a stream.
 
; Stream to Tagged Stream
 
Converts a regular stream into a tagged stream by adding length tags in regular intervals.
 
; Stream CRC32
 
The Stream CRC32 block computes the CRC of the payload and appends the value to it.
 
; Protocol Formatter
The entry for the Format Obj. is 'hdr_format'
 
; Correlate Access Code - Tag Stream
The Access Code is a ''string'' with the 32 bit value.
 
<hr>
==== Testing ====


Terminal 2:
Refer to the [[Packet_Communications_Test_Page#Testing_4|Testing]] section above.
Open another terminal window.
* change to whatever directory you used to generate the flowgraph for pkt_xmt
* execute the following command:
    python3 pkt_xmt.py
* After a few seconds, a window will open with the GUI Time Sink.


Terminal 3:
=== Using an alternate File Source block ===
Open another terminal window.
* change to whatever directory you used to generate the flowgraph for pkt_rcv
* execute the following command:
    python3 pkt_rcv.py
* After a few seconds, a window will open with the GUI Constellation Sink.


Now you can adjust the various channel impairments to see the effect on the received constellation and the ability to synchronize on the received data. The sample rate should be set to 768kHz.
There is an Out-of-Tree (OOT) module which reads a text or binary file and produces a stream output with 'packet_len' tags for GNU Radio. Functionally it replaces a 'File Source' block and a 'Stream to Tagged Stream' block. The advantage of this block is that when the input file size is not an exact multiple of the selected packet length, the remainder at the end of the file is not lost in the 'Stream to Tagged Stream' buffer.


To terminate each of the processes cleanly, click on the 'X' in the upper corner of the GUI rather than using Control-C.
The module is in [https://github.com/duggabe/gr-File_Source_to_Tagged_Stream gr-File_Source_to_Tagged_Stream]. It contains build instuctions and an example flowgraph.


== Using Hardware ==
== Using Hardware ==


If you look at https://github.com/duggabe/gr-control you will see that this tutorial follows the same design. If the user were to build two computer installations with this setup, they could send and receive packets.
If you look at https://github.com/duggabe/gr-control you will see that this tutorial follows the same design. If the user were to build two computer installations with this setup, they could send and receive packets to each other.


== Conclusions ==
== Conclusions ==
Line 398: Line 511:
Based on this tutorial, there are several things which can be concluded:
Based on this tutorial, there are several things which can be concluded:


# BPSK and QPSK are not well suited for burst transmissions due to the need for the receiver to converge on the actual timing and frequency values.
# BPSK and QPSK are not well suited for burst transmissions due to the need for the receiver to converge on the actual timing and frequency values. The longer the times between bursts of data, the more the receiver will need to reconverge on the timing and frequency values for the new burst.
# The longer the times between bursts of data, the more the receiver will need to reconverge on the timing and frequency values for the new burst.
# Since almost all amateur radio communications are half duplex and bursts of data, a more appropriate modulation would be FSK or AFSK. The same packet formatting and recovery would still apply. A proof of concept has been added to https://github.com/duggabe/gr-control with <code>pkt_fsk_xmt.grc</code> and <code>pkt_fsk_rcv.grc</code>.
# Since almost all amateur radio communications are half duplex and bursts of data, a more appropriate modulation would be FSK or AFSK. The same packet formatting and recovery would still apply. A proof of concept has been added to https://github.com/duggabe/gr-control with <code>pkt_fsk_xmt.grc</code> and <code>pkt_fsk_rcv.grc</code>.


Line 407: Line 519:


* The 'Header/Payload Demux' and 'Protocol Parser' blocks seem to work only with the 'digital.header_format_crc' format.
* The 'Header/Payload Demux' and 'Protocol Parser' blocks seem to work only with the 'digital.header_format_crc' format.
* The 'Correlate Access Code - Tag Stream' block seems to require the Default Header Format Obj. and expects a 32-bit access code.
* The 'Correlate Access Code - Tag Stream' block requires the Default Header Format Obj. and expects a 32-bit access code.
 
[[Category:Tutorials]]
[[Category:Version-adaptive docs]]

Latest revision as of 01:24, 24 March 2023

NOTE: Changing the selected GNU Radio version will not change the Table of Contents.


This tutorial presents methods to transmit and receive packet data. As was shown in the QPSK and BPSK tutorials, the received bit data stream could be verified by comparing it to the (delayed) transmitted data. However, there was no way to recover the byte alignment of those bits. That is where packet processing comes into play.

Prerequisites

Notes

See Porting Existing Flowgraphs to a Newer Version to adapt the flowgraphs presented below to your GNU Radio version.

Header Format Object

The following blocks:

use a "Header Format Object", but there is no block with that name. To create such an object, a Variable block is used. The 'Value' field can be one of several chioces depending on what type of header is needed, such as:

   * digital.header_format_default(access_code, threshold)
   * digital.header_format_crc(len_key, num_key)
   * digital.header_format_counter(access_code, threshold, bps)

See specific examples below.

Simulating Packet Comms Using Messages

Basic Packet Processing

In order to grasp the basics of packet processing, this section presents a transmitter and receiver simulation using standard GNU Radio blocks without any modulation or channel impairments.

Building the flowgraph

Build the following flowgraph using the details given below:

Pkt 7 base fg.png


This flowgraph can be downloaded from Media:Pkt_7_base.grc.

NOTE: If using GNURadio 3.8, move the connections coming out of the 'header' and 'payload' ports on the async protocol formatter from the 'print' port of the Message Debug block to the 'print_pdu' port.

Message Strobe

For the Message Strobe to generate a PDU, the Message PMT must be of the form

 pmt.cons(pmt.PMT_NIL,pmt.init_u8vector(9,(71,78,85,32,82,97,100,105,111)))

This specific vector has a length of 9 and the character values of "GNU Radio". It is sent every two seconds.

Async CRC32

The Async CRC32 block computes the CRC of the payload and appends the value to it.

Variable, Id = hdr_format

The Protocol Formatter uses a Format Object to define its parameters. The one used in this example is

 hdr_format digital.header_format_crc(len_key, num_key)

It generates a header with the packet length, packet number, and a 8-bit CRC. No access code is created.

Protocol Formatter (Async)

The entry for the Format Obj. is 'hdr_format'

PDU to Tagged Stream

Converts received PDUs into a tagged stream by adding length tags.

Tagged Stream Mux

The Tagged Stream Mux combines the header and the payload.

Header/Payload Demux

The Header/Payload Demux block synchronizes with the header data and then outputs the payload data.

Protocol Parser

The entry for the Format Obj. is 'hdr_format'


Testing

To test the flowgraph, click the "Execute the flowgraph" icon, or press F6. An example of the output is:

Version 3.9+

***** VERBOSE PDU DEBUG PRINT ******
()
pdu length = 4 bytes
pdu vector contents = 
0000: 0d 10 00 3a 
************************************
***** VERBOSE PDU DEBUG PRINT ******
()
pdu length = 13 bytes
pdu vector contents = 
0000: 47 4e 55 20 52 61 64 69 6f 8f aa 09 f1 
************************************

----------------------------------------------------------------------
Tag Debug: 
Input Stream: 00
  Offset: 13  Source: n/a     Key: packet_len   Value: 13
  Offset: 13  Source: n/a     Key: packet_num   Value: 1
----------------------------------------------------------------------
***** VERBOSE PDU DEBUG PRINT ******
((packet_num . 1))
pdu length = 9 bytes
pdu vector contents = 
0000: 47 4e 55 20 52 61 64 69 6f 
************************************

Version 3.8

* MESSAGE DEBUG PRINT PDU VERBOSE *
()
pdu_length = 4
contents = 
0000: 0d 10 00 3a 
***********************************
* MESSAGE DEBUG PRINT PDU VERBOSE *
()
pdu_length = 13
contents = 
0000: 47 4e 55 20 52 61 64 69 6f 8f aa 09 f1 
***********************************

----------------------------------------------------------------------
Tag Debug: 
Input Stream: 00
  Offset: 13  Source: n/a     Key: packet_len   Value: 13
  Offset: 13  Source: n/a     Key: packet_num   Value: 1
----------------------------------------------------------------------
******* MESSAGE DEBUG PRINT ********
(((packet_num . 1)) . #[G N U   R a d i o])
************************************

The first PDU DEBUG PRINT is the header data containing the payload length, packet number, and a CRC8 of the header.

The second PDU DEBUG PRINT is the payload including a CRC32 of the data.

The Tag Debug shows the length tag and the packet number tag.

The third PDU/MESSAGE DEBUG PRINT is the received payload ("GNU Radio").


Using BPSK with Hardware Simulation (version 3.9+)

In order to create a preamble, a default header, and the payload data in one PDU message, an Embedded Python block was created. The code is contained in the GRC file, but also is shown here:

"""
Embedded Python Block
"""

import numpy as np
from gnuradio import gr
import pmt

class blk(gr.sync_block):
    """Packet Format"""

    def __init__(self):
        gr.sync_block.__init__(self,
            name = "Packet Format",
            in_sig = None,
            out_sig = None)
        self.message_port_register_in(pmt.intern('PDU_in'))
        self.message_port_register_out(pmt.intern('PDU_out'))
        self.set_msg_handler(pmt.intern('PDU_in'), self.handle_msg)

    def handle_msg(self, msg):
        inMsg = pmt.to_python (msg)
        pld = inMsg[1]
        # print (pld)
        mLen = len(pld)
        # print (mLen)
        if (mLen > 0):
            char_list = [85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,225,90,232,147]
            char_list.append (mLen >> 8)
            char_list.append (mLen & 255)
            char_list.append (mLen >> 8)
            char_list.append (mLen & 255)
            char_list.extend (pld)
            # print (char_list)
            out_len = len(char_list)
            # print (out_len)
            self.message_port_pub(pmt.intern('PDU_out'), pmt.cons(pmt.PMT_NIL,pmt.init_u8vector(out_len,(char_list))))

The preamble is 32 bytes of 0x55 giving alternating 1's and 0's. The non-cyclical 32-bit access code is 0xe15ae893 (decimal 225,90,232,147) (binary '11100001010110101110100010010011'). The length of the payload is repeated (16 bits). Then the payload is appended.

Transmitting a Signal

Build the following flowgraph using the details given below:

Pkt xmt fg.png


This flowgraph can be downloaded from Media:Pkt_xmt.grc. The BPSK modulation is done in the same manner as the BPSK Demodulation tutorial. Do a Generate (F5) or Execute (F6) to create the pkt_xmt.py file.

Receiving a Signal

Build a separate flowgraph using the details below:

Pkt rcv fg.png


This flowgraph can be downloaded from Media:Pkt_rcv.grc. For GNU Radio version 3.9 there are two changes: the Polyphase Clock Sync has been replaced by the Symbol_Sync block; and the CMA equalizer has been replaced by a Linear Equalizer and an Adaptive Algorithm.

The output of the Map block is the received bit stream. The 'Correlate Access Code - Tag Stream' block searches for the given access code by slicing the soft decision symbol inputs. Once found, it expects the following 32 samples to contain a header that includes the frame length (16 bits for the length, repeated). It decodes the header to get the frame length in order to set up the the tagged stream key information.

The remaining blocks operate as in the simulation example above.

Testing

When using GRC, doing a Generate and/or Run creates a Python file with the same name as the .grc file. You can execute the Python file without running GRC again.

For testing this system we will use two processes, so we will need two terminal windows.

Terminal 1:

  • since you just finished building the pkt_rcv flowgraph, you can just do a Run. After a few seconds, a window will open with the GUI Constellation Sink.

Terminal 2: Open another terminal window.

  • change to whatever directory you used to generate the flowgraph for pkt_xmt
  • execute the following command:
   python3 pkt_xmt.py
  • After a few seconds, a window will open with the GUI Time Sink.

The output of the transmitter is:

Pkt xmt out.png

The output of the receiver is shown on the GRC console.

***** VERBOSE PDU DEBUG PRINT ******
()
pdu length = 9 bytes
pdu vector contents = 
0000: 47 4e 55 20 52 61 64 69 6f 
************************************

The PDU DEBUG PRINT is the received payload ("GNU Radio").

To terminate each of the processes cleanly, click on the 'X' in the upper corner of the GUI rather than using Control-C.


Using BPSK with Hardware Simulation (version 3.8)

In order to create a preamble, a default header, and the payload data in one PDU message, an Embedded Python block was created. The code is contained in the GRC file, but also is shown here:

  • A numpy array is used in the handling of the incoming PMT message. It must also be put into a byte array before being placed into the PMT u8vector for publishing to the "PDU_out" port
"""
Embedded Python Block
"""

import numpy as np
from gnuradio import gr
import pmt
import array
class blk(gr.sync_block):
    """Packet Format"""

    def __init__(self):
        gr.sync_block.__init__(self,
            name = "Packet Format GR38",
            in_sig = None,
            out_sig = None)
        self.message_port_register_in(pmt.intern('PDU_in'))
        self.message_port_register_out(pmt.intern('PDU_out0'))
        self.set_msg_handler(pmt.intern('PDU_in'), self.handle_msg)

    def handle_msg(self, msg):
        inMsg = pmt.to_python (msg)
        pld = inMsg[1] ## type-> numpy.ndarray
        mLen = len(pld)
        if (mLen > 0):
            ## create a numpy array of type 'int' with preamble and sync word
            tmp_char_list = np.array([85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,85,225,90,232,147],dtype=int)
            ## append length 2x
            tmp_char_list=np.append(tmp_char_list,(mLen >> 8))
            tmp_char_list=np.append(tmp_char_list,(mLen & 255))
            tmp_char_list=np.append(tmp_char_list,(mLen >> 8))
            tmp_char_list=np.append(tmp_char_list,(mLen & 255))
            tmp_char_list_len=len(tmp_char_list)
            ## append original payload
            new_char_list=np.insert(tmp_char_list,tmp_char_list_len,pld)
            new_char_list_len=len(new_char_list)
            ## save final numpy array as byte array, this requires 'import array'
            ## (many thanks to Francis A. for finding this fix)
            byte_array_new_char_list=array.array('B',new_char_list)
            new_bytes_out_len = len(byte_array_new_char_list)
            ## create PMT u8vector using byte array
            new_out_bytes_pmt=pmt.cons(pmt.PMT_NIL,pmt.init_u8vector(new_bytes_out_len,(byte_array_new_char_list)))
            self.message_port_pub(pmt.intern('PDU_out0'), new_out_bytes_pmt)

Transmitting a Signal

Build the following flowgraph using the details given below:

Pkt xmt gr38.png


This flowgraph can be downloaded from Media:Pkt_xmt_gr38.grc. The BPSK modulation is done in the same manner as the BPSK Demodulation tutorial. Do a Generate (F5) or Execute (F6) to create the pkt_xmt_gr38.py file.

Receiving a Signal

Build a separate flowgraph using the details below:

Pkt rcv gr38.png


This flowgraph can be downloaded from Media:Pkt_rcv_gr38.grc. For GNU Radio version 3.8 there is one difference:the Linear Equalizer and Adaptive Algorithm object are not available, the LMS DD Equalizer block is used instead.

The output of the Map block is the received bit stream. The 'Correlate Access Code - Tag Stream' block searches for the given access code by slicing the soft decision symbol inputs. Once found, it expects the following 32 samples to contain a header that includes the frame length (16 bits for the length, repeated). It decodes the header to get the frame length in order to set up the the tagged stream key information.

Testing

When using GRC, doing a Generate and/or Run creates a Python file with the same name as the .grc file. You can execute the Python file without running GRC again.

For testing this system we will use two processes, so we will need two terminal windows.

Terminal 1:

  • since you just finished building the pkt_rcv_gr38 flowgraph, you can just do a Run. After a few seconds, a window will open with the GUI Constellation Sink.

Terminal 2: Open another terminal window.

  • change to whatever directory you used to generate the flowgraph for pkt_xmt_gr38
  • execute the following command:
   python3 pkt_xmt_gr38.py
  • After a few seconds, a window will open with the GUI Time Sink.

The output of the transmitter is:

Pkt tx gr38 gui.png

The output of the receiver is shown on the GRC console.

******* MESSAGE DEBUG PRINT ********
(() . #[G N U   R a d i o])
************************************
******* MESSAGE DEBUG PRINT ********
(() . #[G N U   R a d i o])
************************************
******* MESSAGE DEBUG PRINT ********
(() . #[G N U   R a d i o])
************************************

The MESSAGE DEBUG PRINT is the received payload ("GNU Radio").

To terminate each of the processes cleanly, click on the 'X' in the upper corner of the GUI rather than using Control-C.

Simulating Packet Comms Using Streams

Using Header Format CRC and Protocol Parser

Building the flowgraph

Build the following flowgraph using the details given below:

Str pkt 1 fg.png


This flowgraph can be downloaded from Media:Str_pkt_1.grc.

File Source

The File Source reads a padded text file and outputs to a stream.

Stream to Tagged Stream

Converts a regular stream into a tagged stream by adding length tags in regular intervals.

Stream CRC32

The Stream CRC32 block computes the CRC of the payload and appends the value to it.

Protocol Formatter

The entry for the Format Obj. is 'hdr_format'

Protocol Parser

The entry for the Format Obj. is 'hdr_format'


Testing

The success of this methodology is based on the input file being padded to a multiple of 'packet_len' bytes so that the last few bytes of the source file are not lost in a partial unprocessed packet. To create a padded file for input to the flowgraph, the following Python program can be used:

#!/usr/bin/python3
# -*- coding: utf-8 -*-

# Padded_File_Source.py

import os.path
import sys

Pkt_len = 252
_debug = 1          # set to zero to turn off diagnostics

if (len(sys.argv) < 2):
    print ('Usage: python3 Padded_File_Source.py <input file>')
    print ('Number of arguments=', len(sys.argv))
    print ('Argument List:', str(sys.argv))
    exit (1)
# test if file exists
fn = sys.argv[1]
if (_debug):
    print (fn)
if not(os.path.exists(fn)):
    print('The input file does not exist')
    exit (1)

# open input file
f_in = open (fn, 'r')

# open output file
f_out = open ("padded.txt", 'w')

while True:
    buff = f_in.read (Pkt_len)
    b_len = len(buff)
    if (_debug):
        print (b_len)
    if b_len == 0:
        print ('End of file')
        break
    while (b_len < Pkt_len):
        buff += ' '
        b_len += 1
    # write output
    f_out.write (buff)

f_in.close()
f_out.close()

Note that the 'Pkt_len' in the Python program and the 'packet_len' value in the flowgraph 'Stream to Tagged Stream' block must be the same. The output of the Python program is a file named "padded.txt".

To test the flowgraph:

  • Run the Python program to produce a padded text file.
  • Execute the flowgraph.
  • Examine the output file "output.txt". The output file will be the same as the original file, but will have trailing spaces at the end as a result of the padding.

Using Header Format Default and Correlate Access Code

Building the flowgraph

Build the following flowgraph using the details given below:

Str pkt 11 fg.png


This flowgraph can be downloaded from Media:Str_pkt_11.grc.

File Source

The File Source reads a padded text file and outputs to a stream.

Stream to Tagged Stream

Converts a regular stream into a tagged stream by adding length tags in regular intervals.

Stream CRC32

The Stream CRC32 block computes the CRC of the payload and appends the value to it.

Protocol Formatter

The entry for the Format Obj. is 'hdr_format'

Correlate Access Code - Tag Stream

The Access Code is a string with the 32 bit value.


Testing

Refer to the Testing section above.

Using an alternate File Source block

There is an Out-of-Tree (OOT) module which reads a text or binary file and produces a stream output with 'packet_len' tags for GNU Radio. Functionally it replaces a 'File Source' block and a 'Stream to Tagged Stream' block. The advantage of this block is that when the input file size is not an exact multiple of the selected packet length, the remainder at the end of the file is not lost in the 'Stream to Tagged Stream' buffer.

The module is in gr-File_Source_to_Tagged_Stream. It contains build instuctions and an example flowgraph.

Using Hardware

If you look at https://github.com/duggabe/gr-control you will see that this tutorial follows the same design. If the user were to build two computer installations with this setup, they could send and receive packets to each other.

Conclusions

Based on this tutorial, there are several things which can be concluded:

  1. BPSK and QPSK are not well suited for burst transmissions due to the need for the receiver to converge on the actual timing and frequency values. The longer the times between bursts of data, the more the receiver will need to reconverge on the timing and frequency values for the new burst.
  2. Since almost all amateur radio communications are half duplex and bursts of data, a more appropriate modulation would be FSK or AFSK. The same packet formatting and recovery would still apply. A proof of concept has been added to https://github.com/duggabe/gr-control with pkt_fsk_xmt.grc and pkt_fsk_rcv.grc.

Further Observations

During the development and testing for this tutorial, the following items were observed:

  • The 'Header/Payload Demux' and 'Protocol Parser' blocks seem to work only with the 'digital.header_format_crc' format.
  • The 'Correlate Access Code - Tag Stream' block requires the Default Header Format Obj. and expects a 32-bit access code.