Pushbutton IQ Recorder with descriptive filenames: Difference between revisions

From GNU Radio
Jump to navigation Jump to search
No edit summary
No edit summary
 
(25 intermediate revisions by the same user not shown)
Line 5: Line 5:


* What was the sample rate?  
* What was the sample rate?  
* Center Frequency? SDR Frontend Gain?
* What was the Center Frequency?
* When was it recorded?  
* When was it recorded?  
* What gain setting did we use?
* What were we even trying to capture?
* What were we even trying to capture?


These are all questions we ask after the fact, especially if a lot of time has gone by since we did the recordings.  
These are all questions we ask after the fact, especially if a lot of time has gone by since we did the recordings.  


 
Including those pieces of information in the filename is one way to address this is.
==Introduction==




==Assumptions==
* GNURadio 3.10+
* directory <code>/home/<username>/data/iq_captures</code> is present on filesystem (the flowgraph will automatically find your <code>/home/<username></code> path).
==Goals==
==Goals==
====Create a File Sink with Dynamic Information in the Filename====
====Create a File Sink with Dynamic Information in the Filename====
* timestamp
* timestamp of recording
* radio parameters
* radio parameters
* User Input Note for clarity
* User Input Note for clarity
====Only Record the file on an User Input====
 
====Only Record the file on an User Input Trigger====
* Define User Input as Momentary Switch in GUI
* Define User Input as Momentary Switch in GUI
* Set Conditional Statement in File Sink Block
* Set Conditional Statement in File Sink Block
Line 26: Line 30:
==Content==
==Content==
The flowgraph for this tutorial is shown below along with the GRC file needed if you would like to test it out.
The flowgraph for this tutorial is shown below along with the GRC file needed if you would like to test it out.
This flowgraph is intended for use by anyone with GNURadio 3.10+ installed. It does not use an actual SDR frontend which allows users to test without hardware, but that also means that the values for center frequency and gain are merely representative so, when changed at runtime they won't be reflected in the synthetic spectrum when running the flowgraph. The aim of this tutorial is merely to demonstrate the mechanism. For an example that uses real SDR hardware, try https://github.com/muaddib1984/wavetrap
[[File:Pushbutton_iq_recorder_whole_graph.png]]
[[Media:Iq_recorder_tutorial.grc|Pushbutton IQ Recorder]]


[[File:.png]]
==Create Synthetic Spectrum for the Flowgraph==
This screenshot shows the blocks used to generate some synthetic spectrum with intermittent narrowband carriers:


[[File:Pushbutton_IQ_Recorder.grc|Pushbutton IQ Recorder]]
[[File:Pushbutton_synth_signal.png]]
 
This is what the simulated spectrum looks like when running in GNURadio:
 
[[File:Synth_spectrum_plot.gif]]


==Create a File Sink with Dynamic Information in the Filename==
==Create a File Sink with Dynamic Information in the Filename==
In the following example we will:
In the following example we will:
Use some Pythonic methods to leverage the Runtime Callbacks in our flowgraph which will allow the filenames to change dynamically based on timestamp and radio parameters.
Use some Pythonic syntax to leverage the Runtime Callbacks in our flowgraph and allow the filenames to change dynamically based on timestamp and radio parameters.


We will need to import the appropriate modules to get the user's home directory and create a date/timestamp


[[File:.png]]
[[File:Variables_and_imports.png|300px]]


This portion of the flowgraph:
The filesink contains the filename variable and appends the properly formatted date/timestamp. Since we may want to trigger multiple recordings after starting the flowgraph, the date/timestamp needs to be generated each time the user triggers a recording, so the syntax must be expressed inside block. If we didn't do this, the date/timestamp would only be set once during the initial setup of the flowgraph.


* one
====Filepath====
* two
Then we will use two different variable blocks to define first, the top-level directory (<code>/home/<username></code>) and second, the appropriate subdirectory <code>data/iq_captures</code> for the recordings. This can be changed to your preference.
* three


[[File:Top_level_variable.png|500px]] [[File:Subdir variable.png|500px]]


====Timestamps====
====SDR Frontend Information====
We then create the first part of the filename which contains the SDR Frontend parameters and a user note to describe the capture. These will be updated at runtime in the filename if they are changed.
 
[[File:Filename_variable.png|700px]]


====SDR Frontend Information====
This section of the flowgraph is shown here:
This section of the flowgraph is shown here:


[[File:.png]]
====Timestamps====


The parameters for the File Sink are shown here:
If a different format for timestamps is needed the syntax can be modified to suit your needs. The current format is:  


[[File:.png]]
<code>time.time()).strftime('%Y_%m_%d_%H_%M_%S')</code>
[[File:.png]]
[[File:.png]]


====User Input Note to Describe the Capture====
This can be tested in a python interactive terminal and adjusted to preference. The necessary code to do this is:
*
<syntaxhighlight lang="cpp">
*
import time
*
from datetime import datetime
str(datetime.fromtimestamp(time.time()).strftime('%Y_%m_%d_%H_%M_%S'))
</syntaxhighlight>




====Full Filename and Path====


This part of the flowgraph is shown here:
The below python syntax is put in the "file" parameter of the file sink block. It includes the radio parameters with the date/timestamp. This is incomplete however, as it would record constantly from the time the flowgraph starts. When recording raw IQ we need to be mindful of disk space as raw IQ can take up space fast. In the next section we will show how to trigger the recording with a momentary pushbutton as a safety for our disk space and to keep our recordings limited to only the signals we want to record.


<code>filename+str(datetime.fromtimestamp(time.time()).strftime('%Y_%m_%d_%H_%M_%S'))+".cfile"</code>


[[File:.png]]
In our current example, using the initial parameters, the filename would be:
"RECORDING_NOTE_915000000Hz_1000000sps_50dB_2023_01_04_01_02_39.cfile"




[[File:]]
====User Input Note to Describe the Capture====
*We use the QT Entry Widget block to put an editable text field in our flowgraph's GUI
*NOTE: once the text is changed, the user must hit the ENTER key to update the note
 
[[File:Entry_widget_parameters.png]]




Line 78: Line 102:
==Only Trigger Recording on User Input==
==Only Trigger Recording on User Input==
Using python conditional statement, we can send the I/Q samples to <code>/dev/null</code> until the record button is pressed (and held). When the record button is pressed, the I/Q samples will begin streaming to a file with a timestamp and radio parameters of the flowgraph's state at the time the button was pressed. When released it will send the samples back to <code>/dev/null</code>
Using python conditional statement, we can send the I/Q samples to <code>/dev/null</code> until the record button is pressed (and held). When the record button is pressed, the I/Q samples will begin streaming to a file with a timestamp and radio parameters of the flowgraph's state at the time the button was pressed. When released it will send the samples back to <code>/dev/null</code>
[[File:Momentary_record_button_parameters.png]]
We will also add a QT GUI LED Indicator to turn red when we are recording to disk, it will be green when sending samples to <code>/dev/null</code>.
[[File:LED parameters.png]]


====/dev/Null or Filename with Python Conditional Statement====
====/dev/Null or Filename with Python Conditional Statement====


*
*We will send the IQ samples to <code>/dev/null</code> until the trigger event occurs, at that time the expression will be evaluated to switch the path to the predefined path from the flowgraph.
 
Below is the full python expression to insert in the 'filename' parameter of the file sink block. Note that we simply add a conditional at the end, which is based on the momentary pushbutton's state.
 
<code>filename+str(datetime.fromtimestamp(time.time()).strftime('%Y_%m_%d_%H_%M_%S'))+".cfile" if rec_button == 1 else "/dev/null"</code>
 
[[File:File_sink_parameters.png]]
 
Here is a brief demo of the flowgraph in action showing the subdirectory adding new timestamped files as we click and hold the momentary pushbutton:


Here is a brief demo of the flowgraph in action:
[[File:Pushbutton_iq_demo.gif]]


[[File:.gif]]
NOTE: we use the command <code>watch ls -l</code> in the subdirectory to show the updating file names


==Prerequisites==
==Prerequisites==


* [[Guided_Tutorial_GRC|Intro to GR usage: GRC and flowgraphs]]
* [[Guided_Tutorial_GRC|Intro to GR usage: GRC and flowgraphs]]

Latest revision as of 02:11, 5 January 2023

Application

When doing field work, capturing Raw I/Q for post processing is a helpful way to get the best signal quality of a capture while spending minimal time in a non-lab environment. Changing filenames manually for every capture can be laborious and subject to user error. Furthermore, starting and stopping a flowgraph can be tricky if we are tuning the SDR frontend parameters to find optimal level/frequency/sample rate. If a non-descriptive filename is used such as capture_file.cfile it does nothing to describe the situation to the user during the analysis stage as it doesn't include any information about the RF samples we've captured. This is critical to the post-analysis process.

  • What was the sample rate?
  • What was the Center Frequency?
  • When was it recorded?
  • What gain setting did we use?
  • What were we even trying to capture?

These are all questions we ask after the fact, especially if a lot of time has gone by since we did the recordings.

Including those pieces of information in the filename is one way to address this is.


Assumptions

  • GNURadio 3.10+
  • directory /home/<username>/data/iq_captures is present on filesystem (the flowgraph will automatically find your /home/<username> path).

Goals

Create a File Sink with Dynamic Information in the Filename

  • timestamp of recording
  • radio parameters
  • User Input Note for clarity

Only Record the file on an User Input Trigger

  • Define User Input as Momentary Switch in GUI
  • Set Conditional Statement in File Sink Block

Content

The flowgraph for this tutorial is shown below along with the GRC file needed if you would like to test it out. This flowgraph is intended for use by anyone with GNURadio 3.10+ installed. It does not use an actual SDR frontend which allows users to test without hardware, but that also means that the values for center frequency and gain are merely representative so, when changed at runtime they won't be reflected in the synthetic spectrum when running the flowgraph. The aim of this tutorial is merely to demonstrate the mechanism. For an example that uses real SDR hardware, try https://github.com/muaddib1984/wavetrap

Pushbutton iq recorder whole graph.png

Pushbutton IQ Recorder

Create Synthetic Spectrum for the Flowgraph

This screenshot shows the blocks used to generate some synthetic spectrum with intermittent narrowband carriers:

Pushbutton synth signal.png

This is what the simulated spectrum looks like when running in GNURadio:

Synth spectrum plot.gif

Create a File Sink with Dynamic Information in the Filename

In the following example we will: Use some Pythonic syntax to leverage the Runtime Callbacks in our flowgraph and allow the filenames to change dynamically based on timestamp and radio parameters.

We will need to import the appropriate modules to get the user's home directory and create a date/timestamp

Variables and imports.png

The filesink contains the filename variable and appends the properly formatted date/timestamp. Since we may want to trigger multiple recordings after starting the flowgraph, the date/timestamp needs to be generated each time the user triggers a recording, so the syntax must be expressed inside block. If we didn't do this, the date/timestamp would only be set once during the initial setup of the flowgraph.

Filepath

Then we will use two different variable blocks to define first, the top-level directory (/home/<username>) and second, the appropriate subdirectory data/iq_captures for the recordings. This can be changed to your preference.

Top level variable.png Subdir variable.png

SDR Frontend Information

We then create the first part of the filename which contains the SDR Frontend parameters and a user note to describe the capture. These will be updated at runtime in the filename if they are changed.

Filename variable.png

This section of the flowgraph is shown here:

Timestamps

If a different format for timestamps is needed the syntax can be modified to suit your needs. The current format is:

time.time()).strftime('%Y_%m_%d_%H_%M_%S')

This can be tested in a python interactive terminal and adjusted to preference. The necessary code to do this is:

import time
from datetime import datetime
str(datetime.fromtimestamp(time.time()).strftime('%Y_%m_%d_%H_%M_%S'))


Full Filename and Path

The below python syntax is put in the "file" parameter of the file sink block. It includes the radio parameters with the date/timestamp. This is incomplete however, as it would record constantly from the time the flowgraph starts. When recording raw IQ we need to be mindful of disk space as raw IQ can take up space fast. In the next section we will show how to trigger the recording with a momentary pushbutton as a safety for our disk space and to keep our recordings limited to only the signals we want to record.

filename+str(datetime.fromtimestamp(time.time()).strftime('%Y_%m_%d_%H_%M_%S'))+".cfile"

In our current example, using the initial parameters, the filename would be: "RECORDING_NOTE_915000000Hz_1000000sps_50dB_2023_01_04_01_02_39.cfile"


User Input Note to Describe the Capture

  • We use the QT Entry Widget block to put an editable text field in our flowgraph's GUI
  • NOTE: once the text is changed, the user must hit the ENTER key to update the note

Entry widget parameters.png


Example:

Only Trigger Recording on User Input

Using python conditional statement, we can send the I/Q samples to /dev/null until the record button is pressed (and held). When the record button is pressed, the I/Q samples will begin streaming to a file with a timestamp and radio parameters of the flowgraph's state at the time the button was pressed. When released it will send the samples back to /dev/null

Momentary record button parameters.png

We will also add a QT GUI LED Indicator to turn red when we are recording to disk, it will be green when sending samples to /dev/null.

LED parameters.png

/dev/Null or Filename with Python Conditional Statement

  • We will send the IQ samples to /dev/null until the trigger event occurs, at that time the expression will be evaluated to switch the path to the predefined path from the flowgraph.

Below is the full python expression to insert in the 'filename' parameter of the file sink block. Note that we simply add a conditional at the end, which is based on the momentary pushbutton's state.

filename+str(datetime.fromtimestamp(time.time()).strftime('%Y_%m_%d_%H_%M_%S'))+".cfile" if rec_button == 1 else "/dev/null"

File sink parameters.png

Here is a brief demo of the flowgraph in action showing the subdirectory adding new timestamped files as we click and hold the momentary pushbutton:

Pushbutton iq demo.gif

NOTE: we use the command watch ls -l in the subdirectory to show the updating file names

Prerequisites