Github5561

What do users do when a .grc file behaves differently due to being opened on different GRC version?

- The UX research has been finished

- The draft documentation has been started

- The UI design for the error message needs to be started

UX Research

This work is related to Github Issue 5561.

1. Do research with community to understand what they do when this happens currently. I've put together a short survey to ask users for their experience of this situation, what behaviour was caused, how they dealt with it. Based on the results of that research we create some notifications with advice on how to deal with it.

2. Analysis of the survey quantitative data being done using a collaborative online whiteboard called Miro. You're welcome to leave comments on the Miro board.

3. Write the draft support documentation. Send it to the UX group and to the mailing list for contributions or comments.

4. Do UX design for notification.

Research write-up

Summary

We carried out research with GR users to learn how they dealt with issues arising from the following use-cases:

1. opening a .grc file created with an older version of GNU Radio, in a newer version. 2. opening a .grc file created with a newer version of GNU Radio, in an older version.

We wanted to learn about the following:

1. if they had issues with either of these scenarios happened to them, and if so asked them to describe the situation 2. the most common causes 3. the troubleshooting routes they took 4. how they fixed the issue

This is all obvious!

While this might be obvious to those of you who are expert GR users, it won't be obvious to most people. Also, gathering evidence allows us to make better decisions.

Objective

The objective was to use what we learned to write documentation that supported GNU Radio users of all levels of experience.

Findings

1. This is a common issue!

• All participants experienced issues with both scenarios

2. The documentation needs to be readable by users of different levels of experience

• Users with differing years experience (from 1 to 10 years) all experience these issues
• Users with differing years experience used the same/similar troubleshooting routes and fixes

3. Documentation should provide sign-posting to the community when users can't fix the issue

• Sometimes it wasn't possible to fix the issue. If none of the suggested fixes work, they should be directed to the community for possible help

4. In some cases issues had several solutions

• In terms of troubleshooting an issue, this is an obvious statement to make, but it's useful when working out how to write/structure documentation for users.

5. It wasn't possible to identify clear issue -> troubleshooting -> fix route patterns

• This means the best we can do is provide advice that might possibly duplicative advice, i.e. some causes might have similar troubleshooting paths, fixes.

Causes

Issues were caused by the following: (ranked by how common)

1. Missing blocks/modules

#* Blocks not being installed #* Out Of Tree Modules/Extensions missing/not installed #* Block locations changing/being reorganised between versions #* Blocks being removed e.g. wxwidgets

1. Block/module incompatibilities caused by running multiple different versions of GRC on the same machine
2. Incompatibilities due to changes in block formats over different GRC versions
3. Appropriate alternative blocks not being available in older GRC versions
4. Blocks being renamed over new/older GRC versions
5. Block parameter changes over versions
6. (newer) Blocks not backward compatible
7. Undetermined bugs/issues

Fix/solutions

Fixes and solutions outlined by users were (ranked by how common)

1. Rewrite/redraw the flowgraph

2. change the incorrect block parameter

3. convert flowgraph

4. replace block(s) with another

5. port or rewrite block(s)

6. recompile modules

Troubleshooting routes

Troubleshooting steps mentioned by users were: (ranked by how common)

1. Inspecting the .grc file or block code

2. Reading error messages then doing an Internet search

3. Reading error messages then installing missing blocks/OOT module

4. Reading documentation

5. Debugging code (specified Python)

6. Asking GR community for support

7. Using UI error notifications ("problems are shown in red")

Troubleshooting routes mapped

Mapping what participants told us - what caused the issue -> what they did -> how they fixed it - allowed us to identify their trouble-shooting routes as followed:

It's a bit jumbled but some definite paths existed.

Next steps

Contribute to documentation

Based on the analysis of the qual. data, I think documentation structure as follows would be helpful:

It would provide people with a starting point with the problem they have, then give them suggested ways to fix it.

I've started a documentation page for this issue [[1]]. You're welcome to review it and make changes, improvements.

UI Design for error message

This is the next part to do.

Research method

If you're interested in how we carried out this research, keep reading!

Qual not quant

The data gathered is qualitative - **this is not statistically significant, and is not supposed to be**.

Qualitative data is used to give a deeper understanding about how a smaller number of people deal with a particular issue.

Asking the question

We created a short survey focused on the following scenario:

A user opens a .grc file with an older (or newer) GRC version than the file was created on. When they do the .grc file behaves differently.

and then asked the following questions:

- Has the above scenario ever happened to you?

- If yes, tell us about it.

- How did you go about troubleshooting the issue?

Data analysis

The data gathered in the survey was analysed using the [Method (also called affinity mapping)]. This is very effective for: organising and prioritising opinions and subjective data, to quickly reach a consensus on priorities of subjective, qualitative data.

You can see the analysis process on [interactive whiteboard].

Participants

We had 13 participants take part in the research, from 8 countries.

Criteria

Our only criteria for participation was the person needed to be a GNU Radio user.

This was because we wanted input from users with differing experience levels.

Recruitment

We recruited participants by:

• emailing the survey to
  • the GNU Radio UX Studies group *
  • the GNU Radio user email list
• posting it to the GNU Radio Matrix General chat and ham radio related channels
• posting it to Twitter with GNU Radio relevant hashtags

(* This is a group of GNU Radio users who have expressed interest in taking part in UX research.)