When Stack Overflow conducted a survey of 64,000 software engineers, it found that 1% of their respondents were blind — a far higher percentage than among the total population. Yet it is far from easy for young people with visual disabilities to engage in learning programming in school. In this month’s seminar, Dr Cecily Morrison of Microsoft Research Cambridge shared some of her work in this area. Her talk highlighted the difficulties that children learning to program face if they are blind or have low vision, and the affordances of physical programming tools, in particular Code Jumper.
In her work as a Principal Researcher, Cecily focuses on designing inclusive experiences for people who are blind or have low vision, and she is leading the team that designed Code Jumper (known as Project Torino during its development). She is currently engaged in developing assistive agent technology in Project Tokyo, and she was recently awarded an MBE for her services to inclusive design.
Block-based programming is inaccessible for children with visual disabilities
Block-based programming has become the norm for primary school-aged children who are learning to program, and a variety of freely available environments exist, e.g. Scratch and Blockly. These tools have lots of advantages: discoverability of commands; no syntax errors; and live, imaginative visualisations. But how do you use Scratch if you are blind or have low vision and cannot see the screen?
There are tools that ‘read out’ code in blocks-based environments but — as we experienced in the seminar — their audio output may not readily facilitate understanding. Listening to one line of code at a time can be difficult, for example when trying to understand a loop (let alone a nested loop!). It puts significant demand on listeners’ memory, and children may lack the conceptual cognitive structures to process the audio information. In addition, using screen-based programming environments involves other challenges for blind children: they need to master touch typing, memorise keyboard shortcuts, and understand file systems.
Project Torino to Code Jumper
To address these challenges, Cecily’s team at Microsoft Research started to develop a physical programming tool for primary-aged learners, in a project known as Project Torino. The project started in 2015, and the tool was developed iteratively over the next four years. The team’s goal for this research project was always to generate a tool that is useful and available to all young learners who are blind or have low vision. Thus, in 2019 the research and technology was transferred to the American Printing House for the Blind, and the Project Torino tool was renamed Code Jumper.
In the seminar Cecily described the iterative development of the physical programming tool. It consists of a number of physical pods, including a play pod, rest pod, loop pod, and selection pod. The young learner can feel the difference between the pods by touch and link them together in the right sequence to construct a program. They then use a central pod, known as the hub, to play an audio output of the program they have created. Using this tool they can code tunes, songs, and stories using ready-made sound sets or sounds that they record themselves.
Dials on the pods allow learners to change the parameter values for each program statement, e.g. the number of times to loop. The parameters can also be changed programmatically through the insertion of ‘plugs’ into the dials. For example, a ‘random’ plug can get a random sound to play.
A use case example is coding the song Row, row, row your boat, which is a common nursery rhyme in the UK and USA. By attaching different pods and using the dials, a learner can use a loop to play “row” three times, and then can add pods for the sounds for “your boat”. Constructing a program like this helps the young programmer learn about sequencing and loops.
Several threads can be attached to the central hub, as in the image below, so that children can learn to use multi-threaded programming, as they can in block-based programming environments such as Scratch. The seminar recording below includes some examples of Code Jumper in action!
Five design principles
Cecily described five design principles that her Microsoft Research team used while developing this physical programming tool:
- Persistent program behaviour — When you listen to a program one block/line at a time, it’s hard to get a sense of what it does. Therefore, an important requirement in the design process was that the tool should allow the user to experience the program as a whole. With Code Jumper, the young person can use their hands to follow the program as it executes.
- Liveness — This refers to the responsiveness of the tool. It was important to have instant feedback when programming: with Code Jumper, as soon as you touch one of the pods, you get a response.
- Low floor, high ceiling — This means the tool is accessible to absolute beginners, but it also offers the opportunity to write more complex programs and develop more advanced skills.
- Works across visual abilities — The tool can be used by children with and without vision, and it was designed to be used by learners with multiple disabilities as well as those with low vision.
- Enables progression — The tool can support learners moving from a physical language to a textual language, by enabling them to listen or read their code as they follow its execution.
The ultimate aim of Code Jumper is to open career opportunities in technology.
Evaluation of the tool
As part of Cecily’s research project, her team undertook a nationwide trial to evaluate the effectiveness of Project Torino, with 75 children and 30 teachers. The trial involved a diverse group of students with a wide range of cognitive skills, and the teachers mostly didn’t have much computing experience.
The team developed a curriculum and sent the teachers full course materials along with Torino kits and laptops. A validated instrument was used to measure engagement and motivation, along with teacher-reported learning outcomes.
In the findings from the trial, all teachers (100%) said that they would like to continue using Torino. Students were also very engaged by the project. Students’ self-efficacy in coding grew substantially after exposure to Torino, with a change in the median score from 2 to 4 (of 5) and large effect size (r = -0.730).
Among the qualitative data the team collected, the teacher-reported outcomes included comments about the young people’s use of programming vocabulary (see our previous seminar on the importance of talk in learning to program), and how they improved their problem solving skills. Some teachers also commented on the fact that the physical computing tool generated an inclusive environment in the classroom, as it allowed sighted and non-sighted children to work together.
Overall, our seminar audience found this a very interesting and engaging topic and had lots of questions for Cecily in the question-and-answer session. There is obviously much more to do to ensure that computing is accessible to all children, regardless of any disability or impairment. Research projects such as the one Cecily presented generate useful output in terms of tools for use in the classroom or home, and they also challenge us to think about all our learning materials and their accessibility.
This paper contains more information about the trial. Download Cecily’s annotated slides here, and watch or listen to her presentation:
Join our next seminar
Between January and July 2021, we’re partnering with the Royal Academy of Engineering to host speakers from the UK and USA to give a series of research seminars focused on diversity and inclusion. By diversity, we mean any dimension that can be used to differentiate groups and people from one another. This might be, for example, age, gender, socio-economic status, disability, ethnicity, religion, nationality, or sexuality. The aim of inclusion is to embrace all people irrespective of difference.
In our next research seminar on Tuesday 1 June at 17:00–18:30 BST / 12:00–13:30 EDT / 9:00–10:30 PDT / 18:00–19:30 CEST, we’ll welcome Dr Hayley Leonard and Thom Kunkeler from the Raspberry Pi Foundation team. They will be talking about ‘Why the digital divide does not stop at access: understanding the complex interactions between socioeconomic disadvantage and computing education’.
To join this free event, click below and sign up with your name and email address:
We’ll send you the link and instructions. See you there!
You can now download the first volume of our seminar proceedings, with contributions from our previous guest speakers.
Website: LINK
