Tracing Code: an attempt to understand and improve students' process

Lynda Thomas, University of Wales, Aberystwyth

Motivation

Most CS teachers have had the experience of a student coming to them with a program and saying 'it doesn't work'. I've found over the years that when I start drawing pictures (rough interaction diagrams) and tracing the program out, students are often impatient, then amazed that my approach works. Unfortunately many students don't seem to then carry this forward into their own practice.

First Year programming students at Aberystwyth take multiple choice tests as part of their assessment. Some of the questions on these tests ask the students to trace out what happens when a few lines of code are executed. All test takers are provided with a sheet of scratch paper and these sheets are nearly always returned blank. Yet marks on this kind of question are typically awful.

I've observed that neither I, nor any expert I know, does a tracing problem without drawing some kind of picture. I want to find out what students currently do instead and whether it is possible to change their behaviour to something more productive.

Study's Focal Questions

  1. What do students currently do when faced with a 'tracing problem' (whether on a test or in a programming situation)?
  2. Can we 'scaffold' students with interaction diagrams and thus encourage them to use this technique on their own? Does that improve their understanding and performance?

Confession of 'premature experimentation'

I was thinking about this problem before Bootstrapping last summer and got so thrilled with it that in the autumn term I performed an experiment that attempts to answer the second question. I now realise that what I should have done is try and figure out what is going on with the first question first (.......... sigh). So, I'll write about the second question first because I've done more thinking about it. (And no doubt be chastised in June.)

Question 2: Scaffolding with Interaction Diagrams

Background and Justification

Approach

All the students in the introductory programming class were shown interaction diagrams and examples of their use in tracing programs. We divided the class approximately in half for the experiment. One section, the control group, were given pieces of OO code and asked to trace them in a formative test. The other group were provided with both the code and an incomplete interaction diagrams. We compared the results to see how correct the subsequent tracings were (expecting that the experimental group would do better).

Several weeks later we gave students in both groups further questions that involved tracing. We compared their correctness results again and we also collected their scratch sheets to see if the experimental group used the interaction diagrams technique with which they had been scaffolded.

Analysis

I haven't done this yet - I haven't even glanced at the results. But I intend to do some kind of statistcal analysis to see if scaffolded students did better in the first test and in the second test (t-Test??). In addition I will compare scratch sheets of control and scaffolded students. Not sure what test to use on that?

First Question

Which I should have done first. I think that I should basically get some students and ask them to trace some programs and see what they do. Pretty obvious really, but I'm a bit lost on how to go about this kind of observation. (I know that McCracken had a paper in CSEET 2002 that touched on this kind of observation.)

Bibliography