Acknowledgements

With thanks to all the teachers of Harris Academy, Battersea who took part in this pilot: Tom Harriott, Rosie Lloyd, Juliet Doyle, David Carr, Nicholas Causton-Marks and headteacher David Moody for their support.

Also thanks to Sarah Mason, Janet, Clinton, Hayley Kimball and the rest of the team at the University of Melbourne. Also thanks to UK Transcription and AI Media.

About Nesta

Nesta is a global innovation foundation. We back new ideas to tackle the big challenges of our time. One of these challenges is how to educate our young people for a more automated future workplace, which will require different skills and knowledge.

How do we learn? Are two heads better than one? How can we help our children work better together to solve problems? These questions are too important to leave to chance.

Our Solved! report showed that the ability to solve problems with others (collaborative problem-solving) is more important than ever for future work, and helps kids succeed in the classroom now. It also showed that, despite its promise, it features little in UK school life. That's why we're running a series of experiments to learn more about how educators can do this better. We will be looking at different approaches to supporting collaborative problem-solving both in and out of the classroom. These will range from using technology to aid pupil discussions, to teacher training and sharing platforms to support quality problem-solving.

Nesta is a global innovation foundation. We back new ideas to tackle the big challenges of our time.

We use our knowledge, networks, funding and skills – working in partnership with others, including governments, businesses and charities. We are a UK charity but work all over the world, supported by a financial endowment.

To find out more visit www.nesta.org.uk

Summary

It is now increasingly common practice for teachers to record themselves teaching as a way of reflecting upon their practice, but what about recording the pupils themselves?

Our recent report showed how important it is that students do more collaborative problem-solving at school to get them ready for work. Yet despite strong evidence for its impact, it is rarely taught in schools.

At Nesta we want to support collaborative problem-solving (CPS) practice in the classroom. Over the next year we'll be supporting four pioneering innovators in the field to create new tools and test them with students. In 2016 we embarked on the first of these four experiments, a small-scale exploratory pilot with Harris Academy, Battersea.

Our approach was simple and inexpensive, using dictaphones to produce transcripts in order to examine their effectiveness as a tool for developing pupils' problem-solving discussions. We worked both UK Transcription and AI Media who were able to turn around recordings in just a few days and The University of Melbourne for help with coding them.

Key findings

• Teachers found transcriptions of pupil discussions a useful tool for improving practice, especially on pupils working together and solving problems collaboratively.
• Teachers found it harder than expected to fit even a relatively modest 15 minutes of weekly collaborative problem-solving into their lessons.
• Teachers found the experiment a useful way to reflect upon some of the challenges and opportunities in designing, structuring and facilitating problem-solving activities to ensure meaningful collaboration to take place.

The pilot builds on promising results from Nesta's previous work in particular the EEF's Visible Classroom project, which explored the impact of real-time captioning and teacher lesson transcripts to support teachers' professional development.

It also draws on research into collaborative discussion and self-regulation conducted by one of the teachers at Harris Academy, Battersea (Tom Harriott, Deputy Head of Sixth Form and Teacher of Science) as part of his master's degree.

Introduction

Background

In March 2017 we published our Solved! report which found that collaborative problem-solving (CPS) is an increasingly important skill to teach young people in order to prepare them for the future. Despite strong evidence for its impact, CPS is rarely taught in schools but if structured well it can reinforce knowledge and improve attainment. However, significant barriers exist for teachers implementing this practice, from behaviour management to curriculum coverage, to task-design. For CPS to gain ground, a concerted shift is needed including teacher training, better resources and system level support.

In line with Nesta's research into the positive impact of 'collaborative problem solving' on learning and workforce-readiness we wanted to look at ways to practically support these approaches in the UK school setting.

Harris Academy, Battersea is a mixed comprehensive school based in South London, which has seen a dramatic turnaround in the past two years, having doubled its A*-C pass rate since becoming part of the Harris Federation group in 2014.

The school has a diverse demographic intake with 58 per cent of students eligible for free school meals (FSM), and 52 per cent categorised as having English as an additional language (EAL).

Aims and objectives

The pilot attempted to explore the following questions:

• What conditions (including task design, guidance, classroom set-up) enable good collaborative problem-solving to take place?
• Do teachers find transcripts a useful (and efficient) method for developing their ability to facilitate collaborative problem-solving to take place?

Other questions of interest were:

• Are there lessons to be learned across subjects, or is CPS primarily subject specific?
• Do pupil's CPS skills improve with practice?
• Does it support broader attainment outcomes?

Methodology

The pilot took the form of small-scale action research with six teachers at the school (year 10 to year 12), four maths teachers and two science teachers. Teacher reflection on transcripts was planned to take place in fortnightly record and review cycles.

The resulting discussions were analysed qualitatively through discourse analysis and quantitatively through content analysis which coded key utterances as defined by Haneda and Wells (2010) in their study of dialogic inquiry. Each utterance within the discussions was coded into one of the following; answer, inform, report, describe, suggest, predict, conjecture, explain, conclude and question. These utterances were identified as being a good way to categorise features of collaborative discussion.

Findings and results

Results of baseline survey

In our first meeting with the teachers taking part, they were given a baseline survey which contained a mixture of closed and open questions, to try and ascertain the current frequency with which they set 'problem-solving', 'collaborative learning' and 'collaborative problem-solving' tasks. It also contained Likert scale attitudinal questions to ascertain levels of confidence in teaching and perceived value of each of the approaches.

The survey data showed that while teachers were confident delivering problem-solving tasks, they were less confident when it came to collaborative learning approaches. Our baseline survey found that:

• One-third of teachers 'rarely or never' use collaborative problem-solving approaches; this was compared to five-sixths who engaged in problem-solving exercises 'most days'. (See Figure 1.)
• This correlated with teachers' perceived value to students, with 'problem-solving' on average scoring two points higher than both 'collaborative learning' and 'collaborative problem-solving' approaches.
• On all approaches, teacher confidence was lower than the perceived value of each of the approaches.

This corroborates findings from our recent report which showed that instances of teachers using both collaboration and problem-solving together are rare across most subjects.

Teacher feedback suggested they were looking to gain "more of an understanding about what good collaboration looks like" and develop a "specific approach to collaborative work", alongside the school's emerging approach to problem-solving.

Effectiveness of transcripts and coding

After completing each set of recordings, teachers uploaded them online and they were sent to UK Transcription and AI Media to be transcribed. Once we had the transcripts back we sent them to the University of Melbourne who coded them. We then took the coded transcripts back to our teacher feedback meetings at Harris in order for them to reflect upon their practice. (See Figure 2.)

The coded transcripts from Melbourne contained both data on the number of utterances by each student as well as a qualitative analysis of the data and how the pupils interacted with one another. (See Figure 3 example.)

Our first feedback session with the teachers indicated that on the whole they found both the transcripts and accompanying coding useful. For example:

• It gave them never-before-seen insight into how pupils behaved when they weren't around.
• It allowed them to track individual progress and whether or not students conformed to their expectations.
• It allowed them to see which students worked well together and how the group interacted with one another.
• It allowed them to see whether the problem-solving tasks they set supported good CPS.

Teacher feedback and observations

They found the coding useful in providing a quick overview into whether the task had worked well and the qualitative feedback gave them important insights into how individuals were performing. This usually tied into what the teachers already knew of students' personalities and abilities, where it differed it allowed teachers to examine why the students may be behaving differently.

In general, the utterances that were coded as being associated with high-level collaborative discussion were 'explain' and 'suggest'. This showed that students were not just regurgitating information already given to them (as is suggested by 'report') but that they were interrogating and justifying their viewpoints as well as putting forward suggestions about what the answer could be.

However, one teacher noticed that some coding terms such as 'question' could be misleading, and asking a lot of questions was not necessarily an indication that a student wasn't understanding, with questions deployed for a variety of reasons.

How can students best support one another?

During our feedback sessions with teachers a number of interesting conversations emerged around collaborative problem-solving, including how it could best be structured and supported, as well as its merits and challenges. Some of the key discussion areas are outlined below.

There was some concern when setting group problem-solving tasks that the more high achieving students may work out the answer really quickly, and give it away to other group members before they had a chance to work it through themselves. Therefore groups of students had to be carefully chosen.

Teachers recognised that it was important to help students to support one another and there were some discussions around how this could be achieved. This was something also flagged up during the SPrinG project, a study which worked with schools to improve and imbed collaborative group work across Key Stages 1-3. It recommends 'teaching children to take turns' and share opinions in order to help students break down the process of working collaboratively.

However, as the students taking part in the Harris pilot were of GCSE age, one teacher raised a concern that telling them they must share their opinion can lead to overly forced/laboured questions, i.e. "What do you think, Johnny?"

How then can students be taught to support one another in a way that's both supportive and meaningful? One suggestion was that hint cards could be used to help one another, and a number of teachers in the pilot used these as a way to support students and encourage them to be in charge of their own learning.

What's more important knowledge vs skills?

Could it help to have students write a problem themselves for another student? The consensus was that this was hard and would rely on students having quite an advanced level of understanding of how a problem worked. One of the teachers observed that if students were asked to make a difficult problem in maths they would tend to just make the numbers really big or really small. However, they agreed if students were used to doing this it could be of some benefit but believed that really it represented the next stage of understanding.

Another key discussion topic was around the issue of knowledge vs skills and whether the two were mutually exclusive or could be taught together. One concern was raised around whether collaborative problem-solving was an efficient way to solve problems. Some teachers were unsure how much progress students made with the problems during collaborative discussion, a potential pitfall considering the amount of lesson content teachers are expected to cover.

One of the science teachers found it was a balancing act between ensuring that students participated in the activity for long enough to benefit from the opportunity to develop their skills, but still managed to get to the solution, therefore learning the required content.

He felt it was important for teachers to examine why they were setting a task, was it in order to build problem-solving skills, in which case it is OK that they struggle; or to know the answer and acquire particular knowledge? As the pilot went on he felt it was best to separate these two objectives – problem-solving to develop problem-solving skills, and problem-solving to learn new things, akin to enquiry-based learning.

Whilst in our Solved! report we argue that it is possible for both objectives to be met if embedded from an early enough age and tasks highly structured, we acknowledge that lack of lesson time and coverage of curriculum content is a real issue. These existing demands don't necessarily allow enough time for these skills to be developed.

By the end of the pilot one of the maths teachers remained unconvinced about the value of collaborative activities for problem-solving questions. Although he reported he would now incorporate more collaborative approaches into fluency-based maths questions, he felt an individual approach was still more beneficial to problem-solving questions.

A suggestion of a way to combat this came from another teacher who felt it would be beneficial for students to have some time to think on their own first, before sharing their thoughts on how to tackle it with other group members. It was suggested that this might also be a way to eliminate talk that didn't help with solving the problem, and some discussion emerged around whether students talking more was necessarily a sign that good collaborative problem-solving was taking place.

Teacher Case Study - Rosie Lloyd

Background

Rosie teaches science to years 7, 10, 11 and 12. For her recordings she decided to focus on her year 10 biology class. She wanted to gain more confidence in teaching problem-solving approaches and to 'gain more of an understanding of what problem-solving looked like' in her subject.

Task One

• Two or three students act as junior doctors, trying to diagnose five patients.
• Students must decide what disease each patient is suffering from, what treatment each patient needs and the order in which they should treat them.

This task led to good collaborative discussion, see below for example features taken from the transcripts:

Example One: Pooling their knowledge

Yasmin: She said one of them we did a long time ago. So this must be obesity. Shola: But obesity isn't a disease. It leads to diseases like heart disease. Yasmin: So we can do obesity, coronary. Shona: What is the thing? Diabetes Type 2? Yasmin: How do you know which type it is? Shona: Because Type 1 is in the genes, isn't it? Yasmin: Is it? Shola: Yes.

Example Two: Reaching a consensus and explaining their viewpoint

Teacher: Even if you're not sure about your final decision, try and be confident about how you treat it, OK? Ilda: So, in my opinion, patient number four... Like, the doctor really needs to take care of patient number four, as malaria... Liam: And I think another one that's a lot is patient number two, who has HIV. Ilda: Yeah. But HIV can't, for example, spread around the hospitals. If the disease can spread, then, you know, it may have a really have bad effect on... Liam: But the reason I think patient two needs to be taken care of a lot because of the HIV is because of if they... Because of what the HIV would do to their body. And if they get ill, how they could really suffer from it because of their weaker immune system. Ilda: But again, HIV is spread only in certain body fluids, from a person infected with HIV. These fluids are blood, or rectal fluids, or breastmilk. So, yeah, I mean, HIV is quite important here. But then again, there is malaria, and... And, and, and...mosquitoes can spread it around. Liam: Mm-hm...OK.

Reflections

• Rosie found the transcripts and the coding in particular a really useful way to reflect upon the the task and 'pinpoint any particular problem areas'.
• The next task she set was more maths/problem-solving based which also led to some good collaborative discussions but gave students less opportunity to explain their decisions.

Conclusion

Results of exit survey

At the end of the pilot, teacher confidence and perceived value of all teaching approaches (problem-solving, collaborative and collaborative problem-solving) had gone up (see Figure 4). The biggest increase was seen in the perceived value of collaborative problem-solving approaches and teachers reported a better understanding of what it was and how it could be taught.

We found a discrepancy between the maths and science teachers, with science teachers finding the process of designing problems considerably harder. This was due to the fact that the maths teachers had already designed and set up a large bank of problem-solving questions which they could draw upon, whereas the science teachers had to work harder to work out the ways this could be applied to their subject and design new problems.

Consequently, for maths teachers, there was no change in the frequency with which they would incorporate problem-solving into their lessons by the end of the pilot, as it was already heavily embedded into the subject curriculum. However, for science teachers, by the end of the pilot they had gained more of an understanding of how problem-solving could be applied to their subject and as a result would now incorporate it more into their lessons.

The opposite was true when it came to delivering collaborative learning activities. Here, the science teachers had more experience than the maths teachers who claimed to have 'rarely/never' used collaborative approaches at the start of the pilot. By the end of the pilot the maths teachers expected to incorporate more collaborative approaches into their lessons.

The results show that teachers clearly saw the value in collaborative problem-solving and aim to incorporate more collaborative approaches into their teaching. However, as the pilot progressed it was clear that the teachers found it harder than expected to fit recordings into their working weeks.

In the exit survey, 'Lack of time for designing appropriate problems' was named as one of the biggest barriers to implementation. It seems that the science teachers often didn't have the time to spend to design new collaborative problem-solving tasks, acknowledging that 'a successful collaborative lesson requires a lot of planning'. Even though initially the maths teachers found designing problems easier, as the pilot progressed they realised that the problems they already used were not necessarily the best for generating good collaborative discussion.

'Setting up the tasks/groups' was named as the second biggest barrier to implementation. This is likely due to the time it took to set up the recording equipment and rearranging the class into groups, which required students to change their seating arrangements. Perhaps there is a role for technology to play here in the development of recording equipment that is both cost-effective with minimal imposition on lessons.

Want to try it in your classroom?

Check out our three steps for how you might try recording collaborative discussions in your own classrooms as well as key considerations from our teachers:

1. Get some recording equipment. We purchased Sony dictaphones for less than £30 for the purpose of the pilot. However, you may be able to get some cheaper through a school supplier, or perhaps your school's language department may have some you can borrow. One of the issues we had was around audio quality with parts of some transcripts rendered inaudible due to background noise or some students sitting too far away from the microphone. You may need to arrange your student groups so they are as near to the microphone as possible, but UK Transcription was able to cope with varying audio qualities.
2. Shop around for transcription companies. These can vary quite substantially in price and turnaround times. We used both UK Transcription and AI Media, prices vary depending on turnaround time and audio quality with some companies able to return transcripts within 24 hours if necessary.
3. Group together and share tasks that work. If possible, try and form an action learning set with other teachers so you can share and reflect upon your experiences and tasks. As 'lack of time for setting up appropriate tasks' was listed as the biggest barrier to implementation you may wish to draw upon existing collaborative resources as produced by organisations such as NRICH and Philosophy for Children, as highlighted in the report.

Next steps

Our work with Harris Academy, Battersea backed up the findings from our Solved! report that collaborative problem-solving is not easy for teachers with busy workloads and that more needs to be done to increase teacher training and confidence when undertaking CPS within their classrooms.

We have now commissioned three further follow-up experiments in both the formal and informal sectors to help address some of these issues, which include:

1. Pilot with Rasberry Pi working with ten code clubs looking at whether attempting problem-solving practically or studying worked examples of problem-solving processes results in more success in collaborative digital making.
2. Pilot with NRICH working with ten primary schools in Cambridgeshire to promote collaborative problem-solving approaches, focusing on creating teacher support and training in order to help them utilise their resources.
3. Pilot with Bruno Reddy to create and develop a distribution platform for high quality maths problems, allowing schools to develop their students' problem-solving abilities more easily.

We will evaluate these and further collaborative problem-solving pilots in the future in line with our report recommendations.

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 [email protected] @nesta_uk www.facebook.com/nesta.uk www.nesta.org.uk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091. Registered as a charity in Scotland number SC042833. Registered office: 58 Victoria Embankment, London, EC4Y 0DS.

CC BY NC SA