At the Lighthouse Community Charter School in Oakland California, our research-practice partnership (RPP) team meets monthly to review video data of students engaged in afterschool STEM-rich tinkering activities. Our goal is to develop a shared, evidence-based understanding of the kinds of learning happening in the afterschool program, and ultimately how it can be recognized and leveraged during the school day.

Almost a year into the data collection and RPP data reviews, we’re identifying some compelling student learning pathways that demonstrate a range of ways that Lighthouse middle school students are developing conceptual understanding, problem-solving practices, and creative agency through their participation in the STEM-rich tinkering program. Many of our insights have come through the conversations, among teachers and researchers, about what we can see in the data.

The power of RPPs is that bringing together the disparate analytical lenses (experiences, values, concerns) of both practitioners and researchers broadens the conceptual space in which we can see and make meaning of the data collected in the programs. But fostering these evidence-based reflections hasn’t been completely straightforward and we have learned a few things about collaborative data analysis along the way.

Two of the things we struggled with most were (1) the “grain size” of the data – what kinds of video or what size of field notes excerpts could generate productive conversations in our RPP teams – and (2) making the most of our limited time by keeping the conversation focused on the topic at hand: student learning.

In the beginning, we shared 2-3 minute clips of micro-interactions between student and teacher. We did this (1) because this is what we had at that point and (2) because as researchers we find raw data pretty compelling. Derry et al. (2010) refer to these kinds of video clips as analytic and note their utility for identifying frequencies and patterns across the data. While we loved diving deeply into the micro-interactions between young people and teachers, we found that watching these intense, but somewhat contextless clips, generated RPP conversations that focused on the teacher moves instead of on the student learning. In retrospect, this makes sense: Pedagogical moves are the unit of change for teachers. They are the actions that they can take – to intervene, hold back, share solutions, ask questions – in the moment; and what we were showing was “the moment.” Student learning happens over time.

As we amassed more data, we could begin to portray the arc of individual students’ learning; these clips are what Derry et al. refer to as narrative. We found these representations to be far more generative in terms of focusing conversation on student learning. Many more of the teachers’ comments concerned their views on students’ learning and the evidence in the video data that supported these views. Of course, in the editing process we, as researchers, were making decisions about what to include in the trajectories. But teachers responded with profound insights. An example was an observation that a marker of student learning was when he or she demonstrated the judgment to “walk away” from an idea; i.e., to start over or re-think new projects. This observation adds nuance to current discussions of “grit” and the ideas of perseverance or persistence in the maker movement. It’s not that persistence or grit can’t encompass “walking away,” of course they can and do. But the cultural models and conversations these terms generate tend to valorize gritting one’s teeth and sweating it out, versus exercising the judgment to take a breath, step away and find a more productive path. Our analytical lens and coding scheme now takes this phenomenon into account.

We still use micro-interactional video clips in our coding and analysis, but we no longer use them in our monthly RPP meetings. For us, analytic videos and excerpted fieldnotes have proven to be less generative, perhaps because the before and after of a student’s decision to “walk away” (for example) are lost, whereas narrative video data provide the context in which walking away might be a sign of student understanding and judgment. We wonder if the “before and after” of narrative data, presenting a more recognizable arc of practice in which student learning unfolds over time, can foster deeper evidence-based discussions within an RPP context.

This fall we will be exploring the use of short 3-minute student learning narrative videos as boundary objects to engage classroom teachers with students’ afterschool learning experiences in STEM-rich tinkering.

References
Derry, S. J., Pea, R. D., Barron, B., Engle, R. A., Erickson, F., Goldman, R., . . . Sherin, B. L. (2010). Conducting video research in the learning sciences: Guidance on selection, analysis, technology, and ethics. Journal of the Learning Sciences, 19(1), 3-53.

This collaboration is funded by the Overdeck Family Foundation and the National Science Foundation (Grant DUE-1238253).