Handout: Data Analysis Protocol

1. Observations: Make observations about the data.

a. Begin with factual observations (reflecting is the next step). These types of responses might begin with “I notice...” or “I see...” The group can self-monitor to be sure these responses are observational. Possible indications that a response has moved beyond observation to interpretation or inferencing might be the inclusion of words like, “therefore,” “because,” or “however.”

i. Consider the questions below to invite observations about a given set of data.

1) What are the commonalities or patterns in this data set?

2) What inconsistencies or discrepancies (if any) are evident?

3) What is not represented in the data?

ii. Consider the questions below to invite observations among different data sets.

1) How do the data sets compare to each other?

2) What patterns or similarities are evident across different data sets?

3) What inconsistencies or discrepancies (if any) are evident?

2. Reflect and Make Inferences: Ask participants to reflect upon the observations that have been made and make inferences. Be sure someone captures the ideas and questions raised in this conversation.

a. What assumptions might be connected to what you are noticing in the data?

b. What areas in the data stand out as needing further explanation? Why?

c. What surprises did you find in the data (both positive and negative)?

d. What patterns or themes do you see in the observations?

e. Which of the observations are most relevant to understanding current instructional practices?

3. Questions/Conclusions: Ask clarifying questions or draw tentative conclusions

a. The inferences that the team has generated may raise more questions that need to be answered to support the inferences before a tentative conclusion can be made. If this is the case, generate questions by following the steps below:

i. Brainstorm questions that arise from the observations and inferences the team has made about the initial data set. Record these questions on chart paper.

ii. From this group of questions, identify the questions that must be answered before any tentative conclusions can be made. Record them on a new sheet of chart paper, leaving room next to each question to record more information.

iii. The clarifying questions the team has identified may be answered using the data already collected and displayed. It is also possible that new data will need to be identified, collected, displayed, and analyzed.

iv. For each of the clarifying questions, brainstorm the data needed and record the data element(s) next to each question on the chart paper. (Update the team’s working action plan to capture the work necessary to address the additional data needed.)

4. When the team is ready, ask the participants to summarize the most important findings from the data.