Activity Directions: Fidelity Measurement and Tool Development

Objective: Brainstorm ways to collect fidelity data for the Decision Support Data System (DSDS) and build one or more tools to measure what you have prioritized and have the capacity to accomplish.

Time :2 hours

Preparation

1. Ensure the team has a common understanding of fidelity by completing Action 1, (Data System: Fidelity Data Deeper Dive).

2. All participants will need access to the following documents: 

a. Fidelity Data Sources 

b. Fidelity Brainstorm Activity Guide

c. Prioritizing Tools to Measure Fidelity chart

Part 1 Directions: Brainstorming Fidelity Data Sources

1. Read Fidelity Data Sources to get familiar with the types of fidelity data (Context, Content, and Capacity) and the methods of collecting the data (Observation, Products, and Perception).

2. As a team, or in small groups, work through the Fidelity Brainstorm Activity Guide, following instructions and prompts.

3. Focus on Step 6: Review and Reflect from the brainstorm activity and look for patterns and connections.

· Were multiple methods of collecting evidence generated?

· Were different types of fidelity data generated?

· How will you determine which methods and types of fidelity data are needed to evaluate the effectiveness of our supports?

· Would the data collected be frequent/relevant/actionable enough to impact continuous improvement cycles to support teachers in learning something new? How will the system build the capacity to support this effort?

· From the ideas generated, what are some right next steps to further creating or improving a fidelity assessment?

Part 2 Directions: Prioritizing and Developing Tools

This part guides teams to simultaneously prioritize data for collection and assess the team’s capacity to collect and use that data. Commonly used tools for collecting fidelity data include:

· List of look-fors/checklist (for coaching support, peer observation, lesson plan reflection, etc)

· Survey 

· Interview questions and recording form

· Coaches log template

1. Identify sources that could most likely be used for data collection:

a. Review the sources of evidence across indicators the team added to the Fidelity Brainstorm Activity. Note which sources appear for multiple indicators. 

b. Highlight each row with the same Method of Collecting Evidence with a different color.

c. Reviewing each method, consider which ideas could be combined into one tool. For example, could one observation tool be used for multiple indicators? Note those.

2. Discuss and note implications for using various tools: 

a. What indicators have the highest potential impact? 

b. What do we want staff to focus on right now? 

c. What do we have the capacity to collect and use to make decisions about our supports?

3. Use the Prioritizing Tools to Measure Fidelity chart to record the tools the team has prioritized for development. 

4. Develop the prioritized fidelity tools. 

5. Use the tools to ensure they work as intended. Consider applying the process of Usability Testing

Reference

National Implementation Research Network. Topic 2: Usability Testing. Retrieved 8/30/19 fromhttps://nirn.fpg.unc.edu/module-5/topic-2-usability-testing