Activity Directions: Evaluating Supports

Purpose

Use a protocol or tool to analyze and respond to data in order to evaluate and adjust the supports being provided for implementation. This activity offers two approaches. Depending on the situation, teams may choose one or the other, or do both:

1. Approach 1: Use a protocol to analyze data, evaluate supports, and make decisions about next steps

a. This should be done regularly; ideally, the leadership team engages in this work as part of implementation cycles multiple times each year.

2. Approach 2: Use a process to analyze the organization’s systems and infrastructure to make decisions about next steps

a. This is an organizational capacity assessment. It should also be done regularly, but less frequently than Approach 1.

Approach 1: Use a protocol to analyze data and make decisions

Time

60 minutes or more 

Preparation

Note: Depending on the leadership team, different people may be responsible for different aspects of this preparation.

1. Schedule the leadership team to participate in this activity. 

2. Review the Activity Directions: Data Systems: Preparing for Analyzing and Responding to Data and select one for the team’s decision-making needs or questions. Prepare to facilitate this protocol or process.

3. Select relevant data for analysis and decision-making. Ideally the team will review data that helps the group measure its progress toward meeting each of its process, fidelity, and outcome goals. (See Activity Directions: Planning for a Decision-Support Data System for support in identifying, collecting, and using data for decision-making.)

4. Prepare displays of selected data. (See Activity Directions: Data Displays for guidance on preparing displays.) 

5. Ensure all participants have access to:

a. displays of data being reviewed

b. Any handouts or resources necessary for the protocol that will be used to review the data and make decisions

c. The team’s working Action Plan for revision (see Activity Directions: Develop an Action Plan to Support Instruction [insert link]  for more on developing an action plan; see Action Plan template [insert link] for one option)

Directions

1. Ask participants to review all data being analyzed. 

2. Use the selected protocol to facilitate the team’s conversation. Focus the conversation on making decisions about the supports the team is providing for implementation. Based on the analysis of the data:

a. What should be continued?

b. What should be adjusted? 

c. What should be discontinued?

d. What should be added?

3. Capture decisions made through this process as action plan items. Be sure to articulate steps that ensure the changes will be implemented successfully. 

Approach 2: Use a process to analyze the organization’s systems and infrastructure

Time

3 hours or more 

Preparation

1. Schedule the leadership team to participate in this activity. 

2. Review the Drivers Best Practices Assessment (NIRN) Pages 4-9 outline how to facilitate the assessment, scoring, and next steps. Prepare to facilitate this process.

3. Review the data sources column in the Scoring Rubric, pages 14-27. Compile available data. (See Activity Directions: Planning for a Decision-Support Data System for support in identifying, collecting, and using data for decision-making.)

4. Prepare displays of selected data as necessary. (See Activity Directions: Data Displays for guidance on preparing displays.) 

5. Ensure all participants have access to:

a. displays of data being reviewed

b. Access to the Drivers Best Practices Assessment (NIRN)

c. The team’s working Action Plan for revision (see Activity Directions: Action Planning for more on developing an action plan; see Handout: Planning Template for one option)

Directions

1. Use the facilitator directions in the Drivers Best Practices Assessment to lead the team through completing the assessment of the organization’s infrastructure and systems for supporting implementation.

a. Score each item through consensus vote.

2. Review and use the (a) Total score, (b) Scale Scores, and (c) Item Scores to identify areas of strength and need.

3. Identify priorities to address within a plan.

4. Develop and create an Action Plan that defines immediate and short-term actions focusing on improving the infrastructure activities to support the use of the selected practice and curriculum development.

5. If this is a repeated administration of the assessment, review and update existing plan to continue support for the selected practice and curriculum development.

Source

Ward, C., Metz, A., Louison, L., Loper, A., & Cusumano, D. (2018). Drivers Best Practices Assessment. Chapel Hill, NC: National Implementation Research Network, University of North Carolina at Chapel Hill. Based on: Fixsen, D.L., Blase, K., Naoom, S., Metz, A., Louison, L., & Ward, C. (2015). Implementation Drivers: Assessing Best Practices. Chapel Hill, NC: National Implementation Research Network, University of North Carolina at Chapel Hill.