Action Overview: Evaluate the effectiveness of the professional learning system

This set of resources is designed to help educators evaluate the effectiveness of the professional learning system.

Introduction

The theory of action for professional learning systems outlined earlier in this strategy shows how the system affects educator knowledge, skills and dispositions, teaching and learning environments and equitable academic and experiential outcomes. The theory also shows a series of feedback loops throughout the system (see image below).

These feedback loops are essential to ensuring the system works effectively and is able to increase educator knowledge, skills and dispositions to improve teaching and learning environments and meet the organization’s goal(s) for equitable academic and experiential outcomes for students. In order to help guarantee the system has the desired impact, leadership teams need to be purposeful and strategic in how they

• Design a monitoring process for the system using feedback loops
• Implement and adjust the system as needed to meet changing needs for educators and students

What goes into designing a monitoring process?

Leadership teams need to carefully design a plan for monitoring the implementation of the system to ensure it is done with fidelity and is having the desired impact. The plan should:

• identify the sources of data to be collected regarding learners, educators and the system, and
• determine when and how the data will be collected and reviewed to ensure the professional learning system works as designed.

A variety of descriptive and measurable data about educators, learners and the system will be generated as opportunities are put into place. Teams need to consider how to use this data immediately and over time to improve system operations, measure the effect of the system on teaching and learning throughout the organization and adjust operations over time for greater effectiveness.

Data collected through the monitoring process informs the leadership team and stakeholders about a variety of questions regarding the professional learning system including:

• Is the system being implemented as designed?
• What changes are needed in the system to make it more effective?
• How is the system increasing or changing educator knowledge, skills and dispositions?
• In what ways has the system helped improve teaching and learning environments?
• How is the system helping the organization to meet their goal(s) related to equitable academic and experiential outcomes for students?
• How have equitable academic and experiential outcomes for students changed through system implementation?

The monitoring plan to address these questions should include multiple types of data from multiple sources to create a complete picture of the impact the professional learning system has had on educators and learners and how that system might be improved for greater impact in the future.

How does the leadership team implement and adjust the PLS?

Consistency in implementation is essential to ensure the impact of the professional learning system can be accurately measured and to meet the goal(s). This stage of the process begins by communicating the expectations of the system with stakeholders and putting the infrastructures and supports needed in place. As the system is implemented, it is important that all of the people impacted by the system understand and support the vision behind the design of the professional learning system. Those leading the implementation need to keep the focus on educator and student learning while orienting the system on the results for instruction and student learning.

As the organization implements the system, including the identified monitoring process and feedback loops, it may find that some professional learning experiences are not having the desired effect or are too difficult to implement effectively. Teams may find they need to adjust the competency, organization, or leadership of their professional learning system. The National Implementation Research Network defines these drivers as:

• Competency of those providing learning opportunities related to selection, training and coaching
• Organization of the system related to decision support data systems, facilitative administration and systems interventions
• Leadership of the system which may include either technical and/or adaptive solutions

Leadership teams need to adjust supports throughout these various areas on an ongoing basis to ensure the professional learning system remains effective over time and continues to provide the best supports to make the process meet all of the desired impacts and outcomes.

Suggested Activities

How do educators use Portal resources to Implement, monitor, and adjust the professional learning system?

1. Guide the planning, conduct, and reporting of an evaluation of a comprehensive professional learning system: Activity Directions: Conduct Ongoing Assessment and Evaluation
2. Recommend and prioritize continuous improvements to the Comprehensive Professional Learning System: Activity Directions: Continuous Improvement of a Comprehensive Professional Learning System

Related Content

Continuous Improvement

Data Systems

Sources

Learning Forward. Transform Professional Learning “Evaluating professional learning: Measuring educator and student outcomes.” March 2014

Guskey, T. (2002). Does it make a difference? Evaluating professional development. Educational Leadership, 69(6),

MDE Continuous Improvement Model Questions and Tasks (Step 4 and 5)

MDE Key Components of Continuous Improvement

Killion, J. (2013). Comprehensive professional learning system: A workbook for states and districts. Oxford, OH: Learning Forward.

Hirsh, S., & Crow, T. (2018). Becoming a learning team (2nd ed.). Oxford, OH: Learning Forward.

Park, S., et. al. (2013). Continuous improvement in education. Stanford, CA: Carnegie Foundation for the Advancement of Teaching.

Wholey, J.S., et. al. (Ed.) (2004). Handbook of practical program evaluation Second Edition. San Francisco, CA: Josey-Bass.

Learning Forward Standards for Professional Learning https://learningforward.org/standards-for-professional-learning

National Implementation Research Network Implementation Drivers https://nirn.fpg.unc.edu/learn-implementation/implementation-drivers

Back to Strategy Overview: Professional Learning System.