Component Overview: Data System

Why is a Decision Support Data System needed?

Data systems, in general, support data collection and analysis. Often in education, data systems refer to the technical aspects of how data is organized and stored. In contrast, this strategy focuses on a Decision Support Data System (NIRN, 2016), which is a system that supports identifying, collecting, and analyzing data in order to support continuous improvement. This mirrors a 2010 recommendation from the U.S. Department of Education: “Think of data-driven decision making as an ongoing systematic process rather than a one-time event centered on the acquisition of a data system” (USDOE, 2010, page xix). For the purposes of the portal, all references to "data system" are inclusive of a Decision Support Data System.

Historically, data use in education refers to the processes by which educators examine assessment data to identify student strengths and deficiencies and apply those findings to their practice. An important aspect of this strategy is the use of multiple measures for continuous improvement, including the use of fidelity and programmatic data in addition to student outcome data. Both fidelity and programmatic data help us understand whether we implemented what we determined would improve student outcomes. When the goal is to increase student learning in a standards-based education system, using data to make decisions about how to support changes in instruction and curriculum is paramount. “The moral of the story is, if we want to get different results, we have to change the processes that create the results. Just looking at student achievement measures focuses teachers only on the results, it does not give them information about what they need to do to get different results.” (Bernhardt, 1998.)

When a Decision Support Data System is running well, teams can use data to make decisions about effective instructional practices and the supports needed for educators to implement them successfully.

What are common barriers to using data for decision-making that can be addressed through a system?

Ingram et al (2004) uncovered seven barriers to the use of data to improve practice. They include:

Cultural Barriers:

Teachers and leaders often develop their own interpretation or metric for judging the effectiveness of instruction, and often this metric differs from the metrics of external parties (e.g., state accountability systems and school boards).
Teachers and administrators often base their instructional decisions on personal experience, intuition, and anecdotal information rather than on information and data that is collected systematically.
There is usually little agreement among stakeholders about which student outcomes are most important and what types of data are meaningful.

Technical Barriers:

Teachers can disassociate their own performance and that of students, which leads them to overlook useful data.
Data that teachers want about what they consider to be important academic outcomes are not readily available and are usually difficult to measure.
Organizations rarely provide the time needed for teachers and leaders to interpret and analyze data.

Political Barrier:

Data have often been used politically, leading to mistrust of data, misunderstanding of data, and data avoidance.

What are the purposes of data in a Standards Based Education System?

Developing, maintaining, or refining a data system is important to fully implementing a standards-based education system.

Data help organizations:

Know what’s working and what isn’t within the organization in order to identify solutions to implement. In a standards-based system, this includes using fidelity data about instruction in addition to programmatic and student outcome data.
Make timely decisions for continuous improvement of instruction, curriculum, and learning, including planning, delivering, and adjusting support for educators in implementing effective instruction aligned to standards in order to increase student learning.

Data help individual educators and teams of educators:

Understand where students are succeeding and struggling
Understand the extent to which effective instructional practices are being used as intended and what is needed to support teachers
Understand the impact of instructional practices on student learning, and make adjustments to instructional practices based on that understanding
Respond to student needs and adjust the intensity of instruction using a multi-tiered system of supports (MTSS)

Data help students and families:

Understand where the student is succeeding and where the student can grow
Understand how the system is supporting continuous improvement for the success of all students

(adapted from https://dataqualitycampaign.org/why-education-data/)

What guidance does the Portal provide for developing a decision support data system?

Portal Connections:

Policies and Plans: for support in communicating about data with stakeholders and involving stakeholders in decision-making.

Sources:

Bernhardt, V. L., (1998, March). Invited Monograph No. 4. California Association for Supervision and Curriculum Development (CASCD). https://nces.ed.gov/pubs2007/curriculum/pdf/multiple_measures.pdf, last retrieved August 13, 2019.

Ingram, D., Seashore Louis, K. and Schroeder, R. G. (2004). “Accountability Policies and Teacher Decision Making: Barriers to the Use of Data to Improve Practice,” Teachers College Record, Vol. 106, No. 6, pp. 1258–1287.

National Implementation Research Network (NIRN), AI Hub website, “Handout 28: Drivers Ed - Decision Support Data Systems,” (2016.) https://nirn.fpg.unc.edu/resources/handout-28-dsds, last retrieved August 13, 2019.

U.S. Department of Education, Office of Planning, Evaluation, and Policy Development, Use of Education Data at the Local Level From Accountability to Instructional Improvement, Washington, D.C., 2010. https://files.eric.ed.gov/fulltext/ED511656.pdf, last retrieved August 13, 2019.