The Data Dilemma

By Scott Bray, Ed.S.

from T/TAC Link Lines
November/December 2007

For many teachers and administrators, the mere mention of the word “data” raises their anxiety levels. Educators are drowning in a deluge of data, but data collection is not a new phenomenon. Traditionally, schools have collected data in various forms, ranging from attendance reports and lunch counts to discipline records and assessment scores. However, the passage of NCLB (2001) in conjunction with state-mandated assessments has brought a renewed emphasis on data collection and analysis. The challenge facing educators is discovering how to best use the wealth of available data to positively influence instructional decision making and school improvement.

Most people would consider flying a plane without advanced instrumentation and navigational systems to be foolhardy and dangerous. Pilots are expected to consult weather reports, read their instruments, and communicate with the control tower to gain necessary and valuable information to proceed. The same principle holds true for schools and classrooms. Purposeful data collection is essential to help educators identify patterns, areas of strength, and areas for improvement to ensure the instructional needs of all students are met. Whereas, data analysis is not the only tool available to educators concerned with raising student achievement, it is critical for making well-informed instructional decisions.

Data can be used for a variety of purposes, ranging from guiding teachers’ professional development to determining needed curricular and instructional interventions (Education Commission of the States, 2002). Schools engaged in data analysis find their data useful not only in terms of making smarter instructional decisions but also in creating a meaningful dialogue within their school community. A single data set can reveal a multitude of insights depending on who is reviewing and analyzing the data. Data-driven discussions provide dialogue focused on central questions regarding what is and what is not working and why. Determining what works helps ensure higher levels of performance and achievement for all students by allowing teachers to plan with a purpose to maximize limited instructional time (Gregory & Kuzmich, 2004).

The abundance of data can be daunting, so it is important to begin with a clear focus. There are four major types of data: achievement, demographic, perception, and school process. Most educators focus on cross-referencing achievement data with demographic data in analyzing and comparing various subgroups’ achievement (Bernhardt, 2004). Collecting and connecting various types of data provides a broader perspective and allows for more informed decision making (Education Commission of the States, 2002).

Creating a Plan

Putting data to optimal use requires the creation and implementation of a plan detailing what will be done, by whom, by when, and the measurements used to gauge success. Data teams help facilitate this process by leveraging resources and sharing responsibilities (see the article on creating data teams in this issue of Link Lines). Educators must keep the big picture in mind while identifying data patterns and their implications for students and instruction. Establishing priorities at the front end of the process places focus directly on the areas of need. Forming priorities, determining desired end results, and establishing clear measurements needed to evaluate success creates the foundation of the plan.

Analyzing only year-end assessment data is insufficient. Educators must also identify current levels of performance and create a series of intermediate benchmarks tied to the established year-end goals (Hitch & Jenkins, 2004). Intermediate benchmarks representing areas of focus and growth for shorter periods of time are needed. Benchmark data provide insights into student growth and progress toward the desired end results. These check points are also the times when program modifications can be made if needed.

Once a plan has been created with clear action items listing what will be done, by whom, when, and with specific benchmark measurements, execution of the plan becomes critical. Positive energy alone is not enough to get results. Implementation puts decisions into action and pushes them toward completion even as resistance and unexpected obstacles arise (Welch, 2005). A critical component of executing the plan is monitoring progress at set intervals. Collecting new data is the equivalent of hitting the “refresh” button on your Internet browser; it provides the latest, most up-to-date information. New data, whether from benchmark assessments or other sources, offer support for teachers by providing valuable answers to over-arching questions (see Data-Driven Questions box).

Working with data is an ongoing process. Finding time to debrief with team members is essential. Teams need to meet in order to share their findings and to communicate many “soft” measures. Soft measures include time commitments, impact on morale, measures that worked, and those that were less successful (Hitch & Jenkins, 2004). Debriefing also lays the groundwork for the next data cycle. As data collection and use become part of the school culture, it will be easier to know what questions to ask, how to examine data, and how better to support teachers and students (Moody, Russo, & Casey, 2006).

Data patterns and indicators are powerful tools to help educators become better equipped to solve problems and meet the school’s goals. Schools and teachers should attempt to address only a small number of questions to avoid becoming overwhelmed. Instructional leaders should ensure that school data are easily accessed and appear in user-friendly, understandable formats. The collection and organization of data should not consume excessive amounts of time or resources, and the information collected should be relevant, helping teachers identify “the link between teaching practices and student performance so that high achievement levels can be obtained” (Miller, 2000, p. 15).

The most important factor affecting student learning is still the teacher (Marzano, Pickering, & Pollock, 2001). Providing support and dedicating time for teachers to work together increases their comfort level with data and results in increased use of data during the decision-making process (Lachat & Smith, 2005). Like well-informed pilots, data-wise educators can move forward to create classrooms where students come to realize their potential and believe in their capacity to master academic material.

Data-Driven Discussion Questions
Where are your widest achievement gaps?
How persistent have these gaps been?
Are there significant changes from one year to the next?
Are there differences worth noting between various demographic groups?
Are there major differences between major curriculum areas (math, science, social sciences, and English)?

References
Bernhardt, V. (2004). Data analysis for continuous school improvement. Larchmont, NY: Eye on Education.

Education Commission of the States. (2002). Data driven decision making. Denver, CO:
Author.

Gregory, G., & Kuzmich, L. (2004). Data driven differentiation. Thousand Oaks, CA: Corwin Press.

Hitch, C., & Jenkins, K. (2004). How do I use all this data? Retrieved August 2, 2007, from
http://www.learnnc.org

Lachat, M. A., & Smith, S. (2005). Practices that support data use in urban high schools.
Retrieved August 31, 2007, from www.centerforcsri.org

Marzano, R. J. Pickering, D. J., & Pollock, J. E. (2001). Classroom instruction that works:
Research based strategies for increasing student achievement. Alexandria, VA:
Association for Supervision and Curriculum Development.

Miller, C. (2000, April 28). School reform in action. Paper presented to the American
Educational Research Association Conference, New Orleans, LA.

Moody, L., Russo, M., & Casey, J. (2006). Acting and assessing. In K. Parker, P. Boudett, E. City, & R. Murname (Eds.), Datawise (pp. 156-177). Cambridge, MA: Harvard Education Press.

Welch, J. (2005). Winning. New York: Harper Collins.