An assistant superintendent wraps up a lengthy, data-filled presentation at your monthly board meeting and, quite frankly, you’re relieved when she’s finished. She threw too much information at you and, after 45 minutes, it was becoming increasingly difficult to focus on what she was saying.
While you’re relieved, you’re also frustrated. A lot of the data didn’t mean that much to you, other than to confirm what you already knew – that some of your students aren’t doing well on state test scores. And her report offered no clue as to what your school board can do to support future academic gains.
So what good was all of that data?
You aren’t the first to ask that question. For all of the talk of data-driven decision-making, many new board members – and quite a few board veterans – struggle to make good use of the data available to them. That’s particularly true when looking at data focused on instruction and student achievement. It’s easy enough to look at overall test scores, for example, to learn if district students are reading at grade level. But it’s not so easy to “drill down” into the data and identify exactly why some students are struggling.
It’s even harder to use data to guide your school board in finding the best solutions to improve classroom instruction.
That has to change. Data is a powerful tool for school boards, says Katheryn Gemberling, a nationally known consultant on data-driven leadership and retired deputy superintendent of Maryland’s Montgomery County Public Schools. “It’s really important to understand exactly what data you need as a board – and how you can use that data well.”
Where to start?
One way to look at this issue is to consider the various roles a board can play in the use of data. Even the most novice board members understand, for example, that one of their key roles is to be good stewards of the district – and to hold school personnel accountable for students’ instructional success.
So how does data help the board fulfill this role? What benchmarks or criteria do you use to evaluate the successes and failures of your superintendent and district leadership? And how do you determine whether good instruction is occurring in your schools? If 70 percent of third-graders at your elementary school are proficient in reading, is that a huge success or a disturbing failure?
Such questions help you narrow the selection of data you’ll need. No doubt your board already has determined what information it thinks is important, but you may have your own ideas of what data matters. Or you may be at a total loss in analyzing the data you’re already given.
Not to worry. The first place to start is to ask for someone to sit down with you and bring you up to speed. If you believe additional data might be revealing, raise your ideas at an appropriate time.
Joe Wehrli, director of board development for the Oregon School Boards Association, says any data – and particularly any benchmarks or goals – used for accountability must be mutually agreed on by the board and superintendent.
“They need to be very specific and laser-focused in their discussion with the administration about what achievement expectations they are after … to identify specifically what types of assessments they’ll use,” he says. “If you’re holding district personnel accountable for their work, you’ve got to set expectations …and you have a responsibility to provide the resources for the work.”
Consistency also is important. A school board shouldn’t hesitate to look at new data if it can be useful, but it’s important to track the same data over time. Any progress on test scores – or any evidence of success with a new remedial program – only will be revealed with time.
And apples cannot be compared to oranges. Data needs to include what the district leadership agrees is important. If individual board members start making too many personal requests for data, it’s easy to inundate the central office with additional work.
“There is a tendency to get data in a million different areas,” says Susan Rountree Salter, director of board development for the Alabama Association of School Boards. “But we caution boards to work with their leadership team to figure out what data they need and to prioritize where they want to start looking at data. You can overwhelm a central office in a red-hot hurry.”
Consider the Basics
To determine which data is the most useful, experts say, it’s important to understand a few basic concepts: One is the power of comparisons. Consider that earlier example of an elementary school where 70 percent of third-graders read at grade level. What does that really tell you?
Every time you look at data to determine how your schools are doing, you need to ask, “Compared to what?” Gemberling says. “What was your target? How does this compare to the last time we took this same measure? Are we improving, maintaining the status quo, losing ground? If we didn’t make our target, are we closer than before?”
How information is presented to the board also makes a difference, says Steven Ultrino, a former board member for Massachusetts’ Malden Public Schools. His doctoral studies looked at how school boards can impact student learning. In his former district, he says, school officials use a common format when presenting data, which is useful, for example, in tracking the efforts of principals as they work on their school improvement plans.
That consistency extends to such details as color coding on charts and tables – such as using green for proficient levels of academic achievement or red for below-proficient levels. It is one of many practices, he says, that allow board members to more quickly grasp what the data is saying.
“Data needs to be clear,” Ultrino says. “It needs to be consistent.”
Guiding the Future
Accountability, however, isn’t enough. It’s one thing to put the superintendent on the spot after a three-year effort to boost reading test scores; it’s another to use data to help determine why that effort failed – or what to try next.
Astonishingly, school boards frequently fail to take that extra step. They complain about the results and leave it to the administration to fix, or they set aside the data and turn to anecdotal evidence, gut feelings, or staff recommendations to decide the district’s next response. They don’t recognize how data can help understand the root causes of low test scores, make clear why an intervention strategy isn’t working, and decide whether to revamp or abandon an initiative.
“That’s the biggest ‘ah-ha moment’ I’ve had in 20 years in working with data – that there’s accountability data and instructional improvement data,” says Ronald Thomas, associate director of the Center for Leadership in Education at Maryland’s Towson University. “School boards need [data] to deeply delve into the next set of questions … what do students know, what do they not know, and what are we going to do about it.”
For some, this observation will seem obvious. But it won’t surprise you to learn that few board members have any training in data analysis, so they don’t always know what data to request or how to look at it. Even administrators can be uncertain about how to present the data that will help guide the board to an effective strategy.
It can be a challenge. A simple test score chart, for example, can be sufficient to identify where schools are failing to make adequate yearly progress. But what are the real factors at work? Is it student mobility? A young and inexperienced teaching staff? An incompetent principal? Gemberling says board members need to be cautious when looking for cause and effect. “There’s not necessarily a correlation that one factor is the cause of another,” she says.
That’s why it can be useful to look at data in different ways. Using a chart known as a scatterplot – and comparing school poverty levels against academic growth – might provide some surprising findings. A school serving an affluent neighborhood might report high test scores, yet its students may be making minimal academic gains each year. Meanwhile, a school serving a low-income community may score poorly on tests but is advancing children by a grade and a half each year.
“When you look at a scatterplot, you look at a pattern,” Gemberling says. “There may be many factors [at work] …Nothing is a simple answer in any of this work. But the more comparisons you can make using the data, the better chance you have of getting a handle on what’s happening.”
It’s not always so complicated. Data can take many forms. William Parrett, director of Boise State University’s Center for School Improvement and Policy Studies, tells a story about a district that took a closer look at its calendar and subtracted vacation days, holidays, and classroom time lost to professional development, teacher-parent conferences, early dismissals, athletic events, school assemblies, and the like.
“What this district found, when they counted it up, was that their schools ended up with 13 to 15 eight-hour days of instruction per subject per year.” Parrett says. “Is that enough time to meet your state standards?”
No one expects board members to become experts in data analysis. “The board does not need to be designing this data and then choosing programs or finding solutions to fix problems,” Salter says. “The board needs to say, ‘Let’s look into the problem.’… Then the board turns it over to the administration to [find the right data], to create a solution – a plan for attacking the problem. The board’s role is to consider whether it can afford those things and allocate the resources to make it happen.”
That sounds great as a general principle, but it doesn’t improve your ability to analyze data. So how do you read the data? What are the tricks and techniques to find the answers you need?
Your answers are found in training – and not just for you and your fellow board members. Even if school boards could identify the district’s challenges and develop the best responses on their own, they still need to make sure teachers and administrators can use data to turn the board priorities into day-to-day instructional practices. Your staff needs to understand how to look at data to fix campus problems.
“One of the board’s roles is to enable the superintendent and staff to build up the capacity of teachers to do the kind of data work we’re expecting them to do,” Thomas says.
That’s even more important given the findings of Harvard University’s Ronald Ferguson, who says the widest variations in classroom instruction aren’t found among different schools – but among different classrooms in the same school. That’s true even within your highest-performing schools. Much of the work to bring about instructional change will depend on what happens within the schools.
“The story is inside the buildings,” he says. “It’s how the stronger classrooms differ from the weakest. We spend so much time talking about great schools and troubled schools, yet we’ve got great and troubled classrooms in every building.”
So building human capacity – the ability of principals and teachers to analyze data and put it to use – is a key role of the school board. What’s more, you need to be willing to create a structure that allows staff the time to sit down together and do this work. To make effective use of data, you may need to modify school schedules to create time for teachers and principals to work together to review their data. Your district may need to upgrade its instructional management system or data warehouse to provide the tools for staff to pull it together in a useful way.
The toughest battle may be convincing your board colleagues to invest in their own training, as some undervalue data’s usefulness or their need for training. That’s a huge mistake, Ultrino says. Some of the most valuable training he’s received from the Massachusetts Association of School Committees was in data-driven decision-making. “All of my work on my doctorate over four years didn’t address what I learned in three hours,” he says.“Professional development is key for board members.”
Such efforts will pay big dividends, but Salter notes board members shouldn’t be surprised if they face some staff pushback. “If the board has not been using data, and suddenly they begin to want the data to make decisions and hold people accountable, that can be a very scary thing for your administrative team. That’s why it’s critical for the board and leadership team to have some discussions – work sessions or retreats – to look at why you want to look at the data, what you will do with it, and then reassure the staff that you’re not on a witch hunt.”
It’s a discussion you can’t have too soon, Gemberling says. State and federal policymakers are increasingly intrusive in their school reform efforts, and that should raise some concerns.
“Who is the driving force in education? Traditionally, we’ve seen it as a local [matter] but people are beginning to question that. So one of the things that I’m really convinced of is a local school board that knows how to use data – how to interpret it, and how to communicate its importance to constituents – is the board that will stay in control of its own local governance.”