A New Approach to Old Questions

Institutional research offers a new way to examine what's working and what's not

By Dava Stewart

Institutional research (IR) in independent schools is a relatively new discipline, and as organizations begin using their data to solve problems, questions arise. Is this area a good fit for our school and culture? What can our data be used for? Should we have concerns about privacy? Who should be in charge of an IR initiative?

Maret School in Washington, D.C., established the Center for Institutional Research in Independent Schools (CIRIS), which seeks to provide support for professionals looking for answers to IR questions. Eric Heilman is Maret’s director of institutional research and executive director of CIRIS. CIRIS hosts an annual Summer Fellows Lab, a website, and virtual sessions, and it provides conference materials for school personnel.

“IR can touch on every functional aspect of a school from admissions to student experience and DEI [diversity, equity, and inclusion],” said Heilman. “There are really important and mission-centered initiatives that an IR program can undertake to measure and monitor core aspects of a school’s mission.”

Early Considerations

One of the first questions organizations may have is “Who should be doing IR?” and the answer depends on several factors. Perhaps the most important question is “Who wants to be doing IR?” followed closely by “Who is already doing IR?”

Most independent schools are doing various types of IR around enrollment management and admissions of necessity. Schools track how many applications they get per available spot, their acceptance rates, where students come from, and where they go after graduation. All those metrics are commonly part of a good marketing strategy, so although the discipline of IR is relatively new, many of the foundational concepts on which an IR program is built have existed for a long time.

Academics are a common area for data analysis as teachers and schools track grades, achievement patterns by race and gender, student outcomes, and so on. Increasingly, schools are expanding the idea of success outside of grades, and IR can help track and measure metrics related to student engagement and wellness. DEI is another area where IR may be able to help school leaders and staff better understand problems, potential solutions, and results.

How Should an IR Department be Structured?

Successful IR programs come in many different forms and are staffed by people with extremely varied backgrounds. Indeed, one of the common challenges Heilman noted is that few people possess skills in the five competencies crucial for success in IR: institutional knowledge, data architecture, data analysis, communication, and data culture. There are numerous paths to having expertise in two or three of those competencies, meaning the training, education, and vocational backgrounds of practitioners are widely varied. However, having skills in all five competencies is unlikely to happen without effort.

One characteristic successful IR programs have in common is a strong relationship with the technology department, a necessity in several ways. In many instances, IR is actually part of the tech department, but in other instances, it’s led by a member of the leadership team or faculty—or it may be a completely separate entity.

At The Episcopal Academy in Pennsylvania, for example, IR is led by Erin Rehel, Ph.D., and Josh Berberian. The two have radically different backgrounds. Rehel has a degree in sociology, has worked in academia, then moved to a search and consulting firm specializing in health care and education. Berberian has been a math teacher for 29 years. “People often think we have two people who do IR, but we don’t do it full time,” said Rehel. “My title is director of strategy and IR, and IR is only part of what I do. My role is to identify projects that will have the biggest impact, and then Josh does the execution.”

Berberian teaches math and works with a team to coordinate teaching and learning. “It’s awesome having two people to do IR,” he said. “We always have someone to bounce ideas off of.” He added that because Rehel sits on the senior leadership team, she has a wider view of projects likely to get traction.

At the Gilman School in Maryland, Rachel Gorsky is a full-time institutional researcher situated within the tech department. Her path to IR included a degree in social science followed by a master’s in quantitative research. She worked as an assistant to a professor who ran a survey research initiative prior to entering the world of independent schools, giving her a solid foundation in working with data. 

Gorsky didn’t initially expect to be housed within the technology department. “When I first started, I was very confused,” she said. “But it’s been awesome. It’s great to have direct access to the director of tech if I need clarification or want to talk about a different way to gather data so that it’s in a more useful format.” She also said she reports to both the director of technology and the assistant head of school, and that helps keep the different parts of her job at the fore.

Regardless of the background of the person doing IR or the structure of the department, having a voice within leadership is a necessary element to a successful IR program. Rebekah Sollitto at Poly Prep Country Day School in New York is assistant head for strategic initiatives. “Rather than having an official IR department, I do a lot of projects that are related to IR,” she said.

The advantage to that structure is the ability to put together teams specific to projects. Sollitto said her title provides “a broad bucket” that allows her “to identify projects that are important at any given time.” Being part of the school’s leadership gives her a larger view of what projects are likely to have the greatest impacts.

Crafting Research Questions

For an IR initiative to be successful, it must investigate questions important to the faculty and leadership. In some cases, that means helping teachers craft good research questions; in others, it means investigating data to measure programs and initiatives.

The importance of the research questions is to help define program success and determine which projects to pursue. This requires a careful balance between leadership, faculty, technology, and the IR team. The data itself must also be taken into consideration. Is the data already collected and stored in a database owned by the institution? Can it be gathered from outside sources? Does it need to be collected? What are the privacy implications? Are there ethical considerations?

Another important consideration is who within the school community can most benefit from the research. Denise Uehara, Ph.D., is the principal researcher and knowledge mobilizer at Kamehameha Schools in Hawai‘i and was formerly the inaugural institutional researcher at Punahou School, so she has deep experience contributing to the field of IR in independent schools. Her previous positions at a university and regional laboratory prepared her for the different aspects of IR.

She said part of her job is to “coach teachers to construct relevant and rigorous research questions designed to explore their own practice,” guiding them to consider innovative strategies that will impact their approach to teaching and, ultimately, student learning. Their inquiry questions around teaching and learning align with the school’s initiatives and campus identity.

At Durham Academy in North Carolina, Victoria Muradi, director of strategic initiatives, undertook a project to go deeper into the questions of achievement and gender, race, student birthplace, and other demographic data points to gain a much deeper understanding of diversity. “We built a dashboard for the upper school, so anyone could look and see how students are doing individually. It can show statistics by semester or year and longitudinally; users can filter by demographics or departments,” she said.

Now, staff can begin to see patterns that weren’t obvious before, such as which classes have a wide range in grades and whether demographic factors seem to be associated with grades in certain classes.

The research questions that Kelsey Naughton, director of IR at Deerfield Academy in Massachusetts, addresses are oriented more toward strategic business planning. “Keeping our students’ well-being top of mind, we leverage data that we have in a KPI [key performance indicator] business-intelligence sense and use what we learn to make decisions based on the data,” she said. “How can we use this to make decisions that help our school?”

Some schools are using IR to inform teaching and learning, but Naughton noted that her work does not focus on evaluation and assessment in the classroom. Instead, she said her work examines the question of “how we can be more effective with structures, procedures, and policies.”

How Will IR Impact the School Culture?

“It’s one thing to say, ‘We need to be a more data- informed institution’ and another thing to understand and be ready for the cultural change that’s going to happen,” said Heilman.

Most people don’t associate data research with cultural change or emotional impact, but those are important considerations. In independent schools, especially, there may be concerns about using data. “A lot of people in schools have different reactions to data and numbers,” Heilman noted, adding one common one: “We’ve seen what happens in public schools. Will you judge me based on test results?”

Another impact may be unexpected, and that is sometimes the data provides information that is unwelcome or even unhelpful. While staff and faculty may be aware of areas where the school could be doing better, “seeing it in black and white feels different,” said Heilman.

One researcher shared a story about investigating faculty turnover among teachers of color and white teachers. The narrative within the school community tended to tell a story of higher turnover among faculty members of color, and the research team wanted to understand why and address the issue. What they learned was that, in fact, there was higher turnover among white faculty members, and teachers of color tended to stay at the school for longer periods of time. In the end, the leadership team decided that wasn’t necessarily a helpful piece of data to share, because it could be construed as denying a problem existed. If faculty members perceived a problem, there was a reason; it just wasn’t the reason they expected.

The pandemic presented an opportunity in the IR department at The Episcopal Academy. The school began offering certain summer school courses online for free during the pandemic and discovered doing so changed who had access to summer school. That led to a greater commitment to teaching online. “Summer programming is a huge opportunity,” noted Rehel.

Gorsky noted that many people are simply uncomfortable with numbers and feel that IR can depersonalize things. “Numbers make people nervous,” she said.

Her advice: “Give data a chance. Data is very human in a way it takes a while to recognize.” She suggested that a piece of data uncomfortable to see should be interrogated rather than put aside. “It can be very emotional. Take a deep breath and try again.”

The Endless Possibilities

IR is a nascent discipline, and in many instances, it’s too soon to point to specific impacts of IR in independent K–12 schools. However, using data to improve organizational performance is standard practice in the world of business and higher education.

With independent schools facing challenging demographic and economic trends, IR offers one important path to meeting those challenges. It can facilitate quick recognition of changes or problems, agility, and evidence of the value proposition independent schools bring to the table. “Through our website and appearances at conferences and webinars, we are hoping to share examples of IR success stories from early adopters, so school leaders can see what is possible,” said Heilman.

As for the future of IR? That’s an open question, because it has such potential to be applied in so many ways. With imagination, skill, and effort, the ways IR can be applied in independent schools are virtually unlimited. “We think about IR as iterative process,” said Heilman. “So, you lay out a question or an issue, think about related data, analyze it, send results, and then revisit the question.”