Advancing Measurement and Assessment to Improve People’s Lives

In a classroom, teachers frequently assess their students’ strengths and needs through both formal means like tests, and informal means like casual conversations or observations of students’ interactions. This data-gathering helps inform their decision-making and is a critical component of instruction.

Laura Hamilton thinks of measurement and assessment as functioning in a similar way for researchers and policymakers. “To promote high-quality opportunities for individuals, policymakers need to deeply understand how schools, workplaces, and other organizations support people,” she explains. “High-quality measurement and a thoughtful approach to assessment and data use can inform decisions about how to improve these systems.”

Hamilton is a senior director of education measurement and assessment at AIR. In this Q&A, she discusses the role of measurement and assessment in addressing systemic inequity, the potential implications of using artificial intelligence, and AIR’s work in these key areas.
 

Q: What role can measurement and assessment play in addressing equity and systemic racism across human services programs?

Hamilton: It’s encouraging to see the growing commitment among educators, policymakers, and measurement and assessment experts to ensuring that our work promotes equitable opportunities for all people to learn and thrive. Sometimes that means fundamentally reimagining everything from what we measure to how we use data from those measures to inform decision-making. This is a major shift in the field.

For example, when we use the results of student achievement tests to inform policy, we need to improve how we measure not just student outcomes, but also the learning opportunities that contribute to those outcomes. Some large-scale studies already include a survey component, asking questions about which topics students received exposure to in their classes, or what kind of professional development their teachers could access. These are potential opportunity indicators that we could build into our analyses of large-scale data.

We’re considering the best uses for this type of survey data, which could help identify potential reasons for differences in student performance. Using this lens enables us to think beyond the disparities in performance between groups, to more fully consider how the education system serves those students. My colleague Corey Savage and I wrote about this type of approach in The 74 in response to recent National Assessment of Educational Progress (NAEP) results in Civics and History.

We also need to consider the content and format of our assessments to ensure that they are supporting valid inferences for people from different social, cultural, and linguistic backgrounds. We need to engage a variety of groups, including learners, educators, and the broader community to inform how we design and use these assessments.
 

Q: What developments in measurement and assessment are you following?

Hamilton: New developments in artificial intelligence have major implications for measurement and assessment. They’re likely to impact both how we assess and what we assess. AI will create many opportunities to reduce inefficiencies and costs; it could also support innovations to make test-taking more engaging and less stressful. For example, assessment developers could apply AI tools to create scenario-based tasks that allow students to interact in a simulated environment and engage in complex, collaborative problem solving.

Of course, AI poses significant risks around potential bias, test security, and lack of transparency. For example, when AI is used to score written essays, neither the test-takers nor the other users of those scores will fully understand the criteria that resulted in a particular score. Our measurement and assessment experts will collaborate with AIR’s experts in data science and technology to explore innovations in AI-informed assessment while mitigating potential harms.

AI will also influence the kind of knowledge, skills, and dispositions that young people will need to succeed economically and navigate society. In today’s society, we need to assess a much broader set of competencies and outcomes than in the past. For example, how do we define literacy? What competencies do people need to be proficient writers, given the availability of new AI tools? How should we teach and assess digital information literacy, given the growing threat of misinformation and disinformation? What kinds of jobs should we prepare young people for, and how can we design assessments to establish that they possess the key competencies they’ll need to succeed in those jobs? Answering those questions will require collaboration among researchers, developers, policymakers, and practitioners.
 

Q: How is AIR poised to increase its focus on measurement and assessment?

Hamilton: Measurement and assessment touches so much of AIR’s work, with applications to research and technical assistance in education; workforce development; and youth, family, and community development. AIR already has so much existing work in these areas to draw upon, including our support for the NAEP, as well as evidence on social and emotional competencies, civics, and school climate.

The Center on Advancing Measurement and Assessment at AIR will bring together and build on all this work. In addition to supporting research and technical assistance in specific research areas, the Center will also serve as a hub for innovation and best practices in the field. We will have a centralized repository of resources around measurement, which will help raise awareness of these best practices.

In addition to other researchers, we are excited to partner with educational institutions, government agencies, foundations, and other organizations, to ensure that our work is informed and shaped by a wide, diverse set of priorities and voices. Most importantly, equity underpins all of this work—our ultimate goal with all of these projects is to promote opportunities for all individuals.