A Pragmatic and Unified Approach to Measure Learning Globally

By Silvia Montoya, Director of the UNESCO Institute for Statistics

This blog was also published by the Global Partnership for Education (GPE). 

Last month saw a major leap forward on the global reporting of learning outcomes – on which children are learning, which children are not learning, and why.

At a meeting convened by the UNESCO Institute for Statistics (UIS) in Washington D.C. on June 26, partners representing regional and international learning assessments, as well as development partners, pledged to move forward together to tackle the pressing need for comparable data on learning outcomes at the global level.

With the Sustainable Development Goal for education (SDG 4), the international community has agreed on an ambitious education agenda to track progress. There has been significant growth and improvement in the field of learning assessment across the world. Yet today, it is impossible to provide a global perspective of what children are learning because:

• Not all regions conduct assessments.
• There is no single measure or definition of a minimum proficiency level of learning at any education level.
• The quality and scope of national assessments vary.
• Current assessments are based on different methodological frameworks that may not yield comparable results.   

Exploring the options to measure learning globally

While covering a wide range of issues, the June meeting focused specifically on these comparability issues. There are four options.

Option 1: Every country submits data based on their own choice of assessment and definition of minimum proficiency levels. This means that the resulting data used to monitor SDG 4 will not be internationally comparable.

Option 2: Every country submits data based on their own choice of assessment but the SDG 4 reporting also includes information on the quality of the data and shows how the results compare to a broad learning scale. With this conceptual information, it is possible to make some comparisons across countries.

Option 3: The stakeholders work together to link common items across different assessments. This option provides more rigor than the previous options and makes it possible to develop better cross-linkages to a common reporting scale than option 2.

Option 4: Build a critical mass of comparable data either by encouraging more countries to join existing regional or international assessments or by developing a new assessment for countries that do not have their own.

We must be pragmatic

As explained in previous blogs, the best measures and methodologies in the world will amount to little if countries cannot produce them. We must therefore take a pragmatic approach, which may mean mixing the options.

However, there are some non-negotiables. Most importantly, countries need some degree of comparability to monitor and benchmark their progress towards SDG 4. This means that option 1 is off the table.

In the short term, option 2 offers an interim way of reporting on SDG 4. It would be relatively easy to apply and would offer a global snapshot of learning while development work continues to offer a more nuanced view.

In the medium term, option 3 holds the greatest promise. Together with ACER, the UIS is already laying the groundwork to link different assessments by identifying common items and developing a broad learning scale.

At the same time, countries and development partners can pursue option 4. While the development of a new assessment would take considerable time and resources, efforts are underway, by the Global Partnership for Education (GPE) and others, to help more countries participate in existing assessments. This does not suggest that the regional or international initiatives can replace national assessments, which offer the rich and detailed information needed to guide policymaking and planning with local legitimacy and buy-in. Cross-national assessments complement the national ones while helping to produce global measures of learning for SDG 4. They also help countries strengthen their capacities to develop their own national assessment systems.  

Agreement on a stepping-stone approach

This stepping-stone approach was widely endorsed by stakeholders attending the June meeting.  They understand the political stakes, the technical issues and the need to find a balance between pragmatism and accuracy.

We need to recognize that SDG 4 indicators are barometers – showing which countries (and, for equity’s sake, ideally which segments of which countries) are making progress and which countries need help. Instead of aiming for the most technically rigorous methodologies, we may better serve the world by taking a pragmatic approach to producing the global measures while helping countries improve the quality and use of their national data.  

We proposed some tough deadlines at the June meeting, including a longer meeting to iron out such technical issues and confirm a roadmap for linking assessments, no later than mid-September 2017.

Meanwhile, we need a funding model to roll out to development partners, using various levers to expand their support via regional and international assessments. We also need the active involvement of countries and a partnership model to strengthen their statistical capacities keep us moving ahead together, in the same direction.

There is plenty to keep us busy, but with this new commitment to act in a unified manner, we feel confident that globally comparable data on learning outcomes are finally within our reach.

****
Photo: Grade 3 boys taking a numeracy test, Chiyeke Primary School, Zambia. Credit: GPE/Carine Durand