Helping Countries Measure Learning: Deeds, Not Words

Right now, the Inter-agency and Expert Group on Sustainable Development Indicators is meeting in Geneva to iron-out ways to resolve some of the technical glitches surrounding the indicators. The good news is that education is among the areas in best shape. But we still have our work cut out for us, especially when it comes to learning outcomes.

We know just how messy and complicated it is to measure learning. After working on it for more than 30 years, what have we learned? Measurement demands compromise. Perfect metrics don’t exist, especially in the area of learning, which is shaped by a myriad of factors specific to countries and their communities.

As a starting point, at the national level we must agree and focus on building the capacity for robust learning assessment systems so that countries can own and utilize learning data to drive improvements in the sector.

At the same time, we all share in a global level mandate to design the tools and approaches that will promote measurement of the key targets that were adopted by the global community as part of the SDG agenda.

We may hear criticisms about this “two track” approach – and indeed it will require compromise on all sides and careful attention to bringing these two processes together. But it is important to recognize how much can be gained from such an approach.

It is not too much to say that the life chances of millions of children, youth and adults are riding on it. The world’s leaders have decided that they want measures of learning to monitor progress towards Sustainable Development Goal 4 (SDG 4). We must now support countries to produce the data, providing the best technical solutions available. We must also invest in robust, nationally owned and designed learning assessment systems – linked to sector planning – in order to ensure quality data is used to improve learning.

The Global Alliance to Monitor Learning (GAML), which brings together partners, assessment initiatives, civil society groups, donors and Member States, has been established to coordinate these efforts.  GPE is a proud partner in the GAML alliance, and will contribute to its objectives through its  Assessment for Learning (A4L) initiative.

Here is a summary of the discussion and progress to date on GAML and A4L.

Countries show how we can align existing assessments

Countries have made it clear that they want to continue using their own assessments, methodologies and standards to measure learning:  ones that are relevant to their own unique context and nationally owned. But countries want tools to help them ‘fine-tune’ or align some of the technical components of their assessments. GAML, the UIS and the Australian Council for Education (ACER) are developing a set of learning scales that will help countries use their own data and systems for the global metrics.

The A4L initiative will support a stronger systems approach to learning assessments, building on existing investments in sector planning and analysis, and strengthening the capacity of regional networks and initiatives.

By building on existing resources, we can also help countries that are currently not assessing learning to develop their own national systems and avoid the costs of either starting from scratch or joining a regional or international initiative. GPE will support GAML’s plans to develop assessment modules that can be adapted to meet the specific priorities, contexts and resources of individual countries while still benefiting from best practices and international standards.

Tools for countries to improve the quality of their assessment data

The best metrics in the world will amount to little if countries cannot produce them. So GAML is also developing a set of diagnostic tools to help countries benchmark and strengthen their assessments and improve the quality of their national data while also providing them with the resources needed to report the data according to international standards.

In particular, the Data Quality Assessment Framework (DQAF) is a key tool. It will include three key components:

• Manual of Good Practices in Learning Assessment: an independent set of guidelines based on international standards and principles.
• Evaluation of Alignment Process: an approach to help countries report the data needed for global measures of learning.
• Assessment of Data Process: a consultative mechanism to determine the extent to which a country’s assessment data align with international standards.

The UIS is also streamlining the Catalogue of Learning Assessments, developed with support from GPE, which maps the different assessments countries use to monitor student achievement.  While the current version offers a wealth of detailed information, we found a way to streamline and transform it into a tool to:

• Provide national data that can be used to monitor progress towards SDG 4;
• Help countries evaluate the robustness of their assessment systems and identify capacity building needs; and
• Help development partners allocate technical and financial resources.

Country engagement, ownership, and a systems approach to ensure quality, usable data Significant resources will be needed to sustain quality assessments at the country level and to produce global measures of learning. But we must see them as investments, not costs, which contribute to improve the quality of education. Herein lies the connection between GAML and A4L.

A4L takes a systems approach that focuses on building capacity for national learning assessment systems to measure and improve learning. It looks at all three pillars of a learning assessment system- formative assessments in the classrooms, national examinations and large-scale assessments. The goal is to link investments in strengthening learning assessments to the national planning process and ensure sustainability through integration with education sector plans.

With support from the GAML and A4L initiatives, countries are taking the lead in strengthening learning assessment systems and ensuring that these systems have the capacity to report on their progress on SDG4 targets.

GPE and UIS are committed to this two track approach:  strengthening national capacity, while building robust global standards and tools that help support international measurement and comparability and strengthened national systems clearly focused on improving learning.