Aligning Key Initiatives in Monitoring Learning

In a recent blog, we noted that there is currently a global multiplicity of strong initiatives in generating better data on learning outcomes, oriented at Sustainable Development Goal 4 (SDG 4). And, that’s only in the area of learning outcomes. If one adds other key aspects of SDG 4 and the global and thematic indicators, the number of initiatives is almost too large to catalogue. Even within the limited area of learning outcomes, some of the elements in the various initiatives create synergies, but others might create duplication or at least different proposed ways of doing things, if there is not a forum to clear lessons-learned and the results of different approaches. In our recent blog, we promised to present a systematic analysis that, using a matrix format, compares elements of the various initiatives. This, we hope, would be of use to the institutions involved and the funders asked to underwrite the various proposals.

In this blog we review and compare four initiatives, the Global Alliance to Monitor Learning (GAML), the Assessment for Learning (A4L) initiative, the learning assessment recommendations of the Education Commission (chaired by Gordon Brown, the UN Special Envoy for Global Education). There are also interesting discussion documents from the background work done by the Center for Global Development for the Commission.

The following simplified matrix lays out some of the key actions or tasks that can be deemed important for producing robust, comparable indicators on learning, with specific reference to not just tracking, but moving towards actually achieving SDG4. It is important to note that the content of the matrix, for each initiative, is not a set of policy recommendations from the authors of this blog, but simply a listing of the ideas found in key initiatives. The matrix makes it clear where there is agreement but perhaps also some duplication, and where various initiatives have a unique contribution to make.

Our recommendations could be summarized as follows: a) make it clear what the areas of specialization of the various initiatives are; and, b) where the various initiatives coincide on recommending a certain point, make sure there is clarity as to the approach, or at least a mechanism for reaching clarity.

Specifically, it seems to us that the Commission takes a fundraising approach and proposes that it could raise the funding needed to support work towards measuring SDG 4 in general, and learning outcomes specifically. The Commission coincides with many of the policy recommendations of the other initiatives and cross-references GAML. It seems, also, that A4L has some detailed discussion for how one could make grants to countries and/or regional bodies, while also backing key ideas such as common metrics. It, or the approaches recommended, contain a discussion of financing mechanisms and grants that could be used to disburse funds gathered under the aegis of what the Commission recommends. And, finally, it seems to us that GAML has the most complete proposals for coordinating the overall effort and lining up the technical work needed to underpin the measurement the “brokerage,” convening, and commissioning of technical work on assessment, for instance. In that sense, a fairly clear division of labor could emerge. But making sure that the division of labor emerges will take some conscious work.

One way to delineate amongst these various initiatives would be for UIS to commission a one-off team of experts to validate the various approaches and suggest ways to specialize and develop each initiative’s comparative advantage, so as to maximize synergies and avoid duplication and confusion.

As a last point: sometimes global initiatives take a long time to jell, especially if coordination among various initiatives is necessary. But there are some urgent tasks to be done. We would suggest advancing on those urgent tasks under the aegis of UIS, until bigger issues can be resolved. This would require funding so as to put a “down payment” on the various tools that have to emerge.

Define comparable learning metrics or scales, foster their usage
Without some form of common scale or metrics, it is difficult to judge progress objectively. However, there are many possible approaches to comparability, requiring different degrees of consensus. On this score, the various initiatives agree on the importance of the issue, but only GAML (and associated documents) makes a case for a particular mechanism to resolve it. The language in some initiatives suggests some possible duplication.

International Commission on Financing of Global Education Opportunity	Assessment for Learning (A4L)	Global Alliance to Monitor Learning (GAML)
Included in recommendations. No particular institutional mechanism recommended for resolving the issue of comparability. Thus, the idea is backed, but it is not clear how to achieve the purpose. The Commission recommends age-based measurement, which is not what the SDG language calls for. This would have to be sorted out. Calls for a single indicator, which could be somewhat contentious, depending on one’s understanding of such a single indicator.	A mechanism to disburse grants could cover this area, though it is not a central aspect of the proposal. Proposes that A4L could be a coordinating and convening mechanism.	Central to GAML’s purpose. There is emerging clarity as to the institutional and technical mechanisms to achieve the purpose.

 

Set quality standards for assessment data. Standards for assessment systems capacity (like the World Bank’s SABER)
Even if the goal were to simply report on learning to the global community, strong national systems are key. But having strong national systems that can assist in actually fostering learning, rather than just reporting on it, is itself a strong priority. Strong systems for assuring the quality of data, and the quality of data-producing systems. Guidance on these issues is a global public good with large economies of scale

International Commission on Financing of Global Education Opportunity	Assessment for Learning (A4L)	Global Alliance to Monitor Learning (GAML)
Covered in Recommendation 2 on learning assessments; cross-references GAML.	The A4L initiative would explicitly finance system diagnostics and cross-references SABER approach.	Central to GAML. Proposed as part of the Data Quality Assurance Framework (DQAF). Extensive and detailed discussion documents exist. Data quality assurance frameworks could cover other data systems.

 

Capacity building of national systems
This would be key, even under the minimalist assumption that indicators are only for reporting. But such minimalist assumptions are not the case. Indicators are to be used not just for tracking, but for policymaking to improve national performance and the SDGs.

International Commission on Financing of Global Education Opportunity	Assessment for Learning (A4L)	Global Alliance to Monitor Learning (GAML)
Implicitly covered in call for a global data initiative.	The aspect of “for improving national performance” is a central aspect of the grants mechanism, given its name and content: Assessment for Learning. The activity would also finance diagnostics.	Discussed in considerable detail. Emerging role would be for GAML to provide coordination, brokerage of funding, direct funding, set standards. No mechanism for direct implementation discussed. UIS is generating tools to assess needs for capacity building from countries through the Catalogue of Learning Outcomes and the Data Quality Assessment Framework for Learning Assessments. Generating a data platform where information about countries needs is centralized and can bring in many more actors as supplies. GAML does not delve deeply into the translation of assessment into classroom practice or policy.

 

Actual development of technical tools, most likely contracted out or in partnership with assessment industry
Linking items or a new actual global assessment, data-analytical packages, etc. Large economies of scale and global public good aspects. 

International Commission on Financing of Global Education Opportunity	Assessment for Learning (A4L)	Global Alliance to Monitor Learning (GAML)
Not covered, but implicitly calls on GAML.	Discussed as a possibility for funding.	Discussed in terms of setting overall direction, coordination.

 

Allocate funding to countries for improving systems or to participate in regional/international assessments, indirect support to regional and international assessments
Some poorer countries may require an incentive to participate in regional/international assessments or to participate in linking their national assessments to a global metric or scale.

International Commission on Financing of Global Education Opportunity	Assessment for Learning (A4L)	Global Alliance to Monitor Learning (GAML)
Called for in education data initiative.	Covered, important part of proposal, good ideas for implementation of such grants.	No proposed mechanism or budget specified, but principles of allocation and funding are described.

 

Direct support to regional assessment programmes and existing international assessments
Regional assessment programmes receive widespread support from countries and are therefore a logical way to think about globally-linkable assessment. Supporting such regional programmes could be made conditional on their being linkable to a global scale. No initiative discusses direct support to international assessments.

International Commission on Financing of Global Education Opportunity	Assessment for Learning (A4L)	Global Alliance to Monitor Learning (GAML)
Not discussed explicitly.	Important activity in Asia-Pacific and Africa via NEQMAP and NALA. (See A4L document.)	Not discussed explicitly.

 

Direct (grant) support to development of national learning assessment systems
May be needed in poorer countries to stimulate progress towards assessment systems. Could be made partially results-based. 

International Commission on Financing of Global Education Opportunity	Assessment for Learning (A4L)	Global Alliance to Monitor Learning (GAML)
Calls for support to national assessment systems and connotes willingness to support, does not discuss mechanisms.	Central aspect of original A4L concept note. Discussed in considerable detail, including potential mechanisms, several different types of grants (planning, implementation). Somewhat reduced in importance in current plans for initial period.	Funding of these aspects is discussed. Principles are outlined. But, as yet, no budget or implementation mechanism is discussed.

 

Coordination of the above and other comments
Clearly some degree of brokering and coordination of the above is needed, as these tasks are all somewhat dependent on each other.

International Commission on Financing of Global Education Opportunity	Assessment for Learning (A4L)	Global Alliance to Monitor Learning (GAML)
Calls on UIS and its Technical Cooperation Group.	Makes various proposals that see A4L as providing a coordinating role.	UIS and various committees would govern and coordinate.