Focus on our SDG 4 Data Digest Partners: RTI International

The recent edition of the SDG 4 Data Digest illustrates the range of partners working alongside the UNESCO Institute for Statistics (UIS) to help countries produce and use assessment data to strengthen lifelong learning. This blog highlights the work of one of these vital partners: RTI International, which aims to improve the human condition by turning research knowledge into practice. RTI International’s contribution to the SDG 4 Data Digest provided expert analysis on reading and mathematics assessments for children in the early grades of school.

Understanding foundational skills

Researchers at RTI International have had an interest in measurement for many years and in many areas of human knowledge, including and specifically in education. Our mantra is to measure in order to improve. So, while we value accuracy in measurement, it is the contribution that measurement can make to improving the human condition that we value above all. We collaborate with the UNESCO Institute for Statistics (UIS) for one very simple reason: the UIS mandate complements our own understanding of the importance and use of measurement and data. 

In 2006, RTI International developed a key mechanism to monitor foundational skills: USAID’s Early Grade Reading Assessment (EGRA), followed by the Early Grade Mathematics Assessment (EGMA) in 2009. Both assessments are administered individually and orally to students by trained assessors. And both tools help to identify strengths and gaps in teaching and learning in the early grades of primary school, and chart improvements generated by pilot and demonstration projects or national policies. Their purpose is to provide teachers and policymakers with knowledge to inform practice. With this knowledge, policymakers can support teachers in their efforts to improve the learning trajectories of their students. And as of early January 2019, there is now a UNESCO-sponsored Early Grades Writing Assessment (EGWA).

Learning gaps start early

EGRA and EGMA are relatively easy to understand and apply, and are in the public domain; open-access guidelines for EGRA (available in Arabic, English and French) and EGMA provide researchers with step-by-step instructions for adapting the instruments to new languages and contexts. Following these guidelines, EGRA has been carried out in more than 70 countries and in 120 languages, while EGMA has been carried out in around 25 countries to date.

The results confirm a key concern raised in the UIS SDG 4 Data Digest: global gaps in learning start early. In several low-income countries and across the world’s most vulnerable regions, more than 90% of students in Grade 2 are unable to read a single word of a grade-level passage. Other efforts also covered in the Digest, such as citizen-led assessments, show similar findings.

The global equity crisis

These findings helped to shape the design of Sustainable Development Goal 4 (SDG 4), quality education for all, with its shift in focus from access to education to equitable learning achievement. In the SDG era, the exclusion of most children from acquiring basic reading and mathematics, skills that provide the foundation for all other learning, is not only a global learning crisis but a global equity crisis. It contributes to one of the greatest inequities between the global North and South: the gap in learning achievements.

The EGRA and EGMA experiences also show how countries can use tools that are adapted to their languages and context to monitor issues at the foundational levels of schooling. This reinforces the case for greater investment in early grade learning, particularly for low performers, before the costs of inaction start to increase. Identifying shortfalls at a young age is critical for effective policies and for concrete action.

Concrete results in countries like Kenya

In Kenya, for example, the use of EGRA and EGMA informed the implementation of the Primary Math and Reading (PRIMR) programme, supported by USAID. PRIMR was a partnership between the Kenyan Ministry of Education and RTI International to develop and test several approaches to improve early reading and mathematics instruction, including teachers’ guides and training, student textbooks, and continuous coaching support. Using the results from EGRA and EGMA, researchers and policymakers identified the most effective combination of components to inform the design of a national approach, Tusome (‘Let’s Read’ in Kiswahili), which has been expanded to reach all early grade classrooms across Kenya’s 25,000 public, private and alternative schools.

Both of these efforts, one a pilot and the other a national rollout, have had impacts many times greater than those seen in some wealthier countries, and the rate of improvement in children’s reading is two to three times higher than in control schools. Other non-profits such as Pratham, Save the Children and Room to Read have had a similar impact. EGRA and EGMA have also informed improvements in dozens of other countries.

Some challenges remain

• Comparability. International agencies prize ease of comparison, which can conflict with national desires for contextually-appropriate measurement. This tension between global and local will continue to be a source of debate within the sector. However, while EGRA and EGMA are standardised approaches, each country is free to develop their own performance expectations, defining what they mean by minimum proficiency in reading and mathematics. Education should look to the health sector, which has far more flexibility in its global measurement approaches than those in education tend to assume. 
• Concerns about quality assurance. EGRA and EGMA are relatively easy to apply, but this does not reduce the need for rigorous research design, including proper sampling, administration and analysis. EGRA and EGMA are in the public domain and are adaptable. They also rely on the ability of individual assessors to build rapport with students, while adhering to the testing protocol. For these reasons, the capacity of the entity administering the tools has a direct effect on the quality of the results, likely more so than with traditional paper and pencil assessments. But we believe the benefits of an open access approach far outweigh the risks to quality or the costs of keeping such tools under lock and key.
• The need for more collaboration and sharing of tools: International organizations should encourage more assessment providers to share their approaches more widely. Many EGRA and EGMA results reports can be found on USAID’s Development Experience Clearinghouse, though not all reports contain the survey instruments and some survey results have not been released publicly. While USAID’s Early Grade Reading Barometer website includes results from more than 20 locations, there is no single open access repository for EGRA and EGMA instruments.
• The need for benchmark comparisons to determine what is being compared to what. The Global Alliance to Monitor Learning has been working diligently to build consensus on the reporting and measurement of learning for the Sustainable Development Goals (SDGs). So far pragmatism has prevailed over the desire for perfection and there has been more openness to a variety of approaches that one would otherwise expect from a global organization whose mandate would be made easier if everyone just did the same thing. Those of us working to support countries in measuring against these goals can further help the UIS by ensuring results – including data, instruments and methods – are made publicly available to facilitate comparisons and standard setting.

Where do we go from here?

We do not have time for incremental change: at current rates of progress, it will be decades before every country approaches the levels of learning already achieved by OECD countries. Learning achievements may even be deteriorating in many countries, particularly in countries that have made the greatest progress in expanding access to education and now have millions of additional children in their classrooms.

• We need to learn – very quickly – from countries that have moved up from low levels of performance.
• We need to ensure that schools are teaching at the right level for each child, rather than focusing solely on elite students and ambitious exam standards.
• We need to look beyond the school gates, to address the combination of the disadvantages children experience outside school, poor school quality and discrimination that undermine children’s learning.

Only then can we truly say that we are working to achieve the promise of SDG 4: “to ensure inclusive and equitable quality education and promote lifelong learning opportunities for all”.

About the authors

Luis Crouch, PhD, is a Senior Economist in the International Development Group at RTI International. He has worked on many aspects of education policy and analysis, from household surveys up to Cabinet-level evidence-based policy dialogue. In terms of subject matter he focuses on the foundational years of education. Twitter: @lcrouch1952

Amber Gove, PhD, is Director of Research at the same unit at RTI and is focused on metrics and measurement to inform learning improvement. She has collaborated with government education departments in project design and evaluation, research and data analysis, and policy dialogue. Twitter: @AmberGove

Dr. Crouch and Dr. Gove were both part of the team that developed and worked to deploy EGRA and EGMA with support from USAID.