Strengthening Citizen-Led Assessment Data

As delegates gather for the World Bank South Asia Regional Workshop on Learning Assessment in New Delhi to share knowledge and learning on assessment practices in basic education in the South Asia region, the PAL Network of citizen-led assessment organizations spanning South Asia, Africa and Central America have just released their newly-created network-wide Data Quality Standards Framework.

In the past, citizen-led assessments have been too easily dismissed as lacking the necessary rigour or quality to be compared with other school-based assessments – an issue highlighted during a recent gathering of senior data analysts and leaders from the PAL Network. Speaking at the opening plenary, Dr. Wilima Wadhwa from the ASER Centre, India stressed the importance of PAL Network members establishing systems to ensure robust and reliable data, saying: “When you release data highlighting government failure, the first thing that will be attacked is your methodology. If someone can cast doubt on your methodology or say your data is unreliable or that the sampling hasn’t been done properly and isn’t representative, then people will find a reason not to engage with the findings.”

Catering to the realities of countries in the global South

The fact is that the citizen-led assessment model was designed carefully and deliberately to cater for the realities of countries in the global South. The design reflects a philosophy that differs from that of standard school-based assessments, as citizen-led assessments train volunteers to assess children regardless of their schooling status, using simple tests and tools, sitting one-on-one with the child, in their own homes.

What may appear to be a very simple assessment is actually backed up by highly-sophisticated processes to ensure the reliability of the data generated. These include systematic processes for sampling, partner and volunteer selection, training, monitoring and re-check. In addition, careful data cleaning and other methods are used to validate the data.

As Dr Wadhwa explained: “Documenting and strengthening these processes and sharing best practices is one of the reasons the network exists. As more countries start to conduct citizen-led assessments and the network grows, network members want to learn from each other. It was this shared focus that brought us together a year ago to draft this standards framework. Now we are back here to say – OK, we have this document. How can we support each other to implement it?”

Making the case for citizen-led assessment data to monitor progress in education

For the past three years, the PAL Network has advocated for the inclusion of an early grade indicator in Sustainable Development Goal 4 (SDG 4). For a clear picture of progress, data used to measure this indicator must include all children, regardless of their schooling status. In the wake of the World Education Forum 2015, the PAL Network released a public statement to the Technical Advisory Group on SDG 4, demonstrating the value of citizen-led assessments to track the acquisition of foundational skills for all children. This was followed by an Open Letter to the Inter-Agency and Expert Group (IEAG) in March 2016 with an urgent appeal to retain the draft indicator for the percentage of children in Grades 2/3 who have learned the basics, and a position statement on SDG4 in July 2016.

Since May 2016, network members have been active participants in the Global Alliance to Monitor Learning (GAML), ensuring that citizen-led assessment data are included as a complementary source for reporting against SDG 4 Indicator 4.1.1. Data from citizen-led assessments are also included in the Catalogue of Learning Assessments (CLA) developed by the UNESCO Institute for Statistics (UIS): an important step towards the inclusion of learning data for children in developing countries who are less likely to be found in traditional educational settings.

Ensuring no child is left behind

To ensure that no child is left behind, active efforts are needed to identify the most disadvantaged children. And that, in turn, means  understanding where they are most likely to be found. Across the developing world, the children who are the hardest to reach are often found in the areas that are hardest to reach, and are unlikely to be in school. In places where significant numbers of children are out of school or attending sporadically, citizen-led assessments have significantly improved knowledge of the inequalities that persist in educational access and the acquisition of foundational reading and numeracy skills.

The new PAL Network Data Quality Standards Framework will help member countries improve and ensure technical rigour, with enough flexibility to accommodate the diversity of processes and adaptations to local context that are central to the citizen-led assessment model. The DQSF will be accompanied by implementation and monitoring plans, with member countries supporting each other to meet the minimum required standards, and ensuring that the PAL Network continues to make an important and robust contribution to understanding learning progress for all children.