SDG Indicator 4.1.1: Percentage of children and young people achieving at least a minimum proficiency level (MPL) in (i) reading and (ii) mathematics during primary education (Grade 2 or 3), at the end of primary education, and at the end of lower secondary education.
The sources of data are:
i. Cross national assessments including:
- International assessments data (TIMSS, PISA, PIRLS)
- Regional assessments data (PILNA, SEA-PLM, PASEC, SACMEQ, ERCE)
ii. National assessments: Data are collected through the Catalogue of Learning Assessments (CLA) and/or available in national reports
iii. Assessments for Minimum Proficiency Levels (AMPL): Data are collected through modules that can be administered as a standalone assessment or be integrated into national assessments as a whole booklet or as a rotating booklet.
When the results are not nationally representative, a footnote should be added to the data point.
UNESCO Institute for Statistics
http://tcg.uis.unesco.org/wp-content/uploads/sites/4/2020/09/Metadata-4....
The number of children and/or young people at the relevant stage of education n in year t achieving or exceeding the pre-defined proficiency level in subject s expressed as a percentage of the number of children and/or young people at stage of education n, in year t, in any proficiency level in subject s.
The higher the value of the indicator, the higher the proportion of children or young adults who have acquired the minimum level of meaningful competencies.
Learning outcomes from cross-national learning assessments are directly comparable for all countries which participated in the same cross-national learning assessment. However, these outcomes are not comparable across different cross-national learning assessments or with national learning assessments. A level of comparability of learning outcomes across assessments could be achieved by using different methodologies, each with varying standard errors. The period of 2020-2021 will shed light on the standard errors’ size for these methodologies.
The comparability of learning outcomes over time has additional complications, which require, ideally, to design and implement a set of comparable items as anchors in advance. Methodological developments are underway to address comparability of assessments outcomes over time.
For more information, please refer to the paper ‘Reporting learning outcomes in basic education: country’s options for indicator 4.1.1’.
For additional information, see tables from Annex I
https://tcg.uis.unesco.org/wp-content/uploads/sites/4/2020/09/Metadata-4...
The indicator aims to measure the percentage of children and young people who have achieved the minimum learning outcomes in reading and mathematics during or at the end of the relevant stages of education.
Indicator is published disaggregated by sex.
Other disaggregation such as location, socio-economic status, immigrant status, ethnicity and language of the test at home are based on data produced by international organizations administering cross learning assessment. Parity indexes are estimated in the reporting of Indicator 4.5.1. Information on the disaggregation of variable for Indicator 4.1.1 are presented in the tables in Annex I.
For the first time, the UNESCO Institute for Statistics is publishing regional averages for all learning outcomes indicators for all levels and domains from 2000 to 2022 in the September 2022 data release.
As indicated in the metadata of SDG indicator 4.1.2, the completion rate can be used in combination with SDG indicator 4.1.1 to provide information on the percentage of children or young people in a cohort who achieve a minimum level of proficiency (MPL), and not only on the percentage of children in school who achieve minimum proficiency. Therefore, to reflect the percentage of all children and/or young people who have achieved the minimum level of proficiency and comply with the commitment to leave no one behind, Indicator 4.1.1 can also be disaggregated by the status of completion. However, the information on the percentage of children and/or young people who have reached minimum proficiency does not tend to be available, even though they have left school before reaching the end of primary and lower secondary education, respectively.
Considering that the emphasis of Target 4.1 is to ensure that all boys and girls ‘complete … education leading to relevant and effective learning outcomes,’ it can be assumed that no children and/or young people who have left school before completing primary or lower secondary education have reached the minimum proficiency level expected at that level of education. As a result, the disaggregation by completion status takes the following form (please refer to https://tcg.uis.unesco.org/wp-content/uploads/sites/4/2020/09/Metadata-4...):
Indicator 4.1.1 disaggregated by completion t, n, s =
Indicator 4.1.2 t,n x Indicator 4.1.1 t,n,s
where:
Indicator 4.1.2 t,n = percentage of a cohort of children or young people aged 3-5 years above the intended age for the last grade of each level of education n who have completed that grade, in year t, and achieved or exceeded the minimum proficiency level in subject s.
Indicator 4.1.1 t,n,s = proportion of children and young people at stage of education n, in year t, achieving at least a minimum proficiency level in subject s.
n = the stage of education that was assessed
s = the subject that was assessed (reading or mathematics).
Methodological challenges
The indicator faces the following methodological challenges:
i. Define a minimum proficiency level (MPL)
ii. Harmonize various data sources, including non-official data sources
iii. Define how to include non-completers to assess their level of proficiency
i. Definition of the Minimum Proficiency Levels
A minimum proficiency level (MPL) is the benchmark of basic knowledge in a domain (mathematics, reading, etc.) measured through learning assessments. The minimum proficiency level is measured through the definition agreed in 2018 and was refined in 2020.
To ensure comparability across learning assessments, a verbal definition of MPL for each domain and levels between cross-national assessments (CNAs) was established by conducting an analysis of the performance level descriptors (PLDs) of cross-national, regional, and community-led tests in reading and mathematics. The analysis was led and completed by the UIS and a consensus among experts on the proposed methodology was deemed adequate and pragmatic.
The global MPL definitions for the domains of reading and mathematics are presented in the document ‘Minimum Proficiency Level used to report for indicator 4.1.1’ along with assessment names and the assessment PLDs aligned to the SDG MPL descriptor.
ii. Harmonization of data sources
To address the challenges posed by the limited capacity of some countries to implement cross-national, regional, and national assessments, actions have been taken by the UIS and its partners. The UIS has proposed some options to link assessments together; one of these strategies is the Rosetta Stone, a subject-based psychometric linking approach (new data collection). The second one is the Policy linking approach, which consists on setting benchmarks, or cut scores, on learning assignments to align them with other assessments across countries or contexts (alignment with existing data). While it is an old standard-setting methodology, the UIS and its partners have now extended its use to help countries set benchmarks using the Global Proficiency Framework (GPF) for reading and mathematics, a framework developed by multilateral donors and partners based on current national content and assessment frameworks across more than 100 countries.
An ideal program for reporting on SDG 4.1.1 will have gone through three steps: Conceptual Framework, Methodological Framework, and a Reporting Framework. Each of these contains several complex sub-steps. For various levels and types of assessment, UIS had completed most of this work before accepting the responsibility of being custodian of reporting on SDG 4.1.1. The Protocol for reporting on SDG global indicator 4.1.1 explains each of the activities and outputs and helps to build the tools to generate a minimum level of consistency of education systems’ reporting against Indicator 4.1.1, while retaining sufficient flexibility for education systems to pursue assessment programs appropriate to their context and needs.
iii. Completion status
Combining completion rates with learning outcomes improves our understanding of progress towards Target 4.1. Almost all information regarding learning is school-based and does not consider the completion of the level. The inclusion of completion in the global list offers an opportunity to report according to the completion status. The greatest differences between the SDG 4.1.1 on learning before completion and the disaggregation by completion are found in regions or countries with lower completion and enrolment rates (or children completing and learning) because the adjusted indicator is based on a quality-adjusted completion rate. This also explains why the largest differences occur at the lower secondary level. Globally, 47% of lower secondary students achieve minimum proficiency in reading according to the original SDG 4.1.1 Indicator, but the value for the adjusted indicator would fall to 34% of adolescents completing lower secondary and achieving minimum proficiency in mathematics. See references here.
Protocol for reporting Indicator 4.1.1
In reporting on Indicator 4.1.1, questions may arise in relation to:
- Which content should be measured and what is the percentage of coverage to be covered by a given assessment to be comparable to other assessments?
- What procedures are good enough to ensure quality of the data collected?
- A proficiency scale where all assessments could be informed (and its conversion function or the linking procedure), and a definition of the minimum level for each domain that would allow the estimation of the percentage of students achieving the minimum proficiency level. The Protocol for reporting on SDG global indicator 4.1.1 intends to provide answers to these questions.