Using Scale-Anchoring to Interpret the Young Lives 2016-17 Achievement Scale

Zoe James
Education
Technical notes
YL-TN50.pdf882.37 KB

An important dimension of the Young Lives school surveys in Ethiopia, India, Peru and Vietnam has been the inclusion of assessments in selected cognitive domains. In the 2016-17 secondary school survey, assessments of mathematics and English were administered at the beginning and end of the school year in Ethiopia, India and Vietnam. 

This technical note presents the results of two exploratory ‘scale-anchoring’ exercises, which link items to achievement levels to produce performance-level descriptors of what students have demonstrated they know and can do. The note uses mathematics assessment data from the 2016-17 school survey in India before extending the analysis to include Ethiopia, India and Vietnam in a cross-country scale.

Who is taught by the ‘most effective’ teachers? Identifying unequal patterns of school effectiveness in India

In the past decade, India has seen a dramatic increase in enrolment at both primary and secondary levels of schooling. Policies introduced as part of the Right to Education Act in 2009 have helped children from all backgrounds to attend school, with the greatest increase in enrolment seen among girls, children from disadvantaged social groups, and those in the poorest households – many of whom are ‘first generation learners’.

Yet alongside this positive trend, studies suggest that the Indian education system is subject to vast inequalities in learning outcomes with some children benefitting from education equal to the best schools in many OECD countries, while others appear to have little chance to learn. With education, and perhaps particularly secondary education, thought to be key to breaking the intergenerational transmission of poverty, this raises real concerns that the already wide inequities found within the school system will continue to deepen over time.

This week we launch two reports which look at some of these issues in more detail. The reports make use of data from two recent Young Lives education studies in India - a large-scale school effectiveness study and a smaller classroom observation sub-study – to explore questions of school and teacher effectiveness, and consider how these relate to underlying patterns of inequality and disadvantage.

Through this analysis, we find that children are ‘sorted’ into more or less ‘effective’ schools according to their background. Using ‘value-added’ [i] analysis of school effectiveness data collected from children in Grade 9 in 2016-17, we find that girls, children from poorer families and those with less educated mothers are more likely to attend schools where less learning takes place. Figure 1 shows this pattern in terms of student socio-economic background, revealing that the poorest children attend schools which, on average, add considerably less value than those schools attended by the least poor children. These differences persist even when we control for variations in student background which may impact on opportunity to learn (this is ‘conditional value-added’, shown in the red bars in Figure 1).

[[{"fid":"9312","view_mode":"default","attributes":{"class":"media-element file-default","data-delta":"1"},"fields":{"format":"default","field_file_image_alt_text[und][0][value]":false,"field_file_image_title_text[und][0][value]":false},"type":"media","field_deltas":{"1":{"format":"default","field_file_image_alt_text[und][0][value]":false,"field_file_image_title_text[und][0][value]":false}},"link_text":null}]]

Figure 1: Mean maths value-added by student wealth quintile

Analysis of data from Young Lives’ 2017-18 Classroom Observation study in India provides further evidence of this trend. Using data on the scores attributed to each teacher using the CLASS observation methodology [ii], we find that teachers categorised as having a low score (i.e. those not found to be demonstrating positive teacher-student interactions) are more likely to teach children from disadvantaged backgrounds, such as those with less educated parents or those from poorer families. Figure 2 shows one example of this, revealing that more than 80 per cent of students in classes taught by teachers who have been given a low CLASS score have mothers with no education, compared to just 40 per cent of those taught by those teachers with higher CLASS scores.

[[{"fid":"9316","view_mode":"default","attributes":{"class":"media-element file-default","data-delta":"4"},"fields":{"format":"default","field_file_image_alt_text[und][0][value]":false,"field_file_image_title_text[und][0][value]":false},"type":"media","field_deltas":{"4":{"format":"default","field_file_image_alt_text[und][0][value]":false,"field_file_image_title_text[und][0][value]":false}},"link_text":null}]]

Figure 2: Education level of students' mothers, by teacher CLASS score category

Although we need to remember that issues of confounding in school type, location, and student background data prevent us from inferring too much from these descriptive patterns, these analyses suggest that India’s unequal education system is leading to a ‘double disadvantage’ for some students, who face disadvantage at home and in terms of school quality. This raises real concerns that, rather than challenging existing inequalities and inequities, the education system is instead likely to lead them to deepen over time. Our findings in these reports indicate the importance of identifying policy solutions to address the issue of ‘sorting’ which leads disadvantaged children to attend less good schools, if all children in India are to be given the opportunity to achieve at least a basic level of learning and these inequalities are to be addressed.

This blog highlights some of the findings from two reports published by Young Lives’ education team this week. These are: Young Lives School Survey, 2016-17: Value-added Analysis in India; and Classroom Observation Sub-study, 2017-18: Evidence from India. The analysis discussed in this blog was undertaken in collaboration with Ana Grijalva and Caine Rolleston. For related research findings and updates from Young Lives, please follow us on Twitter @yloxford

 


[i] Value-added is a measure of student progress over a defined period of time. It is designed to compare ‘like for like’, by looking at student progress compared to other students with a similar starting score. ‘Unconditional value-added’ is estimated using student test scores from the beginning and end of the school year. ‘Conditional value-added’ also includes student background characteristics, to take into account that it may be harder for some students to make progress.
[ii] CLASS is the Classroom Assessment Scoring System. This classroom observation tool was developed by Robert Pianta at the University of Virginia. CLASS has been used in many different countries to measure teacher-student interactions in the classroom; the Young Lives Classroom Observation sub-study is the first time this methodology has been used in the Indian context. For details of the design of the Young Lives’ 2017-18 Classroom Observation sub-study in India, please find our technical note here.

Classroom Observation Sub-Study, 2017-18: Evidence from India

Rhiannon Moore
Education
School effectiveness
Country report

There is considerable evidence for declining levels of learning in India in recent years, despite increased enrolment, declining class size and greater teacher availability, but a lot less is known about the cause of this ‘learning crisis’. In this context, understanding the impact of what effective teachers do in the classroom, and how teachers and students interact with and relate to each other in ways which lead to learning, is of huge importance.

During 2017-18, Young Lives undertook a classroom observation study in Andhra Pradesh and Telangana, India, with the aim of helping to unlock the ‘black box’ of the education production function and explore some of the classroom factors associated with differences in student learning outcomes. Building upon estimates of teacher ‘value-added’ generated from the Young Lives 2016-17 school effectiveness survey, the classroom observation study offers the opportunity to understand more about what is happening in the classroom, and how this is associated with variation in student learning gain.

The classroom observations were conducted using the CLASS-Secondary (Classroom Assessment Scoring System) tool for classroom observation. The comprehensive teacher-level data generated by use of the CLASS-S methodology provide detailed aggregate information of some of the teaching practices which make a difference to student learning. This report details some of the key findings from this study, along with a discussion of some of the implications of these.

Effect of preschool education on cognitive achievement and subjective wellbeing at age 12: evidence from India

Renu Singh
Education
Early education
Children's perspectives

This mixed methods study draws upon Young Lives India longitudinal data to analyse whether children who attended private preschools demonstrate higher cognitive skills and enhanced subjective wellbeing at the age of 12 compared to those who attended government preschools in India. Using linear logistic regression models, the analysis reveals that children who attended private preschools have significantly higher mathematics scores and more positive subjective wellbeing than those in government preschools. The propensity score matching technique further substantiates this finding. Furthermore, entering preschool before the age of 4 is shown to have a significant positive association with both cognitive achievement and subjective wellbeing at the age of 12. Given that the recently enacted National Policy on Early Childhood Care and Education recognises early childhood education as the foundation for all future learning and as a sorely neglected area, early childhood education must be prioritised by policymakers across public and private sector.

This research is published in Compare: A Journal of Comparative and International Education (2018). Citation: Renu Singh & Protap Mukherjee (2018): Effect of preschool education on cognitive achievement and subjective wellbeing at age 12: evidence from India, Compare: A Journal of Comparative and International Education

 

Commitment to early education in India

Last month, Galli Galli Sim Sim, the Hindi language adaptation of Sesame Street (known as Sesame Street Workshop in India, of which I am a board member) invited me to share my aspirations for early education reform. They asked what I would like to see prioritised, in response to which I identified an urgency to ‘make early education a justiciable right’.

It is time we recognised the immense importance of early years education as the foundation for all future learning. In India, change must be made in line with recommendations from the 2015 Law Commission Report, which states that the Right to Education Act should be made mandatory ‘with a view to prepare children above the age of three years for elementary education and to provide early childhood care and education for all children until they complete the age of six years, the appropriate Government shall make necessary arrangement for providing free pre-school education for such children.’

But there are barriers to meeting this ask. While the State clearly outlines an ‘endeavour to provide early childhood care and education for all children until they complete the age of six years’ (Article 22), the Right of Children to Free and Compulsory Education Act (RTE), which came into effect in April 2010 failed to make education for children between three and six years of age a legal right.

I share this call for change in line with my role as Country Director of the Young Lives Study in India. Young Lives is a longitudinal study of childhood poverty that has conducted five survey rounds in India since 2002, gathering important evidence to add to the body of knowledge demonstrating that early years investment has long-term benefits, particularly for the most disadvantaged children. 

Analysis of our data shows that pre-school variables (type of school, age of entry, and caregivers’ perception of schooling) have strong associations with the cognitive outcomes and subjective well-being of children even in middle school or at the age of 12. A recent paper highlighted that children who attended private pre-schools are more than twice as likely to finish secondary school than those children who did not attend pre-school.

When comparing the four study countries of Young Lives (Ethiopia, Peru, Vietnam, and Andhra Pradesh and Telangana, India), we see that children across both cohorts (an Older Cohort born in  1994 and a Younger Cohort born in 2001) who engaged in pre-school education performed better in numeracy tests at all ages regardless of the type of pre-school attended. Those who had attended pre-school also had higher levels of pride at eight years of age than those who did not. This is further corroborated by this year’s World Development Report which highlights that children who attend pre-school have higher attendance and better achievement in primary school.

The National Early Childhood Care and Education Policy recognises early childhood care and education as the foundation for all future learning and as a sorely neglected area. Given the importance of standardising quality across institutions catering to children between the ages of three and six years, it is time that the national policy was effectively implemented through a decentralised mechanism to ensure high standards across both the private and public sector.

The Global Education Monitoring Report stated that despite a global focus on early education thanks, in part, to Sustainable Development Goal target 4.2, only 69% of children globally participated in organised learning at the pre-primary level, ranging from 95% in Latin America, the Caribbean, Europe and North America, to 42% in sub-Saharan Africa.

Given that a large number of children in India are growing up in disadvantaged contexts, which may inhibit them from developing to their full potential, early childhood education should be viewed as a key intervention to compensate for environmental deficits, by supporting and strengthening child development. The importance of early years education must be recognised in light of its long-term benefits, in line with recommendations made this January by the Central Advisory Board of Education Committee to extend the Right to Free and Compulsory Education (RTE, 2009) to the under-six population.

To find out more about Young Lives’ education research, please follow @yloxford on Twitter with #YLEducation. You may also be interested in our involvement this week in the 62nd Annual Meeting of the Comparative and International Education Society titled ‘Re-Mapping Global Education South-North Dialogue’. More details are available here.

 

Commitment to early education in India

Last month, Galli Galli Sim Sim, the Hindi language adaptation of Sesame Street (known as Sesame Street Workshop in India, of which I am a board member) invited me to share my aspirations for early education reform. They asked what I would like to see prioritised, in response to which I identified an urgency to ‘make early education a justiciable right’.

It is time we recognised the immense importance of early years education as the foundation for all future learning. In India, change must be made in line with recommendations from the 2015 Law Commission Report, which states that the Right to Education Act should be made mandatory ‘with a view to prepare children above the age of three years for elementary education and to provide early childhood care and education for all children until they complete the age of six years, the appropriate Government shall make necessary arrangement for providing free pre-school education for such children.’

But there are barriers to meeting this ask. While the State clearly outlines an ‘endeavour to provide early childhood care and education for all children until they complete the age of six years’ (Article 22), the Right of Children to Free and Compulsory Education Act (RTE), which came into effect from April 2010 failed to make education for children between three and six years of age a legal right.

I share this call for change in line with my role as Country Director of the Young Lives Study in India. Young Lives is a longitudinal study of childhood poverty that has conducted five survey rounds in India since 2002, gathering important evidence to add to the body of knowledge demonstrating that early years investment has long-term benefits, particularly for the most disadvantaged children. 

Analysis of our data shows that pre-school variables (type of school, age of entry, and caregivers’ perception of schooling) have strong associations with the cognitive outcomes and subjective well-being of children at the age of 12. A recent paper highlighted that children who attended private pre-schools are more than twice as likely to finish secondary school than those children who did not attend pre-school.

When comparing the four study countries of Young Lives (Ethiopia, Peru, Vietnam, and Andhra Pradesh and Telangana, India), we see that children across both cohorts (an Older Cohort born in  1994 and an Older Cohort born in 2001) who engaged in pre-school education performed better in numeracy tests at all ages regardless of the type of pre-school attended. Those who had attended pre-school also had higher levels of pride at eight years of age than those who did not. This is further corroborated by this year’s World Development Report which highlights that children who attend pre-school have higher attendance and better achievement in primary school.

The National Early Childhood Care and Education Policy recognises early childhood care and education as the foundation for all future learning and as a sorely neglected area. Given the importance of standardising quality across institutions catering to children between the ages of three and six years, it is time that a national policy was effectively implemented through a decentralised mechanism to ensure high standards across both the private and public sector.

The Global Education Monitoring Report stated that despite a global focus on early education thanks, in part, to Sustainable Development Goal target 4.2, only 69% of children globally participated in organised learning at the pre-primary level, ranging from 95% in Latin America and the Caribbean and in Europe and Northern America, to 42% in sub-Saharan Africa.

Given that a large number of children in India are growing up in disadvantaged contexts, which may inhibit them from developing to their full potential, early childhood education should be viewed as a key intervention to compensate for environmental deficits, so supporting and strengthening child development. The importance of early years education must be recognised in light of its long-term benefits, in line with recommendations made this January by the Central Advisory Board of Education Committee to extend the Right to Free and Compulsory Education (RTE, 2009) to the under-six population.

To find out more about Young Lives’ education research, please follow @yloxford on Twitter with #YLEducation. You may also be interested in our involvement this week in the 62nd Annual Meeting of the Comparative and International Education Society titled ‘Re-Mapping Global Education South-North Dialogue’. More details are available here.

 

Estimating causal effects of preschool on school dropout using non-experimental data from Andhra Pradesh

This Masters thesis uses data from Young Lives. The author's abstract reads:

There were 112 million children out of school all over the world in 2012. Aware of the benefits of education on future outcomes, it is of utmost importance to reduce the number of children out of school in order to reduce inequalities. While remedial education, conditional cash transfers and demand- and supply-side inputs have been the central point of educational policies in developing countries, preschooling has been barely studied. Using propensity score matching on non-experimental data from the Young Lives project, I estimate the effect of preschooling in the state of Andhra Pradesh, India. The richness and features of the data allow me to account for potential confounders. The estimation suggests that preschool affects dropouts significantly, reducing them by a 7%. Therefore, school attainment can be improved by developing a strong preschool system and providing quality preschool years to children.

Diverging Pathways: When and Why Children Discontinue Education in India

Renu Singh
Education
Working paper

Download working paper here

the current commitment to the Sustainable Development Goals and the push to achieve universal secondary education by 2030, it is important to investigate at what grade or age level boys and girls are discontinuing education, as well as the key reasons for this. 

This working paper examines dropping out of school from a life-course perspective, utilising an ecological model to examine factors affecting school continuity by drawing upon Young Lives longitudinal data in Andhra Pradesh and Telangana, India. Using mixed methods, the reasons cited by children are grouped into three broad categories: (i) pulled out (including to undertake paid jobs and family responsibilities); (ii) pushed out (institution and system-related factors such as distance to school); and (iii) opting out (disengagement with school or institution not caused by the school or institution, or outside pull factors). Listening to the voices of children, the paper analyses push, pull and opt-out factors at both the individual and community level to investigate when and why children discontinue education, and correlates of dropping out, including the role of the community. 

Pull factors account for more than 60 per cent of the reasons given by children who had dropped out of school by the time they were 19 years old, while prolonged absence from school/truancy was the second most cited reason for discontinuation of education. Significant factors such as caste, maternal education, preschool attendance, and opted-out factors emerged as explanatory variables for those discontinuing education before upper-primary education as well as before secondary. However, only caste and preschool attendance were significant factors when comparing children who dropped out before and after higher secondary. 

These findings provide a clear direction to formulate policies and interventions at specific ages. An interesting finding from the multinomial multilevel regression highlights community effects that, after controlling for individual factors, explain around 11 per cent of the variability in dropping out. The fact that distance to public high school is a significant predictor of leaving school, especially at the secondary level, with children being 2.7 times more likely to drop out in communities where schools are further than 5 km away, is a key point to be considered by policymakers.

Click here to download the working paper.

Diverging Pathways: When and Why Children Discontinue Education in India

Renu Singh
Education
Working paper

Given the current commitment to the Sustainable Development Goals and the push to achieve universal secondary education by 2030, it is important to investigate at what grade or age level boys and girls are discontinuing education, as well as the key reasons for this. 


This working paper examines dropping out of school from a life-course perspective, utilising an ecological model to examine factors affecting school continuity by drawing upon Young Lives longitudinal data in Andhra Pradesh and Telangana, India. Using mixed methods, the reasons cited by children are grouped into three broad categories: (i) pulled out (including to undertake paid jobs and family responsibilities); (ii) pushed out (institution and system-related factors such as distance to school); and (iii) opting out (disengagement with school or institution not caused by the school or institution, or outside pull factors). Listening to the voices of children, the paper analyses push, pull and opt-out factors at both the individual and community level to investigate when and why children discontinue education, and correlates of dropping out, including the role of the community. 


Pull factors account for more than 60 per cent of the reasons given by children who had dropped out of school by the time they were 19 years old, while prolonged absence from school/truancy was the second most cited reason for discontinuation of education. Significant factors such as caste, maternal education, preschool attendance, and opted-out factors emerged as explanatory variables for those discontinuing education before upper-primary education as well as before secondary. However, only caste and preschool attendance were significant factors when comparing children who dropped out before and after higher secondary. 


These findings provide a clear direction to formulate policies and interventions at specific ages. An interesting finding from the multinomial multilevel regression highlights community effects that, after controlling for individual factors, explain around 11 per cent of the variability in dropping out. The fact that distance to public high school is a significant predictor of leaving school, especially at the secondary level, with children being 2.7 times more likely to drop out in communities where schools are further than 5 km away, is a key point to be considered by policymakers.
 

 

Education and Learning in Telangana - Preliminary Findings from the 2016 Young Lives Survey (Round 5)

P. Prudhvikar Reddy
Young Lives
Education
Data

This factsheet presents findings from the fifth round of the Young Lives survey of children in 2016 for Telangana state. Young Lives has followed two cohorts of children, born seven years apart in age. This factsheet gives a snapshot of key education indicators for 15-year-olds in 2016 (Younger Cohort) and compares that to the data for 15-year-olds in 2009 (Older Cohort) to show changes in the context of children's education over that 7-year time period. As the key findings show, great progress has been made on expanded access to secondary schooling, but learning levels often remain low. Policies to improve equity should continue to address the position of the poorest children, girls and those from marginalised social groups.