Developing and Evaluating the First Phase of the Grade 12 Mind the Gap Study Guide Series

Description

The aim of the Mind the Gap impact evaluation was to establish broadly if its learner self-instruction materials made a difference to Grade 12 learner performance in the high stakes NSC examination, and consider taking them to scale if they did. It asked: did the materials improve learner performance?, was the improvement had in all 4 subjects?, and, which learners benefited most from the intervention?

Adopting an impact/RCT design and using secondary data sourced from a website with NSC results, evaluators found the materials did improve performance, that performance was improved in 2 of the 4
subjects, and that more able learners benefited most form using the materials. Findings were not statistically significant, and thus need to be treated cautiously especially in discussion to take the materials to scale. In this regard authors should also have noted the usual practice in RCTs to produce statistically significant findings and test for reliability, amongst others, before a discussion of use. As it
stands, the evaluation appears a rigorous application of an impact evaluation which has been carefully implemented, but its claims may overstate what can be done with its findings. As impact evaluation is a recognized approach by government and in the evaluation literature, it has the advantage of being open to debate, unlike approaches which are not. And it should be noted this application of impact evaluation is unusual as it relies on secondary sources of data and not, as in more usual applications, on test development, testing on site, and the like. The assessment score of 3.30 appears about right for the evaluation as it stands, which is likely to have benefited from critical comment from peers external to the DBE.

Information

Evaluation Number:
420
Score:
3.5
Report Approval:
21 October 2013
Published:
03 February 2014
Initiated By:
DBE materials developer
Undertaken By:
Dr S Taylor, DBE
Evaluation Period:
September 2012 to October 2013
Evaluation Area:
Education
National Outcome:
National Outcome 1: Improved quality of basic education.
Commissioned By:
National
Evaluation Type:
Impact
Evaluation Subject:
Programme
Geographic Scope:
Provincial

Evaluation Documents

File Name
Developing Evaluating Mind the Gap Technical Report 2013.pdf
Developing and Evaluating the First Phase of the Grade 12 Mind the Gap Study Guide Series.pdf

Similar Evaluations

Evaluation of the effectiveness and efficiency of Western Cape Archives and Records Service publicity and marketing

The purpose of the evaluation was to assess the effectiveness and efficiency of the WCA&RS publicity and marketing processes, inclusive of the National Archives Week, in providing equitable access...
Evaluation Number:
415
Report Approval:
18 May 2012
Published:
20 May 2016
Initiated By:
Department of Cultural Affairs and Sport

The Impact of the Introduction of Grade R on Learning Outcomes

This Report reports on the research undertaken. It includes a literature review of the available evidence of the impact of early learning on learning outcomes (Chapter 2); a re-estimation of the...
Evaluation Number:
NEP/12/428
Report Approval:
17 June 2013
Published:
20 May 2016
Initiated By:
Department of Performance Monitoring and Evaluation and Department of Basic Education

Report on the Evaluation of the National School Nutrition Programme (NSNP)

The purpose of the evaluation was that given the importance of nutrition in the promotion of good health and effective learning for learners, the Public Service Commission (PSC) deemed it necessary...
Evaluation Number:
234
Report Approval:
01 March 2008
Published:
01 June 2016
Initiated By:
Public Service Commission

'Opening locked doors'- Evaluating peer education in schools in the Western Cape Province: A mixed methods longitudinal study

The purpose of the evaluation is to 'assess the impact of a structured, time-limited, curriculum-based, peer-led education programme to high school learners (Grade 8) in schools in the Western Cape...
Evaluation Number:
433
Report Approval:
14 February 2014
Published:
24 May 2016
Initiated By:
The Global Fund, The Western Cape Provincial Government, and the Centre for the Support of Peer Education

Expanding Opportunities for South African Youth through Math & Science: The Impact of the Dinaledi Program

The National Department of Education has been collaborating with the World Bank since 2008 to answer the question: did and can a program like Dinaledi significantly contribute to this agenda? The...
Evaluation Number:
402
Report Approval:
01 December 2010
Published:
25 May 2016
Initiated By:
The World Bank and National Department of Education