Developing and Evaluating the First Phase of the Grade 12 Mind the Gap Study Guide Series


The aim of the Mind the Gap impact evaluation was to establish broadly if its learner self-instruction materials made a difference to Grade 12 learner performance in the high stakes NSC examination, and consider taking them to scale if they did. It asked: did the materials improve learner performance?, was the improvement had in all 4 subjects?, and, which learners benefited most from the intervention?

Adopting an impact/RCT design and using secondary data sourced from a website with NSC results, evaluators found the materials did improve performance, that performance was improved in 2 of the 4
subjects, and that more able learners benefited most form using the materials. Findings were not statistically significant, and thus need to be treated cautiously especially in discussion to take the materials to scale. In this regard authors should also have noted the usual practice in RCTs to produce statistically significant findings and test for reliability, amongst others, before a discussion of use. As it
stands, the evaluation appears a rigorous application of an impact evaluation which has been carefully implemented, but its claims may overstate what can be done with its findings. As impact evaluation is a recognized approach by government and in the evaluation literature, it has the advantage of being open to debate, unlike approaches which are not. And it should be noted this application of impact evaluation is unusual as it relies on secondary sources of data and not, as in more usual applications, on test development, testing on site, and the like. The assessment score of 3.30 appears about right for the evaluation as it stands, which is likely to have benefited from critical comment from peers external to the DBE.


Evaluation Number:
Report Approval:
Monday, 21 October 2013
Monday, 03 February 2014
Initiated By:
DBE materials developer
Undertaken By:
Dr S Taylor, DBE
Evaluation Period:
September 2012 to October 2013
Evaluation Area:
National Outcome:
National Outcome 1: Improved quality of basic education.
Commissioned By:
Evaluation Type:
Evaluation Subject:
Geographic Scope:

Evaluation Documents

File Name
Developing Evaluating Mind the Gap Technical Report 2013.pdf
Developing and Evaluating the First Phase of the Grade 12 Mind the Gap Study Guide Series.pdf

Similar Evaluations

Evaluation of the effectiveness and efficiency of Western Cape Archives and Records Service publicity and marketing

The purpose of the evaluation was to assess the effectiveness and efficiency of the WCA&RS publicity and marketing processes, inclusive of the National Archives Week, in providing equitable access...
Evaluation Number:
Report Approval:
Friday, 18 May 2012
Friday, 20 May 2016
Initiated By:
Department of Cultural Affairs and Sport

The Impact of the Introduction of Grade R on Learning Outcomes

This Report reports on the research undertaken. It includes a literature review of the available evidence of the impact of early learning on learning outcomes (Chapter 2); a re-estimation of the...
Evaluation Number:
Report Approval:
Monday, 17 June 2013
Friday, 20 May 2016
Initiated By:
Department of Performance Monitoring and Evaluation and Department of Basic Education

Report on the Evaluation of the National School Nutrition Programme (NSNP)

The purpose of the evaluation was that given the importance of nutrition in the promotion of good health and effective learning for learners, the Public Service Commission (PSC) deemed it necessary...
Evaluation Number:
Report Approval:
Friday, 16 September 2016
Friday, 03 November 2017
Initiated By:
Public Service Commission

'Opening locked doors'- Evaluating peer education in schools in the Western Cape Province: A mixed methods longitudinal study

The purpose of the evaluation is to 'assess the impact of a structured, time-limited, curriculum-based, peer-led education programme to high school learners (Grade 8) in schools in the Western Cape...
Evaluation Number:
Report Approval:
Friday, 14 February 2014
Tuesday, 24 May 2016
Initiated By:
The Global Fund, The Western Cape Provincial Government, and the Centre for the Support of Peer Education

Expanding Opportunities for South African Youth through Math & Science: The Impact of the Dinaledi Program

The National Department of Education has been collaborating with the World Bank since 2008 to answer the question: did and can a program like Dinaledi significantly contribute to this agenda? The...
Evaluation Number:
Report Approval:
Wednesday, 01 December 2010
Wednesday, 25 May 2016
Initiated By:
The World Bank and National Department of Education