Skip Links
 

document downloads button image

programme evaluation brief button image

Evaluating impact

AET Project Brief –Evaluating the impact of the AET Programme

Evaluating the impact of the AET Programme - NOW CLOSED.


The Autism Education Trust was awarded contracts by the Department for Education (DfE) for the development and delivery of an integrated programme of training and materials over the period 2011/15. 

An extension to the current contract until April 2016 will build on the success of the AET’s two previous programmes. This brief sets out an evaluation strand in the 2015/16 programme focussing on the impact of AET programmes at a school and pupil level.  


Purpose

To conduct interviews in 4 schools that have in engaged with the AET programme to assess the impact on their practice. Additionally  the robustness and of the newly developed AET Progression Framework will be will be trialled in respect of 20 pupils to track the effectiveness of links between improved provision and pupil outcomes. As well as evaluating the impact of the programme the data gathered could be used as exemplars of the application of the new progression framework.


Background

The Autism Education Trust

The Autism Education Trust (AET) was founded in 2007 by Ambitious about Autism (formally Tree House), The Council for Disabled Children (CDC) and The National Autistic Society (NAS) and has received consistent support from both governments since that time.

The  vision of the AET is that all children and young people with autism should receive an education which enables them to engage in society as active citizens and that their families and the professionals who work with them are informed, supported and equipped to enable this to be achieved.

In order to achieve this, the AET has a mission to improve the education of young people with autism.

The AET achieves its mission through:

• the identification and promotion of high standards in autism education practice
• the identification and promotion of effective practitioner competency in support of those standards
• providing a platform for training in effective practice in support of improved competency
• disseminating information regarding evidence-based and good practice approaches/services to policy makers,  commissioners and practitioners
• providing  an interface between users, practitioners and policy makers in respect of service development
• ensuring that full and proper engagement of stakeholders informs its work and facilitating the engagement  of stakeholders with government

The AET is in a unique position to carry out these activities because of:

• Its independence
• The partnership it supports between voluntary bodies and voluntary bodies and the maintained sector
• The active role of people with autism in the work and governance of the AET.
• Its national brief
• Its  focus on practical activity in education contexts


The 2011-13 and 2013-15 AET programmes

In the summer of 2011 the DfE awarded the AET funding to deliver an ambitious 2 year programme to raise workforce competency through:

• Delivering autism training for school staff  at level 1 (general awareness), level 2 (for staff working with young people with autism on a daily basis) and level 3 (for specialists such as SENCOs)
• Integrating this training with national standards for schools for autism education.
• Developing a competency framework to guide professional development

Birmingham University were commissioned to develop the training materials and the standards and 7 regional hubs were commissioned to deliver the training. The programme was rolled out successfully. By the end of the programme in March 2013: 
• Over 10,500 school staff had been trained at level 1, over 850 at level 2 and nearly 200 at level 3.
• The AET Standards were published in May 2012 and there was strong interest and positive feedback from schools from the outset. There is now evidence of the standards being used in schools and strategically in some LAs on a whole LA basis.
• An AET Competency Framework was published in September 2012 which provides a structure for professional development. We are now receiving regular feedback from schools that this supporting staff CPD in this area of expertise.

A final evaluation (2013) from University of Warwick CEDAR unit found:

For level 1
• Very positive feedback  with robust evidence of staff moving on to level 2 as a result if undertaking level 1 training.
• Level 1 AET training materials have been successful in raising the knowledge, awareness and understanding of the majority of the participants, including those with previous experience of working with pupils on the autism spectrum.
• There was a highly significant rise in pre- and post-training mean total score on the Knowledge Quiz (p < .001).
• Over 8 out of 10 thought the training was ‘worthwhile’.

For level 2 and 3

• Very positive feedback with over 84% indicating the training was worthwhile and had increased their knowledge
• 93% of level 2 delegates and 97% of level 3 delegates  were interested in further training

The DfE awarded the AET a contract for a new 2013-15 programme that built on this success. It extended it by:
• Developing new Early Years  and Post 16 Training materials and associated Autism Standards and Competency Framework
• Commissioning 4 Early Years and 4 Post 16 Training hubs to deliver the programme
• Developing the AET Standards for use by parents in selecting schools and having a constructive dialogue with staff ion their child’s school.
• Developing a toolkit for LAs and other organisations who are using the AET training, Standards and Competency Frameworks

The diagram below sets out the broad structure of the 2015/16 AET offer. It brings together work produced in the 2011/13 and 2013/15 programmes and links these with the new developments in the current 2015/16 programme.  

 


 

Evaluating the impact of the AET programme at a school and pupil level

Evaluation has been integral to the last two programmes and has played a key part in both guiding development and providing accountability. The bulk of evaluation and QA processes for the existing programme have now been transferred AET as part of its ongoing operations. This has included delegate feedback, structured observation of training sessions and analysis of portfolios from delegates submitted for accreditation.

As a result of these functions being taken over in-house by the AET this independent evaluation strand is significantly smaller than in previous years. However it is still of critical importance to the programme.

In response to feedback from Ofsted the brief will include a focus on school change and pupil outcomes arising from the embedding of previous programmes. This strand would link closely with the development of progress measures for pupils with autism above and coordinated work with the team leading that work will result in significant economies of scale and improvements in effectiveness. The report will focus on 5 schools who have applied the training and track developments from that training.

Additionally the newly developed AET progression framework will be trialled in respect of 20 pupils to track links between improved provision and pupil outcomes. As well as evaluating the impact of the programme the data gathered could be used as exemplars of the application of the new progression framework.



Integration with other project strands in the new 2015/16 programme

This strand of the AET 2015/16 programme links with other strands of the programme.  In particular work will need to be coordinated with the following strand of the 2015/16 programme:

Research and development to produce a progression framework and associated training module measuring the progress of children and young people with autism

The need to for research that can be applied by teachers to track the progress of children and young people with SEND has been identified as a research priority by DfE in its latest paper in March 2014. This strand of the AET 2015/16 programme is an important response to the sharper focus on outcomes set down in the new SEND Code of Practice. This priority has also been identified consistently by AET training hubs.

The first part of the progression framework project is a review of the literature which has a practical application to the work of education staff tracking the progress of children and young people with autism and using this information to guide their educational offer. The second part is to produce a framework that can be applied in a wide range of settings for pupils with autism with diverse needs. The final part of the brief is to develop a training module for the AET training hubs that can support staff in applying the framework in their own situations. The role of the evaluation team in providing formative feedback in terms of identifying robust and effective and achievable outcomes data is key to both pieces of work.

 


What are the key deliverables?

The contractor will produce a final report covering the following three elements:

1. Describing and assessing the impact of the programme in 4 schools. Ideally these should include both special and mainstream schools in primary and secondary phases.
2. Design an online questionnaire to gather quantitative information on impact from a less selective sample on an anonymous basis to be sent out by the AET as an e-bulletin and by all hubs to schools trained.
3. Tracking the links between the programme, improved provision and pupil outcomes using the new AET Progression Framework
4. Feed into the development of the new Progression Framework and assess the benefits of the new Progression Framework and make suggestions for its further improvement.

It also expected that the contractor will liaise with other relevant Programme Partners as necessary including interim feedback to the autumn AET Programme Partners meeting.



What key skills / experience are important to us?

The chosen organisation will be able to offer:
• Ability to undertake high quality evaluation which is of practical use for all our stakeholders
• An excellent track record in working in similar scale action research projects in the area of SEND and the delivery of CPD.
• Ability to understand the wider policy framework for SEND and CPD in the context of new legislative and systemic changes.
• Bidders should state where their research team includes researchers with autism and how their skills will be used in this work.



When do we need it?

The delivery timetable is set out below:

 Activity

  By When

 Contractor appointed

 June 2015

 Commencement of contract

 June  2015

 Schools and pupils identified

 September  2015

 Fieldwork including the application of the new AET Progression Framework

 December 2015

 Final report to AET Programme Board

 March 2016

 


Budget and course cost structure

The total funding for the contract inclusive of any VAT payable, all production costs and all expenses is £10,000


Further information

Organisations who are interested in competing for this work should submit the proforma available to download here by Friday 22nd May 2015.   Shortlisted applicants will be interviewed on Monday 15th June and Tuesday 16th June 2015.

For an overview of the AET please see the AET website at www.autismeducationtrust.org.uk . For individual queries please email our Administrator, Joanne Driver at Joanne@autismeducationtrust.org.uk . We will either respond to your query by email or arrange a phone call if necessary.

 


Relevant links

The AET programme