Education, health and care plans: parents and young people survey - technical report

Experiences of Education, Health and Care plans

A survey of parents and young people

Technical report

March 2017

Lorna Adams, Angus Tindle, Sabrina Basran, Sarah Dobie and Dominic Thomson, Iff Research; Dr Deborah Robinson and Dr Claire Shepherd, University of Derby

Contents

List of figures 3

1 Background and aims 4

2 Sampling 5

2.1.1 EHC plan data classification errors 8

2.1.2 Contact details 9

2.2 Sampling for the pilot stage 10

3 Questionnaire design and cognitive testing 11

3.1.1 Cognitive testing of the questionnaire 11

4 Piloting 15

5 Mainstage fieldwork 18

5.1.1 How parents and young people took part 18

5.1.2 Mainstage response rates 20

5.2 Number of responses by local authority 22

6 Weighting 23

7 Data analysis 31

7.1 Coding 31

7.2 Data editing 31

7.3 Cross-tabular analysis 32

Appendix A – Ethnicity variable 34

List of figures

Figure 1.1 Outline of fieldwork approaches 16

List of tables

Table 2.1: Npd and Ilr returns used for mainstage sampling 6

Table 2.2: Approach for establishing ‘new’ EHC plans 6

Table 2.3: Profile of mainstage sample 7

Table 2.4: Instances of possible re-classification over time 8

Table 2.5: Schools with a high proportion of EHC plans 9

Table 3.1: Breakdown of cognitive interviews by age and primary SEN of the child 12

Table 5.1 Response rate by approach 20

Table 5.2 Telephone survey outcomes 21

Table 5.3 Response by local authority 22

Table 6.1 Comparison of starting sample and completed interviews 24

1 Background and aims

The Department for Education commissioned a survey of parents and young people with an Education, Health and Care plan (EHC plan).

The aim of the survey was to build a representative national picture of how parents and young people in England were experiencing the Education, Health and Care assessment process and the resultant EHC plans, to assess whether delivery of EHC plans properly reflected the intentions set out in the Children and Families Act 2014 and the accompanying SEND Code of Practice.

Specifically, the survey sought to do the following:

• Understand the extent to which children, young people and parents experienced EHC needs assessments and EHC plans as they are intended to be experienced;
• Understand variation in experiences by groups with different characteristics;
• Explore how satisfied they were with the process and the resultant EHC plan, and

the extent to which this varied by local authority and by groups with different characteristics;

• Identify how the Education, Health and Care assessment process and the resultant EHC plans could be improved;
• Understand any variation in experiences by local authority, to identify where action

was needed to improve experiences, and where, conversely, there were examples of best practice from which others could learn.

The key findings of the survey are presented in the main report, published alongside this document1.

There were three key stages to the study:

• Cognitive testing of the questionnaire, testing content and also different formats (online, telephone, postal);
• A feasibility stage to test the likely mainstage survey response rates by conducting a small-scale survey using different contact strategies;
• A mainstage which involved collecting the views of parents, children and young people who had received an EHC plan in the calendar year 2015.

This Technical Report presents detailed information on the methodological approach for the study at each of the key stages, including information on sampling, data analysis and weighting.

1 Adams, L. et al (2017) Experiences of Education, Health and Care plans: a survey of parents and young people. Department for Education, Research Report RR657

2 Sampling

All sample for the survey was drawn from two official databases: the National Pupil Database (Npd), a dataset containing the details of all pupils in state schools, nurseries and alternative education providers; and the Individualised Learner Record (Ilr) which records information on learners in publically-funded further education.

These databases do not cover children/young people attending private providers and hence these are an omission from the starting sample. They also do not cover children/young people studying at Higher Education Institutions2.

The mainstage sample was made up of children and young people with an EHC plan that was created in the calendar year 2015.

Because no indication is given in the Npd collections or Ilr of the date/period in which an EHC plan was produced, multiple returns of each collection were obtained in order to identify ‘new’ EHC plan cases by comparing individuals’ records over time.

The first step was to identify which returns ‘framed’ the sampling window as either a ‘baseline’ file, marking the period directly up until the sampling window (where any EHC plans flagged would have been created before the sampling window), or ‘main’ file, marking the close of the sampling window (where any ‘new’ EHC plans present would have been created within the sampling window).

The table below shows the returns that were used to derive the starting sample.

2 The possibility of supplementing the sample with records held by the Higher Education Statistics Agency was explored but this was not possible.

Table 2.1: Npd and Ilr returns used for mainstage sampling

To ascertain which EHC plans found in the main files were ‘new’, a process of elimination was then applied. This is summarised in Table 2.2:

Table 2.2: Approach for establishing ‘new’ EHC plans

The sampling approach described above resulted in a total sample size of 65,172 records with an EHC plan in 2015. Table 2.3 shows the breakdown of this starting sample by source and key demographics - including whether the child/young person for whom the EHC plan was created was aged under 16 (in which case the survey approach was made to their parent) or 16+ in which case the young person themselves was initially given the opportunity to take part themselves.

Table 2.3: Profile of mainstage sample

It was not possible to contact a small proportion (6%) of the mainstage starting sample during the research. Among the 9,612 Ilr records with an EHC plan that was created in 2015, 4,162 were excluded because of their contact preferences (they had opted out of being contacted by telephone and either post or email), bringing the total amount of eligible sample down to 61,010.

2.1.1 EHC plan data classification errors

This starting sample included records for some individuals who did not actually have EHC plans. This was because some education providers had misunderstood how pupils with SEN should be recorded in the school census following changes to support for them set out in the Children and Families Act 2014. New SEN data categories were introduced to the school census: ‘SEN Support’ and ‘EHC plan’. Children with SEN whose needs were being met without a SEN Statement (i.e. those receiving School Action and School Action Plus) were to be re-classified into the new ‘SEN Support’ category. Those pupils with SEN Statements were to be re-classified to the ‘EHC plan’ category, after they had transferred from a SEN Statement to an EHC plan (following an EHC needs assessment). The transfer process should have been carried out in stages in schools, with pupils being transferred by year group. The data should have reflected this with a mix of SEN Statements (for those waiting to transition to a plan) and EHC plans (for those who had transitioned or who did not have a SEN Statement and required anew EHC plan). However, for some schools, very high proportions or all of the pupils with SEN Statements were identified as having an EHC plan, suggesting that they had been incorrectly reclassified in one go without reflecting the staged transfer process.

Analysis conducted in the preliminary stages of this study showed that some institutions had over time been re-classifying individuals flagged as having an EHC plan to another SEN marker again. Just under a third (29%) of EHC plans shown in the Autumn 2014 School Census were no longer marked as such in the Summer 2015 Census; while a significantly lower proportion of EHC plans shown in the Spring 2015 Census were no longer marked as such in the 2016 Spring Census (18%).

Table 2.4: Instances of possible re-classification over time

ˆCompared with Summer Census 2015 ˆˆCompared with Spring Census 2016

As the absolute number of EHC plans almost doubled between Autumn 2014 and Spring 2015 Censuses while the proportion of EHC plans being re-classified had decreased over this period, there was a strong suggestion that the ‘worst’ period of data misclassification was over. However, it was not possible to predict at the outset with confidence the exact number of records within the mainstage sample that were not truly EHC plan cases.

Analysis of Spring Census 2016 data at the provider level indicated that there were still providers where a high proportion of children with SEN were flagged in the Npd as

having an EHC plan – albeit to a much lesser extent than in the 2015 release. As Table 2.5 shows, there were six schools in the 2016 School Census where 50% or more of their SEN pupils were marked as having an EHC plan3.

Table 2.5: Schools with a high proportion of EHC plans

Records from these schools could have been excluded from the mainstage survey sample as there was an element of doubt over the accuracy of these EHC plan markers. In terms of the impact this would have had on the starting sample size:

• Excluding schools where 50% or more of SEN pupils were flagged as having EHC planss would have removed 122 records from the mainstage sample;
• Excluding schools with 60% or more would have removed 90 records;
• Excluding schools with 70% or more would have removed 32 records.

The decision was made not to exclude any records as there was no certainty about the inaccuracy of these EHC plan flags and so on balance it was preferable not to ‘lose’ any legitimate cases from the mainstage sample.

2.1.2 Contact details

The Ilr contains a range of contact details but the Npd contains only postal addresses. As a small scale exercise to explore augmenting these contact details, we approached a small number of local authorities to understand whether they would be willing/able to

3 Three schools had 100% of their SEN pupils flagged as having an EHC plan, but the absolute number of pupils affected are so low (one pupil at one school, two pupils at each of the other two schools) that they have not been included in this analysis.

explore appending the contact details that they hold about individuals in their area that have received an EHC plan to the sample generated through the Npd/Ilr.

Six authorities were contacted in February 2016 with a request for data for individuals the authority had produced an EHC plan for in 2015. Ultimately, this approach was found to be an unrealistic means of sourcing the contact details of individuals with EHC plans for a number of key reasons:

• Difficulty getting hold of the right local authority officer(s) able to agree such a data request;
• The level of resource required by the authority to fulfil the request – and the impact this would have on timetabling;
• Resistance by the authority due to concerns over data protection.

As a result, the survey used the contact information available on the Npd/Ilr supplemented by a telephone look-up exercise to obtain additional telephone numbers.

2.2 Sampling for the pilot stage

The aim of the pilot stage was to test the methodology outlined for the mainstage, without diminishing the ‘pool’ of potential mainstage respondents. The pilot sample was therefore compiled of children and young people with an EHC plan that was created before 2015 to avoid any overlap with the mainstage sample.

As for the mainstage, the pilot sample was drawn from two sources: the main School Census collection in the Npd and the Ilr.

The pilot sample was made up of 2,400 records for children and young people who had an EHC plan created in 2014.

3 Questionnaire design and cognitive testing

This first stage of the study involved extensive questionnaire development in conjunction with the DfE project team and Advisory Group, and assessment of available sample sources.

The questionnaire was designed by Iff Research and Derby University. It asked parents and young people about their experiences of the Education, Health and Care assessment process and the resultant EHC plans, with a particular focus on aspects where the SEND Code of Practice sets out what the process should be like4. There were two slightly different versions of the questionnaire5 – one for young people aged 16+, and one for parents/carers of children aged up to 15 years old6.

A number of different resources were consulted and referenced during the questionnaire review stage, including but not restricted to:

• The SEND Code of Practice 2015;
• The Personal Outcomes Evaluation Tool (Poet) for children and young people with Education Health and Care Plans (EHC plans);
• Qualitative data from a study by Ask Research, mapping user experiences of the EHC plan process.

In addition, the questionnaire was reviewed by a panel of experts7. It was also tested in detail with young people and their parents during a ‘cognitive testing’ phase.

3.1.1 Cognitive testing of the questionnaire

The aim of the cognitive testing was to check whether the question wordings were understood as intended, and whether the questions were asking about things that the parents and young people would be able to answer.

Specifically, the cognitive testing explored with parents and young people:

1. This focus on whether parents and young people had experienced specific elements set out in the Code

of Practice was in part informed by the findings of Skipp and Hopwood (2016). Their qualitative study, Mapping user experiences of the Education, Health and Care process, established that parents tended to score individual components of the process (referral, assessment etc.) differently to how they scored their satisfaction with the process overall. It also found that parents’ overall satisfaction varied according to their degree of understanding of what the process should be like – meaning that parents could express high levels of satisfaction with an experience of the process that did not appear to be in line with the Code.

1. Versions of the final questionnaires for parents/carers and young people are presented in the appendices

to the main report. See Adams, L. et al (2017) Experiences of Education, Health and Care plans: a survey of parents and young people. Department for Education (Research Report RR657)

1. Where the child or young person named on the EHC plan was aged 16-25 years, the questionnaire was

designed for them to fill out, where they were aged under 16, the questionnaire was largely designed for the parent or carer to complete on their behalf. This version of the questionnaire also included a section for the child/young person to complete themselves if they chose to, with or without their parent/carer.

1. From Derby University, the DfE project team and its Advisory Group.

• The comprehension and appropriateness of questions, exploring any obvious difficulties associated with particular survey modes;
• Parent and young people’s experience of the whole questionnaire (especially their thoughts on whether they thought the questionnaire allowed them to accurately and fairly present their feelings about the process of getting an EHC plan);
• Whether the questionnaire worked in online, paper and telephone formats.

The cognitive study sample was compiled using the National Pupil Database (Npd) and Individualised Learner Record (Ilr), as well as using the University of Derby’s network of contacts in the SEND arena. As the mainstage of the survey covered EHC plans created in the calendar year 2015, the cognitive testing phase only drew on EHC plans created in 2014 in order to preserve the mainstage sample.

The cognitive testing was carried out by the project team at Iff Research and Derby University between February and March 2016. Interviews were carried out face to face and typically lasted for an hour to an hour and a half – depending on the speed with which the parent or young person was able to complete the questionnaire. Interviews were conducted in London, East Sussex, Staffordshire, Birmingham, Leicestershire and Derbyshire.

In total 36 interviews were conducted:

• 18 interviews with parents of children aged under 16;
• 18 interviews with young people aged 16+.

Within these main cohorts, quota targets were set to ensure a good spread of characteristics between different sub-groups. Among parents these targets were based on the age of the child in receipt of an EHC plan, among young people these were based on the primary need of the individual (available as an Ilr sample marker). Interviews with respondents attending a range of educational settings were achieved.

Table 3.1 shows how the interviews were split between the parent and young people sub-groups.

Table 3.1: Breakdown of cognitive interviews by age and primary SEN of the child

Each cognitive interview consisted of two key stages:

• The parent or young person reviewed the introductory letter and completed the

questionnaire in one of three ways - reflective of the three survey modes that would be employed in the pilot and mainstages (i.e. as postal, online and interviewer-administered telephone surveys8);

• An in-depth discussion led by the interviewer, exploring the parent or young person’s experience of completing the questionnaire.

Each survey mode was tested with both parents and young people.

At a general level, the response to the questionnaire in the cognitive testing was positive and parents and young people welcomed the chance to share their views and for the most part, there were no difficulties answering the questions.

A few areas for improvement emerging from the cognitive testing included:

• Some minor presentational issues across the different modes tested (online,

postal, telephone) – for example making sure signposting was clear in both the postal and online formats, breaking up longer banks of questions using agree/disagree scales for the telephone interview due to difficulties recalling the scale;

• In the letter, a need for signposting to support / further information such as the use

of a separate ‘FAQs’ section which also included data protection reassurances, to keep the upfront text introducing the research simple.

• In the questionnaire itself, some parents and young people struggled with some of

the question wording or terminology – for example young people struggled with the term ‘provision’; and parents asked for clarification on whether or not the question about the educational setting attended referred to the present setting or the setting the child/young person attended when the EHC plan process was first started.

• Some aspects of parent and young people’s experience were not being captured

by the questions – for example making more than one request to get the EHC plan.

• The different stages of the EHC plan process were not always clear in parent and young people’s minds – for example reviewing the EHC plan or being offered a

8 As the purpose of the cognitive testing was to gauge respondents verbal and non-verbal reactions to the survey questionnaire it was important for the interviews to be carried out face to face. Where the survey mode being tested as an interview-administered telephone survey, this was explained to the respondent and the questionnaire was read aloud by the interviewer to the respondent in the same way as they would over the phone. Other considerations were made to make the cognitive testing ‘true to life’ – for example the interviewer did not make eye contact whilst reading the questionnaire script, to replicate the experience of answering the questionnaire via telephone.

Personal Budget. Similarly, parents and young people weren’t always sure which authorities/bodies were meant to be responsible for different stages of the process.

• The questionnaire did not always provide an option for respondents to say that a

particular part of the process was not relevant to them e.g. in terms of engagement with staff and services - young people generally had not had involvement in this.

• Difficulties identifying one ‘primary’ type of SEN for the child/young person – it was

generally felt that individuals had multiple areas of need and it was not possible to identify a single ‘main’ need.

• Some sensitivities around the framing of ‘forward-looking’ questions and feeling

these weren’t always relevant, and a strong sense that the EHC plan itself would not change the child/young person’s future – the emphasis would need to be on the content of the EHC plan and specifically the help/support it sets out.

A report on the results of the cognitive testing findings was sent to the Department for Education Advisory Group and a discussion meeting held to go through the suggested changes to the questionnaire. These were agreed and the questionnaire and letters were revised on the basis of the feedback.

4 Piloting

As mentioned earlier, the contact details available on the Npd and Ilr are limited. Hence, it was decided that the survey would need to take a mixed-methodology approach to maximise response.

A pilot exercise was conducted to explore which combinations and sequences of survey methodologies obtained the best response rate.

The only contact details that are available for all records in the Npd and Ilr are postal addresses. Hence the options available to include all eligible parents and young people in the survey were:

• A ‘post-to-web’ survey where parents and young people are sent an introductory letter which provides them with a log-in and password for an online survey;
• A paper-based survey involving a questionnaire sent out in the post.

However, self-completion exercises through either of these routes typically achieve relatively low response rates and hence it felt beneficial to include a telephone element to boost response rates (even though telephone numbers were only available for part of the sample).

Three approaches to combining these methodologies were trialled as below (and illustrated in Figure 4.1):

• Approach A: Sending a postal questionnaire initially (including in the covering

letter details for accessing the survey online). Then sending a reminder letter two weeks later with the details for accessing the online survey. One week after the reminder email, attempting to contact by telephone all non-responders for whom a telephone number was held.

• Approach B: Initially only offering the ‘post-to-web’ option but issuing a paper

questionnaire with a reminder letter two weeks after the initial invitation to participate. Then one week later moving to a telephone approach as outlined above.

• Approach C: offering the ‘post-to-web’ option but then moving straight to the

telephone stage after around 2 weeks. This approach did not use the postal version of the questionnaire at all.

Figure 4.1 Outline of fieldwork approaches

A starting sample of 800 records (554 parent and 246 young person) was used for each of the three approaches i.e. a total pilot sample of 2,400.

The pilot fieldwork was carried out between 11th April and 23rd May 2016. A reminder was sent out (for Approaches A and B) two weeks following the initial launch, from 25th April. Telephone fieldwork for all three contact strategies was carried out between 3rd May – 23rd May.

Email addresses were available for 106 records in the pilot sample (taken from the Ilr sample). A reminder email was sent to these records to coincide with the reminder letter being posted. The email invite was based on the reminder letter, with text around how to access the online survey modified accordingly.

A dedicated helpline and email account was set up which was routed directly to the project team and checked on a daily basis, for any respondent queries and also to monitor opt-outs to the survey. Any opt-outs were removed from the contact database to ensure they did not receive any reminder letters/emails and were not contacted during the telephone fieldwork (127 people).

In total, 317 parents and young people were interviewed during the pilot stage. The response rates to each of the three contact strategies is shown in the table below:

Table 4.1: Response rate by approach

The key findings from the pilot exercise were that:

• The approaches that included a paper questionnaire option (Approaches A and
• obtained a considerably higher response rate;
• The overall response rates for Approach A and B were similar but Approach B;
• Resulted in more interviews completed online and fewer by telephone which is more cost-effective;
• Achieved a higher response from young people.

On this basis, the decision was taken to proceed with Approach B for the mainstage survey i.e. a letter to participate in the survey containing a link to the online survey, followed by a reminder letter including the link to the online survey and a paper questionnaire, and finally telephone fieldwork.

5 Mainstage fieldwork

The mainstage survey focused on children and young people with an EHC plan that was created in the calendar year 2015 (when the survey took place, this was the most recent full calendar year for which data was available).

The Npd and Ilr databases identified 65,172 children and young people who had an EHC plan put in place in 20159 and these cases formed the starting sample for the survey. Of these, it was not possible to contact 4,162 Ilr records due to their contact preferences, bringing the total eligible mainstage sample down to 61,010 records.

The survey approach was an attempted census of these records.

The survey fieldwork was carried out from 25th July to 28th November 2016. A total of 13,643 parents and young people took part.

5.1.1 How parents and young people took part

Letters were initially sent to the families of all those in the starting sample. These letters explained the purpose of the survey and how the survey data would be used, and invited the recipient to take part by accessing an online survey.

The initial invitation letters were sent out on 22nd July 2016. This letter included an option for individuals to ‘text-in’ and opt to be contacted during the telephone fieldwork stage. In total, 275 individuals used the text-in option – of these 197 texted in before the reminder letters were sent (i.e. between 25th July and 19th August 2016) and were therefore excluded from the reminder mail out.

After four weeks, those who had not responded were sent a reminder letter (19th August 2016). This again contained details for accessing the online survey, but also included a paper questionnaire booklet and a reply-paid envelope, providing individuals with the option of taking part in this way.

The initial invitation was also sent by email as was the reminder to those individuals we had email addresses for: 2,870 people received the initial email invitation and the reminder email was sent shortly after to those who had either not completed the survey or not opted out.

Of the eligible sample of 61,010 records, telephone numbers were obtained for 23,217 records. Of these, 20,152 were eligible for contact during the telephone stage – this included those who had not completed the survey after the initial invitation or reminder letter/email and had a telephone number available. Those individuals who had either

9 This figure is known not to be entirely accurate because of a degree of data reclassification errors where some education providers had incorrectly re-categorised all their SEND pupils as having an EHC plan.

opted out of the survey or informed Iff that they did not have an EHC plan in place (127 people) were not contacted.

The telephone element of the mainstage was started six weeks after the reminder mail out (fieldwork period: 3rd October 2016 to 28th November 2016) 10. We attempted to contact by telephone all non-responders for whom a telephone number was held.

It was also possible to request the questionnaire in an EasyRead format; to ask to be interviewed face-to-face; or to request an interview in a language other than English. Nobody completed the survey in EasyRead format, although it was sent out to a small number of people who requested it. In total there were eight foreign language interviews completed (two Bengali, two Urdu, two Gujarati, one Polish and one Spanish.).

In the majority of cases (48,053 of 61,010 records that were eligible to be contacted) the child/young person with the EHC plan was under the age of 16, and hence the invitation letters and telephone calls were in the first instance addressed to their parent or carer. However, 12,957 records were for a young person aged 16 or over. In these instances, the young person themselves was approached to take part.

Where the survey invitation was sent to the parent or carer, they were invited to consult the child or young person with the EHC plan about the answers to give, if they felt this was appropriate. There was also one section of the questionnaire (‘Section H’) sent to parents/carers that focused on the experiences of the child or young person specifically: parents/carers were encouraged to fill in this section with the child/young person’s input, or to answer the questions from their point of view, as far as possible.

Where the survey invitation was sent to the young person, they were invited to complete the questionnaire either themselves, or – if they preferred – with their parent/carer, or with their parent/carer answering on their behalf. The majority of responses come from a parent or carer on behalf of the young person. Overall, of the 13,643 responses received:

• 10,675 were from parents/carers answering principally about their own

experiences of the EHC plan process (on behalf of a child/young person aged under 16);

• 2,246 were from parents/carers answering on behalf of a young person aged 16+; and;
• 722 were from young people aged 16+ answering about their own experiences.

10 A small-scale telephone exercise was carried out before the mainstage proper telephone fieldwork. The exercise was carried out over two weeks (16th August to 31st August 2016) to contact the 275 records that had ‘texted-in’ to opt for a telephone interview as it was felt that it was important to keep these individuals engaged with the study. For example those who had texted in after the initial invitation letter was sent in July would otherwise have not received a follow-up call until sometime between September and November 2016.

5.1.2 Mainstage response rates

From a starting sample of 65,172 records, 13,643 completed interviews were achieved at a response rate of 21%. When looking at the 61,010 records that we were able to contact for the research (i.e. excluding the 4,162 Ilr records that were excluded because of their contact permissions), the response rate was 22%.

The starting sample of 65,172 records contained some individuals who had incorrectly been flagged as having an EHC plan, largely as a result of the EHC plan data classification issue discussed in the sampling chapter. Outcomes from the telephone interviewing stage made it possible to estimate the proportion of records incorrectly labelled in each local authority. Using these figures produced an estimate of the true population of EHC plans at 57,112.

Using this adjusted population figure gives a response rate of 24%.

Table 5.1 shows the number of responses achieved at each stage of the fieldwork period (i.e. online, postal and telephone).

Table 5.1 Response rate by approach

Table 5.2 shows a full breakdown of the sample outcomes from the telephone survey. Among the 20,152 records loaded for telephone fieldwork (i.e. those that had a telephone

11 This figure has been calculated using the mainstage starting sample of 65,172 records to give the most accurate estimate of the true population of EHC plans. 12 This is based on the 61,010 records that were eligible to be contacted.

number and had not taken part online or via the postal survey), 26% completed a telephone interview.

Refusals were low and accounted for just 4% of the starting sample, whilst contact was not made with 24% because their telephone number was incorrect.

Table 5.2 Telephone survey outcomes

Approach Questionnaire

Total

type

Overall the response rate to the mainstage survey was higher than that of the pilot. There are a number of possible explanations for this:

• The lag between EHC plans being issued and fieldwork was smaller for the

mainstage sample than for the pilot sample (which included only EHC plans created in 2014);

• The fieldwork period for the mainstage fieldwork was longer;
• Some promotional activity was undertaken in advance of mainstage fieldwork. The

key findings from the pilot were published on the Iff website and in a DfE news letter to SEND contacts in local authorities. The National Network of Parent Carer Forums also promoted the mainstage survey while it was live to raise awareness, particularly among parents and young people eligible to participate.

5.2 Number of responses by local authority

Ideally the survey would have obtained robust results for each local authority. However, the size of the starting sample and the availability of contact details limited the extent to which this was possible.

Although all local authorities were represented in the starting sample, the number of EHC plans per authority varied considerably. At two authorities, fewer than 5 EHC plans had been created in 2015, whilst Hertfordshire, Birmingham and Kent had the largest amount of sample with over 12,000 records each.

As Table 5.3 shows, at least 50 interviews were achieved in 71% of local authorities. Of the remaining 43 local authorities where fewer than 50 responses were achieved, the average number of records available for interview was 160 though there were four local authorities where there were fewer than 50 records available.

Table 5.3 Response by local authority

6 Weighting

The approach to the survey was an attempted census and hence there was no need to apply any design weights.

However there were slight variations in the profile of the starting sample and the profile of achieved responses as a result of;

• Varying levels of contact details provided (meaning not all could be approached for the telephone stage, for example);
• Some groups being more likely to respond than others.

To explore the impact of these factors on the achieved sample, the profile of the starting sample was compared with the profile of achieved interviews for the following demographic variables:

• Sample source;
• Age of the child/young person for whom the EHC plan was for;
• Ethnicity of the child/young person13;
• Gender of the child/young person;
• The primary SEN type of the child/young person with an EHC plan (available as a market on the Npd and Ilr sample)

This comparison is shown in Table 3.1. As the table shows, at an overall level, the profile of achieved interviews was similar to the starting sample in terms of age, gender and primary SEN of the child/young person with the EHC plan.

A slightly lower response was achieved from respondents where the child/young person with the EHC plan was non-white, this group made up 25% of the starting sample but 22% of the completed interviews.

A lower response was also achieved from respondents where the child/young person with the EHC plan was aged 16-25 (26% of the starting sample but 22% of the completed interviews). This can be explained by the 4,612 Ilr records that were excluded from the research because of their contact preferences. When these ineligible records are excluded from the calculation, the pattern of response is much closer (21% of the starting sample compared to 22% of completed interviews).

13 Ethnicity of the child/young person was available as a marker on the Npd but not the Ilr. A ‘nationality’ marker was available on the Ilr sample. For the purposes of weighting, the categories from the ethnicity and nationality markers were cross-referenced and combined where considered appropriate, to produce a a new ethnicity marker that could be applied to all records from both the Npd and Ilr.

Table 6.1 Comparison of starting sample and completed interviews

There were also variations in the profile of the starting sample and achieved interviews by local authority as Table 3.2 shows.

Table 6.2: Comparison of starting sample and completed interviews by Lea

Given the expectation that there would be some geographical variation in EHC plan processes between local authorities, it was important to correct for the slight differences in the profile of the achieved and starting sample by local authority in order to produce results that were accurate at a national level. We also decided to correct for the small difference in the profiles by ethnicity.

The weights were applied using a cell weighting approach based on a local authority by ethnicity grid as shown below.

Table 6.3 Example of weighting grid

Local authority White British/Irish Non-White Missing

LA1 LA2 LA3 Etc.

For each cell the weight applied was calculated in the following way:

• The proportion of responses in each cell was calculated for both the achieved

sample and the starting population. For example in the achieved sample, the number of responses by those in LA1 from a White British/Irish ethnic background were calculated as a proportion out of the whole achieved sample (i.e. out of 13, 643), and for the total population.

• The proportion in the achieved sample was divided by the proportion in the

starting population to give a weight for that cell. This weight was then applied to each record in the cell e.g. to all records in LA1 from a White/British ethnic background. Weights were calculated at 15 decimal places.

For the purposes of applying the weighting, the population figures used were adjusted to try to take account of data reclassification errors made by some education providers referred to earlier which meant the starting sample of 65,172 cases over-estimated the size of the actual population of EHC plans issued in 2015. During the mainstage fieldwork, a record was kept of parents and young people who responded to the survey saying that the child/young person did not have an EHC plan in place. This information was then used to produce a more accurate estimate of the ‘true’ population of EHC plans created in 2015.

Using outcomes from the telephone stage, we were able to look at the proportion of those who we had contact with that turned out not to have an EHC plan in place14 for each local authority. We then applied these proportions to the starting sample to produce an estimated ‘true’ population of 57,112 EHC plans. The data on ethnicity by local authority was re-scaled to this total and these figures were used for the weighting calculations outlined above.

The lowest weight applied to any cell was 0.1876934913 and the final highest weight was 2.8238065562.

Table 6.4 shows the minimum and maximum weights calculated for each ethnic group: Table 6.4 Minimum and maximum weights for each ethnic group

14 As part of the telephone interview, care was taken to explain to respondents what an EHC plan is and to differentiate between an EHC plan and Statement of SEN to ensure there was no misunderstanding. The telephone data relating this is therefore considered a reliable source of information. For this reason, we did not exclude records from the population of EHC plans in 2015 who reported not having an EHC plan via the online or postal survey.

7 Data analysis

7.1 Coding

Several questions in the survey offered the option of providing an ‘Other (please specify)’ response. Such responses were reviewed and, where possible, assigned to an existing code (‘backcoding’). Where backcoding was not possible – and if similar or identical responses emerged among the ‘other’ answers – additional codes were added to reflect the emerging themes.

The survey also contained two open-ended questions, without a pre-developed codeframe. Here parents and young people were asked what went well during the process of getting an EHC plan and what didn’t work well. These responses were recorded verbatim. Once the fieldwork period had finished, code frames were drawn up to reflect the common themes which emerged for these questions.

7.2 Data editing

The following data edits were applied where necessary to postal, online and telephone responses:

• Where the parent or young person revealed as part of their response to the open-

ended questions or as a comment elsewhere that they did not have the final version of their EHC plan yet – their answer to the opening question about whether or not they have an EHC plan was amended accordingly. This closed the survey as they were not considered eligible to participate;

• Where the parent or young person had responded that the EHC plan had been in

place for more than 3 years – their answer was re-coded as ‘don’t know’ (this was not a plausible response as EHC plans were introduced in 2014);

• Where the given date-of-birth of the child / young person made them significantly

over the age of 25, their response was re-coded as ‘don’t know’ (this was not a plausible response as EHC plans are available for individuals up to the age of 25 only).

Two questions in the survey asked parents or young people whether they had a) heard of / been told about sources of information, advice and support and complaints processes and b) whether they had gone on to use these. At these questions, the following data edits were applied as ‘logic checks’:

• The parent or young person said that they had used a particular service but left

blank the question of whether or not they had heard of / been told about it – these responses were coded that they had heard of / been told about the service;

• The parent or young person had heard of / been told about at least one of the pre-

coded services, but left blank the question of whether they had used any of them – these responses were coded that they had used ‘None of these’ services;

• The parent or young person had heard of / been told about ‘None of these’

services but left blank the question of whether they had used any – code that they had used ‘None of these’;

• The parent or young person had responded ‘don’t know’ to the question of

whether they had heard of / been told about any of the pre-coded services, but left blank the question of whether they had used any – code ‘don’t know’ for the question of whether they had used any.

7.3 Cross-tabular analysis

Once the mainstage fieldwork period had finished, a set of data tables were produced to a specification agreed in advance with the DfE and this formed the cornerstone of the analysis process.

The following cross-breaks were used throughout the set of tables:

• Age of child / young person;
• How long the EHC plan process took;
• Whether or not the child / young person had a SEN statement before applying for the EHC plan;
• Whether or not the EHC plan was provided following the family’s first request for one;
• Types of need covered by the EHC plan, particularly comparing Plans which covered education, health and social care with Plans covering education only;
• Primary SEND need;
• Region;
• Ethnicity of the child / young person;
• Number of EHC plans issued by the Local Authority in 2015;
• Number of appeals to a SEND Tribunal per 10,000 of 0-18 population in a Local Authority;
• Index of Multiple Deprivation, particularly comparing the top and bottom quintiles.

The data tables included significance testing and highlighted differences that were significant at the 95% level. Two types of tests were performed:

• z-tests comparing the findings for the sub-group with the remainder of the sample
• t-tests comparing the findings for a sub-group with all other sub-groups within that

category (so for example comparing results for EHC plans created for young people aged 16+ with all other age groups).

Tables were analysed with consideration given to the original research objectives and the questions that the research set out to answer. Significant differences highlighted in the tables were analysed in the context of the research objectives and not reported indiscriminately. However a full set of tables is also available on request (see final page for contact details).

Appendix A

Table showing how the new ‘ethnicity’ variable was created by cross-referencing and combining codes on the Npd for ethnicity and Ilr on nationality. Ethnicity Code (Npd/Ilr)

© Department for Education, 2017

Reference: DFE-TR657

Isbn: 978-1-78105-726-1

The views expressed in this report are the authors’ and do not necessarily reflect those of the Department for Education.

Any enquiries regarding this publication should be sent to us at: Michael.Dale@education.gsi.gov.uk or www.education.gov.uk/contactus

This document is available for download at www.gov.uk/government/publications