Evaluation hero

Service Evaluation

A service evaluation will seek to answer “What standard does this service achieve?”. It won’t reference a predetermined standard and will not change the care a patient receives.

Where an evaluation involves patients and staff, it is best practice to seek their permission, through consent, for their participation. The degree to which consent is taken can vary from asking someone if it is OK to seek their feedback on the service, through to a formal process using consent and information leaflets.

What is a service evaluation?

Many people make claims about their services without sound evidence to inform their judgements. A well planned and executed service evaluation will provide:

  • Evidence to demonstrate value for money.
  • A baseline from which to measure change.
  • Evidence to demonstrate effectiveness.
  • Evidence to demonstrate efficiency.
  • Evidence to demonstrate benefits and added value.

Service evaluations do not require NHS Research Ethics review but do need to be registered with, and approved by, the Research and Evidence Department before they can commence. This will include checks that the project is feasible (e.g., the service has capacity, appropriate timeframes) and carried out in line with Trust standards. Registration ensures the R&E Department know what type of activity (research and service evaluations) is happening in which services.

Please note, the Trust primarily processes data to deliver healthcare. The data cannot be used for other purposes unless the law allows ‘secondary processing’. Research and service evaluation are both considered secondary processing.

Data collected through service evaluations must be obtained, recorded and stored as defined by all relevant data standards, including The General Data Protection Regulation (GDPR) and the Data Protection Act 2018. At all times your evaluation must be designed in such a way that privacy is built into all aspects that involve data.

A great source of information is also available from the Evaluation Works.

What do I need to do to undertake a service evaluation?

Be clear about what you want to evaluate

This will shape how you conduct the evaluation and define what information you will need to collect and where the data is. Your project should also link to at least one of the Trust’s Quality Priorities.

 

Identify all stakeholders

Stakeholders in the success of your evaluation may include, amongst others service managers, commissioners, staff and patients. Be sure to engage them as early as possible to ensure they understand your work. Involving them early will help you shape your application to ensure it is successfully delivered.

 

Plan your project

Planning is paramount and needs to include an honest assessment of how long it will take to produce the necessary paperwork and seek approvals. You must also consider how and where data will be recorded and stored. Wherever possible you should use non-identifiable data. Data must be anonymised* or pseudonymised and kept secure within the Trust. The Information Commissioner's Office define ‘anonymised data’ as “…data that does not itself identify any individual and that is unlikely to allow any individual to be identified through its combination with other data.” Where agreed by the Research & Evidence Department, only fully anonymised data may be taken outside the Trust (e.g. to be stored on university servers).

All project members – Trust employees and those on honorary contracts – are required to have completed the Trust’s annual Information Governance (IG) training. A paper version can be provided where members do not have electronic access. Project members receiving anonymised data only are not required to complete the Trust’s IG training.

 

Develop your paperwork

The Service Evaluation (SE1) form sets out the information you need to provide. This covers the aims, objectives (what you will do to meet the aims), methodology (including data handling), analysis, and dissemination. Please follow the additional guidance on the SE1 form so that the form is completed in full.

Depending on how you plan to gather the data for your evaluation, you may need a participant information sheet and a consent form. You should seek appropriate informed consent from participants if you are asking them to do more than what is in the Friends & Family Test. Consent should be explicit verbal or written (written consent where participants are identifiable or where their identifiable data is involved, or qualitative methods are being used). The Research and Evidence (R&E) department can help you determine what you will need; templates are available.

All forms must be version controlled to ensure approved ones are being used.

Your project must be approved by, and registered with, the R&E department.

Prior to submission, applicants are encouraged to discuss their proposed project at one of the Trust’s monthly Research Clinics via MS Teams. This can help to improve the application. Please contact research@nottshc.nhs.uk to book on to a Research Clinic.

Apply to register your project with the Research and Evidence department

The completed SE1 form and any additional documents (e.g., copies of data collection forms and any interview topic guides) should be sent to research@nottshc.nhs.uk. Incomplete forms will be returned to the applicant to request completion. The review will not start until we have received a completed SE1 form and all relevant accompanying documents listed on the SE1 form. Please contact the R&E Department if you need advice.

Following an initial review from the R&E Compliance team, it is likely you will be invited to discuss your application at a Service Evaluation Review Panel (SERP). This is a supportive discussion to help approve the project. The SERP will comprise representatives from relevant departments (e.g., R&E, Information Assurance) and allow applications to be processed more efficiently as it gives the applicant the opportunity to address any queries or concerns.

Applicants are expected to address any outstanding actions resulting from the SERP and submit responses. The application will be approved when any actions have been completed.

For non-Trust staff, a research passport / Letter of Access may be required where the applicant does not hold a contract with the Trust. This will be addressed at the SERP.

Applicants must not start their service evaluation until they receive written approval from the R&E Department.

 

Conducting your project

With good planning and engaged stakeholders your project should run according to plan. Where we see problems arise it is usually because of something that was not considered, or planning was not thorough enough.

Where relevant, the service evaluation lead will be asked to provide ‘recruitment’ figures quarterly to the R&E Department. This is so we can monitor the level of evaluation activity in the Trust. The study team should notify the R&E Dept when they have stopped recruiting participants.

Where amendments to the evaluation are necessary (e.g., timeframes, changes to the study team), the applicant should discuss and agree these with the R&E Compliance Team beforehand. This should prevent deviations to the approved study.

 

Writing up your project

It is important that your project is written up at the end to ensure what was learnt can be shared and used to make improvements. A Final Report (FR1) template is available. This is the minimum feedback that should be provided. You should send the final evaluation report to research@nottshc.nhs.uk so that we can put a copy on our Connect page.

In addition to providing the final report, the study team should present to the service or other relevant Trust meetings if requested.

We also want to capture the impact of your service evaluation so please give some thought as to how this can be achieved.

Forms, templates and guidance documents for service evaluations

 

 

Rate this page or report a problem

Rating
Rate this page or report a problem