Evaluation

Introduction
How do we know that the Open Government Pioneers UK initiative is making a difference?
 * The evaluation of the core activities funded by the Big Lottery Fund UK will be delivered in partnership with an external consultant with expertise in this area selected by the Project board. Budget has been allocated for the consultants fee.
 * The evaluation of the equality activities will involve an equality impact assessment and further consultancy budget for specific equality expertise has been allocated for this.

Aims and objectives
We want the learn from the mistakes we make and build on the successes we create. Is there evidence that a diverse range of citizens and civil society are any closer to engaging the decisions that affect them as a result of the Open Government Pioneers Project UK?

Objective
Track progress towards the following four big outcomes.
 * 1) Empowered engaged citizens, leveraging their knowledge of SDGs.
 * 2) Political representatives have greater awareness of SDGs and types of actions required.
 * 3) Government officials have a better plan for involving citizens.
 * 4) Open government seen as instrumental, and a common good towards progressing SDGs.

Overall approach

 * To use an open and agile approach to monitoring, learning and evaluation
 * To assess how successful the project is at opening up government progress against the Sustainable Development Goals to citizen participation
 * To inform future activities that support Open Government approaches to delivering the Sustainable Development Goals

How should the project be evaluated?
Proposed evaluation framework

Agile learning approach

 * Initial data collected in the first quarter of the project will form a baseline (against the Data subjects in the table below)
 * An assessment will be made of any changes when reported (quarterly, monthly or annually as appropriate). Advice from Reference Group partners will be sought as appropriate.
 * A recommendation will be made to the Project Board by the Delivery Group to change project elements based in this assessment.

Independent evaluator
We need to ensure that the project success is properly and independently assessed by an independent evaluator. This is for the following reasons:
 * The use of an independent evaluation consultant was agreed by project core partners and specified in the bid to the Big Lottery Fund UK. There will be an expectation of this by all partners who have reviewed the application and the plans which was published on this Wiki at that time
 * An independent evaluation will allow a more objective evaluation of the success of the project than project core partners interviewing or getting feedback themselves from project beneficiaries
 * A single independent evaluation rather than separate evaluations in each UK home nation will allow us to stay within budget and capture differences and common experiences across the nations.
 * We will use the process towards recruiting the independent evaluator to generate a conversation between existing and prospective delivery partners on our evaluation approach. In particular, we will seek advice on how to get a clearer line of sight between the project output delivery and progress towards outcomes

We will openly advertise and recruit an evaluation partner (consultant) to deliver the independent evaluation, and another evaluation partner to deliver the equalities evaluation. The project's Delivery Group's recommendation to the project board will be to use a single evaluator for both the main and equalities evaluation of the project. This is to recognise the importance of embedding equalities in the overall evaluation approach. It would also fit into the outcome Empowered engaged citizens, leveraging their knowledge of SDGs, emphasising the focus on engaging people whose voices are least heard.

In order to inform the process used to recruit the evaluation consultant, we will draw on the pilot approach we used to recruit the equalities trainer for the project team. This involved an open approach, where potential delivery partners self-identified their interest on this Wiki on the Partners page. They were subsequently added to the Opengovernment Pioneers Project Partners forum and were encourages to comment on the overall process and brief openly to other partners before making their pitch.

Access the evaluator tender brief here.

The evaluation will involve interviews with project partners, project participants and government, and will be supported by the data collected as part of the project's monitoring activities and a discussion with the project board and input from project's forums around evaluation priorities. The evaluator will be expected to make recommendations about how to continue and develop the engagement of citizens and civil society in using open government to secure progress towards SDGs.

Self-evaluation
It is important that this project does not rely completely on the independent evaluation to understand the difference it is making.

Therefore:
 * 1) The UK Conference will include a workshop component to facilitate stakeholder analysis, where project participants collectively deliberate around what works and how the project is making a difference
 * 2) The Activity sprint on international learning in Summer 2018 will also provide international level learning about what works and external perspective on how the project is making a difference based on International webinars with Pioneer civil society cohort
 * 3) Crucially, the independent evaluation will be accompanied by six monthly self-evaluations by each core partner for each home nation which will be captured and reported through this Wiki.

How will we ensure a strong equalities focus?
This Open Government Pioneers Project is primarily about raising awareness and supporting the implementation of the SDGs through Open Government principles and methods, with an emphasis on reaching groups who have difficulty in having their voices heard.

We will use the following approaches to ensure an equalities focus is embedded at all stage and all activities of the project:
 * Prioritising budget and engagement with partners and groups supporting those without a strong voice in opengovernment and SDG discussions
 * Securing strong buy-in to this project from a variety of equalities-focused civil society organisations
 * Ensuring all project activities seek wider engagement than the 'usual suspects' - e.g. using livestreaming, non-jargon language, accessibility checks for digital platforms.
 * Ensuring all project activities are as open and accessible as possible at the design stage to people who face additional barriers to engaging the decisions that affect them.
 * We will learn from the existing evidence base available to us, starting with the research and reports referenced at Analysis and theory of change

Project Impact
Updated Jan 2018

Overall impact assessment

 * The project has facilitated a significant renewal of interest in the Sustainable Development Goals which had been flagging since the launch of the SDGs in late 2015.
 * The project has also increased interest and visibility in the Open Government Partnership and action plans amongst the policy communities in Scotland and Wales.
 * The project has secured a greater appreciation by government and civil society participants across the UK and has engaged a broader constituency of interests in the open government partnership of the link between OGP and SDGs in Northern Ireland and England.
 * The project is beginning to broaden awareness and interest in OGP and SDGs in the broader public and media. There has been a steady increase in engagement with the structures and planned engagements that are now in place to progress this over the previous six months of the project.
 * The project has for the first time brought engagement around open government and SDGs to the attention of young people, and people engaged with minority and disability groups.

Engagement statistics
Stats are cumulative totals, unless otherwise specified