Project Evaluation

Programme & Project Evaluation and Assessment focuses on three main project stages defined below. At each stage we focus on identifying how projects can deliver value for money whilst delivering intended business case benefits. An important aspect of the service is that information from past projects is translated into lessons learned to improve current and future programmes.

ASSAy (Assessments Structured Approach)

An ASSAy is a structured review of a project or programme. We make use of tools and techniques appropriate to the project and consultants with direct experience of the field under review.

Our ASSAy toolkit can be applied in the planning, delivery and post project phases:

A. Pre Project Evaluation: ASSAy can be used to assess if a project or intervention is worthwhile in a broad sense i.e. will it have the social and economic benefit expected. ASSAy can establish if the "business case" is justified, achievable and, given the planned approach to delivery, will deliver the intended benefits.

B. Project in Progress Evaluation: ASSAy can be used to assess and evaluate a project that is currently underway, its impact to date, providing an assessment of the project's likelihood of achieving its original objectives.   Lessons learnt may be applied to ensure that the project meets its original goals or seeks to meet modified goals. Assessment may support a decision to continue with delivery or to curtail delivery at an earlier stage on failing projects where recovery is no longer achievable within project budgets or timescales.

C. Post Project Evaluation:   ASSAy can be used to evaluate projects for their actual economic and social benefit as against the original objectives set for the project.

Focussed and Disciplined Approach

A typical approach to evaluation & assessment involves a combination of Financial Analysis, Economic Benefit Assessment, Project Document Review and Stakeholder Interviews and/or Surveys.

When working with the public sector, AP Benson's evaluations and reviews follow and comply with the Treasury's Green Book on Appraisal and Evaluation, the Guidance on Impact Evaluation Frameworks for Regional Development Agencies and also use a variety of technical approaches including statistical sampling and econometric tools.

The detailed approach for an ASSAy review is tailored to the needs of each client, but is based on the following broadly outlined plan:

Goals 

Define the original goals set out for the programme/project

Measures 

Establish measures of performance that can be used to evaluate the programme/project

Design 

Design an evaluation approach from a collection of tools and techniques within ASSAy that assess the effectiveness of the programme/project in meeting its goals

Collection

Select data collection methods from the Evaluation and Assessment Techniques and Frameworks toolkit ("Assay Fundamentals") that target the key data areas involved in this assessment.

Process 

Evaluate data gathered during this assessment and model using ASSAy Fundamentals to identify key trends, issues and outputs

Conclusion 

Use the data and evaluations to conclude on the benefits (qualitative and quantitative), outcomes and costs of the programme/project and its true utility to the funder including reporting results.

ASSAY Fundamentals (Techniques and Frameworks)

The ASSAy Fundamentals approach is both comprehensive and flexible using techniques that are designed to meet the needs of a wide range and complexity of projects and programmes. ASSAy Fundamentals covers:

Documentation

Review of documents, (including but not limited to):

a. Original market appraisal documents
b. Funding agreements
c. Research reports into original need, opportunity, failure etc
d. Programme strategy and approach
e. Programme monitoring reports, feedback & financial & other outputs

Financial Impact Analysis

a. Review of performance against original budget
b. Review & analysis of output and monitoring data
c. Review of key programme results

Stakeholder/Beneficiary Feedback

Interviews 
a. Surveys of beneficiaries
b. Interviews with project and programme sponsors, programme managers, support staff and other participants

Stakeholder Consultation 
a. Workshops, focus groups & consultations with main stakeholders
b. Consultations with key individuals

Beneficiary Surveys 
Consultations, interviews and surveys with beneficiaries, intended beneficiaries, potential beneficiaries and unintended beneficiaries.

Economic Benefit Analysis

a. Analysis of economic data, GVA (gross value added), job creation and protection, land regeneration, social inclusion etc
b. Updates to original assessment of economic benefits in light of the experience to date

Logic Profiling 

a. Development of logical approach to programme implementation
b. System mapping and analysis

Benchmarking 

Benchmarking against similar programmes and projects in the UK and overseas.