How to Strengthen Entrepreneurship Policy Evaluation? Getting more reliable evaluations and learning from evaluations (CIMR debates in Public Policy)
When:
—
Venue:
Online
Join the Centre for Innovation Management Research on Wednesday 4 December, 1 pm – 2.15pm, online for an hybrid lunchtime seminar on Policy Evaluation. The debate is part of the CIMR Debates and Workshops in Public Policy series.
Chair: Dr. Federica Rossi, Associate Professor of Economic Policy, Università di Modena e Reggio Emilia and Deputy Director, Birkbeck Centre for Innovation Management Research
Speakers:
-
Dr Jonathan Potter, OECD
-
Shannen Enright, Assistant Principal, Data and Evaluation Unit, Department of Enterprise, Trade and Employment, Ireland
-
Prof. Marc Cowling, Professor of Economics and Productivity, Oxford Brookes Business School, Oxford Brookes University, UK
Abstract
Learning from evaluations of government policy is critical for improving policy impacts. However, there are some important obstacles. First, much of the impact evaluation literature in the field of entrepreneurship and SME policy cannot be considered reliable, because of a lack of use of control groups. Second, even in reliable studies, it is difficult to compare policy effectiveness across the portfolio of alternative measures because of a lack of common metrics and a lack of cost effectiveness information across different evaluations. Third, governments often lack processes to integrate evaluation into policy development processes.
How can academics and policy makers improve on this situation? The OECD has produced an guidance document for governments and evaluators internationally in the form of the Framework for the Evaluation of SME and Entrepreneurship Policies and Programmes 2023. Among its key innovations are:
- Promoting a portfolio approach to evaluation of SME and entrepreneurship policies across government in order to compare effectiveness of different approaches – e.g. hard (financial) versus soft (non-financial) policy interventions. This means not just undertaking reliable evaluation across the whole government policy portfolio, but also using common metrics (sales, employment, start-ups, survival) and including expenditure information. This requires some common approaches by academic evaluators.
- Developing an Evaluation Quality Score (EQS). The first (2007) version of the Framework used David Storey’s 6 Steps to Heaven focused on nature of the control group. The 2023 version adds the EQS, which stresses additional issues like accounting for survival/non-survival in evaluations and researchers making efforts to secure policy learning from their studies.
This event will present the Framework, give the example of how the Irish government promotes reliable evaluation and evaluation learning, and debate the implications for researchers – including how to evaluate reliably, how to create a common corpus of evaluation that will enable decisions to be made about relative effectiveness of different measures, and how to work with policy makers to secure policy change from evaluation evidence.
Biographies
Chair – Federica Rossi
Dr Federica Rossi is Associate Professor in Economic Policy at Universita’ di Modena e Reggio Emilia, Italy, and Honorary Research Fellow at Birkbeck Business School. Her current research interests comprise: knowledge exchange between research and industry, including with the arts and cultural sector; digital technologies and innovation; policy evaluation particularly in relation to innovation, science and technology policy. She has collaborated with many organisations including the OECD, the National Centre for Academic and Cultural Exchange, the World Intellectual Property Organisation. She has authored numerous articles in highly ranked peer-reviewed journals. She has also published two books, edited three special issues and contributed to numerous edited volumes.
Speaker - Jonathan Potter
Jonathan Potter is Head of the Entrepreneurship Policy and Analysis Unit at the OECD Centre for Entrepreneurship, SMEs, Regions and Cities. He directs OECD work streams including country reviews of SME and entrepreneurship policy, evaluation approaches for SME and entrepreneurship policy, policies for inclusive entrepreneurship including support for youth and women, and policies for entrepreneurial ecosystems at national and regional levels. He has worked at the OECD since 1997 and holds a PhD from the University of Cambridge. He prepared the first OECD Evaluation Framework (2007) with David Storey and the second Framework (2023) with David Storey and Ondrej Dvoulety.
Contact name:
CIMR Events and Communications