How leadership development is changing – and its effect on measuring impact
High-performing organisations have a strong commitment to developing the skills and capabilities of their staff, especially those in positions of leadership[i]. However, there is increasing concern in many organisations about the efficacy of leadership, with 6 out of 10 organisations considering that they have failed to implement change because of poor leadership capability[ii].
For some years, organisations have been addressing this failure through leadership development programs. Yet, for many of these programs, insufficient evidence – both qualitative and quantitative – exists to determine whether the programs have been effective in addressing this failure of leadership [iii]. As such, organisations require a strategy by which leadership development can be measured to ensure value-for-money and organisational impact.
Challenges for modern leadership
21st century organisations operate in an increasingly volatile and unpredictable environment with intractable challenges for which there is no single ‘right’ solution[iv]. In this context, leaders are faced with a work environment characterised by high levels of complexity and change requiring exceptional levels of personal maturity, people leadership and change agility[v].
At the same time there are increasing expectations and demands of leaders in business and in the broader political and social context. Where are the good leaders to come from? How are they to be educated and supported in their work?
These challenges have driven changes not just in the most important capabilities required of leaders but also in the very foundations of what is required to develop leaders[vi].
Addressing these challenges: developing leaders and leadership
To address these challenges, leaders need to be exposed to the new skills and expectations of leaders, with development focusing upon:
- strengthening self-insight and emotional regulation as a leadership foundation;
- developing capacity to monitor and challenge their own and others’ thinking and behaviours;
- building capacity to deal with ambiguity, the complex environment and uncertainty;
- developing the relationship skills required to influence and collaborate effectively;
- ensuring delivery of enterprise-wide culture and performance across functions; and
- creating a cohort of people to support learning and further development[vii].
Strategies for developing these skills need to ensure that change is not only individual – for the benefit of leaders – but also organisational – for the benefit of shareholders and stakeholders[viii]. Leadership development can no longer be simply described as an individual characteristic or capability but rather as a collective or group activity using a relational, strategic global and complex social dynamic[ix].
Increasingly, another responsibility for organisations is to measure the impact of leadership development programs, identifying whether this development is contributing value to the organisation.
Measuring impact: moving beyond ‘happy sheets’
An observer of the leadership development industry has suggested that, “a dirty secret about the leadership industry is that its offerings are rarely evaluated” [x].
For far too many leadership development programs, participants are surveyed on completion of individual programs, using so-called ‘happy’ or ‘smile’ sheet surveys, in the belief that these will measure results and impact. However, surveying participants on the final day of a program does not provide organisations with the insights and data which connects development activities to real value and organisational impact.
Even where a structured evaluation methodology is used, this is often focused only on the changes made by the individual participant – not their impact on the organisation’s performance. And in a more complex global business environment, simple remedies for the development of leaders and for the measurement of its impact have become increasingly unviable. What is required is a strategy by which leadership development can be measured and evaluated to ensure organisational impact is achieved.
Building an impact measurement strategy
Measuring the value or results achieved from leadership development has always been seen as relatively difficult because required changes need to be made to long-held knowledge and behaviours. It has been estimated that only 3% of organisations measure the relationship between business programs and business results[xi].
Measurement of the impact of a leadership development program is an integral part of understanding its contribution to a company’s bottom line. The impact of many leadership development activities are not adequately measured against the company’s overall goals and objectives. Understanding the specific goals which an organisation is trying to achieve and how this impact will be measured is crucial.
There are four integrated steps in this process, developed by Amanda Martin who has been measuring and reporting on the impact of leadership development programs for more than 20 years.
These four steps are drawn from experience, together with a review of the academic research and program evaluation case studies. The questions, within each step of the process, support measuring the impact of leadership development beyond the level of individual participant to include results for the organisation:
- embedding measurement into the design of the development from the beginning;
- establishing program learning goals so that learning activities clearly contribute to individual and organisational change and results;
- developing a robust measurement methodology creating multiple points of evidence at the same time as recognizing the reality of organisational life; and
- reporting results to the people who have sponsored, supported and been a part of the development both celebrates the outcomes and provides accountability for their investment in the program.
Designing your impact measurement strategy
- Embed measurement into the design of the development activity
Key questions to consider during the design of a leadership program or development activity are:
- What are the metrics that will tell the organisation that the program has been successful?
- How will this program contribute to the organisation’s outcomes and strategy?
- How will the program deliver on the organisation’s HR strategy?
- Who are the key stakeholders for this impact measurement strategy?
- How does the program fit with the overall leadership capabilities required for this organisation?
- What are the assumptions underlying the design of the program and how will the design contribute to these objectives [xii]
These questions contribute to the development of robust program objectives, reflecting what the organisation is aiming to achieve, right from the early phases of the program design.
This is one of the most difficult stages of measuring impact – these questions are not easy to answer and may feel frustrating when stakeholders ‘just want to get the program going’. It’s no wonder that the American Society for Training and Development (now the Association for Talent Development), amongst others, has found little evidence that program measurement is regularly embedded into program designs[xiii].
- Establish program learning goals
At this stage, program developers should be asking:
- How will this program support participants to build their capability and to deliver on the organisation’s strategy?
- What are the most effective learning activities that will ensure these learning goals are achieved?
- How will these learning activities build on each other to deliver impact?
The temptation is to focus on the instruments, methods and tools that support development such as 360 degree feedback, executive coaching, digital learning, and specific learning models. However, bolting these together without considering how they support learning doesn’t deliver a well-designed and impactful program.
- Develop a program measurement methodology
Embedding measurement into the learning design means asking:
- What data will be needed to measure these program objectives?
- Which measurement methods will help identify the required changes in knowledge or behaviour?
- What is the cost of undertaking these measurement methods?
- What is the best timing for collecting data?
- How will this data contribute evidence of the link to the program objectives?
- How will the data be used to support iterative learning design, participant learning and engagement with program stakeholders?
- What else is happening in the organisation that may affect the data collection or results?
While being founded in the rigour of research, methods for measurement of impact also have to be balanced with the relevance of organisational life and must be pragmatic.
It’s also important to identify whether the resulting data can be integrated, analysed and reported in a timely way to tell the story of the impact of the program for key stakeholders.
- Report the results
In this stage, understanding who the program stakeholders are is crucial, as this guides reporting and insights:
- What is the data saying about the program results when measured against the program objectives and learning goals?
- Who needs to be involved in reviewing the data and in identifying insights?
- What are the key insights: what is the analysis showing or not showing about the program impact for participants, their teams, their direct line managers, and for the enterprise more broadly?
- What are the recommendations for change arising from the evaluation results and key insights?
Taking action: translating leadership development into impact
Understanding the impact of leadership development programs is crucial in an economic environment where organisations are under increasing pressure.
Learning and development programs are often the first to be cut in these circumstances yet there is significant evidence to demonstrate that investing in the ongoing development of leaders is essential to organisational success.
Understanding the impact and effectiveness of development is foundational in identifying what has been achieved and also what can be achieved, and it’s time that more than 3% of organisations use more effective approaches to measurement than “happy sheets”.
[i] Boedker, C. Vidgen, R., Meagher K., Cogin, J., Mouritsen, J., Runnalls, J.M., (2011) Leadership, culture and management practices of high performing workplaces in Australia: the high performing workplaces index. Society for Knowledge Economics
[ii] Deloitte: The Leadership Premium March 2012, p. 5
[iii] _Avolio, B. J., Walumbwa, F. O. & Weber, T. J. 2009. Leadership: current theories, research, and future directions. Annu Rev Psychol, 60, 421-49.
[iv] _O’Connell, P. K. 2013. A simplified framework for 21st century leader development. The Leadership Quarterly, Article in Press, 1-21.
[v] Allen, S. J. & Middlebrooks, A. (2014). The challenge of educating leadership expertise. Journal of Leadership Studies, 6, 84-89.
[vi] Van Velsor, E. (2008). A Complexity Perspective on Leadership Development. Complexity Leadership Part 1: Conceptual Foundations. M. Uhl-Bien and R. Marion. Charlotte, North Carolina, Information Age Publishing Inc. 1: 333-346.
[vii] Bunker, K. A., Kram, K. E., & Ting, S. (2002). The young and the clueless. Harvard Business Review, 80(12), 80–87.
[viii] Day, D. V., Fleenor, J. W., Atwater, L. E., Sturm, R. E. & Mckee, R. A. 2014. Advances in leader and leadership development: A review of 25years of research and theory. The Leadership Quarterly, 25, 63-82
[ix] Uhl-Bien, M. & Marion, R. 2008. Introduction: Complexity Leadership-A Framework for Leadership in the Twenty-First Century. In: Uhl-Bien, M. & Marion, R. (eds.) Complexity Leadership Part 1: Conceptual Foundations. Charlotte, North Carolina: Information Age Publishing Inc.
[x] Kaiser, R. B. & Curphy, G. 2013. Leadership development: the failure of an industry and the opportunity for consulting psychologists. Consulting Psychology Journal: Practice and Research, 65, 294-302. p296
[xi] McGonagill, G & Pruyn, P W. 2010 Leadership development in the US: Principles and Patterns of Best Practice, Bertelsmann Stiftung Leadership Series
[xii] Packard, T and Jones, L. 2015. An outcomes evaluation of a leadership development initiative. Journal of Management Development, Vol 34, No. 2 pp 153-168 Emerald Publishing Limited.
[xiii] Packard, T and Jones, L. 2015. An outcomes evaluation of a leadership development initiative. Journal of Management Development, Vol 34, No. 2 pp 153-168 Emerald Publishing Limited.