Five months into a new leadership program for first‐time managers, an HR team is breathing easy.
The launch and follow‐up sessions were well‐received, with enthusiasm spilling over into use of mobile and self‐paced online courses. A second cohort is slated to begin soon.
Then, the discussion turns to measuring impact — finding ways to present the initial good news and demonstrate value over time.
Unfortunately, this team has missed the best window for getting an evaluation started. The most useful and effective evaluations are those that are designed at the beginning — parallel to designing your program or initiative — and not tacked on later.
“You can never go back to the beginning,” says Tracy Patterson, director of CCL’s Evaluation Services Center. “It is much better to think carefully about evaluation at the onset than try to work from a retrospective lens.”
CCL addresses evaluation early on in conversations with HR leaders looking to create or expand their leadership development efforts. Evaluation is threaded through the discovery, design, and implementation process. Based on experience, research, and best practices in the field, CCL suggests that clients take steps in the earliest stages of program planning and design:
1. Engage evaluation stakeholders. The primary purpose of evaluation will stem from the needs of stakeholders, so identifying who they are and learning about their expectations, concerns and perceptions is important. Stakeholders could be those who are involved directly — participants or delivery team — as well as those who could invest in the initiative or are affected by it.
Then, determine whether all stakeholders are in agreement about the initiative and aligned in terms of outcomes or intended impact. Without this step, you miss an opportunity to plan for and collect valuable information. When you put the effort in at the beginning, you can increase the likelihood that the results will be both relevant and helpful.
2. Connect with the initiative design. Evaluators need a solid understanding of the organizational challenges driving the need for leadership development and who the initiative targets and their leadership needs. Then, the more difficult part: figuring out the specific measurable objectives of an initiative.
Evaluation design is based on knowing the connection between each program element and what it is trying to achieve. What knowledge and skills is the program designed to develop? What actions or behaviors reflect those desired lessons learned? How will we know learning has occurred? What is the definition of success? What degree of improvement is expected?
When this step is taken early on, it also can strengthen the design of the initiative. It can reveal details or disagreements that had been overlooked as well as unnecessary elements or gaps in the program design.
3. Focus the evaluation. Narrowing the scope to what is most critical to measure is the next step. It is easy to end up collecting more information than you need or miss collecting something that ends up being of great interest to your stakeholders if you breeze through this part of the process.
Plus, you can also avoid conflict down the road if you can simplify and summarize the evaluation at this point so stakeholders get another chance to understand and buy into what you are doing.
4. Confirm resources and expectations. Are resources available to match expectations? Often the two are not aligned. Taking a reality check at this point allows time, people and budget to be realistically factored into the evaluation process.
5. Determine data collection methods. Surveys, interviews, focus groups, observation and documentation are common data‐collection methods and each has advantages. Different combinations will work best, depending on what data will be gathered, from whom, how and when. This is also the time to talk through the burdens placed on the people providing data, address issues of data security and confidentially, and adjust the timeframe for data analysis and interpretation based on the final design.
6. Plan for communication of results. There are many ways to share evaluation information and often several audiences. Determine the best way to communicate upfront, so it can be factored into the planning. The goal is to put useful data in the hands of stakeholders when they need it and in a way that encourages discussion and action.
Evaluating leadership development is always a complex challenge. All leadership development outcomes are difficult to measure. Taking these 6 steps in the beginning gives you the best opportunity for success.