Gina Abudi recently posted a worthwhile blog entitled “Using ROI to Evaluate Project Management Training” on ProjectSmart.CO.UK. While the case study she shares involves a law firm, the key points of the initiative apply to those in any industry, non-profit or government organization. As a trainer, curriculum developer and someone who often works with managers who are interested in improving the project management skills of their employees and/or colleagues, here are the key elements that I noted in her case study.
1) Objectives for the Training–The client and the service provider took time and worked together to define exactly what was wanted as a result of the training and how progress toward those objectives was to be measured. At Schedule Associates we like to ask the training sponsor, “How do you want the participants to behave differently as a result of the class?” While no training class alone can change behavior, a well-designed course in conjunction with the appropriate pre- and post-class activities can go a long way toward affecting change.
In addition, we define different levels of learning objectives, roughly aligned with the first three skill levels in Bloom’s (1) cognitive domain.
- “Awareness” indicates that the attendee is cognizant of the issue upon completion of the course, but does not have a thorough understanding of it or the ability to apply the corresponding techniques.
- “Understanding” indicates that the attendee could explain the topic to another individual in a manner that suggests that they could utilize the information in the workplace and apply it if necessary.
- “Application” indicates that the attendee has received hands-on practice with the technique/issue in the course and could perform that work in a hands-on fashion in the workplace following the course.
2) Client Involvement–We are often surprised at how difficult it is to get the training sponsor to spend time with us defining what it is they would like to achieve as a result of the class. Sometimes this may be due to the fact that the training is being coordinated by someone other than the true sponsor, an administrative assistant for example, while other times the priority just isn’t there. This is particularly true for small one-time training efforts that are well below the scale of what Gina describes in her post.
Nonetheless, without sponsor involvement is is unlikely that the efforts to support the learning outside of the classroom will take place and therefore the ROI will be unlikely to be achieved.
3) Holistic View of the Learning Process–The client and the service provider identified actions to be taken outside of the classroom in order to support, encourage, and measure the learning. The action plan assembled and tracked by participants is a great example of this. When education is viewed as only what happens within the training room (or in the virtual classroom online) it is likely to fall well short of the desired ROI goals. Pre-course assignments, action plans, subsequent working group sessions to apply what was learned in the classroom, and other such activities significantly improve the chances of retention, application, and a real return on investment.
4) Appropriate Time & Resources–We are often asked to provide training on Microsoft Project, a complex software application for developing and managing project schedules, staffing plans and spending plans, in a one day class. We consistently tell the requestors that, except in unique situations, we don’t provide such training in less than two days because you simply can’t learn what’s necessary in order to use that tool in the real world in under two days. In fact, we find that when given a choice, the overwhelming percentage of students who take our open-enrollment classes on Project sign up for three rather than two-day sessions. Longer sessions provide a more relaxed pace, more extensive coverage of the material and, most importantly, more hands-on time which is so vital for the learning process.
The client Gina speaks of allotted three days for the project management training and didn’t try to squeeze it into four hours or some other short timeframe which wouldn’t have allowed for the accomplishment of the learning objectives.
5) Support for the Supporters–The case study alludes to the development of a half-day program to educate managing partners of the firm on how to support project initiatives. We’ve all seen project managers and others who, despite their best efforts, were unable to be successful in their environment because of the organization above them. The managers of the organization have a large influence on the internal culture, the priorities of projects and other work, and the expectations for quality, customer satisfaction, and other metrics. If management doesn’t understand how to support and prioritize what class attendees learned and the importance of doing so then the learning is likely to wither on the vine.
6) Time & Resource Commitment–Establishing learning objectives, getting buy-in to them, evaluating training results and assessing the data takes a significant amount of time and commitment of resources. Without that commitment the ROI won’t be known, but also almost certainly won’t be achieved.
Of course, as with any such effort there is the danger that the Hawthorne Effect in which performance improves simply because of the attention that’s being paid to the subjects is the cause of some of the increased performance and that the ROI from training will diminish over time as performance reverts to earlier levels. This possibility makes it all the more important that concrete changes be put in place as a result of the training (i.e. different methodologies, templates, etc.) and that management supports the new processes and results of the training on an ongoing basis.
Finally, it’s interesting to note that managers at Microsoft responsible for the development of Microsoft Project cite a 50% reduction in the time to learn Microsoft Project as a result of good quality training(2), a result that squares nicely with Abudi’s finding of a 200%+ return on investment.
(1) Benjamin Bloom, editor, M. D. Englehart, E. J. Furst, W. H. Hill, David Krathwohl, The Taxonomy of Educational Objectives, The Classification of Educational Goals, Handbook I: Cognitive Domain, 1956.
(2) Heard first-hand at Microsoft’s Microsoft Office Project conference, Seattle, WA, 2008.