[.ca] Return on Investment in Training and Performance ... (ISBN 0750676019)

This book is a GREAT return on your investment:
Dr. Phillips has simplified a very complex concept. His step-by-step recommendations to conducting ROI studies are clear and concise. However, caveat emptor! He simplifies ROI so well that it seems relatively easy; but beware: there are great hurdles to leap when conducting ROI studies. While Phillips does cover some of the most serious obstacles one may face, such as getting management buy-in, isolating training effects, and handling soft data, conducting a ROI study is by no means a cake walk. For instance, ROI is more than a fifth level of evaluation after Kirkpatrick's four. It should be conducted at all levels of evaluation, which Phillips does suggest. Furthermore, ROI, or any evaluation effort for that matter, should not be viewed as merely a summative attempt; it should be conducted in an ongoing formative manner. Every project should have an evaluation component that parallels each task through the lifecycle of the project. Additionally, in my opinion, Dr. Phillips' conservative approach to ROI is the greatest selling point. He accounts for error in all his measurements. For instance, when collecting self-report data, he has the respondent allocate a confidence weighting to their estimates. This confidence value weights the response while taking into account error. Furthermore, when calculating values for hard data or converting soft data (i.e., work habits and attitudes) to monetary benefits, he offers formulas that result in a range and suggests that the lowest, most conservative value is reported. Converting soft data to monetary benefits can be painstaking, but Phillips very eloquently addresses the conversion. By obtaining estimates from stakeholders (with a confidence weighting, of course), soft data can contribute to the overall calculation of return on investment. Therefore, your final conservative monetary return is not only based on hard data, but it is based on the less tangible elements of your organization as well. If Phillips' approach to calculating ROI is too quantitative or laborious for your organization, you should consider calculating the return on expectations. Return on expectations can be assessed via a concept mapping (a multidimensional scaling approach) technique derived by Dr. Bill Trochium at Cornell University. Visit www.conceptsystems.com to review his technique. With Dr. Trochium's visual approach, you have stakeholders (i.e., stockholders, executive management, clients, employees, etc.) set expectations for your organization and then assess the alignment between set expectations and the actual performance of end groups. The result is a graphic pattern match that is easily interpreted and empowers decision-makers at all levels. Concept mapping has some of the same problems inherent in ROI, such as obtaining management buy-in. However, it is much easier to conduct than an ROI study, is based on a sound measurement techniques, and produces graphical results that, when considered collectively, illustrate the expected bang for your organization's buck

A very useful guide to first timers in the area of ROI:
The formula Phillips uses is a good one and he illustrates his model with many practical examples. There is a particularly good chapter on isolating the effects of training and this answers many of the objections raised to this sort of model where dollar amounts are calculated. The problem with the model is that it is mainly retrospective which means that you are closing the stable door after the horse has bolted in some cases. Having said that, it does deal with topics such as the collection of post program data in a thorough and clear way. If you are interested in intangible benefits of training then there is a chapter which deals with this topic. I would recommend that you also read the ASTD publication which Jack Phillips edited called 'Measuring Return On Investment'. Here there are 17 case studies which demonstrate how you can do ROI studies in the real world. Both these books are vital for anybody interested in the whole area of ROI on investment.

A good RETURN ON YOUR INVESTMENT!:
Dr. Phillips has simplified a very complex concept. His step-by-step recommendations to conducting ROI studies are clear and concise. However, caveat emptor! He simplifies ROI so well that it seems relatively easy; but beware: there are great hurdles to leap when conducting ROI studies. While Phillips does cover some of the most serious obstacles one may face, such as getting management buy-in, isolating training effects, and handling soft data, conducting a ROI study is by no means a cake walk. For instance, ROI is more than a fifth level of evaluation after Kirkpatrick's four. It should be conducted at all levels of evaluation, which Phillips does suggest. Furthermore, ROI, or any evaluation effort for that matter, should not be viewed as merely a summative attempt; it should be conducted in an ongoing formative manner. Every project should have an evaluation component that parallels each task through the lifecycle of the project. Additionally, in my opinion, Dr. Phillips' conservative approach to ROI is the greatest selling point. He accounts for error in all his measurements. For instance, when collecting self-report data, he has the respondent allocate a confidence weighting to their estimates. This confidence value weights the response while taking into account error. Furthermore, when calculating values for hard data or converting soft data (i.e., work habits and attitudes) to monetary benefits, he offers formulas that result in a range and suggests that the lowest, most conservative value is reported. Converting soft data to monetary benefits can be painstaking, but Phillips very eloquently addresses the conversion. By obtaining estimates from stakeholders (with a confidence weighting, of course), soft data can contribute to the overall calculation of return on investment. Therefore, your final conservative monetary return is not only based on hard data, but it is based on the less tangible elements of your organization as well. If Phillips' approach to calculating ROI is too quantitative or laborious for your organization, you should consider calculating the return on expectations. Return on expectations can be assessed via a concept mapping (a multidimensional scaling approach) technique derived by Dr. Bill Trochium at Cornell University. ... With Dr. Trochium's visual approach, you have stakeholders (i.e., stockholders, executive management, clients, employees, etc.) set expectations for your organization and then assess the alignment between set expectations and the actual performance of end groups. The result is a graphic pattern match that is easily interpreted and empowers decision-makers at all levels. Concept mapping has some of the same problems inherent in ROI, such as obtaining management buy-in. However, it is much easier to conduct than an ROI study, is based on a sound measurement techniques, and produces graphical results that, when considered collectively, illustrate the expected bang for your organization's buck