Business Evaluation Strategy

Relating Training to Business Performance: The Case for a Business Evaluation Strategy

William J. Tarnacki II

Eileen R. Banchoff

For many years organizations have been professing that the key to a truly sustainable competitive advantage is an engaged and talented workforce. “Managers are fond of the maxim `Employees are our most important asset.’ Yet beneath the rhetoric, too many executives still regard—and manage—employees as costs. That’s dangerous because, for many companies, people are the only source of long-term competitive advantage” (Bassi & McMurrer, 2007, p. 115). This professed realization has pushed organizations to establish training programs (and even corporate universities) that provide opportunities for employees to develop skills and competencies related to their existing (or sometimes future) roles in the organization.

These training programs have evolved tremendously over time, becoming much more sophisticated and oriented toward creating a well-rounded workforce. Unfortunately, these training and development (T+D) efforts have not kept pace with the changing demands of business. In fact, T+D departments have evolved to be separate entities from the operations of the business, basically managing a repository of training options versus partnering with business and operational leaders to customize solutions based on evolving business needs. Recent attempts to broaden T+D efforts to encompass performance improvement (PI) are a much needed, long overdue, uphill climb. Unfortunately again, today’s business leaders are looking to their human resources (HR), PI, and T+D colleagues to operate at a much higher level and to develop a new language around the expectations and the demands of the business.

If our field of practice is changing (albeit slowly), it stands to reason that the traditional evaluation methods (see Table 4.1) we use to measure transfer from our training programs (skills and knowledge) are also too narrow to measure business results. These evaluation methods are being taught and even trained in the context of another narrow model—the ADDIE instructional design model (analyze, design, develop, implement, and evaluate). Evaluation strategies and tools, based on T+D and ADDIE, limit our ability to understand the overall business model and associated metrics in order to offer robust, impactful, meaningful evaluation results that help manage the business.

Table 4.1 Traditional Evaluation Types

To help build the case for a more business-focused evaluation strategy, this chapter will trace the real-life professional journey of Joseph Williams (fictional name) as he matures from a young, academically prepared ADDIE advocate to a sophisticated human resources strategist and business partner. Mapping Joseph’s experiential journey demonstrates how his perspectives changed to align his training results with business results without completely revamping the long-trusted tools and methods he learned in graduate school.

STUDY OF PERSEPCTIVES

The Academic Perspective

It was May 1997 and Joseph Williams had just graduated with a master’s degree in instructional technology from a very reputable urban university. Basking in the glory of his new degree, Joseph was excited about applying all the wonderful concepts and practices he had just spent two plus years learning. He left the graduation ceremonies eager to employ ADDIE and its corresponding evaluation techniques to become the hero he knew some organizations were desperately waiting for.

While completing his coursework in the evenings, Joseph spent his days as a foreman and quality specialist on the shop floor of a small manufacturing organization. In this academic preparation period, he worked diligently to incorporate several new skills and techniques into his day-to-day manufacturing activities. But he soon ran into the brick wall of lack of interest and engagement and found that this organization was just too small and had too few resources to “hear his voice.” In order to begin applying his new wealth of knowledge and expertise, he decided he had to move on to a bigger and more strategic enterprise.

The Novice Perspective

Soon after graduation (in 1997), Joseph jumped at an opportunity to work in the training and development department of a large automotive company’s finance division. From his first day in the new white-collar environment, Joseph was sure he could make a difference by analyzing, designing, and delivering training programs that would have an immediate, positive impact on this global finance business.

Alas, it did not take long for Joseph to realize that the full application of the instructional design model was not part of the expectations of his new job. Instead, management directed him to use his ISD background to oversee projects that addressed needs they had identified during an analysis performed five years earlier! He also soon discovered that his deliverables were predetermined (seminar courses in a standard format) and that the implementers (the project trainers) would have minimal involvement in the design, development, and evaluation processes. Joseph held the title of “Instructional Designer,” but he soon acquiesced to being just a project manager.

The “siloed” nature of the firm’s T+D structure was perplexing to Joseph, but there were several other issues that also seemed even more counter-intuitive to this fledging ISDer.

· First, Joseph could not understand why design and development activities were focused on “old” information, especially when the organization was in the process of a major restructuring. For some time now, many department customers were indicating that much of the proposed content was out-of-date due to recent technological and organizational improvements.

· Second, he did not know why T+D operated as its own function, was not integrated with the operational areas of the business, and was only minimally assimilated with the broader human resources function.

· Third, Joseph struggled with the reality that there were stand-alone needs analysis, design, development, and delivery functions, but evaluation was not its own entity. And worse yet, evaluation was not conducted with any depth besides basic Levels 1 and 2 or summative evaluation (even formative evaluation was minimally applied).

Table 4.2 illustrates Joseph’s foundational perspectives—how different types of evaluation were taught and aligned during Joseph’s (and most professionals’) graduate preparation.

Table 4.2 Application of Traditional Evaluation Types

A Second Academic Perspective

Joseph knew the business had problems, but his formal ISD preparation was not robust enough to help him fix them, so in 1998 he set off to get a master’s in business administration (MBA). He hoped this additional degree would give him a better understanding of how a business operates and how he could better apply his instructional systems design background to those operations. As he progressed through the finance-focused graduate program, things became much clearer in terms of how identified gaps have a negative effect on organizational results. At the turn of the century, the concept of human performance technology (HPT) was emerging in his ISD world, providing a new perspective on how to address performance gaps with instructional and non-instructional interventions. But, again, even with HPT and a deeper understanding of overall business concepts and functions, Joseph was still struggling. He was discovering that much of what happened was not a result of “pulling” business priorities to create a true linkage to results, but rather a “pushing” of functional priorities based on limited, higher-level evaluation data and a desire to justify turfs and budgets. Was this how all businesses operated? Was HR and, more specifically, T+D, simply a budgeted function of the business or could it be a strategic, value-generating consulting partner? To find out, Joseph decided to move up in his career.

The Generalist Perspective

While continuing to build his business acumen through the MBA program, Joseph ventured into an HR generalist role in 2000 with the same large automotive manufacturer, serving both the finance arm and the parent manufacturing organization. At last, Joseph was positioned to get more exposure to the operational workings of the business, and he was able to work closely with the business leaders to deploy solutions to address problems to close performance gaps. What a difference from his previous position in which he was a specialist working for the T+D function and evaluation strategy and execution were performed in a vacuum.

Joseph was getting used to being a human resources generalist, but he still knew he was not truly operating at the strategic level and really utilizing all the skills and knowledge he had learned through his new MBA and previous ISD programs. As a result, Joseph changed jobs once again in 2003 and accepted a role with a large, global, diversified industrial organization. From his first day on this new job, he felt he was really part of the leadership team and a key decision-maker and influencer in the organization (at least in the plant location in which he led the human resources function). Consequently, he learned a new way of looking at evaluation. Joseph began to understand that evaluation, as he learned it (see Table 4.2), was a very narrow application of the term, especially in relating training or general HPT interventions to business performance.

The Experienced Perspective

Joseph spent more than two years as a plant human resources manager with one diversified industrial organization and then an additional two plus years with another as a division human resources manager. Both experiences proved invaluable. Joseph learned that evaluation started with an established baseline. In other words, having a solid evaluation strategy was contingent on (1) knowing what was important to the business, (2) knowing how the business measured what was important, and (3) knowing how the organization or function was performing against specified metrics.

Once he began aligning current performance to strategic business goals and metrics, Joseph knew he could develop a solid evaluation strategy that would measure the success of the targeted interventions. At long last . . . this was something Joseph did not gain exposure to in his previous assignment as an instructional designer. His previous training and experience in evaluation was narrowly focused on being able to demonstrate that learning occurred, rather than focusing on the more meaningful metric of how the learners’ enhanced job performance affected the business overall. He also was able to recognize that he had been treating training as an intervention focused solely on accomplishing specific department metrics instead of concentrating on something that was devised in partnership with the internal business customer (process owner). He concluded that this was in large part due to the segregated nature of the T+D department from the actual operations of the business.

Joseph soon learned that metrics started at the top of the house and cascaded down through the organization so that each facility and function could be aligned according to what was important. This type of top-down alignment ensured that evaluation was focused on the quantitative and qualitative priorities, as expressed in the organizations’ business models. Evaluation became more than just focused on one particular aspect of a cycle (for example, formative evaluation as focused on certain portions of the ISD model). It was really ensuring that any evaluation eventually tied back to the overall business metrics no matter what the focus of measurement was. As a member of the operational leadership teams for the two large diversified industrials, Joseph learned a better way to integrate his business and instructional technology educational experiences (2003 through 2007).

Joseph found, however, that the evolution of business toward talent management, employee engagement, and business consultation by the human resources function was much broader than where many HR, T&D, and PI professionals were focused. He also observed that evaluation became an overused tool, overcomplicating the nature of business because everything became a target for evaluation, even if there really was not a valid business reason or linkage to business results. Evaluation was performed for evaluation’s sake instead of determining the true value derived from performing an evaluation by relating interventions such as training, recognition, succession, and recruiting to overall business performance. This prompted him to consider a new opportunity that would allow him to take everything he had learned, good and bad, and demonstrate the type of human resources focus and alignment that would close all the gaps in the evolution of the function he had observed.

The Business Perspective

Joseph’s frustrations with continuing to perform wasted activities and employ non-value-added measurement systems drove him to accept a new position as director of human resources and organization development at a much smaller, private organization (in 2008). Having spent most of his career in manufacturing, Joseph was excited about getting into a different type of organization—the information publishing and services industry. He was thrilled with the proposition of owning the development of the human performance improvement and development strategy for the organization, aptly called the “talent management” strategy. This was finally his chance to take all the great practices (perspectives) he had been exposed to and “lean out” those things that he considered waste, to produce what was bound to be considered an industry benchmark talent management system—at least in his mind!

Immediately after accepting the new position, he realized that the organization (a business that was growing very quickly) had nothing in place to build on. The non-existent program aspects did not pose a serious issue for him. But he did find the non-existent systems infrastructure—so critical in linking all aspects of a talent management and an employee engagement strategy together—to be particularly stressful.

Business Model. Beginning with this context, Joseph knew the business and its respective leaders were not looking for a narrowly focused T+D program, or a time-consuming HPT gap assessment, to determine immediate, critical solutions. They were looking for a full-scope talent management program that aligned to the business’s philosophy, values, vision, and mission, as well as to their long-term organizational strategy. And they expected this to be built very quickly, with minimal resources and a focus on achieving their high-level business metrics. Joseph knew that training and development was a component of this, as well as some gap assessment and closure, but he also knew that they were small components in the grander scheme of things. He, therefore, set out to understand the business, which started with a grasp of the organization’s business model, as seen in Figure 4.1.

This metrics-based, three-tiered way of thinking was critical to help Joseph understand exactly how corporate leadership defined the culture and essence of the company, how they envisioned the future of the organization, and how they measured success. Joseph knew that this was the basis for each functional strategy (including his HR operations) and the related evaluation strategy of all interventions or solutions that his group would have to deliver. Once Joseph established a foundational understanding of the business, he embarked on getting to know each function and location.

Figure 4.1 Business Model.

HR Strategic Model. Through his extensive U.S. travels, Joseph came to understand that some specific training programs were important to the firm’s overall talent management program. But T+D was still a microcosm of the employee engagement strategy the organization needed in order to continue to prosper and meet its long-term objectives. His vision of a truly strategic HR partnership with the business began to take shape and he was able to see how the three pieces fit together in his mental HR strategic model (see Figure 4.2) to culminate in a high-performance culture and organization.

Figure 4.2 Strategic HR Model.

· Engagement and satisfaction would have to address the environment in which people work and interact and needed to include communication, culture and change management, reward and recognition, and community, team, and employee relations (see Figure 4.3).

· Talent management is integrated strategies to increase workforce productivity through having the right people (capacity and attitude) with the right skills and knowledge to meet current and future business needs. The integrated cycle had to be a continuous flow of finding talent inside and outside the organization and continuously cultivating that talent, and it needed to include assessment, identification, development, acquisition, and alignment (see Figure 4.4).

· Functional excellence would then become the focus on data-driven continuous improvement across the enterprise and needed to include lean/six sigma, business consulting, coaching, and competency modeling.

Figure 4.3 Employee Engagement Model.

Figure 4.4 Talent Management Model..

The Strategic HR Model (Figure 4.2) allowed Joseph to demonstrate to the organization that managing talent effectively was at the core of ensuring an engaged workforce. Following this strategy would allow each function to perform at its very best to drive improved business performance. Together these models helped Joseph visualize and enforce with the organization that training and development are important but are minute in the context of how the organization benefits from its human capital.

Joseph reflected on how he used to “push” training programs based on the internal demands of the department versus the ever-changing needs of the business. He cringed to think how many businesses used to (and still do) create giant repositories of training to try to cover any potential employee need, rather than only producing what was actually critical to achieve true success. He finally found himself in a position to change these outdated practices and satisfy the organization’s hunger for customized solutions. If, by starting with the business’s overall reason for existing, he could work with individuals and managers on solutions that always linked back to those reasons. Joseph came to understand that evaluation was not just about figuring out whether people were learning, or were happy, or whether an intervention produced a return on investment (justifying financial investments versus showing whether solutions were working and serving as a means for continuous evolution and improvement). He now knew that he had to first determine what aspects of the model needed to be measured and then focus on how each piece of the puzzle was contributing to a pre-defined desired outcome, all starting with the organization’s business model.

Strategic Business Model. The following systematic breakdown is how Joseph approached evaluation as a tool for his new human resources business strategy (see Figure 4.5):

1 Start with a comprehensive understanding of the overall business model.

2 Align the employee engagement program to the business model.

3 Assess the major components of the employee engagement program to establish a baseline to measure against (performance to metrics and cultural gaps).

4 Build a plan for talent management that capitalizes on the employee engagement program and linkages to the business model.

5 Assess the culture and environment to establish a baseline to measure against (performance to metrics and talent gaps).

6 Develop the gap closure strategy utilizing basic PI principles and practices.

7 Identify the evaluation needs and measurement tools based on business-defined desired outcomes.

8 Evaluate (and improve):

· External sources of support and solutions for appropriateness

· Processes for internally designed and developed solutions

· Employee and manager attitudes toward solutions, culture, organizational strategy, and level of engagement

· Knowledge transfer related to personnel development solutions

· Job performance and individual and team capability and capacity

· Effects of solutions on functional and organizational performance

· Perpetual alignment of solutions to achieving the business model value proposition

“