Oh expert in the crystal ball, please tell all...tell all!
Questions abound in the learning and performance world. Here are a few we have been asked several times recently along with our answers. We welcome additional questions as well as comments and reactions to our responses.

To ask your question, please click on the crystal ball.

To access our Ask Harold Archives, click here.

I was wondering if you could point me in the right direction in finding a case study (or studies) that compare the learning results of Instructor-led Training (ILT) versus Web-based Training (WBT)? I am currently working on my Masters thesis in which I am comparing the two, and I am trying to find some numbers that back up my theory that ILT is more effective because of the live human interaction, etc. I remember you mentioning at a conference I attended that well-designed WBT is just as effective as ILT, but I'm still skeptical. Any resources that you could recommend would be greatly appreciated. Answer

A colleague of mine and I are working on the challenges new hires face in organizations. We were wondering when is it worthwhile to hire a star performer? And, the flip side of that, if you are a star performer, should you stay put versus move to other positions and/or organizations? Answer

You were referred to in a webinar I attended as a source in stating that only around 30% of know-how and skills contribute to improvements on performance. This being the case, what are the other factors that make up the other 70% and where may I find this reference? Is it contained in your research or in a publication? Answer

I am conducting a study on the workload of university professors. Besides time-on-task, what other components of the workload should be measured? For example, you and I could spend the same amount of time on a task but your results would certainly be of higher quality than mine. Or, you might spend lesser time on a task than I and, again, your results would be of better quality. Applied to professors, what is the relation between time-on-task and performance? Answer

According to a leader in my organization, “HPT” and “coaching” are the exact same thing. In fact, according to the leader, the two expressions are synonyms and one hundred percent exchangeable. I was under the impression that “coaching” was a trend in Europe (most books I have seen come from France or England) and, although it was about performance improvement, it was not rooted in theory and research, and was only systemic but not systematic. Actually, I thought it was a trend started by a few athletes “importing” their coaching expertise into the business world. Would you be kind enough to give me your impressions on this? Answer

I am a senior in university and want to go into human capital consulting after graduation in May. I was wondering what positions are open for recent graduates. Answer

What basic elements should be included in a "training plan" for a job in which the development of technical and soft skills are required? Answer

What is the difference between a competency and a skill? Answer

In what kind of company would the technical skills of top managers be more important than human relations or conceptual skills? Answer

I'm a training professional working in the pharmaceutical industry and am seeking a basic competency model detailing what performance looks like for classroom trainers. Ideally the model would describe performance at three different levels: basic/fundamental, intermediate and advanced. Can you point me in the right direction? Answer

I'm putting together a pre-test and post-test for one of our training programs. It will be given to some, but not all of the participants to determine if the participant knowledge increased as a result of the training. Since the training we do is not based on clear, measurable objectives, the only test I can create is a knowledge test of the content that is covered. Do you have any information on when a pre- and post-test is effective and when it is not? Answer

Has HPT had a well-documented link to ethical business or corporate social responsibility? If so, who has made this link and where is it documented? Answer

I was wondering if you could point me in the right direction in finding a case study (or studies) that compare the learning results of Instructor-led Training (ILT) versus Web-based Training (WBT)? I am currently working on my Masters thesis in which I am comparing the two, and I am trying to find some numbers that back up my theory that ILT is more effective because of the live human interaction, etc. I remember you mentioning at a conference I attended that well-designed WBT is just as effective as ILT, but I'm still skeptical. Any resources that you could recommend would be greatly appreciated.

This is a tough comparison. There are so many variables that go into creating a learning session. The delivery mechanism is not the main differentiator. In a large number of media and delivery system comparison studies, the general conclusion drawn has been that "all other things being equal, there is no significant impact from the medium with respect to learning effectiveness." Richard E. Clark has written extensively on this.

What you are trying to compare is a lesson in ILT with one in WBT. How do you maintain perfectly equal design elements? You can't. Long ago, there were attempts (and occasionally you see more recent ones) to effect comparisons between live classroom instruction and some form of CBT. These went nowhere. Culick and Culick pushed this agenda in the 1980s, attempting to demonstrate superiority of CBT. Robert Kozma at the University of Michigan also tried that CBT was more effective for learning. Richard E. Clark demolished their arguments.

I recommend focusing on specific design aspects in live and web-based instruction to determine if they have similar effects. Examples are use of inference, dynamic versus static examples, some form of questioning and forms of feedback. It's the design of the instruction, not its delivery mode that makes the difference.

A colleague of mine and I are working on the challenges new hires face in organizations. We were wondering when is it worthwhile to hire a star performer? And, the flip side of that, if you are a star performer, should you stay put versus move to other positions and/or organizations?

To begin, being a star performer does not necessarily imply that this person is only limited to stardom in one setting and will not perform well elsewhere. If the high performance is based on intimate relationships with various parts of the organization, developed over a long period of time or a deep familiarity with processes and systems unique to the organization, then transfer of success to a new environment is less likely. If, on the other hand, the person operates mostly as an individual contributor or as a leader of direct reports who are there to carry out support tasks for this person, then the success factor can be more readily transported elsewhere. A high performing rocket design engineer, a researcher scientist with deep, specialized knowledge, a star sales person or hair dresser with his or her own system and personal connections are examples of stars that can be more readily moved from one work environment to another. However, this is not a dichotomy. Rather, it is a continuum. The key factors are the sources of success. The more dependent on the current organization’s environment, systems and circumstances, the less portable the high performance.

The second question: “If you are a star performer should you stay put versus move to other positions and/or organizations?” Obviously, there is a linkage between this question and what I wrote above. What the research suggests for star performers is that they not be blinded by their current success, but rather they should build a broad network of connections and contacts outside of their specific work environment. They should also invest in their interpersonal, organizational and communications skills to increase their ability to transport their successes elsewhere. To this point, Groysberg has found that women appear to be more successful with their star portability than men. He suggests that this may be due to women more frequently maintaining broader relationships and networks. Because high performing women tend to meet more obstacles in the workplace, they also tend to weigh more factors than men do, especially cultural fit, values, and managerial style in making job changes. You might want to go to http://www.bnet.com/2439-13070_23-186751.html for more on this.

You were referred to in a webinar I attended as a source in stating that only around 30% of know-how and skills contribute to improvements on performance. This being the case, what are the other factors that make up the other 70% and where may I find this reference? Is it contained in your research or in a publication?

The degree to which skills and knowledge variables affect human performance at work strongly varies with the context of a performance situation and the nature of the desired performance. Overall, however, both research and documented professional practice have shown that there are a host of factors affecting workplace performance. I refer you to several books: Thomas Gilbert (1996), Human Competence: Engineering Worthy Performance; Geary Rummler and Alan Brache (1990) Improving Performance: How to Manage the White Space on the Organization Chart; James Pershing (2006), Handbook of Human Performance Technology. All of these volumes contain a great deal of information concerning the array of factors influencing how people perform. What emerges in a consensus fashion is that about 75 - 80 percent of these are of an environmental rather than an individual nature. Here are the ones most frequently cited: lack of specific performance expectations, conflicting expectation priorities; lack of timely and specific feedback with respect to expectations; lack of timely access to required information; task interferences; inadequate tools and resources; unclear or counterproductive policies, processes and procedures; inappropriate or even counterproductive incentives and consequences; poor or inappropriate selection of performers; lack of perceived value to perform; threats in the environment; environmental obstacles (physical; administrative; emotional); language and cultural issues.

There are many more, but these are the ones that are most commonly found. As you examine the literature on human performance technology, you will encounter these and many others.

I am conducting a study on the workload of university professors. Besides time-on-task, what other components of the workload should be measured? For example, you and I could spend the same amount of time on a task but your results would certainly be of higher quality than mine. Or, you might spend lesser time on a task than I and, again, your results would be of better quality. Applied to professors, what is the relation between time-on-task and performance?

You pose a question that has multiple answers. Let me change your question to a more general one and in the process define some terms. Let's start with word "performance." In my world of the workplace, I and most human performance technologists define this critical term as "valued accomplishment derived from costly behavior." [See Thomas Gilbert's book (1996) Human Competence:Engineering Worthy Performance.] Many others such as Geary Rummler, Peter Dean and Harold Stolovitch and Erica Keeps have explored this definition and applied it in a number of workplace instances.

To elaborate, performance is a function of what you do and what you achieve. The doing is the cost portion (effort, money, resources, time). What you achieve, the valued accomplishment, is the benefit or desired result you obtain from the expenditure of the costly behavior. Gilbert proposes his "Leisurley Theorem" which, in brief, says that the best performance is one where we obtain the greatest valued accomplishment with the least effort. This certainly includes time expenditure. The corollary to this theorem is that the more time you require to achieve a valued result, the less "worthy" you are as a performer (worth being the ratio of value to cost and time being a critical cost factor).

Continuing in this vein, in McKinsey's study on the War on Talent (2001), the authors suggest that top performers are 70 percent more productive (achieve more in the same units of time) than average performers. I have been involved in studying exemplary performers in the retail automotive world. What I discovered is that top performers in, for example, automotive sales, are more than twice as productive as the average. The top 20 percent of sales consultants in our study sell 2.1 vehicles to 1 compared to their average colleagues. And this occurs month after month.

In an internal study at the European Patent Office, communicated to me privately, they discovered that their top patent specialists, most with PhDs and/or very highly recognized technical competencies in specialized fields such as medicine, pharmacology, telecommunications and computer sciences, are three times more productive than their average colleagues. That is, they are able to process patent requests and bring them to closure more rapidly with no less quality (e.g. number of appeals, clarity of decision, numbers of communications) than the others.

You have certainly seen the expression, "If you want a job to be well done, give it to the busiest person." In studying appropriate workload, you have to begin by defining accomplishments. In any given field there are "stars." These are top performers who demonstrate what is possible in a given amount of time. They are the models for setting standards. I recommend that you identify these top performers based on sets of generally acceptable performance criteria. Then study what these exemplary persons do and how much time they spend to achieve their valued results. Publications, teaching scores (too frequently neglected), PhDs produced, significant contributions made, research funds obtained, frequency of citations in respected publications, invitations to speak and invitations to consult are some of the success criteria. They can be made specialty dependent.

To conclude, I recommend starting with accomplishments, deriving exemplary standards and then, based on those who achieve the most with the least amount of wasted time, build portraits of appropriate workloads

On a personal note, as an active professor, I was aware of the four requirements for promotion: research and publication; teaching; external notoriety (e.g. leadership positions in my field; invitations to speak and conduct seminars, consulting to professionals in my field; awards); internal contributions (e.g. committee work, administrative work; university leadership positions; mentoring of colleagues). They were fairly clear and, I believe, appropriate. From my perspective they required balancing and equal attention. Find those who do well in all of these and analyze their time expenditures. Also study those who are average and those who do poorly. I believe you will find valuable data for your study.

According to a leader in my organization, “HPT” and “coaching” are the exact same thing. In fact, according to the leader, the two expressions are synonyms and one hundred percent exchangeable. I was under the impression that “coaching” was a trend in Europe (most books I have seen come from France or England) and, although it was about performance improvement, it was not rooted in theory and research, and was only systemic but not systematic. Actually, I thought it was a trend started by a few athletes “importing” their coaching expertise into the business world. Would you be kind enough to give me your impressions on this?

What nonsense! HPT is a field. Its purpose is to engineer systems that result in performance all stakeholders value - performers, management, customers, regulators and the community at large, as appropriate. It is systemic in its vision and approach, scientific in its base, systematic and orderly in its methods, and draws form all relevant sources to achieve optimal, verifiable results desired by all stakeholders.

Coaching is simply one intervention to achieve performance. It's central purpose to is to increase skills and knowledge and to guide those whose performance requires improvement. By-products of coaching is increased accuracy and fluency through task-focused feedback, increased value and interest with respect to the targeted performance and increased confidence in being able to perform at desired levels.

Comparing coaching to HPT is like comparing shoes to fashion. Yes, there is a tangential connection - very tangential. Please refer the professor to The Handbook of Human Performance Technology.

I am a senior in university and want to go into human capital consulting after graduation in May. I was wondering what positions are open for recent graduates.

This is a tricky issue. One rarely finds openings for "human capital consultants" per se. I recommend reviewing the ISPI and ASTD job banks located within their websites. The nature of the jobs (the descriptions) provide cues as to what they are hunting for. Also look for positions related to performance consulting, human resource management and human resource development. Sometimes, you find a position that sort of fits what you are looking for. Once in the job, you can begin to demonstrate the value-add of your perspective.

Another route is to contact performance consulting companies and offer to do an internship there. Then, show your stuff. If you're great, they will want to hire you. McKinsey, Accenture and KPMG or Deloite are the big guns. Search out some of the smaller consulting groups as well.

You might also want to contact companies with strong track records in human capital management and inquire about positions. Nordstrom's, Federal Express, Lexus and General Electric are a few names that come to mind. CDW is moving in this direction.

Read the journals to see who are doing excellent work. Don't hesitate to contact them. If you have limited experience, go relatively cheap to gain experience. You can always renegotiate based on your demonstrated worth to the organization.

What basic elements should be included in a "training plan" for a job in which the development of technical and soft skills are required?

Training plans come in so many varieties that it is hard to respond specifically. The best practice approach is to build a performance map (a job analysis) of the job. This includes the major performances to be demonstrated by the incumbent and then broken down into successively more elementary components. It resembles a task analysis. The next step is to assess each individual based on the performance requirements of the job. This means that each person within a given job classification can (and really should) have an individual learning and development plan.

Finally, you have to determine where the gaps lie and which ones are skill/knowledge deficiencies. At this point you can create the training/learning/development plan.

I like my training and development plans to contain five components. Based on the job, I start with Essential Skills and Knowledge Development. These are the ones absolutely required to perform adequately in the job. These are comprised of three sub-components: technical, conceptual and interpersonal. The higher up the management chain the person is, the more conceptual and interpersonal skills and knowledge are required. But once again, it depends on the assessment of each individual. One person may need greater technical skill development, another more interpersonal. Believe it or not, we sometimes have to add "Social" or even "Public Relations" skills depending on expectations of the performer. The next set are the Development Skills and Knowledge. These include those skills and knowledge that help to shore up both strengths and weaknesses. We want our performers to strengthen what they are already good at and develop more in those areas where weaknesses will be detrimental to expected and future performance. Finally, I like to include a section called Personal Growth Skills and Knowledge. This is where each individual can devote some time to explore avenues related, but not central to, the current job. They are career enhancement pursuits and must be approved for the organization to support these.

With respect to how one develops these, a variety of alternative means can be specified. These include, but are not limited to: internal training sessions, task force or committee participation, joining a mentoring program, reading, temporary assignments, outside courses and degree programs, participation in projects, special assignments, self-monitoring activities, and even having lunch with certain people or going out on visits.

The plan should include not only a specific plan of action, with specified activities, but also a Timeline and Agreed Upon Review Points with accountabilities.

The plan, as you can see, includes training elements as necessary, but is far more of a development path. That is appropriate in this age of knowledge workers in which lock-step routine jobs are largely disappearing. A plan should take people, in light of what is expected of them, from where they are to where we - and they - would like them to be. No two persons are identical. Neither should be the plans to help them perform optimally.

What is the difference between a competency and a skill?

A competency is an ability to perform that is required by a job. You always create competency models based on named jobs. Hence, competencies are derived based on a set of external requirements. A skill is something that an individual is able to do. We can inventory and assess the skills of people. Imagine that we want to determine what skills you possess. We may identify a broad range of these including calculation skills, analytic skills and a number of physical dexterity skills to name a few. Now imagine that we are hiring circus clowns. We can create a competency model based on exemplary clown performers. These are what we consider are needed to perform well as a clown. We specify these competencies in a job search and then match the skills of job candidates against the competency requirements. We can also hunt inside the organization to discover whether we can develop internal candidates with skills that show potential to meet the competency requirements of the job with training. You may have skill in juggling balls. It is irrelevant for most jobs. However, it may be a competency requirement for the clown position. To summarize, skills are what you have - the ability to do things. Competencies are what a job requires.

Please don't confuse either of these with characteristics, knowledge or values. These are different. Too many problems arise when we mix up all of these terms and define them poorly. The general rule is hire for characteristics and values as these are extremely difficult to develop or alter in a person. Train to build skills that match competency requirements.

Here is a link to a brief article I wrote that is somewhat relevant to your question: http://www.nxtbook.com/nxtbooks/mediatec/tm0707/ (article on page 16).

In what kind of company would the technical skills of top managers be more important than human relations or conceptual skills?

This a tough on; there is no fixed rule for this. However, in general, small, entrepreneurial companies that have some form of cutting edge technical capability can often grab a lot of new business because of the leaders' recognized technical competencies. The innovation excites customers and the demand is such that business acumen, interpersonal skills and even communication or marketing skills are of lower priority. Often, the organization takes on the form of a cult with the leaders as technical gurus surrounded by dedicated disciples who work crazy hours to build success.

As the organization grows, this highly personalized, technically focused adventure begins to outgrow the initial visionary and innovative thrust that launched it. Reality and day-to-day concerns emerge. To survive beyond the launch requires more mature conceptual, business, marketing, financial and interpersonal skills. Other players enter the market and the cutting edge uniqueness of the service or product now faces competitors. Demand softens and the realities of the competitive marketplace begin to diminish the focus on the technical aspects alone of the enterprise. Financial supporters have to be wooed and appeased; new people who are brought in have expectations that must be met beyond the great vision of the start-up. The leaders find themselves faced with the hard-core and mundane requirements of running a business. Not all visionary and technically brilliant entrepreneurs can make the shift.

Hence, to finally answer your question, those companies that have caught fire because of the technical brilliance of their leaders (e.g. General Electric, Hewlett-Packard) come to mind first. Those companies that are largely technically driven also require leaders who are highly skilled in their technical areas. Professions such as law, accounting, medicine and certain fields of consulting, including Human Performance Technology, certainly must have technically capable leaders to inspire and generate credibility in the marketplace. However, in all of these instances, as maturity and growth occur, other skills must be added. Sustainability requires that conceptual, interpersonal, business and a wide range of organizational and management skills be present. Without these in the leadership (or if the leaders are wise, then brought in to help in decision making), the company is unlikely to prosper long term. Technical skills may be necessary. However, in the long run they will probably not be sufficient.

I'm a training professional working in the pharmaceutical industry and am seeking a basic competency model detailing what performance looks like for classroom trainers. Ideally the model would describe performance at three different levels: basic/fundamental, intermediate and advanced. Can you point me in the right direction?

There are many beliefs and models out there for you to pick and choose from. However, if you are looking for one that is structured, has a long track record, is both process and performance centered, and has been used with a large number of organizations, I recommend the program provided by the Canadian Society for Training and Development (CSTD). Here is the link, http://www.cstd.ca/certification/index.html. This is a program to certify trainers at increasingly higher levels that has been around for many years and has had solid success. I believe that CSTD offers a variety of arrangements for organizations.

Another example of a series of graduated modules for building ever-increasing training capabilities is the Trainer Competency Track (TCT). The link to this is http://www.nystc.org/committee/minutes/pdt/competency.htm. It is arranged in five levels and, at the top, includes performance consulting, organizational development and many other areas for professional growth. You'll have to do some investigating to learn more about the modules and the model that underlies it. My knowledge is only cursory concerning the programs.

Another "model" is the one devised by CHART, the Council of Hotel and Restaurant Trainers. CHART is an industry-wide non-profit group that encompasses most of the hospitality organizations and has been around for years. As you can imagine, this is an area in which a huge amount of training occurs. Here is a link to a "white paper" about the CHART model: http://batrushollweg.com/Insights&Research/WhitePapers/BlueprintForTrainerDevelopment.pdf

I could pepper you with a host of models. Many are industry specific. I leave you with a link to the directory of the Society of Pharmaceutical and Biotech Trainers, http://spbt.klickit.com/local/files/vendor%20directory/vendorproj_proof2.pdf. It has a section of trainer programs arranged in a hierarchical format. The Society itself has tackled issues such as trainer development, performance improvement and competency modeling. As this is closer to your organization's universe, I recommend connecting with SPBT and getting your hands on what it has produced.

I'm putting together a pre-test and post-test for one of our training programs. It will be given to some, but not all of the participants to determine if the participant knowledge increased as a result of the training. Since the training we do is not based on clear, measurable objectives, the only test I can create is a knowledge test of the content that is covered. Do you have any information on when a pre- and post-test is effective and when it is not?

First, let's define pre-test and post-test. A pre-test is a post-test equivalent that is administered prior to submitting "subjects" to a "treatment." In the training world, this means that you create a test based on what it is that trainees are to master as a result of the training and administer it before and after the training. This allows trainees who have already mastered portions of what is included in the training to skip these. To ensure that trainees have not memorized the responses from the pre-test when you administer the post-test, you should create a different but equivalent set of test items for each of the tests. To do this requires some knowledge of both test item construction and a bit of statistics to ensure equivalence of tests. Usually, we construct test item banks and then draw from these at random to ensure equivalence. It may sound complicated, but it is not so bad. You need to have someone help you with this until you know what you are doing.

Now for the more important matter. Training has long history of being criterion-referenced. This means that specific and measurable criteria for mastery are established, normally derived from performance-based task analyses that lay out the required outputs and outcomes of a job (or set of tasks within a job). Training is designed to ensure that trainees meet these criteria. The criteria are expressed through verifiable objectives that are trainee centered, contain the verifiable performance and state a standard of acceptable performance given the level of the trainee group. Test items are created to perfectly match the objectives. Our book, Engineering Effective Performance Toolkit, details all of this with lots of job aids.

If you do not have performance-based objectives, it is difficult to create any kind of valid test. In addition, testing declarative (talk-about) knowledge of a content area that requires procedural (doing, using) knowledge is of little use except to say, "I tested them." There is very little correlation between being able to talk about what you can do and being able to actually do it. The declarative knowledge test will only be useful for those portions of the content that are to be remembered verbally. They have very little impact on performance results (e.g., talking about the features and benefits of a product versus actually being able to demonstrate the product in a convincing manner).

As for sampling trainees, that is fine if the sampling is random and meets sampling criteria. You can find guidelines for this on the web or in books on testing and sampling.

To conclude, what is the purpose of the activity? If it is to show that there are knowledge gains from a course so that someone can say,"See, they learned," then the approach being taken will work. Trainees will probably remember something that they did not know before. If it is to verify whether or not the training had an impact on their ability to perform, then the results of such an exercise will not be helpful.

As a side note, we are currently conducting a research study on transfer of training. As has been found elsewhere in many studies, declarative knowledge learning and retention is high. The trainees learn and retain enough to do well on the tests and significantly better than on the pre-tests. However, when we examine application on the job, the portrait changes markedly. They may "know," but they do not "apply." We are studying the variables that facilitate or inhibit on-job transfer.

Has HPT had a well-documented link to ethical business or corporate social responsibility? If so, who has made this link and where is it documented?

Your questions about HPT and ethical business and social responsibility are very relevant in today's highly competitive enterprise climate. As HPT professionals, not only do we have to act ethically (and be seen to do so), we must also make sure that we provide guidance and counseling to our clients that in turn help them to act appropriately. I cannot vouch for a "well-documented link" between the writings on HPT and corporate social responsibility, but the theme has often been addressed in HPT publications. All of the Handbooks of Human Performance Technology have chapters dealing with ethics and standards. The latest one, edited by James Pershing, is no exception. Please check out chapter 44, Standards and Ethics in Human Performance Technology, written by Ingrid Guerra. In particular, examine the section: Social, Corporate, and Professional Consciousness starting on page 1,032. The references section is also a great starting place.

You might wish to contact Ingrid at Wayne State University. Another person who has long been interested in the ethical dimensions of out field is Peter Dean in the College of Business, University of Tennessee. He was a senior fellow in the ethics program of the Wharton School's Department of Legal Studies at the University of Pennsylvania for several years.

Like most professions, HPT is very concerned about its practitioners operating ethically and helping clients do things correctly. From my own consulting experience, I have unfortunately had to deal with potentially unethical situations. In some cases, I assisted the clients to make the right decisions. In others, I had to walk away from the projects because I could not in good conscience participate in activities that were not in accordance with HPT's ethical standards. I made my case, attempted to change the clients' directions and when I could not, simply withdrew - but documented the reasons for my withdrawal to the client. There are other HPT professionals who have shared similar cases with me over the years.

ASK HAROLD !