Figure 1-2

Introduction 
The effectiveness of distance learning and computer based training has received a lot of hype in recent months, but how well does it work for the type of hands on training needed by NDT inspectors? Over the past four years, the US Air Force has spent over $1 million attempting to provide refresher training to NDT inspectors via computer based training. This paper describes the lessons learned during this process.

 

Background
In 1995 the US Air Force NDT program was facing a serious problem. Retirements and separations were quickly depleting the ranks of experienced NDT inspectors. Traditional methods for training new NDT personnel in technical schools were filling many of the vacant positions, but these new personnel did not have the experience required to perform all necessary inspections. Many junior personnel were unable to acquire the needed experience because there were no senior level personnel available to train them. In an attempt to help solve the problem, the Air Force awarded a $1 million contract to develop 60 hours of computer based training refresher courses on the following topics: basic metals, basic math, NDT overview, ultrasonic refresher, eddy current refresher, and several equipment simulation modules.

 

---

With Web delivery, you no longer need to reproduce and distribute CDs. 

---

The contract was awarded to a company with a proven track record of NDT expertise and successful computer based training development. The result was a 10 CD-ROM set with photos, narration, text, graphics, animations, quizzes, and interactive exercises that were intended to provide refresher follow on training to NDT inspectors. These multimedia computer based training courses were made available to approximately 1600 Air Force inspectors at 200 NDT labs. These computer based training courses have also been purchased by hundreds of corporations to augment their NDT programs. Since 1995, the Air Force has contracted out another seven custom NDT computer based training courses that have also been widely distributed on CD-ROM.

To help assess the effectiveness of this computer based training, users have been encouraged to provide feedback and complete surveys. The lessons learned recapped below are based on a compilation of these surveys, feedback, and numerous discussions and interviews with users.

 

Lessons Learned

Acceptance of Computer Based Testing by Inspectors Met Expectations

When we began developing these custom computer based training courses, we expected some resistance from NDT inspectors. After all, people are often resistant to change. However, we believed that most would quickly see the benefits of what we were trying to do, and this is indeed how it unfolded. There was some resistance at first, but our feedback indicates that approximately 80 percent of the 1600 people who have access to this computer based training now use it regularly with few complaints. Most use it before annual recertifications (although the computer based training is not required by the Air Force, many shop supervisors have established local requirements to complete the training annually), and to refresh their memory before complex or seldom performed procedures.

 

Comprehension and Retention Exceeded Expectations
Shop supervisors indicate that the computer based training does a better job of refresher training than traditional lecture training. The computer based training provides better explanations of NDT concepts and procedures than normally provided by an in house instructor, enabling students to more quickly understand the lesson material. Most studies reveal that computer based training result in a 25 to 40 percent increase in comprehension and retention over traditional lecture training. To be honest, we had some doubts about whether NDT computer based training would prove as successful. We need not have worried.

 

Computer Based Testing Does Not Satisfy All Training Requirements
Although the computer based training has proven to work well for refresher training, it has not yet been proven to satisfy all training requirements. Comprehensive computer based training courses for initial NDT training are probably several years away, and even then we expect a continuing need for the computer based training to be augmented with instructor led and hands-on training.

 

Usable Life Cycle is Longer Than Expected
When we first developed the computer based training, we expected a requirement to perform major revisions every three years. However, only minor revisions were required in the first three years. We now expect major revisions every five to seven years. Therefore, we calculate the useable life cycle to be six years.

 

Return on Investment Exceeded Expectations
With a useable life cycle of six years instead of the predicted three, the return on investment is much higher than we initially expected. The initial 60 hours of NDT computer based training cost $900 000 to develop, or about $15 000 per courseware hour. We estimate that 1300 personnel use at least 20 hours of the computer based training annually in resulting in 26 000 training hours per year. Over the six year life cycle, this will result in 156 000 hours of training. When we add in the cost for revisions, duplication, and distribution, total cost of providing this training (excluding student time) will be approximately $1.2 million. We are therefore looking at an end cost of about $7.70 per training hour. This is significantly less than the cost of traditional NDT training courses.

 

Computer Based Testing Can Significantly Improve Rapid Reaction Capability
At the same time we began developing the NDT refresher computer based training, a new critical NDT procedure for the C-141 aircraft had just been developed. The Air Force needed to train NDT inspectors quickly at many different locations on this new procedure. Flying all of the inspectors to a centralized location to provide the training was too expensive, and sending the few qualified inspectors to each location to provide on site training would take too long. It was therefore decided to build a custom multimedia computer based training package on a stand alone CD and to distribute copies to all locations. We arranged to take numerous digital photos of the inspection procedure, and developed a one hour computer based training lesson clearly illustrating how to perform this procedure. We then distributed copies to 12 different locations. The result was that new training was developed and delivered to all of the inspectors within two months. With advances in the ability to produce and deliver computer based training via the Internet, rapid reaction training for new NDT procedures can now be implemented in about one month, a significant improvement over traditional training approaches.

 

Clear, Concise, Illustrated Explanations Were Most Important
As we began developing the refresher NDT computer based training in 1995, we tried different approaches to obtain the most cost effective training results. Our primary focus was on improving comprehension and retention, while decreasing development costs. It quickly became apparent that, for NDT training, clear concise illustrated explanations were the most important element of this training. We segmented each procedure into a series of discrete steps, succinctly explained each step in simple English via on screen text bullets and a synchronized audio narrative, both of which were integrated with graphic illustrations and digital photos. When appropriate, we provided an on-screen hyperlink to a portion of the corresponding technical order (detailed written procedure) to provide optional additional reading for those accustomed to learning new procedures the old fashioned way. By keeping the program simple, we not only improved comprehension and retention, but we also further reduced development costs. We found this approach could be quickly duplicated for almost any NDT procedure.

 

Narrated Digital Photos are More Cost Effective Than Full Motion Video
Although we did include full motion video depictions of portions of some NDT procedures, we quickly learned that video was often not the most effective approach. Most video segments were replaced with a series of digital photos with a synchronized audio narrative and text bullets to reinforce key points. The digital photos allowed us to better emphasize critical points, and proved much easier to edit and refine than video. Now that much of this NDT computer based training is being developed for the Worldwide Web, the advantages of narrated digital photos over full motion video have increased. When accessed over the Web via a 56 kilobits per second (kb/s) modem, video quality is marginal at best, while a series of digital photos with synchronized streaming audio provides great quality with no interruptions.

 

Interactive Exercises Were Not as Important as Expected
Although we continue to develop interactive exercises to allow inspectors to practice what they have learned via onscreen simulations, we do not include as many of these as we used to. We have learned that interactive exercises can often dramatically increase computer based training development cost while only marginally improving comprehension and retention. We now design interactive exercises only when needed to improve learning or when they can be developed with little impact on cost.

 

Automated Training Records Management is Very Important
Although we have included a complete automated student tracking system with our computer based training from day one, it turns out that such an automated training records management capability is much more important than we initially envisioned. In the NDT world, providing the training is only half the battle; the requirement is not met until the training has been adequately documented. Fortunately, when we initially designed the automated student tracking database, we built in much more capability than had been specified. If not for this, we would have had to back up and modify this database on several occasions.

 

Capability for Local Customization Was Not as Important as Expected
When we initially designed the computer based training, we believed that many shops would want to customize the training to their specific environment. Therefore, we built in capabilities to easily add text notes to any computer based training screen, and to hyperlink any computer based training screen to locally produced Microsoft PowerPoint presentations. To date, these capabilities have been seldom used. It appears most NDT shops are primarily interested in improving training on the basics, with local customization provided via discussion and on the job training after completing the computer based training.

 

Computer Based Testing Can Now be Delivered Effectively Via World Wide Web
In 1995 when we began this project, the World Wide Web was still in its infancy. Standard Web access was via a 14.4 kb/s modem, and we determined that the multimedia training we needed to provide could not be delivered effectively via the Web. Therefore, we focused on delivery via CD-ROM. Today, with only minor cosmetic modifications, the same quality of multimedia training can be delivered over the World Wide Web to anyone with a 56 kb/s modem. Optimizing a computer based training course for such Web delivery normally increases development cost by about 5-10 percent, but this initial cost increase is quickly offset by savings in maintenance and distribution. With Web delivery, you no longer need to reproduce and distribute CDs, and you no longer need to worry about users inadvertently using an outdated CD.

 

Summary
When we began this project, there were those who argued that the NDT community would not accept computer based training. We believe they have been proven wrong. Some also argued that computer based training could not replace traditional NDT training. Since we have always agreed with this argument, we did not attempt to replace such training totally. Instead our goals were to develop cost effective computer based training to provide refresher training and to provide training on new NDT procedures to qualified inspectors. The record clearly shows that these goals have been met. If your organization does not yet incorporate computer based training into at least portion of its NDT training and certification program, we propose that now is the time to consider it.

 

* Karta Technologies, 1892 Grandstand, San Antonio, TX 78328; (210) 681-9102; fax (210) 681-9198; Web site www.karta.com/services/trainingsolutions/project_cbt1.stm.

 

Copyright © 2000 by the American Society for Nondestructive Testing, Inc. All rights reserved.

[ Materials Evaluation ]