JULY 2006
Progress is Both Evolutionary and Revolutionary
By Quinn Sutton - Editor
Recently I had the opportunity to drive the family mini van across country. It was a four-day journey right out of a "vacation" movie starring Chevy Chase. It also put me in touch with "progress".
Each day I calculated the distance we had traveled, how much farther it was until the next - whatever. And I began to get a sense of progress. It was not a complete change from one place to another - it was a methodical and tedious process of simply moving forward based on a plan. This was at first a bit painful and I was impatient.
Then on a whim, somewhere in Ohio, we turned off the road to see a historical monument dedicated to the early pioneers who had crossed the same area some 150 years earlier. We learned that some 800 pioneers had died at that spot after dredging three months in mud, up to their knees at times, to make it to this temporary rest stop. We made the exact journey on a paved road, in an air-conditioned van in about four hours.
I was struck by both the similarities of our journeys and the vast differences. While we had been unknowingly following their trail, counting our progress as the mile markers went by, their progress was measured by completely different standards. Our course was similar but dramatically different in technique. I came to realize that progress comes in both small steps and great leaps.
When I contemplate how far we have come in the computer-based testing industry, I would dare say that in spite of all our progress, we are still pioneers. We're just beginning to see what is possible, just getting the kinks worked out. Now, we are ready for the bold application of technology that will revolutionize the way we do things. I am sure if those early pioneers could see how easily I traversed their trail and under what conditions, they would have been speechless. They were simply too consumed in the realities of their day-to-day world to envision the fantastic. That is where I think we, the computer-based testing industry, stands today. We're too busy to consider - what if?
At KRYTERION, we're working on truly revolutionary contributions to the testing industry. Although it is too early to disclose the details, let me say we are committed to making the vision of a dramatically different testing experience a reality - and you will soon see what I mean. Nevertheless, take advantage of the evolution and start taking control, start your journey and begin to make a difference.
Some say it's impossible - we call it progress.
Mature Market or Simple Stagnation
By Gary Clark
You don't have to listen too hard to the computer-based testing organizations' messages to start believing that running a cost effective, secure testing program is like climbing Mt. Everest or some other equally perilous and challenging endeavor. Now, while there's no question that creating and maintaining a secure testing program requires expertise, there is simply no excuse for the state of the industry being in the state that this industry is in today.
I've heard that the industry is "mature" and we need to resign ourselves to mature-market dynamics. I don't believe that the current state of the industry is mature. I believe it is simply stagnant. This is no doubt due to those who are stuck in a quagmire of legacy systems, antiquated methods and petrified management philosophies that they can't figure a way out. Looking at the rate of technological advancement, it is clear that there is a lot of innovation yet to be implemented and I contend that we are at the beginning of the curve - not at the end of it.
Lack of Control
The computer-based testing industry today is faced with a growing problem that simply hasn't been addressed by the current mindset. Test sponsors today rarely, if ever, have the kind of control over their programs that they'd like to have. The ability to publish on the fly, run real-time reports from anywhere, do something about cheating and piracy are capabilities that every test sponsor ought to have. Yet, despite over 20 years of computer-based testing our industry has managed only to create a system that for the most part simply automates the paper and pencil tests that have been around, well, since paper and pencils.
Other industries such as telecommunication, banking, computing or personal entertainment are fostering innovation and progress while becoming more customer-centric and lowering costs. I am challenged to name similar significant innovations that have come to the computer-based testing industry over the last 15 years. There just haven't been many significant innovations.
Who Pressed Pause?
To be sure, we brought some of this on ourselves. By using multiple vendors to deliver a single certification or licensure exam we played a part in the stifling of innovation. Test sponsors could no longer implement new and innovative ideas unless all involved vendors could implement these ideas contiguously. To do otherwise would create unequal and arguably unfair tests - a big (maybe the biggest) no-no in every psychometrician's playbook. In retrospect it would have been better for test sponsors to commit to one vendor for administration of a given test and be prepared to move their business if not satisfied with the early commitment. This strategy would have allowed for innovation, as changes made would not have differentially affected any one sponsor's test takers. Test delivery providers would have had more flexibility to innovate in the same manner as so many other high tech industries.
To be fair we must acknowledge that while wide distribution channels, efficient data collection and reasonably good reporting have in fact emerged, these features simply have not kept pace with the breath-taking innovations we've seen in so many other high tech industries over the last 20 years (wireless telephones, ATMs, thumb drives, portable mp3 and video players, plasma and LCD screens, internet commerce).
Time for Progress
Perhaps the most troubling aspect of the industry facing us today is complacency. I believe that the status quo thinking has resulted in a generally accepted low level of innovation and service. The existing models are antiquated, are too similar to each other, take existing markets for granted and continue to ignore opportunities requiring innovation. If we were the music business we'd still be listening to cassette tapes and 12" LPs.
Unfortunately lack of creativity, lack of innovation, lack of service and lack of leadership have paralyzed our industry. As a result, we're 20 years behind other industries and it's time to move forward, catch up and embrace progress.
Fortunately this means there is no where to go but up and forward. We at KRYTERION will continue to question the way things are and more importantly continue to provide solutions that will bring the computer-based testing industry to its innovation revolution. Make sure you stay tuned to PROGRESS as we move this industry into the next century. Ultimately you will be the one to benefit the most. It's about time. Your time.
Newsletter Archive
Upcoming Webinar
How to Improve Your Program's Progress
Top 3 Ways to Use Item Statistics
Join us for a 30-minute discussion on ways to improve your program's progress. Learn how to incorporate the latest innovations in technology and advanced techniques into your testing programs. Find out how you can make PROGRESS work for you!
Register here
Technology Spotlight
It's About Time. Your Time.
Our latest Webassessor release was designed with you in mind. We're
focusing on time - your time and how to use it efficiently. Webassessor
gives you more control and detailed insight into your program so you can
gather, report, distribute, analyze and respond on your schedule.
Some of the innovative enhancements include:
- Forensics data capture - support for security audits.
- Usability upgrade - improved user interface.
- Item creation enhancements.
- On screen item statistics.
- Enhanced reporting - greater program control.
Psychometric Corner
Q - Is having detailed information on my items critical to the success of my exam?
A - Decisions are only as good as the information upon which they are based. An exam is only as good as its individual items. Do you have the right type of information to make good decisions about your items and significantly improve your exam?
- Do you know which items are "model items" so you can create higher quality items in less time in the future?
- Do you know which answer options should be changed to "fix" an item?
- Have you asked your Test Takers what they think of each item? Do they feel it measures important knowledge and/or skills?
Send to a Friend
Click here to send to a friend
We Want to Hear from You!
Please send us your questions, comments or editorial ideas to Quinn Sutton at pr@kryteriononline.com.
Press ContactQuinn Sutton |