October 2007
Einstein at Work: How to Use Reports Like a True Program Manager Genius
Winston Churchill once said, "True genius resides in the capacity for evaluation of uncertain, hazardous, and conflicting information." If you manage a testing or certification program, congratulations, you are a genius!
Like most program managing geniuses, you understand the importance of collecting data and reporting on your program. It's important to have your data at your fingertips, because if knowledge is power then reporting is the extension cord of your testing program. How can you power your program effectively and efficiently if you can't plug it in to the data that you are collecting on a daily basis?
Example of Real Life Genius at Work
John is the administrator for a certification program. The board was meeting the following day and they requested some last-minute information about the program. The board wanted to know where their Test Taker base was and how many in each area. They also wanted a year-to-date program revenue report. In the paper-pencil world this would have taken hours, maybe even days
Fortunately John had real-time online secured access to his testing data. He was able to run a quick and comprehensive report. He picked his fields - test name, city, state, country, complete date, and score - and ran a report. The data was filtered to include YTD data and grouped the data by state.
He then ran a financial report based on the test, payment method, purchase price and date purchased fields. He then constrained the data to the YTD. He assembled and ran the reports in about 120 seconds, exported to and formatted in Excel in 90 seconds and sent it to the appropriate people in another 60 seconds. How long does it take you to access and analyse your data?
Making Your Data Work for You
Aspiring testing program Einsteins may ask, "Now that I have the raw data, what reports are most important?" Critical reports for your program will depend on your setup and needs. However, there are few reports that are important to most programs. These include:
Weekly
- Test Taker Registrations - identify trends in registrations and correlate them to your efforts to increase registrations
- Assessment Results - How many passed and who, where and when?
Monthly
- Financial - Monthly test program revenue
- Test Taker Feedback - End of test comments or survey data
- Marketing - Test Taker names, addresses and emails
Quarterly
- Item Analysis - How are the items and test form performing?
- Demographics - Where is your Test Taker base?
Yearly
- Total tests taken, passing rates, financials
When creating reports remember the words of the Author and Professor Martin Henry Fischer, "Knowledge is a process of piling up facts; wisdom lies in their simplification. "Clear, concise and accurate reports are the "Theory of Relativity" for program managers in that all of the data can be consolidated into more basic terms (like Einstein's E=mc2).
Using tools like Webassessor, KRYTERION's online, secured test delivery system, anyone with access to the Internet and the proper administrative rights can run any of these reports at any time. The Webassessor system records detailed level data on your testing program and can quickly produce Excel, PDF and CSV exports on the data. In addition, your favourite reports can be saved and run over and over, for free.
Proactive Program Management
Effective reporting allows you to be proactive and creative. It empowers you with the data to make and justify your program management decisions.
Now that's genius
KRYTERION Releases Webassessor 7.0
KRYTERION Webassessor(TM) provides robust, online secured testing tools and services that extend testing beyond traditional delivery venues.
KRYTERION Inc. released an enhanced version of its acclaimed technology - Webassessor 7.0. This release represents a critical milestone for KRYTERION's implementation of its aggressive vision to provide a viable extension to high-stakes testing delivered exclusively via testing centers.
"We envision making the test taking experience as convenient for the Test Taker as possible, while increasing test delivery security," said David Foster Ph.D., CEO of KRYTERION. "Although traditional testing centers will always play an important role in our overall testing solution, we are committed to using technology to provide High-stakes testing where you live, learn and work. Webassessor 7.0 is a critical step towards achieving this vision."
The Webassessor Solution
KRYTERION has a successful track record in helping organizations create new certification programs and providing secure, online test delivery technology, via Webassessor.
Webassessor is a complete testing tool for all Program Administrators, Item Authors, Test Developers and Test Takers. Through a powerful, highly secure and easy-to-use interface, Webassessor allows users to take control of their testing and certification programs.
Key NEW features that allow Program Administrators to take greater control of their testing programs include:
Item Development and Security
Administrators can:
- Control access to their item banks in order to manage item security in a multi-author environment.
- Store test item information that is specific to the testing program. Additionally, Program Administrators can use custom item fields for tracking item-specific information such as expiration dates, item difficulty and reference materials.
- Manage items through the development process using Item Workflow. Administrators can also store and track the change history for every item.
- Easily create and edit items with the addition of an HTML editor. Text formatting, tables and lists are standard features.
- Preview items to see exactly how they will appear to their Test Takers.
Administrators can:
- Outline how they want to manage registrations, including who registers, when to register and how to register.
- Define who can register for an assessment by setting the retake rules and prerequisites.
- Determine assessment registration and delivery by setting the registration and delivery availability windows and catalog availability.
Administrators know exactly when and where KRYTERION delivers their test.
Test Delivery Options
Administrators can define their delivery options and their delivery location by selecting where they would like KRYTERION to deliver their proctored assessment using the Webassessor Delivery Channels. Administrators can choose from KRYTERION's HOST Locations or their private locations or set geographical restrictions on where KRYTERION delivers their test.
Control, Convenience and Confidence
"Webassessor is the only product of is kind on the market," said Quinn Sutton, Vice President of Marketing for KRYTERION. "The degree that Webassessor allows our clients to directly control their testing and certification program data is unheard of and we'll continue to make progress for our clients and the testing industry. With each new release of Webassessor we will integrate innovative technology to increase security, enhance data control and provide convenient test delivery options for Program Administrators and Test Takers alike. Some say it is impossible - we call it progress."
Webassessor is Available Now
For detailed information on Webassessor 7 please visit www.kryteriononline.com/webassessor.htm
Testing Connections
Here are just some of the things you should know about in the testing industry...
news.bbc.co.uk/1/hi/england/london/6269210.stm
www.caveon.com/resources_news.htm
www.testpublishers.org/europe.htm
Comments?
Please send us your comments or editorial ideas to euonline@kryteriononline.com
E-assessment Market Survey
Click Here to take the E-assesment Market Survey
Newsletter Archive
Upcoming Events
Assessment-tomorrow
Learning and skills Sector
23 November 2007
Cavendish Conference Centre
London
For more information visit:
www.london2007.assesment-
tomorrow.com
Assessment-tomorrow
E-learning Awards
Thursday 8 November 2007
Park Lane Hotel
Piccadilly, London
For more information visit:
www.elearningage.co.uk/
goawards.htm
Other Events
http://www.c4lpt.co.uk/Conferences/
index.html
Psychometric Corner
Q. Is having detailed information on my items critical to the success of my exam?
A. Decisions are only as good as the information upon which they are based. An exam is only as good as its individual items. Do you have the right type of information to make good decisions about your items and significantly improve your exam?
- Do you know which items are "model items" so you can create higher quality items in less time in the future?
- Do you know which answer options should be changed to "fix" an item?
- Have you asked your Test Takers what they think of each item? Do they feel it measures important knowledge and/or skills?
Our guess is that you do not. Just imagine, having a report that allows you to view the test item, item statistics and comments in a single report on demand for any period of time (one day, one month or one year...). This means knowing if the person who provided negative feedback on an item got the item correct and how he/she did on the overall test. With this type of information in a single location you can make good decisions about building and rebuilding your item pools and exams over time.
Send to a Friend
Click here to send to a friend