webqem blog

The Challenge

Hyundai Motor Company Australia had a requirement to assess 1400 technicians across Australia within a month. Their specific requirements were:

• Assessment for three different technician levels.
• Each assessment would be timed for 60 minutes and consist of 40 multiple choice questions that would be pulled randomly from its own unique pool of up to 120 questions.
• Each assessment would consist of varying quantities of questions, based on three levels of difficulty: easy, even and hard.
• Marking of the assessment would only occur once the participant had indicated that they’d completed or if the exam had timed out.
• Participants would be allowed to review the exam results once completed.
• The assessment participants would be tracked on their number of attempts.
• For Hyundai’s analysis, the solution also needed to collect data on the questions, such as how often a question was successfully answered and which answers were selected for each question.

The Solution

Adobe Connect (formerly Macromedia Breeze) was chosen as the Learning Management System (LMS) to deliver the assessment. Connect Training has tools to create, deliver and manage courses and curricula; and manage, track and report on participants. Connect Training also conforms to the AICC standards widely used by many LMS’.

Another two Adobe products, Captivate and Presenter, which are used to build content and quizzes, were integrated with Connect Training. Neither Captivate nor Presenter however, offer random questions or timed exams which was part of Hyundai’s requirements. In light of this, webqem built a custom solution for Hyundai with Adobe Flash that conformed to the AICC standards. Each assessment is configured by an individual XML manifest file that allows Hyundai to set variables such as the maximum amount of time for the exam and the number of questions to ask from the pool.

Standard quizzes produced by Captivate and Presenter are marked on a question by question basis as they’re answered, which means that there is no review process before an assessment is completed. The solution built by webqem is non-linear, meaning participants can navigate through to any question that has already been attempted, review their answer and change if required. Marking does not occur until the participant has finished reviewing their answers.

Another issue addressed by the webqem solution is the full reporting of questions and answers to Adobe Connect. Standard Captivate, Presenter and Flash quizzes only report the question text and the id of the chosen answer. Due to Hyundai’s requirements, webqem’s solution reports all details of the 40 questions selected randomly from the 120 question pool. The question copy and the full copy of all answers are reported to the Connect Training server for each participant.

Hyundai can create new course content in an Excel spreadsheet and convert this into XML, used by Adobe Flash and Connect. The XML, Flash and JavaScript components are then uploaded as content for a course, in Adobe Connect. Finally, participants are uploaded in a CSV file into Adobe Connect, and the course is ready for training.