Where's My Clicker? Bringing the Remote into the Classroom

By Gina M. Copas

The Classroom Performance System (CPS) is a software and hardware package developed by eInstruction Corporation that allows instructors to receive and instantly grade homework, reviews and tests. The software allows the instructor to construct and organize a set of classroom material to be delivered in a fun and fast-paced environment. The hardware consists of numbered remote response pads that allow students to beam their answers to a receiver unit. The software can then generate detailed reports about how each student performs in class (eInstruction Corp).

CPS provides two modes of assessment: (1) the Teacher Managed mode, and (2) the Student Managed mode. The Teacher Managed mode lets the teacher control the pace and students' progress through the questions. Students read the test questions that are displayed on a projection screen and beam their response into the receiver unit. The instructor manages the assessment by advancing the display of the assessment questions - the pace, therefore, is at the discretion of the instructor. Below each test question, the instructor has the option to display the response pad indicators which show by number which response pads have successfully beamed in an answer (see Figure 1).

 

 

 


Figure 1. Response Cycle Feedback Grid for a 32 Student Class (indicates which response pads have successfully beamed a response).

Figure 1. Response Cycle Feedback Grid for a 32 Student Class (indicates which response pads have successfully beamed a response).

Alternately, the Student Managed mode allows the instructor to distribute printed materials to the students who then “beam in” their responses at their own pace. The projection screen (shown in Figure 2) displays the number of each response pad (top) and the item number of the next question (bottom) on the assessment.

Figure 2. Student Managed Assessment Answer Window (indicates by response pad the question number that is to be answered next).

Figure 2. Student Managed Assessment Answer Window (indicates by response pad the question number that is to be answered next).

The objective of this pilot usability study was to obtain some initial usability and student satisfaction information regarding the CPS in an actual classroom setting. The intent was to determine what aspects of this product are well-received and which would warrant further usability testing.

METHOD

Participants/Materials

Sixteen graduate students (7 male and 9 female) enrolled in a research methods laboratory section in the psychology department at Wichita State University participated in this pilot usability and satisfaction study. None of the participants reported having used a remote response testing system prior to this study, however, 60% of the participants reported having used an online testing system (i.e., Blackboard).

The students were given a non-graded test on a course-relevant topic using the Teacher Managed Assessment mode. Questions were displayed on the screen one at a time and the participants beamed in their responses. Participants received instant feedback on the correct response to each test item. Participants were then given a satisfaction survey using the Student Managed Assessment mode. The survey questions were printed on paper but the participants beamed in their responses at their own pace. The projection screen only displayed the response pad number and question numbers. Participants were encouraged to write open-ended comments on the paper questionnaire. Responses from 2 of the 16 participants were not included in the final analysis because the participants made input errors when entering their responses.

RESULTS

Difficulty Rating

In rating the CPS system, 7% reported the system to be “difficult”; 52% of the participants reported it to be “easy” and 38% reported a “neutral” response (See Graph 1).

Figure 3. Difficutly rating for CPS system.

Entering Responses

All participants believed they were able to successfully beam their response to the receiver in one attempt per question, however, only 72% of them reported that they could “very easily” tell if the response had been successfully received by the system. In the Student Managed mode, 93% found it “easy” to determine which test question they were currently answering, while only 7% found it to be “very difficult” (See Graph 2).

Changing Responses

With regard to the option of being able to change a response that has already been beamed to the receiver, 64% of the participants found it to be “easy”, while 7% found it to be “difficult” and 29% reported that they did not try to change their responses (See Graph 3).

Figure 5. Difficulty rating for changing responses.

Privacy

Regarding the privacy aspect of electronically responding to test questions displayed on a projection screen, only 36% of the participants reported they felt their responses were still “private”; 29% believed their responses were “not private” and 29% responded “neutral” (See Graph 4).

Figure 6. Privacy ratings of beamed responses.

Instant Results and Assessment Mode Preference

Regarding the importance of receiving immediate feedback on test performance, 43% of the participants reported that it was “important” to get instant test results, while 57% reported that it was “not important” to receive instant test scores. Of the two CPS response modes, 36% of the participants preferred the Teacher Managed mode and 64% the Student Managed mode (See Graph 5).

Figure 7. Preference of assessment modes.

Compared to Other Test Methods

When comparing use of an automated test system and the traditional pencil-and-paper test taking method, 71% of the participants indicated they would prefer an automated test taking system, while the remaining 29% preferred the traditional method. Reported preferences between this CPS system and other online test systems for multiple choice and true/false testing were evenly split, with 50% in favor of CPS, etc.

DISCUSSION

It appears that most of the participants of this pilot study enjoyed using the Classroom Performance System by eInstruction Corporation. General verbal comments indicated that the novelty of the system was refreshing and fun. Other usability and satisfaction comments made by the participants included:

“In the instructor-managed test situation we were rushed through the questions, this was frustrating and affected performance. I kept my finger/thumb on the selection/button I chose so I could see if I got the correct answer during the instructor-managed session.”

“Better than Blackboard - Blackboard is less efficient than this. I am always worried something will crash or not be linked using Blackboard. I have more confidence in the remote system.”

“The instructor-managed approach showed more clearly when you answered, but I liked that you could go back on the student-managed approach.”

“Placement of the receiver could be an issue - making sure it's high enough so everybody can get to it. Privacy of answers might be more problematic.”
“I don't like the immediate feedback. For “lower” students, this could possibly lead to learned helplessness. I don't like having everyone “seeing” the rate of entry - this provokes competition and carelessness in favor of speed.”

“Instructor-managed could use a timing device so that you know how long you have left to answer. Student-managed seems made for a test/quiz situation, I wouldn't want to wait for others. I Prefer instructor-managed as a study guide learning format.”


“The letters on the remote should be vertical, not horizontal. That makes it more like a test and not confusing.”

“The questions should be across the top. Then the answers should be mapped according to the remote.”

 

Some of these comments address the potential mapping problem between the test question as displayed on the projection screen and the order of possible responses on the response pad itself. The questions were written horizontally with the possible multiple choice responses (A through D) listed vertically below the question. Conversely, the response pad had the response buttons grouped in horizontal pairs (i.e., A and B, C and D), which was incompatible with the display of the test question.

Further studies are planned for the investigation of these and other general usability issues with the Classroom Performance System. Items to be addressed will include:

REFERENCES

eInstruction Corporation (no date). Classroom performance system user's guide. (Version 3.12)