Usability News

July 2002, Vol. 4 Issue 2

Usability News is a free web newsletter that is produced by the Software Usability Research
Laboratory (SURL) at Wichita State University. The SURL team specializes in software/website user interface design, usability testing, and research in human-computer interaction.
Barbara S. Chaparro, Editor


Article PDF Print Article E-mail the Usability News Article


Bringing the Chat Room to the Classroom


By Mark C. Russell & Charles G. Halcomb

One of the more interesting —and perhaps underused —tools provided by the Blackboard online education support system (www.blackboard.com) is the"virtual classroom." For those unfamiliar with Blackboard, it is a web-based server software platform which is used by many institutions, including Wichita State University, to supplement traditional classes. Blackboard provides many tools and resources for both instructor and students, including ways to organize assignments, lecture notes, external world wide web links, etc. The"virtual classroom" is essentially a 'chat' application with access restricted to students enrolled in the class. Students are able to log in from any computer with internet access and a java-enabled web browser.

virtual classroom

In the Spring 2002 semester, we used the virtual classroom on several occasions in a graduate-level Human Factors psychology course to discuss various topics related to Human-Computer interaction (HCI). The purpose of the discussions was to stimulate students in the class to think about issues surrounding topics presented in the text, and to motivate the students to read and think about the material from the text and other sources. We were interested in exploring the communication patterns of the class in terms of amount of participation on and off-topic, the role of a designated mediator, and the role of the course instructor and contrasting this with the face-to-face discussion experience.

Table 1. Course Details

Course name: Software Psychology; a seminar course in the human factors doctoral program offered through the Psychology Department at Wichita State University. The instructor sometimes refers to the course as Introduction to Human-Computer Interaction.
The forum: Blackboard's virtual classroom chat software (provided by Tutornet.com, Inc.). Students logged in from remote locations or from computers located in the actual classroom.
The task: The class was instructed to discuss a specific topic related to human-computer interaction; not a formal debate—opinions welcome, evidence not required but encouraged.
Instructions: A student was designated as facilitator and asked to begin and guide the conversation. The instructor participated as well.



Method

We reviewed the transcripts of two class sessions (automatically archived by Blackboard) and recorded the following:

  • number of posts (individual segments of text submitted for others to see on the computer screen) made by each participant;
  • type of entry, whether they be on or off-topic, etc.;
  • the differences between how the designated mediator compared in participation to the actual class instructor.

    In addition, we interviewed the students about their personal impressions of the virtual classroom, and compiled the results into a table outlining the perceived advantages and disadvantages of this classroom format.

Analyzing the sessions

In order to standardize the analysis, the following operational definitions were used:

(1) Contributory comments: any posting which appeared to be adding to the conversation at hand in some meaningful way, whether it be new information, opinion, or even encouragement or agreement with the postings of others. Almost any 'on subject' comment fell into this category.

(2) Miscellaneous comments: the 'aside' bits of conversation that served no actual purpose, such as jokes, greetings, and other class business unrelated to the topic of interest.

(3) Guidance comments: questions or statements which directed conversation in some way, posted either by the facilitator or the instructor.

Any posting of text by a student, regardless of length, was included in the count and categorized according to content.

Facilitator vs. Instructor

In an effort to encourage student participation and differ the virtual format from a more traditional instructor lecture, a student was designated as the facilitator for each session. This facilitator was to make suggestions for material to discuss and keep the overall discussion on track. The facilitator also added to the conversation and invariably made miscellaneous comments just like everyone else. However, the instructor also attempted to direct the conversation at various times.

It was very difficult to precisely determine exactly how many conversation 'threads' were in existence at any one time, nor was it exactly clear to what extent there was any effect on the conversation by the facilitator/instructor competition. Therefore, for our purposes here we simply stuck to the numbers of posts and their general classifications.

Results

Table 2 shows a summary of the number of various types of posts made by the students, facilitator, and instructor during the two sessions examined. This rough analysis is not an attempt to compare the two instances, but rather a simple attempt to get an idea of individual/group participation. Keep in mind, the only assignment for the students was to engage in a meaningful discussion about the assigned topic; there was no other specific task, and it was not a formal debate. Tables 3 and 4 show specific details for Session I and II.

Table 2. Summary of participation by session.

  Session I Session II
Total contributory posts
272
247
Total miscellaneous posts
51
112
Facilitator"guidance" posts
8
10
Instructor"guidance" posts
9
7
Total posts
340
376
Average number of posts
26
29


Table 3. Session I Details

When: Near the beginning of the semester
Duration: 1 hour and 30 minutes
Participants: 11 graduate students and the course instructor
Topic: The role of models/theories in the practical world
Student Participation: Lowest number of posts: 3
Highest number of posts: 68 (facilitator)

Comments: One of the expectations of the virtual classroom environment was that student participation would be increased or at least made more equal. One of the ways in which virtual environments usually foster this equality is through the ability of participants to submit posts simultaneously and by keeping the author of the post anonymous. In this session, we had the former, but not the latter; students were identified on-screen by name. There was no reason, however, to keep the students anonymous, nor is there any reason to believe that participation would have changed had anonymous posts been permitted. More likely, participation was dictated more by the student's comfort with the conversation topic and other less technology-related reasons.

Table 4. Session II Details

When: Approximately six weeks after"Session I"
Duration: 1 hour and 18 minutes
Participants: 12 graduate students and the instructor.
Topic: A comparison of"natural" versus computer languages.
Student Participation: Lowest number of posts: 2
Highest number of posts: 58 (facilitator)

Comments: There are no striking differences between the two sessions, except for an increase in the amount of miscellaneous or off-topic comments made by the students in Session II. This is certainly not an unnatural thing to occur, as the semester had progressed and the students undoubtedly had become more comfortable, not only with each other, but with the virtual format as well. In addition, these miscellaneous comments accounted for just under 30% of the total number of posts. Whether this seems an inordinate amount of side-talk would of course be a matter of opinion. But anyone who has stood in front of a classroom full of eager, young minds and asked for comments only to be confronted by a sullen silence disrupted only by the plaintiff chirpings of a lone cricket might eagerly take this necessary evil in exchange for increased class participation. Also, since each participant was seeing the posts of the other participants typed onto a computer terminal, it would seem likely that irrelevant comments could be more easily ignored than would be the case in a face-to-face discussion.

Student Impressions

At the end of the semester students were interviewed on their impressions of the virtual class experience as they compared to the traditional face-to-face class sessions. Table 5 summarizes their comments in terms of perceived advantages and disadvantages of this format.

Table 5. Student Perceptions of the Virtual Classroom

Advantages Disadvantages

Allowed more participation.

Participation voluntary—'quiet people' could still stay quiet

"More" talk does not necessarily mean"better" talk
Reading/typing time delay gives you time to compose your thoughts So much text to read; you could easily get behind before posting a response
Class discussion was informal, comfortable A more formal moderator arrangement might work better, eliminate excess talking
Instructor participation was welcome Instructor's comments and questions shifted conversation dramatically; instructor was answered instead of facilitator
International students participated more than in regular class, possibly understood better Misspellings can cause confusion
Not hard to stay on topic; side comments not perceived as excessive in number Several conversations going at the same time can be confusing
Able to type in comments as you think of them without waiting Lack of non-verbal information; could not detect emotional nuances; not always sure about jokes
Time seemed to pass quickly; discussion more engaging than lecture Discussion could seem aimless depending on facilitator style and task specificity


Instructor Impressions

The instructor of the course was also asked to provide his impressions of how the 'virtual' discussion compared with the actual face-to-face discussion and how well, in his opinion it had worked. His impression was that the virtual discussion had elicited considerably higher volume of comments from a more representative cross-section of the class and that for the most part the discussions stayed focused on target. He did comment that in the future, it might be possible to structure the discussion in such a way as to make the outcome even more consistent with the goals. He was interested in the extent to which his presence impacted the outcome and has suggested that in the future we take a look at how the role of the instructor should best be formulated to achieve the best possible outcome.

Discussion

By no means can one presume that we have provided any definitive answers to the question of whether or not there is utility in the use of the computer-mediated discussion group in a classroom setting. However, it is our hope that you will be struck, as we were, by the generally positive reaction to the experience. The instructor thought that the virtual sessions were successful. The goal had been to increase participation by all students and provide the motivation to learn the course content throughout the semester without the need of an examination. The observations of the students and instructor as well our survey demonstrate that their was an increased participation in the discussions.

We have noted that in our sessions there was a facilitator and that the instructor participated. While it remains for future studies to define and evaluate the impact of instructor participation, it is clear from this experience that, whatever the consequences of instructor participation, the instructor was able to participate without completely dominating the discussion as so often happens when the discussion is face to face. Also, looking at it from the viewpoint of the instructor, having the ability to document the participation of each individual student and ask that all of the students review and evaluate what took place during the discussion was an important and unexpected benefit afforded by this format. If there were no other positive benefits, the instructor felt like the virtual sessions were successful.

We all were surprised by the students participation in the process. Even those students who verbalized a dislike for the more 'impersonal' nature of the communication participated actively in the discussions. Another major advantage of the procedure was that students could be asked to review the archives following each session and submit a summary or evaluation of the discussion as an individual project. This ability to 'replay' the discussion and reflect on or add comments after the fact was something that was viewed by both students and instructor as a positive outcome. Even the simple act of summarizing the threads of conversation that had occurred served to reinforce the positive aspects of the experience.

In summary, this single classroom activity produced two behavioral products: (1) first, the virtual group discussion itself, complete with the increased motivation for individual participation and the benefits of collective group insight on the discussion topic; and (2) the capacity for the individual student to go back and summarize and reflect on the experience served to extend the benefits of the activity beyond the classroom and served as a catalyst for many of the students to help in the consolidation of the ideas and thoughts which had emerged during the discussion. It seems likely that, with an awareness of this dual capability afforded by this format, in the future, these sessions can be formulated to take better advantage of the group collaboration with full understanding that an individual response will also be required.

These virtual discussions took place in the classroom in the context of an 'on-campus' course where the student met in a face-to-face setting, and interacted with the instructor in an actual classroom setting, the virtual classroom format as described in this paper could also be developed to provide an enriched interactive opportunity for the student who might be taking the course as a 'distance education' course. This becomes more important as economic and other factors conspire to make the off-campus course more and more necessary for many seeking to further their educational experience.

There are, of course, questions which still beg for answers. For example:

  • Do the multiple threads of conversation commented on by a number of the students, actually distract, or are the just an annoyance which with time will become an experience that will drop from awareness?
  • How can the sessions best be structured to take advantage of the archiving feature to extend the activity beyond the group discussion to an individual learning effort?
  • To what extent should the instructor participate?
  • In what way does participation by the instructor add or detract from the educational benefit of the experience?

These are all questions that a more formal examination of this virtual classroom should help answer. The internet is often billed as the 'information highway', so let us find ways to use tools such as 'chat' forums to help us 'transport' more educational opportunities to the student.

 

Article PDF Print Article E-mail the Usability News Article


Related Articles:

SUBSCRIBE to Usability News!