Usability News - 7.1 2005 -- Easy Print Version

Often credited with creating a first impression, fonts are classified according to unique typographical features (serif, sans serif, etc) and overall appearance. The combination of appearance and typographical features frequently lead graphic artists and typographers to describe typefaces using personality traits (“less cuddly, more assertive,” Berry, 2004). In a recent BBC audio program (Peacock, 2005), fonts were depicted as feminine and masculine, among other traits. Typographers and designers are often interested in the typeface personality or “typographic allusion” which refers to “the capacity of a typestyle to connote meaning over and above the primary meaning which is linguistically conveyed by words” (Lewis & Walker, 1989, p. 243).

Studies have investigated the perceived personality of fonts for printed text; however, similar studies have not been conducted regarding onscreen text. We are investigating the perceived personalities and uses of a variety of fonts used onscreen.

As subscribers to Usability News, we thought it would be interesting to include your data in our analyses. We are eager to include diverse populations to better generalize our findings to those who are designing and developing for digital media.

Please take the time (30-45 minutes) to complete our online survey by clicking the following link (this link will open a new browser window):

Click on this link to take the survey at http://www.shaikh.us/fontstudy/

We thank you for your time and your interest. Results of the survey will be posted in our next issue of Usability News.

Berry, J.D. (2004). Now read this: The Microsoft ClearType font collection. Seattle, WA: Microsoft Corporation.

Lewis, C., & Walker, P. (1989). Typographic influences on reading. British Journal of Psychology, 80, 241-257.

Peacock, I. (Speaker). (2005). From Arial to Wide Latin (Radio Broadcast). London: BBC Radio. (Available online: http://www.bbc.co.uk/radio4/science/fromarialtowidelatin.shtml)

The amount of text-based information available online is steadily increasing. Universities are offering more online classes; publishers are releasing more online books and journals; and consumers are gathering online news, weather, and product information daily. Therefore, the examination of the factors that influence the readability of online text is very important. In a recent survey, Shaikh and Chaparro (2004) evaluated the reading habits of Internet users across five document types (journal articles, news, newsletters, literature, and product information) and found that users preferred to read journal articles in printed form, but other documents such as online news, newsletters, and product reviews in online formats.

Research has shown that many aspects of physical layout of online text impact reading performance and satisfaction; Dyson (2004) gives an excellent review. One such factor that has been studied is line length, or the number of characters presented per line of text. Research investigating line length for online text has been inconclusive. Several studies found that longer line lengths (80 – 100 cpl) were read faster than short line lengths (Duchnicky and Kolers, 1983; Dyson and Kipping, 1998). Contrary to these findings, other research suggests the use of shorter line lengths. Dyson and Haselgrove (2001) found that 55 characters per line were read faster than either 100 cpl or 25 cpl conditions. Similarly, a line length of 45-60 characters was recommended by Grabinger and Osman-Jouchoux (1996) based on user preferences. Bernard, Fernandez, Hull, and Chaparro (2003) found that adults preferred medium line length (76 cpl) and children preferred shorter line lengths (45 cpl) when compared to 132 characters per line.

Shaikh and Chaparro (2004) found that 62% of respondents preferred to read news onscreen rather than on paper. The goal of this study was to investigate the effects of line length on reading performance of online news articles. This study investigated the effects of four line lengths (35, 55, 75, and 95 cpl) on reading speed, comprehension, and satisfaction.

Twenty college students from a Midwestern university volunteered to participate in the experiment and were compensated with $25. All participants had 20/40 or better unaided or corrected vision. Participants reported visiting websites daily, had experience reading online, and were regular users of the Internet.

News articles with approximately 375 words (M = 367.9) were selected from MSN® and Yahoo® news. Average reading level for each passage was at the 12.0 grade level. The news articles covered a variety of topics ranging from computer viruses to innovative automobiles. The passages used Arial 10-point font with 12-point interlinear spacing between lines. Each paragraph was separated with an additional line break as is typically done on web pages. Hyphenation was not used in the text.

A total of 19 comprehension questions were written for each article including one title, main idea, and structure question and three factual and incidental questions, and 10 recognition questions (see Dyson & Haselgrove, 2001). A background questionnaire was used to collect demographic information as well as computer and Internet usage patterns. A Reading Satisfaction Evaluation consisting of 11 questions on a 7-point Likert Scale collected user reactions to the passage layout, perceived eyestrain, and fatigue. A Post-Experiment Questionnaire was used to determine layout preference.

Following the completion of the consent form and the background questionnaire, participants were seated approximately 52 cm from the screen. The pages of text were displayed using Internet Explorer 6.0 in full screen mode to reduce distractions of tool bars, scroll bars, and menu items; paging (with no scrolling) was used to navigate the passages. The HTML pages contained “Next” and “Previous” buttons that users clicked with the mouse to navigate. All HTML pages were accessed from the computer hard drive to minimize download time.

Participants read a total of eight short news articles presented in counterbalanced order for line length. Two news articles were presented in each line length. Participants completed the set of comprehension questions after reading each passage. The Reading Satisfaction Questionnaire was given after each set of short passages. Following the last passage, the Post-Experiment Questionnaire was administered.

Reading Speed. Reading time was converted to words per minute. Results from a one-way within subjects ANOVA showed that there was a significant main effect of line length on reading speed, F(3, 57) = 3.45, p = .02, η² = .15. The 95 cpl (M = 178.82, SD = 41.83) articles were read significantly faster than any of the other line lengths (35 cpl M = 167.21, SD = 33.66; 55 cpl M = 167.38, SD = 33.96 ; 75 cpl M = 169.44, SD = 33.48) (Figure 1).

Reading Efficiency. Reading efficiency was computed by multiplying the reading speed by the percent correct on the comprehension questions. Results of a one-way within subjects ANOVA showed a marginally significant effect of line length, F(3, 57) = 2.68, p = .06, η² = .12. Significant differences were found between 35 cpl and 95 cpl with the 95 cpl being more efficient.

Overall Comprehension. To compute an overall comprehension score, raw scores were adjusted per question type to yield a total comprehension score of 6 (based on the six types of questions). In doing this adjustment, each type of question was worth a total of one point. Results from a one-way within subjects ANOVA showed no effect of line length on overall comprehension.

Comprehension by Question Type. Results from a one-way within subjects ANOVA revealed a significant main effect of question type on comprehension, adjusted F(3.98, 75.69) = 10.193, p = .001, η² = .35. Participants scored significantly lower on the structure questions than title, main idea, and factual questions across all line lengths (Figure 2).

Satisfaction & Preference. There was no main effect of line length on overall satisfaction. There were also no significant differences in preference. However, 60% of participants chose either 35 cpl (30%) or 95 cpl (30%) as the most preferred line length. On the other hand, 100% of participants chose either 35 cpl (45%) or 95 cpl (55%) as the least preferred line length.

This study examined the effects of line length on reading performance. Reading rates were found to be fastest at 95 cpl. Readers reported either liking or disliking the extreme line lengths (35 cpl, 95 cpl). Those that liked the 35 cpl indicated that the short line length facilitated “faster” reading and was easier because it required less eye movement. Those that liked the 95 cpl stated that they liked having more information on a page at one time. Although some participants reported that they felt like they were reading faster at 35 cpl, this condition actually resulted in the slowest reading speed.

Circulation of newspapers at 814 of America’s largest daily newspapers declined 1.9% from September 2004 to March 2005 (Shin, 2005). This decline is part of a 20-year trend in newspaper circulation and is due, in part, to the increased use of the Internet and other forms of media (cable, satellite, etc). As users continue to choose online news sources, it is imperative to understand factors that contribute to improving the overall online reading experience for news. Participants were able to read news articles significantly faster while maintaining high reading efficiency using 95 cpl. Despite the fact that there were no differences in satisfaction scores, a line length that supports faster reading could impact the overall experience for users of online news sources.

Note: For additional information on this study, please see the proceedings of the Human Factors and Ergonomics Society's 49^th (2004) Annual Meeting, Orlando, FL.

Bernard, M. L., Fernandez, M., Hull, S., & Chaparro, B. S. (2003). The effects of line length on children's and adults' perceived and actual online reading performance. Proceedings of the Human Factors and Ergonomics Society 47th Annual Meeting, 1375-1379.

Duchnicky, R. L., & Kolers, P. A. (1983). Readability of text scrolled on visual display terminals as a function of window size. Human Factors, 25, 683-692.

Dyson, M. C. (2004). How physical text layout affects reading from screen. Behaviour and Information Technology, 23, 377-393.

Dyson, M. C., & Haselgrove, M. (2001). The influence of reading speed and line length on the effectiveness of reading from screen. International Journal of Human-Computer Studies, 54, 585-612.

Dyson, M. C., & Kipping, G. J. (1998). The effects of line length and method of movement on patterns of reading from screen. Visible Language, 32, 150-181.

Grabinger, R. S., & Osman-Jouchoux, R. (1996). Designing screens for learning. In H. van Oostendorp & S. de Mul (Eds.), Cognitive aspects of electronic text processing. Norwood, NJ: Ablex Publishing Corporation.

Shaikh, A. D., & Chaparro, B. S. (2004). A survey of online reading habits of Internet users. Proceedings of the Human Factors and Ergonomics Society 48th Annual Meeting, 875-879.

Large high-resolution displays can now have resolutions of over 1900 pixels, resulting in extremely long lines of text. One way to resolve the problem of very long text lines is to divide the text into multiple columns, thus decreasing the width of each individual line. Some sites even allow users to customize pages into one, two, or three columns (for an example, see http://www.travelyellowstone.com).

To date, research investigating the optimal use of line length, multiple columns, and text justification is inconclusive. Longer line lengths typically result in faster reading times (Duchnicky and Kolers, 1983; Dyson and Kipping, 1997; Dyson and Kipping, 1998; Dyson and Haselgrove, 2001), but research suggests medium to short line lengths typically may result in higher comprehension (Tinker, 1963; de Brujin, de Mal, & van Oostendorp, 1992; Chaparro et al., 2004). In terms of columnar text, the research supports both long single columns of text (Dyson and Kipping, 1998), and multiple short columns (Lam et al., 2000) while preference seems to lie towards multiple short columns (Bouma, 1980; Dyson and Kipping, 1998; Andreyev & Martynov, 2000).

There has been little research done on the impact of text justification on online reading. Priestly (1991) and Hartley (1986) suggest that printed materials should be left-justified because the intra-word spacing remains constant, which helps slower readers identify words more quickly. However, Priestly notes that the question of whether instructional material is best presented in a full-justified or left-justified format is still debated. Full justification requires that both margins be aligned. To accomplish this, large spaces must sometimes be made between words, or words must be hyphenated, both of which can reduce reading speed and comprehension. Left-justified text is generally preferred because it removes the need for hyphenated words.

In summary, the major findings regarding line length, justification, and multi-column displays and online reading have been:

This study investigated the effects of multi-column displays and justification on reading performance and satisfaction of an online narrative passage.

In this study, participants read text displayed either as a single column (90 CPL), a 2-column format (45 CPL each), or 3-column format (30 CPL each) presented using either full or left justification.

Sixty-six undergraduate students with normal or corrected vision participated in the study. Participants had an average age of 22.8 years. Data were collected on the participants’ amount of online reading experience and types of materials read online. Eighty-nine percent reported using the Internet a few times per week or more. Forty-eight percent of participants indicated that they read online for two to six hours per week, 15% reported reading online for seven to fourteen hours per week, and nine percent reported reading online for fifteen hours or more per week.

H.H. Munro’s (“Saki”) short story “The Lumber Room,” a passage of 2191 words, was used as the reading material. The story had a readability score of 9.6 on the Flesch-Kincaid Grade Level statistic. Six versions of the story were created: a single column with a width of 90 CPL, a 2-column format with column widths of 45 CPL, and a 3-column format with column widths of 30 CPL, each presented in either a full or left justification format (See Figures 1 – 6).

Passages were presented on Dell Dimension 4600C desktop with a 17” display running 1024 x 768 screen resolution. Each of the passages was presented on six consecutive pages; users clicked on an arrow at the bottom of each page to advance to a subsequent page or return to a previous page. No scrolling was required. Users read the passages at a distance of approximately 60 cm and passages were displayed in 10pt Verdana font.

The design of this study was a 2 x 3 randomized block design with matched subjects. Column format (single, 2-column, and 3-column) and Justification (Full, Left) were the independent variables.

Participants were matched across conditions based on reading speed predetermined by a online reading test. Participants were then asked to read the short story displayed in one of the six different column conditions (one, two, or three columns, in either a Full or Left-justified format). After reading the story, participants answered sixteen comprehension questions (fifteen multiple-choice and one recall question with ten units) about the story, modeled after those used by Dyson and Haselgrove (2001). The dependent variables of interest were:

A 2x3 randomized block ANOVA found a significant interaction for justification x number of columns, F(2,20) = 3.812, p = .040. η² = .276, 1 – β = .624 (see Table 1). Post-hoc Tukey’s HSD comparisons showed that reading speed was significantly faster for two-column full-justified (M=269.33) than for one-column full-justified, (M=224.31), and significantly faster for one-column left-justified (M=266.43) than for one-column full-justified (M=224.31), or three-column full-justified (M=227.60) (see Figure 7).

A 2x3 randomized block ANOVA found no significant main effects for reading efficiency though an interaction approaching significance for justification x number of columns was found, F(2,20) = 3.124, p=.066. η² = .238, 1 – β = .534.

To compute an overall comprehension score, raw scores were adjusted per question type to yield a total comprehension score of 6 (based on the six types of questions). A 2x3 randomized block ANOVA found no significant main effects or interaction for either justification or number of columns for total comprehension (Table 2).

A 2x3 randomized block ANOVA found no significant main effects or interaction on overall satisfaction.

Originally, participant reading rate was not considered as an independent variable in this study. Dyson & Kipping (1997) propose that fast and slow readers use different reading strategies that may impact comprehension. They suggest that faster readers are able to scan narrow columns more efficiently and increase their comprehension. Based on this idea, the fastest readers in this study were compared to the slowest readers by condition. Reading speed, reading comprehension, reading efficiency, and reading satisfaction were then analyzed using a 2 x 2 x 3 ANOVA.

In general, results from these analyses indicate that the fastest readers benefited most from the 2-column justified text, while the slowest readers performed best at the 1-column left justified format (see Figure 8). In addition, satisfaction levels were found to be higher for the fast readers at the two-column full-justification condition than the other conditions.

The purpose of this study was to examine how multiple columns and text justification impact online reading in terms of reading speed, comprehension, and satisfaction of a narrative passage. Results from this study showed that reading speed was significantly faster for two-column full-justified text than for one-column full-justified text. Post-hoc analyses showed that it was the fastest readers that benefited the most from this format.

Slower readers showed their fastest reading and highest reading efficiency at the one-column left-justification condition. This may be because the very short lines impeded the reader’s ability to take in an optimal amount of information at each fixation. Gutherie & Wigfield (2000) assert that a slow reader may lose all information about the beginning of a sentence from short-term memory before he or she has read to the end. Slow readers may also have had difficulty “keeping their place” with the multiple line length conditions for this same reason.

Numerous factors contribute to the ease of online reading. As more types of documents become digital, and more people read from online sources, finding the optimal presentation of these materials will become critical in the near future. Reading and comprehension tests are moving into the digital domain and studies such as this one can help researchers and designers know which types of layouts are best for their audience. A two-column full-justified format is best for situations where fast readers are the primary audience. However, for more a more equalized setting, a layout with no strong advantages or disadvantages may be best.

The results of this study suggest that there is not one best way to present text online. Although fast readers performed best at the two-column full-justified condition, slow readers benefited from a single column non-justified layout. This suggests that users should have the option to customize web pages to suit their reading needs. To date, there are very few sites that allow users to tailor information displays into multiple columns or adjust justification styles or font types. Giving users these options can help increase their reading speed with the text, and eventually may lead to a more optimal online reading experience.

Andrevey, V., & Martynov, A. (2000). Effects of splitting text into multiple columns. Dept of Computer Science Class Project, Univ. of Maryland, College Park, MD. Available at: http://www.otal.umd.edu/SHORE2000/multicol/index.html

Bouma, H. (1980). Visual reading processes and the quality of text displays. In Ergonomic Aspects of Visual Display Terminals, E. Grandjean and E. Vigliani (Eds.) Taylor & Francis: London pp. 101-114.

De Brujin, D., De Mul, S., & Van Oostendorp, H. (1992). The influence of screen size and text layout on the study of text. Behaviour and Information Technology, 11, 71-78.

Duchnicky, J.L., & Kolers, P.A. (1983). Readability of text scrolled on visual display terminals as a function of window size. Human Factors, 25, 683-692.

Dyson, M.C., & Haselgrove, M. (2001), The influence of reading speed and line length on the effectiveness of reading from a screen. International Journal of Human-Computer Studies, 54, 585-612.

Dyson, M.C., & Kipping, G.J. (1998), The effects of line length and method of movement on patterns of reading from screen. Visible Language, 32, 150-181.

Dyson, M.C., & Kipping, G.J. (1997), The legibility of screen formats: Are three columns better than one? Computers & Graphics, 21, 703-712.

Hartley, J. (1986). Designing Instructional Text. 2nd edition. Kogan Page, London.

Lam, K., Lam, Y., Liu, J., & Shin, U.G. (2000). Reading Comprehension and Rate: One Column vs. Three Columns. Dept of Computer Science Class Project, Univ. of Maryland, College Park, MD http://www.otal.umd.edu/SHORE2000/columns/index.html

Priestly, W. (1991). Instructional typographies using desktop publishing techniques to produce effective learning and training materials. Australian Journal of Educational Technology, 7, 153-163. Available at: http://www.ascilite.org.au/ajet/ajet7/priestly.html

Tinker, M.A. (1963). Legibility of type. Ames, IA: The Iowa State University Press.

Eye-tracking has been used in usability testing for many years, beginning primarily with cockpit design testing (Fitts, Jones and Milton, 1950). Such early studies were valuable in establishing some assumptions about the relationships between eye-movement data and the cognitive activity of a user. Frequency of fixations was assumed to be related to the importance of the control, while the duration of fixations was related with ease of interpreting information.

In recent years, eye-tracking has been adapted to usability studies involving web-based stimuli (for review see Duchowski, 2003; Hyönä, Radach and Deubel; 2003). Usability professionals and independent research groups have been attempting to identify the specific contributions of eye-tracking to website design and usability. Many such studies involve the identification of the specific targets of the users’ visual attention to different parts of the interface, or Areas of Interest (AOI). The data relative to these AOIs can then be compared in different ways, including order, number and duration of fixations to each AOI.

As part of a continuing investigation into the contributions of eye-tracking to usability testing, user eye-movement data was gathered during testing of three e-commerce websites. The data was examined in terms of various AOIs on the screens and was used to uncover more information about how the users interacted with these interfaces.

Thirty-six undergraduate and graduate students at Wichita State University (26 female, 10 male) at Wichita State University participated in this study. Seventy-two percent of the participants were between the ages of 18 and 26 years. A Pentium IV-based PC computer with 96 dpi, 17” monitor with a resolution setting of 1024 x 768 pixels was used. The monitor was integrated with the Tobii 1750 eye-tracking system, which was used to detect and collect participant eye-gaze data during testing. The Tobii 1750 eye-tracker samples the position of the user’s eyes on an average of every 20ms (i.e., 50Hz) and is characterized by the unobtrusive addition of the eye-tracking hardware (e.g., high resolution camera and near infra-red light-emitting diodes) to the monitor frame. This design aspect helps promote more natural user behavior by not placing unnatural restrictions on participants (e.g., helmets, head-rests, etc.). Three e-commerce websites specializing in the sale of educational toys were used in the study: Mastermind Toys, Toys to Grow On, and Wonder Brains.

Participant performance data was compared on three usability measures: time on task, number of pages visited, and perceived difficulty (Likert-type scale, 1 = “Very Easy”, 5 = “Very Difficult”). The eye-movement data collected during testing included the number and duration of visual fixations relative to different AOIs on each site’s homepage. The results discussed below are for two of the tasks performed during usability testing of the sites.

This was a simple information search task that requested users to find contact information for the toy company. On this task Site 2 (MasterMindToys.com) had significantly higher scores for both perceived difficulty (F (2,33) = 5.459, p. = .009) and for Time on Task (F (2,33) = 9.841, p. < .001) (see Table 1).

The specific links that lead to the company contact information were located in different areas of the homepages for each respective site. Site 2 (MasterMindToys.com) had the proper link located among a list of other hyperlinks on the right hand side of the page. By way of contrast, Site 1 (Toystogrowon.com) had the link at the bottom of the page, and Site 3 (WonderBrains.com) had the link located in multiple locations at the top and bottom of the page. The elevated time and difficulty scores for Site 2 suggest that users may have had some preconceived notions of where to find certain common webpage components on a homepage (i.e., contact information), which resulted in poorer performance when the link was in an unexpected location. This view is substantiated by research into web objects and user schemas (Bernard, 2001; Bernard 2002).

Eye-tracking data for this task was used to explore how users went about searching for these particular hyperlinks on the different homepages. Based on the average time to first fixation received to the individual AOIs on the homepage, the users of Site 1 started their visual search for the appropriate link in the center of the page then examined other web components along the edges the page, such as the top shopping cart links and left navigation bar (Figure 1).

A similar bias for searching for the contact information link at the top of the page is shown by the AOI data for Site 2, suggesting that expectations as to this type of link’s typical location worked against the users when the link is located elsewhere, such as the links on the right side of the page, which were typically fixated upon last by users (Figure 2).

The ClearView eye-tracking software can represent the areas of the screen receiving either more fixations or receiving the longest dwell times in a color-coded “hotspot” image of the interface, with the closer to red the more fixations occurred in an area of the interface and intensity decreasing with movement down the spectrum.

Inspection of these hotspots on the homepages for this task relative to the AOI definitions reveals a very distributed pattern of user eye-movement on Sites 1 and 2 (Figures 3 and 4). This suggests that users directed their visual attention to several areas other than that where the correct link was located. For whatever reasons, it was these areas that users searched most frequently and longest for the correct link in addition to the area where they eventually found it.

On the other hand, the hotspots for Task 1 on Site 3 were more concentrated and directed primarily at the correct links located at the top and bottom of the page (Figure 5).

This task asked participants to find a puzzle for children ages 5 and up. There were significant differences observed for difficulty (F (2,33) = 3.593, p. = .039), time on task (F (2,33) = 5.564, p. = .008), and number of pages (F (2,33) = 4.949, p. = .013) (see Table 2), with Site 1 (Toystogrowon.com) having the highest scores. These results suggest that there was something about Site 1 that made it more difficult for users to shop by age than on the other two sites. These findings are corroborated by a review of the AOI data for this task.

This task could be completed in two different ways on all three sites: (1) using the categories on the left navigation bar to find a puzzle and narrow the search from there; or (2) using the shop by age option available from the homepage. On Site 1, the shop by age option was accessed through a single hyperlink, a solution used by only one participant. Based on the average time to first fixation for each AOI, the area where this link was located was typically attended to later than many other areas (Figure 6).

Examination of the fixation and dwell time data in terms of hotspot maps of the homepage during this task assists with the interpretation of this data (Figure 7). Although the left navigation array received the most attention, the “shop by age” option in the upper navigation array was minimally noticed, primarily by the one user that clicked on this link. Higher fixation rates or dwell times in this area would be indicative of users finding the option but perhaps not recognizing its function, and thus failing to choose it. The relative lack of visual attention to that area suggests that few users other than the one who chose the “shop by age” option ever fixated upon it.

Figure 7. Hotspot map of the homepage showing areas of highest fixation for Task 2

On Site 2 (MasterMindToys.com), all the participants but two used the “shop by age” option which was broken down into a range of age-related links on the top of the left navigation bar. This area of the homepage received the majority of the users’ visual attention (Figure 8). The product category links on the left navigation bar and the “shop by age” links above them were attended to relatively late on average, but this did not seem to impact performance for most participants. There were some aspects of the page not useful in this task that received considerable visual attention, including the large picture of the store interior and the center links to the right of the picture, similar to Task 1 on this site (discussed above). This observation is confirmed by the hotspot map of the homepage for Task 2 (Figure 8), and may indicate that these links are distracting and/or listed in a manner that is difficult to quickly glance through and eliminate as potentially useful during a search task. However, the overall design of the page and placement of the age-related links was conducive to a successful search for products by age, an assessment supported by the eye-tracking data.

The users of Site 3 (WonderBrains.com) were more evenly split on the use of the “shop by age” option, though it was less popular and the age-related links received less attention than the standard toy categories higher up on the navigation bar (Figure 9). In fact, examination of this hotspot map confirms that the most of the visual attention directed toward the homepage during this task involved the appropriate links—either categorical or age-related—and little else. This implies that the design of Site 3’s homepage was conducive to shopping (at least for this task) and held few visual distractions for participants.

Much can be learned about the nature of the user experience in the comparison of user visual attention afforded to different areas of web interfaces. Specifically, the use of eye-tracking data allows usability professionals to gather more information about what areas of interest (AOIs) are potentially: (1) most eye-catching; (2) most informative; (3) most frequently ignored; and (4) most distracting. In this way eye-tracking can be used to determine not only whether users are successful at navigating to a particular page of a website, but also where and how they search the interface for a targeted link or web object. This use of eye-tracking data as supplemental information to standard usability testing can be very informative as to both user expectations and resulting design recommendations.

The results of this study emphasize the importance of interpreting user eye-movement data in usability testing within the context of the interface being evaluated. In future studies, AOI eye-tracking could also be utilized not only within a single webpage or interface screen, but also across different pages of a site (see Pan et al., 2004), multiple exposures to a site (see Josephson and Holmes, 2002), and between different stages of a complex usability task. These comparisons could also be performed both in terms of iterative studies between different versions of web page/site and comparative studies between entirely different websites.

Duchowski, A. T. (2003). Eye tracking methodology: theory and practice. London: Springer.

Fitts, P. M., Jones, R. E. & Milton, J. L. (1950). Eye movements of aircraft pilots during instrument-landing approaches. Aeronautical Engineering Review, 9(2), 24-29.

Hyönä, J., Radach, R. and Deubel, H. (2003). The mind’s eye: Cognitive and applied aspects of eye movement research, Amsterdam: Elsevier Science.

Josephson, S. and Holmes, M. E. (2002). Visual attention to repeated images on the World-Wide Web: Another look at scanpath theory. Behavior Research Methods, Instruments, & Computers, 34(4), 539-548.

Pan, Hembrooke, Gay Granda, Feusner & Newman, (2004). Determinants of web page viewing behavior: An eye-tracking study, Proceedings of the Eye tracking research & applications symposium on Eye tracking research & applications, 2004, San Antonio, Texas.

Children between the ages of 6-12 are using the Internet more and more as a resource for finding information for school projects. The UCLA Center for Communication Policy (2003) reports that approximately 75% of children who used the Internet in 2002 went online at their school, an increase of about 12% from 2000. Educational website creators have the challenge of designing for a wide range of ages and abilities. Arsenenault (2004) tested the usability of 24 commercial websites for children elementary-aged children in a private school in Montreal, Quebec. She found that while children preferred an interactive website, too many frames caused confusion and tables with elaborate listings and poorly organized links were cognitively demanding (especially for those under ten years old). The use of icon metaphors, excessive animation, and required plug-ins also were reported to be problematic to this population.

Bilal (2001) carried out an experiment with 17 children in an East Tennessee school using Yahooligans! as a test site. She found that browsing required less cognitive load than keyword searching. Searching by keywords often resulted in failure because of misspellings or inappropriate search terms. Bilal concluded that the children’s limited success on completing the assigned tasks was due to the Yahooligans! design not being based on children’s developmental level and cognitive abilities.

To further understand what aspects of educational websites influence performance and satisfaction, we evaluated childrens' success, navigational efficiency, and satisfaction with three educational websites^* - Enchantedlearning.com®, Factmonster.com™, and Infoplease.com’s Homework Center™ (see Figures 1- 3). EnchantedLearning.com™ is a site designed for children by Enchanted Learning Software. The Infoplease.com Homework Center and FactMonster.com are sites sponsored by Information Please, a division of educational publisher Pearson Education. These sites were chosen because they offered similar content but different interfaces to the users. FactMonster.com was a very colorful and graphical site while The InfoPlease.com Homework Center was primarily text-based. EnchantedLearning.com offered a mix of graphical and textual links and generous areas of whitespace.

A total of 30 elementary-aged children (17 females, 13 males) ranging between 7 and 11 years of age (M = 8.67) volunteered for this study. Participants were recruited from local communities such as churches, scout troops, and after-school programs. All the participants had at least one year of experience using computers and the Internet specifically.

Participants were asked to complete a series of seven informational search tasks on one of the three websites. Age of the participants was balanced across websites. The tasks were as follows:

Tasks were presented in random order and took approximately 30 minutes to complete. After all tasks, participants were asked to complete a modified version of the System Usability Scale satisfaction survey (Brooke, 1986) using a “smiley-face” Likert scale derived from the Wong-Baker pain rating scale (1998) to rate their satisfaction with different aspects of the website. In addition, participants' success, time on task, and search efficiency were measured. Search efficiency, or lostness, was defined by the number of pages traversed beyond the optimum number of pages to complete a task. Time and efficiency data was gathered by the tracking program Ergobrowser™.

Table 1 shows the percentage of participants that were successful on each task for each site. Factmonster.com had the highest rate of overall success (67.5%) followed closely by Enchantedlearning.com(62.8%) and then the Infoplease.com Homework Center(47.6%).

There were several tasks that seemed to suggest that the level of difficulty varied across website. Finding contact information proved to be extremely difficult on Enchantedlearning.com (10%) and the Infoplease.com Homework Center (33.3%) primarily because this information was located at the very bottom of the page. Participants using the Infoplease.com Homework Center also had trouble finding information on Mars, Sir Edmund Hillary, and the Italian language. The primary problem was the overwhelming number of link choices on this site and terminology that did not match what the participants' were thinking. For example, the information on how to say thank-you in Italian was not found by any of the participants using this site. This information was located under the "Almanac > Writing and Language > Basic Phrases in Other Languages" links. Most participants looked for it under the "Speaking and Listening" link on the home page rather than "Almanac." Participants looking for information on Sir Edmund Hillary on FactMonster.com clicked on the "People" link and then "People Fun Facts" instead of the correct link of "Biographies". Those looking for this information on the InfoPlease.com Homework Center clicked on "Social Studies" or "History" instead of "Biographies".

Table 1. Success rate for each task by website (bold indicates least successful tasks)

Results from a one-way ANOVA showed that the participants using FactMonster.com were able to complete all seven tasks the quickest - an average of 7 minutes faster than the InfoPlease.com Homework Center (F(2,27) = 3.76, p<.05) (Figure 4). Time to complete all tasks on EnchantedLearning.com was not significantly different from the other two sites. Average lostness was not significantly different across sites with EnchantedLearning.com having an average of 3.3 pages beyond the optimal path, FactMonster having 4.2, and the InfoPlease.com Homework Center having 4.8 (F(2,27)=.884, p=.42).

Overall, participants rated EnchantedLearning.com, FactMonster.com, and the InfoPlease.com Homework Center equally well (26, 29, 29 respectively out of 44 total. The higher the number, the more satisfied the participants). The only individual satisfaction question that showed some difference across sites was in regard to the colors of the site. Participants using FactMonster.com were more positive about the colors than the other sites. Given that each participant worked with only one site (and had no other sites to compare it to), it was not too surprising that there were no differences in satisfaction. Our experience at SURL has been that children, in general, tend to be positive about a website even when they do not perform well.

Across all three sites, it appeared as though the older children had an easier time finding the information than the younger children (see Figure 4). This could be due to the fact that they (i) had more experience in school using the Internet and (ii) they had more knowledge of what the correct answer to the questions were and (iii) they had an overall better vocabulary which helped them understand the site's terminology. Despite this fact, all sites were designed for children in K-12. Educators of children in this age group (especially those less than 10) need to be very cognizant of a site's content and terminology to make sure it is best suited for their classroom.

The results obtained suggest several things. Overall, the success rate on each website was low with this population. Factmonster.com was highest at 67.5%, which was better than the 43% with the InfoPlease.com Homework Center, but still not good odds for a child attempting to complete a homework assignment. The following were noted as important features that contributed to the participants' overall success with the sites:

Page Length: Many of the web pages on all three sites were long. Participants, especially the 7 and 8-year-olds, tended not to scroll down the page even when reminded that they could do so before the start of each task. This was especially evident with the task to find contact information. Both the Infoplease.com Homework Center and EnchantedLearning.com had its “contact us” link at the bottom of the homepage. Factmonster.com's higher success rate on this task was due to the fact that its Help icon was immediately visible at the top of the screen.

Too Many Choices: The Infoplease.com Homework Center offered many links on each page. For example, on the "Find the origins of Mars" task, once a user clicked on the Science category, there were many topics and sub-topics to choose from ranging from the earth sciences to technology. Users reported it to be "messy" and confusing, regardless of age. By contrast, Factmonster.com had fewer category links and was easier for participants to initially find a keyword related to what they were looking for.

Terminology: Enchantedlearning.com and FactMonster.com used simpler and more descriptive words to identify topics than the InfoPlease.com Homework Center. Factmonster.com used titles like “People” and “Word Wise” which were readily understood by the youngest children. However, they and the InfoPlease.com Homework Center also used terminology that was not as familiar to some children like “Biography” and “Almanac”.

Font size and pictures: The younger children liked the larger font size and pictures offered on Enchantedlearning.com. This was a contrast to the Infoplease.com Homework Center that used a smaller font size and few pictures to guide the children.<

	Columns
	One	Two	Three
Full Justification	224.31 (39.76)	269.33 (61.43)	227.60 (41.77)
Left Justification	266.44 (40.52)	242.87 (35.14)	246.29 (70.31)

	Columns
	One	Two	Three
Full Justification	4.64 (0.92)	4.25 (0.90)	3.99 (1.01)
Left Justification	4.28 (1.17)	3.77 (0.95)	4.14 (1.08)

Measure	Site 1 (Toystogrowon)	Site 2 (MastermindToys)	Site 3 (WonderBrains)
Difficulty	1.08 (0.29)	1.81 (0.60)	1.67 (0.77)
Time on Task (sec)	40.12 (10.59)	61.94 (17.1)	42.66 (10.84)

Measure	Site 1 (Toystogrowon)	Site 2 (MastermindToys)	Site 3 (WonderBrains)
Difficulty	2.58 (1.08)	1.72 (1.00)	1.67 (0.65)
Time on Task (sec)	108.09 (36.59)	61.31 (27.4)	67.54 (45.64)
Number of Pages	8.00 (4.11)	5.08 (1.5)	5.17 (0.94)

	EnchantedLearning	FactMonster	Infoplease
Find the state capital of Wyoming	100	81.8	66.7
Use the search field to find location of Great Wall	80	100	100
Find site contact information	10	63.4	33.3
Find origin of Mars	70	63.6	33.3
Find a multiplication table	70	72.7	77.8
Find info on Sir Edmund Hillary	70	36.4	22.2
Find how to say “Thank You” in Italian	40	54.5	0
Average	62.8	67.5	47.6