Usability News - 7.1 2005 -- Easy Print Version

Summary: This study examined the effects of enhanced layout (headers, indentation, and figure placement) on reading performance, comprehension, and satisfaction. Participants read text passages with and without enhanced layout. Results showed that reading speed and comprehension were not affected by layout, however, participants were more satisfied with the enhanced layout and reported it to be less fatiguing to read.

The readability of online text has been shown to be influenced by a number of typographical variables including text size and type, line length, density, margins, and white space (see Dyson 2004 for a review). In the last issue of Usability News, we reported on the effects of white space, manipulated through margins and leading (Chaparro et al., 2004). Results from this study showed that participants read the text with optimal white space slower, but comprehended more than the text with poor use of white space. In another study, Chaparro & Bernard (2001) found that moderate amounts of white space distributed throughout a webpage resulted in higher user preference but no performance differences when compared to low and high amounts of white space for a series of search tasks.

An important source of white space on a textual page occurs around headers, embedded figures, and paragraphs. This white space influences the overall layout of a page and is recommended for optimal reading (National Cancer Institute, 2003). This study examined the effects of such white space on reading performance and comprehension. Participants read online text passages with and without an "enhanced" page layout. Enhanced page layout included optimal use of white space with headers, indentation, and figure placement.

Twenty college students (10 male, 10 female) with normal or corrected vision participated in the study. Seventy percent of the participants reported visiting web sites daily, 15% reported visiting web sites a few times a week, and 15% reported visiting web sites a few times a month. Primary activities reported were email, browsing, searching for information, online games, online music, and instant messaging. Thirty-five percent of the users reported reading online 7 – 24 hours per week, 15% reported reading 2 – 6 hours per week, and 50% reported reading 0 – 1 hours per week.

Text passages chosen from SAT and ACT practice examinations of approximately 800 words (M = 830.17, SD=33.55) were used in the study. The passages were created in a graphical format by expert typographers to incorporate enhanced page layout including optimal use of headers, indentation, and figure placement. An example of the two conditions is shown in Figure 1. Text passages were randomly presented using a Visual Basic 6.0 program which captured reading time. Passages were presented on a Dell Inspiron 5100 Laptop with a 15” display running 1400 x 1050 screen resolution. Passages were set up on two pages; users clicked on an arrow at the bottom of each page to advance to a subsequent page or return to a previous page. No scrolling was required. Users read the passages at a distance of approximately 50 cm.

Participants were randomly assigned to first read passages from either the Enhanced Page Layout or the Poor Page Layout reading condition. In each condition, they spent approximately 30 minutes reading 3 documents. They were asked to read each document at their own pace. After reading each passage, the participant was given eight comprehension questions about the passage. Participants were permitted to go back to the passage to look up the answers to the questions, but were advised that they only had 5 minutes to do so. After reading all 3 documents, the participants completed a questionnaire regarding their reading satisfaction. Participants then took a short break and then repeated the procedure with the 3 passages for the other condition. After both conditions were completed, participants were shown a sample page with images of the 2 conditions and asked to state their preference as to which layout they liked best. The order of the conditions and passages was counterbalanced across participants.

Reading performance. Reading time was averaged across passages for each condition and converted to words per minute. Comprehension scores were computed as a sum score out of a total eight possible. Paired-samples t-test revealed no significant differences across conditions for reading speed (t(19) = .41, p > .05) or comprehension (t(19) = .96, p > .05) (Table 1).

Satisfaction. Paired-samples t-test revealed significant differences across conditions for the satisfaction questions related to layout (t(19) = 4.11, p < .01) and mental exhaustion after reading (t(19) = 2.49, p < .05) (Table 2). Scores reflect a 7-point Likert scale score; the higher the score, the more satisfied the participant reported being.

Table 2. Mean (SD) Satisfaction Scores Across Conditions (Scale 1 – 7; high score indicates higher satisfaction)

Preference. Results from a Wilcoxon Z test showed a significant preference for the Enhanced Page Layout layout, Z(N=20) = -3.58, p < .05. Overall, 90% of the participants preferred the Enhanced Page Layout format while 10% preferred the Poor Page Layout format.

Results from this study showed that, interestingly, reading performance or comprehension was not influenced by the quality of the page layout, despite the fact that participants often had to sometimes read around a photograph in the poor layout passages. Participants reported higher satisfaction and less mental fatigue with the enhanced layout passages than the poor layout passages. Not surprisingly, preference was also in favor of the enhanced layout. This study demonstrates how resilient readers are to poorly formatted online text. Poor use of white space does not impact reading performance. Higher satisfaction and preference of the better layout, should not be discounted, however, since such variables influence whether a user continues interacting with a website or simply moves on to one with better visual appeal.

Acknowledgement: This study was funded by a grant from the Advanced Reading Technology team at Microsoft Corporation.

Chaparro, B. and Bernard, M. (2001). Finding Information on the Web: Does the Amount of White Space Really Matter? Proceedings of the Tenth Annual Usability Professionals’ Association Conference. (also available at http://psychology.wichita.edu/surl/usabilitynews/
2W/whitespace.htm)

Dyson, M. C. (2004). How physical text layout affects reading from screen. Behaviour and Information Technology, 23, 377-393.

National Cancer Institute (2003). Research-based web design and usability guidelines. Retrieved on January 28, 2004, from the National Cancer Institute’s Usability.gov Web site: http://www.usability.gov

Metaphors and Website Design: A Cross-Cultural Case Study of the Tide.com Stain Detective

This research will be presented at the HCI-International 2005 in July. For more information see http://www.hci-international.org/.

Summary: This study investigated the generalization of a home metaphor used in the Tide.com Stain Detective (Nelson & Hibner, 2003) to middle-class Indian females. The stain detective was developed with American women based on a card sorting activity. A similar card sorting activity was conducted with six Indian females. Results showed that the Indian participants grouped the stains by the amount of work that was required to remove it, rather than by the location where it occurred.

Tide, a product of Proctor and Gamble (P & G), is a well-known laundry product. In India, Tide and Ariel (both P & G products) control 11% of the domestic detergent market (P & G, 2004). In March of 2004, P & G launched a campaign to gain more control in India by reducing the price of detergent by 20-50% resulting in a 200-gram pack of Tide selling for about $0.22 (P & G, 2004). An increase in the market share is likely to result in an increased number of Indian users logging on to the Tide website. Tide.com features a stain detective to provide users with advice on removing stains while doing laundry. The Tide.com stain detective was developed based on a card sorting activity in which clusters “mapped to rooms in a house where the stain would occur” (Nelson & Hibner, 2003, p. 1323). As a result of this, the stain detective uses the metaphor of a prototypical American house to facilitate users’ search for a particular stain type (Figure 1).

Interface designs that rely on metaphors should taking into account cultural diversity of targeted users (Marcus & West-Gould, 2000). Interfaces need to reflect “the values, ethics, and morals of the target users” (Ford & Gelderblom, 2003, p.220). According to Marcus (1998), metaphors provide a visual meaning of concepts through words and images. Duncker (2002) emphasizes that metaphors have to match the target user’s mental model of physical objects. When users feel a sense of representation they are able to benefit from and feel comfortable using metaphors.

The question arises as to whether the house metaphor applies to other cultures. The typical Indian house and living arrangements are structured very differently than the average American home. The house metaphor presented on Tide.com is not consistent with a typical urban Indian house. In addition to bedrooms, the Tide.com house has a playroom, nursery, bathroom, office/den, kitchen/dining, garage/shop, and laundry/basement. Urban Indian families often live in patrilineal joint families (men are related) consisting of 4-8 family members. The typical higher-middle class Mumbai house has a 1-3 bedrooms, a kitchen, a combined living/dining area, and 1-2 bathrooms.

The main objective of this study was to compare the categorization of stains by Indian middle class females to American females to determine if the Tide.com Stain Detective (Nelson & Hibner, 2003) is a valid metaphor generalizable to India.

Six Indian females ranging in age from 26 to 46 (M=34.67) were recruited to participate in a card sorting activity of 92 common household stains. All participants labeled themselves as middle class ($300-500 USD/month) or above based on income. All participants indicated they were fluent in English and had at least the equivalent of a Bachelor’s degree.

The card sorting activity was carried out at a private residence in Mumbai, Maharashatra, India. Participants were compensated with a small gift. All participants completed a background questionnaire before completing the card sort and a post-session questionnaire asking how familiar they were with the stains presented and asking them to list stains that were present in Indian life but not represented on the cards. After completion of the study, participants were interviewed regarding the perceived usefulness of a house metaphor for this task.

The results of the card sort revealed an inherent metaphor that differed from that used by the American participants. The clusters created by the Indian participants did not rely on the location of stain occurrence within the house. The majority of participants created clusters based on the amount of work necessary to remove the stain. These groups were often assigned names such as (1) easy to remove, (2) more challenging to remove, (3) leave for the servant, and (4) requires professional laundry service.

In the post-experiment interview, participants were asked if they could group the stains based on location of occurrence in the house. None of the participants felt comfortable using a house metaphor. Due to the size of the Indian homes, the rooms are used for a variety of purposes with the living room often serving as a bedroom at night and the bedroom often serving as playroom and office during the day.

A cluster analysis using EZCalc™ revealed clusters with some similarities to those found on the Tide.com website. Participants were also able to list a variety of stains commonly encountered in India that were not presented (e.g., mango, tumeric, henna).

The results of this case study clearly show that care must be taken in website design when metaphors are used as the primary means of interaction. The major question raised by the results is to what degree the lack of congruity between the house metaphor and the removal-difficulty metaphor would interfere with the actual use of the stain detective. A usability test of the Tide.com stain detector with this population would reveal specific cultural difficulties with the metaphor. According to Duckner (2002) users need to feel a sense of representation in order to comfortably use metaphors; usability testing would indicate whether the Indian users have this sense of representation which would enable them to fully benefit from the house metaphor.

Duckner, E. (2002). Cross-cultural usability of the library metaphor. In Proceedings of JCDL, 223-230. Portland, OR: ACM Press.

Ford, G., & Gelderblom, H. (2003). The effects of culture on performance achieved through the use of human computer interaction. In Proceedings of the 2003 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologies on Enablement through Technology (SAICSIT), 218-230. South African Institute for Computer Scientists and Information Technologies.

Marcus, A. (1998). Metaphor design in user interface. Journal of Computer Documentation, 22(2), 43-57.

Marcus, A., & West-Gould, E. (2000). Crosscurrents: Cultural dimensions and global web user interface design. ACM Interactions, 7(4), 32-46.

Nelson, T., & Hibner, S. (2003). A user-centered approach to redesigning a web-based utility: Tide.com’s stain detective. In Proceedings of the Human Factors and Ergonomics Society 47th Annual Meeting, 1322-1325. Denver, CO: HFES.

P&G cuts detergent prices in India. (2004). Retrieved September 15, 2004, from Business Courier Web site: http://www.bizjournals.com/cincinnati/stories/2004/03/01/daily16.html

Tide Stain Detective
http://www.tide.com/staindetective/selectStain.jhtml

Summary: This study discusses the contributions of eye-tracking data to traditional usability test measures for first-time usage of websites. Participants viewed the homepages of three different websites. Results showed that eye-movement data supplemented what users verbally reported in their reactions to a site. In particular, the eye-tracking data revealed which aspects of the website received more visual attention and in what order they were viewed.

As eye-tracking systems have become both more sophisticated and more affordable, there has been an increasing interest in the use of eye-tracking within the software usability testing domain (see Byrne, Anderson, Douglass, & Matessa, 1999; Kotval & Goldberg, 1998; and Hornof & Halverson, 2003).

Eye-tracking studies that have focused on web-based stimuli have gathered eye-movement data while participants were engaged in typical information search tasks on web pages (Cowen et al., 2002; Goldberg, Stimson, Lewenstein, Scott & Wichansky, 2002; Josephson & Holmes, 2002). Despite an increased amount of research in this area, it is still not entirely clear exactly how much eye-movement data contributes to the evaluation of a user interface.

As a part of a comparative usability test between three websites of a similar domain, participant eye movements were recorded during the users’ initial introduction to each site’s homepage. The eye-tracking data was examined to gain additional insight into how users initially viewed a site and what page elements attracted their visual attention.

Thirty-six undergraduate and graduate students (26 female, 10 male) at Wichita State University participated in this study. Seventy-two percent of the participants were between the ages of 18 and 26 years. A Pentium IV-based PC computer with 96 dpi, 17” monitor with a resolution setting of 1024 x 768 pixels was used. The monitor was integrated with the Tobii 1750 eye-tracking system, which was used to detect and collect participant eye-gaze data during testing. The Tobii 1750 eye-tracker samples the position of the user’s eyes on an average of every 20ms (i.e., 50Hz) and is characterized by the unobtrusive addition of the eye-tracking hardware (e.g., high resolution camera and near infra-red light-emitting diodes) to the monitor frame. This design aspect helps promote more natural user behavior by not placing unnatural restrictions on participants (e.g., helmets, head-rests, etc.).

Participants were seated approximately 60 cm away from the computer monitor. After calibration, participants were presented with the home page of one of the three e-commerce websites used in the study and were asked to take about 1 minute to look over the site and verbally report their impressions as to the site’s purpose and what audience the site was targeted toward. (Participants completed a series of tasks with each site, but only the analysis of this initial task is discussed in this article.)

All three of the websites used in this study were e-commerce sites that specifically dealt with the sale of educational toys. During this exercise, the participants’ responses were often little more detailed than, “it is a site for toys” and “it is aimed at parents”, but this was to be expected. In a standard website usability test, this would simply be a means of determining whether users are able to recognize the purpose or domain of the site simply from the homepage. However, under a typical usability study (that does not gather eye-tracking data), the only data gathered during such an exercise is whether or not the design and information of the homepage sufficiently communicates the purpose of the site to users. The addition of the eye-tracking data allows designers to better understand exactly what users are looking at in order to make sense of the site.

ClearView™ was used with the Tobii 1750 to examine the users’ eye-movements by means of a post-hoc analysis of visual fixations falling within different areas of each homepage. Specifically, captured images of each homepage are manually divided into natural segments of interface real estate, e.g., the company logo, search field, or navigation menu. The eye-tracking data is then examined in terms of fixations recorded within these segments, or areas of interest (AOI). The fixation data for these AOIs can be compared in a number of ways, including: (1) the order in which each AOI received its first fixation; (2) the number of fixations recorded in each AOI; and (3) the cumulative dwell time of fixations recorded in each AOI. This provides a way to determine what aspects of the homepage users generally look at first, for how many times and for how long when they first encounter the site.

Site 1: Toys to Grow On (www.toystogrowon.com)

The Toys to Grow On homepage was divided into 10 different AOIs (Figure 1). User eye-movement data was reviewed, and it was determined that some AOIs were fixated upon earlier, more often, and longer than others (Table 1).

Table 1. Area of Interest data for homepage

The first AOIs typically receiving fixations from this group of users were the large areas in the center of the page: (1) the picture of the child on the bike; (2) the big Toys To Grow On logo to the right of the picture; (3) the block of text directly beneath the picture; and (4) the Lakeshore logo below the text. These areas are not only the largest areas and those dominating the center of the page, but they also generally provide the most information about the nature of the site, which follows the purpose of this orientation task. Figure 2 shows the general order of first fixation to the different AOIs.

Another output of the eye-tracking data is a representation of the areas of the screen receiving either more fixations or receiving the longest dwell times in a color-coded “hotspot” image of the interface (see Figure 3). Specifically, the closer to red the more fixations occurred in an area of the interface, with intensity decreasing with movement down the spectrum (see Figure 4). Inspection of these hotspots relative to the AOI definitions reveals that, although the large picture in the center of the page was fixated upon earliest and received the most fixations on average, the portion within that AOI most fixated upon was the child’s face, and not the toy motorcycle he is riding. This indicates two things: (1) just because certain AOIs are larger than others, that does not mean that visual attention (i.e., concentration of fixations) is evenly distributed across the AOI; and (2) it may be that photos of toys posted on the homepage to generate interest in the company’s products might be more effective if they emphasize the toy over the child model.

Figure 3. Hotspot map of the homepage showing areas of highest fixation count

Figure 4. Example legend for hotspot maps; red = more fixations, green = fewer, etc.

It is interesting to note, however, that the Lakeshore logo, which was typically fixated upon 4th in sequence, received an average of 6 fixations per user and was dwelled upon for an average of 3 seconds, actually provides very little in the way of information about this particular site. If it were the intention of the website’s owners to emphasize their company’s relationship with Lakeshore, this strategy was successful. However, if it is more important for users to focus on the explanatory text or the navigation menu options, this logo may be considered a distraction.

Site 2: Mastermind Toys (www.mastermind.com)

The Mastermind Toys homepage was divided into 18 different AOIs (see Figure 5). User eye-movement data was examined in a manner similar to the first site. Table 2 shows a summary of the fixation data by AOI.

The first AOIs on the homepage typically receiving fixations from this group of users were those running along the top of the page: (1) the “live help” information—not a link but merely a 1-800 number listing); (2) the top “Mastermind Toys” logo; and (3) the search engine (see Figure 6). This was generally followed by fixations on the large areas in the center of the page: (4) the picture of the inside of the store; and (6) the main listing of links to the right of that picture. All these areas were not fixated upon very many times or for very long, however, as the areas given the most visual attention—in addition to the large picture of the inside of the store—were the various products pictured down the center of the homepage. Remarkably, even those items appearing far “below the fold” were extensively examined.

It is interesting to note that users depended primarily on the pictures of the products displayed on this homepage in order to determine the purpose of the site, rather than focus on the listing of links on the left navigation bar. This supports the apparent intent of the site’s owner(s)/designer(s) to generate interest in the site by drawing attention to some available products, but also brings into question the effectiveness of the lengthy left navigation bar. Hotspot analysis of this page showed that the portion within the large store AOI most fixated upon was the text “10 stores to serve you”, and not the toys in the picture (see Figure 7). Again, this indicates two things: (1) just because certain AOIs are larger than others, that does not mean that visual attention (i.e., concentration of fixations) is evenly distributed across the AOI; and (2) if text in this area is quickly and frequently attended to, this might be a good place to provide more critical information (e.g., sales, popular toy names) rather than a generally descriptive statement about the number of stores in the chain.

Site 3: WonderBrains (www.wonderbrains.com)

The WonderBrains homepage was divided into 18 different AOIs (see Figure 8). User eye-movement data was examined in a manner similar to the first site. Table 2 shows a summary of the fixation data by AOI (Table 3).

	ENHANCED PAGE LAYOUT	POOR PAGE LAYOUT
READING SPEED (WPM)	185.60 (47.22)	183.38 (51.36)
COMPREHENSION	5.32 (.99)	5.08 (1.31)

SATISFACTION	ENHANCED PAGE LAYOUT	POOR PAGE LAYOUT
Easy to read	5.20 (1.15)	4.75 (1.37)
Ability to concentrate	5.30 (.98)	5.20 (1.40)
Satisfied with text size	5.25 (1.29)	5.05 (1.50)
Satisfied with layout *	5.50 (1.79)	3.25 (1.71)
Crispness of text	4.97 (1.25)	4.75 (1.09)
Physical level after reading	4.15 (1.08)	4.20 (.95)
Mental level after reading *	5.50 (1.35)	4.70 (1.03)
Confidence in comprehension	5.25 (1.12)	4.75 (.97)
Level of eyestrain	4.90 (1.33)	4.40 (1.53)
Like to read textbook material with this text	4.05 (1.54)	3.25 (1.71)
Like to read leisure material with this text	4.15 (1.75)	3.55 (1.88)
Like to read news material with this text	4.35 (1.63)	4.00 (1.92)

AOIs	Average Fixation Order	Average Number Fixations (SD)	Average Dwell Time in seconds (SD)
Picture	1	11.5 (2.89)	329.85 (0.84)
Big logo	2	8.5 (2.50)	385.89 (2.06)
Text	3	14 (2.3)	771.21 (3.96)
Lakeshore logo	4	6 (3.09)	192.63 (0.45)
Upper logo	5	1 (1.15)	48.3 (0.26)
Left navigation	6	3.5 (5.25)	63.21 (0.34)
Shopping cart	7	0.5 (1.41)	265.98 (7.02)
Upper navigation	8	1 (1.73)	70.38 (0.29)
Bottom navigation	9	0 (0.50)	10.74 (0.0)
Search fields	10	1 (1.26)	23.88 (0.534)

AOI	Average Fixation Order	Average Number of Fixations (SD)	Average Dwell Time in seconds (SD)
live help/phone no.	1	0.25 (0.87)	0.06 (0.22)
top logo	2	0.67 (1.78)	0.31 (0.92
Search	3	0.25 (0.45)	0.08 (0.14)
store picture	4	34.67 (12.39)	8.12 (3.26)
top cart view	5	7.00 (12.78)	1.55 (2.40)
center links	6	25.50 (8.53)	7.21 (2.47)
top items	7	33.75 (14.08)	7.44 (2.76)
view cart contents	8	0.33 (0.49)	0.03 (0.07)
your account	9	0.75 (1.14)	0.17 (0.26)
checkout now	10	1.00 (1.21)	0.18 (0.20)
links above nav bar	11	1.08 (1.44)	0.30 (0.61)
holiday blurb	12	3.92 (5.62)	0.99 (1.69)
bottom links	13	2.83 (3.90)	0.59 (0.90)
middle items	14	14.67 (8.96)	3.83 (2.00)
shop by age links	15	3.67 (3.77)	1.02 (1.04)
shop by brand links	16	4.08 (7.51)	2.00 (3.64)
shop by category links	17	3.42 (4.44)	1.17 (1.53)
below fold items	18	41.33 (28.37)	10.79 (5.08)

AOI	Average Fixation Order	Average Number of Fixations (SD)	Average Dwell Time in seconds (SD)
Logo	1	12.92 (9.89)	0.08 (0.22)
Description	2	17.92 (11.31)	2.92 (1.31)
Header