In a previous issue of Usability News, we examined the use of paging vs. scrolling for the presentation of search results. In that study we found that a moderate amount of scrolling (fifty links per page) was preferred to search result presentations that required either a large or small amount of scrolling (one-hundred links per page and ten links per page, respectively). Participants were also fastest at searching in the fifty-link condition.

In this study, we examined the use of paging vs. scrolling in reading passages, including participants' reading comprehension in paging and scrolling conditions. Dyson and Kipping (1998) found that participants read through paged documents faster than scrolled documents, but showed no differences in reading comprehension.

A Pentium II based personal computer, with a 60 Hz, 96dpi 17" monitor with a resolution setting of 1024 x 768 pixels was used. Content for the passages came from Barron's "Passkey to the ACT" (2001) study guide. The articles had a mean of 713.7 words (S.D. = 36.83). The passages were presented in 10 point Verdana. Participants’ performance was tracked using Ergobrowser™ software.

Participants

Fifteen participants (3 males, 12 females) volunteered for this study. They ranged in age from 18 to 40, with a mean age of 23.3 (S.D. = 6.11 years). The median Web use for the participants was 7-14 hours per week (100% used the Web a few times per week or more).

Procedure

Participants were asked to read one passage from each of three distinct layouts: one passage presented over four web pages, ("Paging", see Figure 1), one passage presented over two separate web pages with no scrolling necessary, ("Full", see Figure 2), and an entire passage presented on one web page, ("Scrolling", see Figure 3). 

After reading each passage, participants were given five multiple-choice comprehension questions related to the given passage (e.g., "In making decisions about parent-child relationships, judges are often guided by the principle that: [A: the family unit is the basic unit of society]"), and following those, five search task items (e.g., "What did the economic needs of families and communities necessitate until the end of the 19th century?"). Participants were not permitted to refer to the passages when answering the comprehension questions. Information in the search tasks had to be found within 5 minutes in order to be considered correct. Participants moved through the pages using the "forward" and "back" button presented on the page in the paging and full conditions, or by clicking on the scrollbar in the scrolling condition. The Paging condition required the users to click a page-forward button 3 times, the Full condition required the users to click on a page-forward button 1 time, and the Scrolling condition required user to click a maximum of 7 times on the window scroll button. The layouts were counterbalanced using a Latin square design. The layouts were stored on a local server, virtually eliminating download time in each condition.

After finishing all questions for each condition, participants answered a satisfaction questionnaire. The questionnaire consisted of a 6-point Likert scale, with 1 = “Disagree” and 6 = “Agree” as anchors. The questionnaire items were: "The layout made it easy to find information," "This site was visually pleasing", "The arrangement of this site promotes comprehension," "I am satisfied with this site," and "The layout looks professional." After participants completed the questionnaire for all conditions they ranked the three layouts for general preference.

A within-subject ANOVA design was used to investigate participant performance (mean reading time, comprehension, and search completion time) and perceived ease of use of the three conditions. A Friedman χ² was used to analyze preference for all three conditions.

Reading Task Completion Time

Evaluation of the average time (in seconds) taken to read each passage revealed a significant difference between the three groups [F (2,28) = 3.42, p < .05] (S.D. Paging = 72.4, S.D. Full = 53.5, S.D. Scrolling = 57.89; See Figure 4). Post hoc analysis found reading time in the Paging condition to be significantly slower than the Full or Scrolling conditions.

Comprehension Questions Correctly Answered

Evaluation of the number of comprehension questions correctly answered revealed no significant differences between the three conditions [F (2,28) = 1.213, p = .31] (S.D. Paging = 1.32, S.D. Full = 1.16, S.D. Scrolling = 1.40; See Figure 5).

Search Task Completion Time

Evaluation of the time (in seconds) taken to complete the search tasks revealed no significant differences between the three groups though there was a trend for the search time in the Paging condition to be longer than that in the Full or Scrolling conditions  [F (2,28) = 1.795, p = .185] (S.D. Paging = 92.83, S.D. Full = 139.43, S.D. Scrolling = 83.48; See Figure 6).

Perceptions of Site Efficiency

Well-suited for Reading

No significant differences were found in the perception that a particular condition was more well-suited for reading [F (2,28) = .079, p = .924]. (S.D. Paging = 1.33; S.D. Full = 1.40; S.D. Scrolling = 1.53). (See Figure 7).

Easy to Find Information

No significant differences were found for participant satisfaction between the conditions in terms of user perceptions' that information was easy to find [F (2,28) = .547, p = .585]. (S.D. Paging = 1.43; S.D. Full = 1.47; S.D. Scrolling = 1.33). (See Figure 8).

Arrangement Promotes Comprehension  

No significant differences were found for the perception that a particular layout promoted comprehension [F (2,28) = .483, p = .622]. (S.D. Paging = 1.37; S.D. Full = 1.42; S.D. Scrolling = 1.37). (See Figure 9.).

Layout Preference  

Analysis of layout preference using a Friedman χ² showed no significant preference for any of the conditions [Friedman χ² (2, N=15) = .133, p = .936.] (See Figure 10.)

The findings from this study show that participants using the paging condition took significantly longer to read the passages than either the full or scrolling conditions. Participants also showed no significant differences in their ability to answer comprehension questions correctly, nor in their perceptions or satisfaction of the reading conditions. This is somewhat surprising given that previous research has shown paging to be typically faster than scrolling (Dyson & Kipping, 1998). One might have expected that the Scrolling condition may be more time consuming given the users had to click the scroll bar more times to view the entire passage than they did in the Paging (3 clicks) or Full (1 click) condition. However, several users commented that they were more accustom to scrolling when reading documents on the web. It may be that since participants had more exposure to scrolling they were able to read through the documents more quickly using that mode of navigation. Participants stated that they found the Paging condition to be "too broken up," and that they had to "go back and forth" quite a bit to search for information. It is possible then, that for searching as well, viewing more of the document on a single screen facilitated easier scanning.

Ergobrowser™, Ergosoft Laboratories© 2001

Barron's Passkey to the ACT. (2001). George Ehrenhaft (Ed.) Barron's Educational Series: New York.

Bernard, M. L., Baker, J. R., & Fernandez, M. (2002).  Paging vs. Scrolling: Looking for the Best Way to Present Search Results. Usability News, 4.1. ../usabilitynews/41/paging.htm

Dyson, M. C., & Kipping, G. J. (1998). The effects of line length and method of movement on patterns of reading from screen. Visible Language, 32, 150-181.

Just as Hansel and Gretel created a breadcrumb trail to find their path back home, today’s internet user often finds a need to get back to a website’s previously visited locations; hence, the cyber “breadcrumb trail” was created. This textual representation of where and how information is located within the website allows the user to link to major categories of information along a continuum of sequential order. For example, the breadcrumb path for a leather chair on the Office Max website would be: Home > Furniture > Chairs > Leather Chairs. The location of this navigational path is often prominently placed in the upper left quarter of the website page.

In general, breadcrumbs serve two purposes: 1) they provide information to the user as to where they are located within the site, and 2) they offer shortcut links for users to “jump” to previous categories in the sequence without using the back key, other navigation bars, or the search engine. Breadcrumb paths give the site location information and links in a backward linear manner.   Navigation methods, such as search fields or horizontal/vertical navigation bars, serve to retrieve information for the user in a forward-seeking approach. As suggested by Marchionini (1995), systems that support navigation by both browsing and analytical strategies are most beneficial to users since various patterns, strategies, tactics, and moves associated with both types of strategies are normally used.

Breadcrumb trails, according to Bowler, Ng, and Schwartz (2001), help users browse and search for information as well as provide paths back up the web hierarchy. Toms (2000) found that the search tool was most useful in focused searching; while menus were considered essential for browsing. She also suggests that users need both a stable orienting device, such as a menu, to facilitate pathways through the site, as well as a system that supports scanning to smooth the progress of the search (Toms, 2000). However, research on breadcrumbs as a navigation tool has been limited. Studies researching breadcrumb navigation have used simple site structures which have been designed conditions with or without breadcrumb navigation bars for the study. These studies found the availability of breadcrumb navigation bars results in more efficient site navigation and satisfaction (Bowler, et al. 2001; Maldonado & Resnick, 2002). 

In an attempt to better understand how, when, or even if, users use breadcrumb navigation in a real-time online environment, we conducted an exploratory study to determine if participants use breadcrumbs when given a list of items to find on a website. In addition, we were interested in monitoring what other navigational methods they use to access information or e-commerce items (i.e., back button, navigation bars, search). 

In this study, we evaluated participants' navigational strategies, user satisfaction, and general preference for two sites that utilize “breadcrumbs” – Google Directory directory.google.com, a web information search site, and Office Max www.officemax.com, an e-commerce office supply site.  In addition to breadcrumb navigation, the Google directory (Figure 1) is organized in main and sub-categories and offers search capability.

 
Participants using Office Max (Figure 2) also had access to a breadcrumb navigation, search, a hierarchical left side vertical navigational structure, and a top of page horizontal navigational bar. 

Participant performance was measured by the number of clicks, "Back" button selections, searches, and time to complete the ten tasks. This data was gathered by the tracking program Ergobrowser™. Data to collect breadcrumb usage was via Mousetrack 1.0 from LGSoft. Pentium 4-based personal computers, with a 60 Hz, 96dpi 17" monitor with a resolution setting of 1024 x 768 pixels on a campus network were used to access the sites.

After completing the tasks on each site, participants answered the End-User Computing Satisfaction (EUCS) instrument (Doll, Xia, & Torkzadeh, 1994), which was adapted for web usage and consisted of 12 satisfaction questions using a 1-5 Likert scale.

Seventy-two participants (13 male, 59 female) with an average age of 23.8 volunteered for the usability study. All participants were familiar with the web – 83% reported searching for information at least weekly, while 64% had purchased a product online. Participants were asked to find ten items on the Office Max site and place them in their shopping cart. In addition, they were asked to find and record answers to ten information search questions on Google. The order of the sites was counterbalanced across participants.  Participants had approximately 20 minutes per site to search for the information. The tasks were organized such that items from similar categories were grouped together so that use of the breadcrumb would enhance efficiency (i.e., the 1st five items in the OfficeMax task list were all from the category Furniture). After tasks for both sites had been completed, users answered additional questions related to their knowledge of breadcrumbs, recall of whether a breadcrumb was present on the sites, and recall of the path to a specific item.

Information Search Tasks (Google Directory):  

1. What is the height of Luke Skywalker from the movie Star Wars?

2. What character did DeForest Kelley play in Star Trek V – The Final Frontier?

3. Name two actors in the cast of Toy Story 2 animated movie.

4. What is Michelangelo’s last name?

5. How old was Claude Monet (artist) when he died?

6. What company developed the Elite Force action video game?

7. Did the Star Trek video game, Generations, get a “good” or “bad” review from the Game Zone website?

8. Who developed the Monopoly board game?

9. In the area of psychology, what did Al Gore’s handwriting analysis reveal about his personality?

10. What do the letters, D, I, S, C, stand for in the DISC personality test?

E-Commerce Tasks (Office Max): 

Items that you need:

1. 1 high back office chair

2. 2 file cabinets (lateral, 2 drawer)

3. 1 file cabinet (vertical, 2 drawer, letter)

4. 1 home/office safe

5. 1 leather guest chair

6. 1 scientific calculator

7. 1 cordless phone with caller ID

8. 1  8-outlet surge protector

9. 2 boxes of hanging file folders

10. 4 packages of post it notes
     

Results from a paired samples t-tests between sites revealed significant differences across the sites in terms of satisfaction, number of breadcrumbs, searches, back clicks used, time, total clicks and recollection of seeing a breadcrumb path.

Participants were queried as to the biggest problem they encountered with each site.  The results found that “Not being able to find the information I am looking for” was the most cited problem by 60.8% of the participants in the Google directory search; in the OfficeMax site, 41.9% of the participants found their biggest problem was “It takes too long to view/download pages”. The overall satisfaction totals for both sites revealed participants were significantly more satisfied with the OfficeMax site (M = 57.52, SD = 8.17) than the Google directory site (M = 44.18, SD = 13.32), t(70) = -7.33, p < .01, for the search tasks.

We calculated the optimal number of clicks necessary to find the items for Office Max to be 67 (including 8 breadcrumb links) and 2 backs; for Google the optimal path was 68 clicks (including 7 breadcrumb links) and 8 backs. Figures 3 through 7 show the average number of breadcrumbs used, backs, total clicks, searches, and completion time for the two sites. As can be seen, the average breadcrumb usage is fairly low for each site (M = 2.11, SD = 2.46 for Google; M = 3.25, SD = 3.75 for Office Max), t(62) = -2.06, p < .05.  The back button was used significantly more in the Google directory (M = 59.25, SD = 30.23) than for OfficeMax (M = 16, SD = 18.21), t(62) = 10.31, p < .01. The total number of clicks was high (M = 152.54, SD = 64.75 for Google; M = 94.07, SD = 29.09 for Office Max), t(62) = 6.71, p < .01. Although participants were instructed not to use the search to access information or products, 61% used this method in Google (M = 5.89, SD = 7.09) and 47% in Office Max (M = 3.00, SD 4.23), t(70) = 3.57, p < .01. Significant differences were also found for the task completion times between the two sites. The questions for the Google directory site took longer to find (M = 1652.98, SD = 482.91) than the products to purchase on the OfficeMax site (M = 917.02, SD = 256.58), t(62) = 10.73, p < .01. 

Figure 4. Average number of times Search was used.
 

Figure 6. Average time to complete ten tasks for each site.
 

The median split of breadcrumb usage was found to be 4.0, within a range of 0 to 21, 28  (48%) of the participants were categorized into non-breadcrumb user (<4) and 31 (52%) as a breadcrumb user (≥4). No significant differences were revealed when breadcrumb users were compared with their efficiency data--total clicks, backs, searches, and time. 

When participants were asked if they recalled seeing a breadcrumb path (with an example provided) seventy-nine percent of participants recalled seeing one on Google (M = 1.61, SD = .49) and 40% recalled seeing one on OfficeMax (M= 1.20, SD = .40), t(70) = 5.33, p < .01. (Figure 8). 

Figure 8. Number of users who reported recalling a breadcrumb path on each site.

This exploratory study was conducted to determine whether participants used the breadcrumb trail as a navigational tool within a site. We found the overall usage of the breadcrumb in site navigation to be low. Breadcrumb users were not found to be more efficient than non breadcrumb users. Participants used a variety of navigational means, such as Back, left and top navigation bars, and searching to find the information instead of or in addition to the breadcrumb tool.

Eighty-nine percent of the participants reported seeing a breadcrumb path on the sites; however, it is not known if the participants understood the function of the breadcrumb path. Although we designed the tasks to aid the users' learning by choosing items that repeated the same main and subcategories, participants did not utilize the breadcrumb tool. Future studies should investigate whether users understand the purpose of the breadcrumb path, and if training to use this navigational tool aids their search efficiency and mental model of the site. 

The tasks in this study were designed to allow participants to efficiently use breadcrumbs to find the answers to questions in the Google directory search, or find the products to purchase on the Office Max task list. However, there were significant differences between sites in terms of satisfaction and all dependent measures. These differences were not entirely surprising given the site size and opportunities for new browser windows to open in the Google directory search for which using the back button was necessary. However, the optimal path clicks for both sites was relatively similar (67 total clicks, 2 backs, 8 breadcrumbs for OfficeMax and 68 total clicks, 8 backs, and 7 breadcrumbs for Google). Yet the average number of clicks in comparison was extremely different (94 total clicks, 16 backs, and 3 breadcrumbs for OfficeMax and 153 total clicks, 59 backs, and 2 breadcrumbs for Google). The results suggest there is reason to further investigate the differences between navigational strategies for an e-commerce task versus an information search task.

Participants in this study also reported seeing a breadcrumb path more often in the Google directory search, yet they used this navigational tool less on that site. This finding is even more interesting considering that participants reported their biggest problem on Google was not being able to find the information. Although it would appear from past research and theory that the breadcrumb path aids the user in navigation efficiency and their place within the site, our results did not find that to be the case.  The results of this study put forward many new questions regarding the use of breadcrumb and efficiencies in navigation for future research.

Bowler, D., Ng, W., and Schwartz, P. (2001).  Navigation bars for hierarchical websites. Retrieved 01/20/03 from University of Maryland, Student HCI Online Research http://www.otal.umd.edu/SHORE2001/navBar/index.html

Doll, W. J., Xia, W., & Torkzadeh, G. (1994). A confirmatory factor analysis of the end-user computing satisfaction instrument. MIS Quarterly, December, 453-461.

Ergobrowser™, Ergosoft Laboratories © 2001.

Maldonado, C. A. & Resnick, M.L. (2002).  Do common user interface design patterns improve navigation?  Proceedings of the Human Factors and Ergonomics Society 46^th Annual Meeting, 1315-1319.

Marchionini, G. (1995).  Information seeking in electronic environments.  Cambridge:  Cambridge University Press.

Mousetrack 1.0, LGSoft, 2002.

Toms, E. G. (2000).  Understanding and facilitating the browser of electronic text.  International Journal of Human-Computer Studies, 52, 423- 452.

Cascading versus Indexed Menu Design

by Michael Bernard & Chris Hamblin

If there is one basic truism about the Web it is that every designer has their own opinion concerning the best method for presenting menu items on a web page. Two common ways to present menus are to either hierarchically cascade the menu items upon mouse-over, or to simply place most, if not all, of the menu items in a categorical index. Cascading menus have the advantage of requiring little screen real estate. However, they have been much maligned for several reasons. First, it is sometimes difficult to use for the reason that users must precisely control their mouse movements in order to select the correct menu item. It becomes increasing difficult with the number of levels a user must navigate. Second, cascading menus hide menu information until the user positions the mouse over the menu level above it (see Walker, 2000).

Index menus, on the other hand, tend to take up valuable real estate. They also can present an overwhelming amount of menu information at one time. Unfortunately, there has been very little empirically validated research comparing different types of menu layouts since the the advent of the Web (see Norman, 1991). In response to this lack of research, this study sought to compare user performance and satisfaction of two basic types of cascading menu layouts to a categorical index menu layout.

In addition, since users have different reasons for searching on the web, this study also sought to examine user performance for two types of searches: the direct search, where the item being searched for is explicitly known, and the implicit search where the subject material being searched for is known, but the specific item is not.

METHOD

A Pentium II based PC computer, with a 60 Hz, 96dpi 17" monitor with a resolution setting of 1024 x 768 pixels was used. The participants’ performance was tracked using Ergobrowser™ software.

Participants were instructed to search three types of menu layout conditions. In one condition, the menu items were arranged in accordance to a categorical index menu layout (Index, see Figure 1). Another condition arranged the items in accordance to a drop-down or "horizontal" cascading menu layout ( Horizontal, see Figure 2). A third condition arranged the items in accordance to a vertical cascading menu layout (Vertical, see Figure 3). The menu links were the same size (10 pt. Verdana) in all three conditions. There were no delays in the presentation of the submenu for both the Horizontal and Vertical cascading menus. In addition, both cascading menus had one level of depth and had the same submenu items cascade from it. These menu items corresponded to the items that were presented in the categorical index menu.

Figure 1. The Index menu layout for an electronics website condition
 

Figure 2. The Horizontal menu layout for an electronics website condition
 

Figure 3. The Vertical menu layout for an electronics website condition

 
Participants

Eighteen participants (8 males and 10 females) volunteered for this study. They ranged in age from 20 to 49, with a mean age of 32.6 (S.D. = 8.1 years). Eighty-nine percent for the participants use a computer daily, and most (72.2 %) use the Web 25 hours or more per week.

Procedure

Participants were presented with three web sites, each with a different menu item layout condition. The website domains consisted of either an online electronics, general merchandise, or a children's toy store.

In order to present a realistic search environment, participants were presented with both directed search and browsing questions. For each condition, participants were instructed to search for specific information pertaining to six explicit and six implicit task questions. The explicit task questions directly stated in the question the specific merchandise item to be searched (such as, "You want to buy a Nikon 8-mm camcorder"). The implicit tasks required participants to search for an item that was implied in the task question, but was not explicated stated (such as, "Your parents are having their 40^th anniversary and you would like to give them a gift. They always talked about going down the Mississippi River and viewing the plantation homes"). Each question had to be properly answered within five minutes to be considered correct. Participants could repeatedly search until they found the correct information by using the Back button, or until the time expired. The menu layout conditions, the web domain, and the task questions were counterbalanced by means of a Latin square design.

After finishing all the questions for each condition, participants answered a satisfaction questionnaire. The questionnaire consisted of a 6-point Likert scale, with 1 = “Disagree” and 6 = “Agree” as anchors. The questions were as follows: "I felt lost/disorientated while searching for information," "The site was easy to navigate," "The menu structure was frustrating to use." After participants answered the respective questionnaires for each condition they ranked the four link arrangements for general preference.

Results and Discussion

A within-subject ANOVA was used to investigate actual performance (mean task completion time and search efficiency) and perceived performance for four types of menu item conditions. Post hoc comparisons were done using the Bonferroni test. The preference for each condition was measured by means of a Friedman χ².

Task Completion Time

Assessing the time needed to complete the search tasks found a significant layout main effect [F (2, 34) = 5.39, p < .01] in that participants searching in the Index condition found task information more quickly then the other two conditions. Participants searching with explicit task questions found the information significantly faster than searching with the implicit task questions for all three conditions [F (1, 17) = 25.42, p < .001]. There was no task by menu item layout interaction (see Figures 4, 5, and 6 for means and standard deviations).

Figure 4. Explicit task completion time (S.D.) for explicit task questions

Figure 5. Implicit task completion time (S.D.) for implicit task questions

Figure 6. Task completion time (S.D.) for both implicit and explicit questions

 
It is somewhat surprising that the task completion time differences were so great. That is to say, the results revealed a 6.4 second, average per-task time difference between the Index and the Horizontal conditions. Moreover, the average completion time difference between the Index and the Horizontal conditions increased to 8.5 seconds per task when participants searched using implicit task questions. These differences can obviously be quite considerable over a extended period of time. For example, according to the Nielson/Net Ratings (2003), the average home Internet user accesses 40 pages during a typical surf of the Web. Thus, the difference in time between these two menu arrangements would be 4.27 minutes for an average search.

Perceived Disorientation

Examining perceptions of search-related disorientation revealed a marginal main effect approaching significance [F(2, 34) = 2.84, p = .072], suggesting that the Horizontal menu layout was perceived as being the most disorientating of the three menu item layouts (see Table 1 below for means and standard deviations).

Perceived Ease of Navigation

Examining the perception that a specific menu layout was easier to navigate found no significant differences between the three layout conditions (see Table 1 below for means and standard deviations).

Perceived Frustration 

Examining the perceived frustration of the participants with regard to the use of a specific menu layout conditions found no significant differences between them (see Table 1 below for means and standard deviations).

Table 1. Impressions of Link Locations - Mean (S.D.) (1 = “Disagree” and 6 = “Agree”)

 
Link Arrangement Preference

Analysis of the participants’ preference for each menu layout revealed no significant differences in ranking. However, examining the number of times a participant chose a particular menu layout as a first preference choice did indicate that the Index layout was particularly favored over the other Horizontal menu item layout (see Figure 6).

Figure 6.  First preference choices of participants.