Mark that e-Page! The Usability of making Notes, Bookmarks, and Highlights in Three E-Readers
Summary. E-Readers, or electronic reading devices, are becoming a household item; however, there exists little research on the usability of these devices. This study investigates the ease-of-use of three e-Readers (iPad, Kindle 2, and Nook 1st Edition) for basic e-Book navigation. Participants (N = 17) were asked to complete a series of book tasks (e.g., opening a book, bookmarking a page, etc.) on the three e-Readers. For each device, participants were asked to rate the task difficulty and the facilitator recorded completion time and number of taps to complete the task. Participants also rated each device in terms of perceived workload and satisfaction. After using all three e-Readers, participants were asked to rank each device on a series of attributes (e.g., screen size, bookmarking, glare, etc.) and overall preference. Results revealed that the iPad was better than the Nook and Kindle on all performance measures, task difficulty, perceived workload, satisfaction, and preference. The only task in which the iPad proved to be more difficult than the Nook and Kindle was the note-making task. Suggestions for improving future e-Readers are discussed.
E-Readers are becoming a device people use every day. E-Readers allow the owner to carry hundreds of books at once on a small portable device and order books instantly through online stores. To keep up with tablet devices, e-Readers are now offering apps, music listening capabilities, web browsing, and the ability to play games.
Amazon.com has reported selling more e-Books than paper books throughout different time periods in the past couple of years (Kennedy, 2010). In 2010, Barnes & Noble and Amazon reported increased sales which is partially due to sale of their e-Reader products and accessories (Barnes & Noble Reports Fiscal 2011 Third Quarter Financial Results, 2011; Glover, 2011). Furthermore, for every 100 paperback books Amazon.com sold in 2010, 115 e-Books were sold (Glover, 2011).
As e-Book and e-Reader sales increase, companies making e-Readers may have to develop better and more user-friendly devices to get consumers to purchase their device over another. What do users like or dislike about current e-Readers? What problems do users encounter when they are trying to complete specific tasks while reading an e-Book on an e-Reader? Answers to these questions may help companies in designing the next top-selling e-Reader.
The purpose of this study was to evaluate three competing e-Readers (iPad, Kindle 2, and Nook 1st Edition) for usability, preference, satisfaction, and workload. All three devices provide the capability to not only read books, but also to make bookmarks, highlight text, and keep notes.
The participants in this study were 17 (12 females and 5 males) Wichita State University students, who participated for course credit. Participants’ ages ranged from 18 to 59 (M = 25.8; SD = 12.5). Fifteen of the participants were right-handed and 2 were left-handed. None of the participants reported having any prior e-Reader experience before the experiment.
Three e-Readers were evaluated in this study, an iPad (using the iBook application to view e-Books), Kindle 2, and Nook 1st Edition (See Figure 1 for a piture of each device and Table 1 for device dimensions). While the Kindle 2 and the Nook 1st Edition have been replaced by successor touch screen devices, both have been praised as quality devices for simple book reading. In addition, these devices are somewhat cheaper than their successors and, as a result, are more likely to be used by a student population like the participants in this study.
All e-Readers were utilized by participants in portrait mode. A web camera and Morae (version 3.1) were used to record the interactions with the e-Readers for each task and a stopwatch was used to record time for each task.
Figure 1. The three e-Readers tested in this study from left to right: iPad, Kindle 2, and Nook 1st Edition.
Table 1. The dimensions of the three e-Readers.
|Dimensions (mm; Height x Width x Depth)|
Participants completed all tasks on each of the three e-Readers. The presentation order of the three e-Readers and the user tasks was counterbalanced across participants to control for order effects (See Table 2 for tasks and task descriptions). The time to complete each task was recorded with a stopwatch, along with the number of taps to complete a task. Task success (whether or not a participant successfully completed a task) was also recorded. Participants were allowed up to two minutes to complete each task. After each task, participants were asked to rate the difficulty of the task on a 5-point Likert scale (1 = very easy, 5 = very difficult).
Table 2. Tasks and descriptions of each task.
|Open Book||Open the book|
|Get to Page||Navigate to a certain page within the book|
|Add Bookmark||Add a bookmark to the current page|
|Make Note||Make a note on the first sentence of the page|
|Text Search||Find a certain sentence within the book|
|Highlight||Highlight this sentence|
|Turn Page||Turn five pages ahead|
|Change Text||Change the size of the text by two font sizes|
|Locate Bookmark||Locate the previously made bookmark|
|Delete Boookmark||Delete this bookmark|
After completing all the tasks on one e-Reader, the participants were asked to complete two questionnaires. One questionnaire was a subjective workload assessment (NASA-TLX), which asks for ratings on six dimensions (mental, physical, performance, temporal, effort, and frustration) on a 21-point Likert scale. The other questionnaire was an online satisfaction survey (adapted SUS from Brooke, 1996). Once the participants completed all the tasks and the questionnaires for each e-Reader, they were asked to rank their preferences for different attributes of the e-Readers, as well as their overall preference (See Table 3 for attributes and descriptions of the different attributes). The presentation of the eight attributes was randomized across participants.
Table 3. Attributes and descriptions of attributes.
|Legibility||Legibility of the text|
|Glare||Glare on the screen|
|Menu||The structure of the menu|
|Highlight||Highlighting on the device|
|Notes||Making a note on the device|
|Bookmark||Adding, removing, and finding a bookmark|
|Search||Searching for a particular text or page|
|Screen Size||Size of the screen|
Task Success. Successful and unsuccessful completion of each task was recorded for each participant for each device. Figure 2 shows the frequency of success for each task on each device. The iPad had three tasks ("Turn Page", "Open Book", and "Change Text") that all participants successfully completed, the Nook had one ("Delete Bookmark"), and the Kindle had none.
Figure 2. Frequency of completion for each task on each device (N=17).
Completion Time. A repeated-measures ANOVA revealed that there was a statistically significant difference between the average time (average time for a participant across all tasks on one device) it took participants to complete all tasks on each device, F(2,16) = 8.924, p < 0.01; η2= .358. Post-hoc analysis indicated that participants completed the tasks faster on the iPad than on the Nook and Kindle. Further repeated-measures ANOVAs were conducted for completion times for each task across devices (See Figure 3). Results showed that in performing the “Change Text” and “Turn Page” tasks, participants were faster on the iPad than on the Nook and Kindle. Participants were also faster on the iPad than on the Kindle when performing the “Get to Page" task and faster on the iPad than the Nook when performing the “Open Book” task (p < .05).
Figure 3. Average completion time per task on each device (N=17).
Note: Error bars represent +/- 1 standard error.
A repeated-measures ANOVA revealed that there was a difference in perceived difficulty of all the tasks across the devices, F(2, 16) = 4.812, p = 0.02; η2 = .231. Post-hoc analysis showed that participants reported the difficulty of the tasks to be lower on the iPad than on the Kindle and the Nook. This was true for the “Change Text”, “Open Book”, “Turn Page”, and “Get to Page” tasks. Interestingly, the iPad was rated as more difficult on the “Make Note” task, p < .05. No difference between the devices was found for the “Highlight”, “Add Bookmark", "Delete Bookmark”, or “Locate Bookmark” tasks (See Figure 4 for average task difficultly ratings).
Figure 4. Average task difficulty rating per task on each device (N=17).
Note: Error bars represent +/- 1 standard error.
An adapted version of the System Usability Scale (SUS) was used to gather satisfaction ratings, which range from 0 to 100. A higher number indicates a greater level of satisfaction with the e-Reader. A repeated-measures ANOVA was calculated, which revealed a significant difference between the devices on total satisfaction scores, F(2,16) = 16.276, p < 0.01; η2 = .504. Post-hoc tests test revealed that participants had higher satisfaction with the iPad than the Kindle and Nook. Figure 5 shows the average overall satisfaction score for each device.
Figure 5. Average satisfaction score for each device (N=17).
Note: Error bars represent +/- 1 standard error.
Participants were asked to rank their preferences of the different devices along eight different attributes and overall preference. Friedman’s test was run to see if there were any differences among the devices on overall preference and the result was significant, x2(2) = 16.353, p < 0.01. Post-hoc analysis indicated that the iPad was ranked higher than the Nook and Kindle in terms of overall preference. Similar results were found for the following attributes: “Screen Size”, “Searching”, “Bookmarking”, “Highlighting”, and “Legibility of Text”. No difference was found among the devices on “Note Making”, “Menu Structure”, and “Glare”. See Figure 6 for the frequency of first rankings among the devices on the different attributes.
Figure 6. Frequency of first rankings among the devices on the different attributes (N=17).
Results from this study showed that for many e-Book usage tasks the iPad was preferred and resulted in better performance. The exception to this was for the “Make Note” task where the Nook and Kindle were found to be easier. In addition, the tasks “Add, Delete, or Locate Bookmark” or “Search Text” were found to be comparable across devices.
On the Kindle and Nook, there is an option within the menu to “Add note or highlight”, but on the iPad one must touch on the e-Book’s text to make a menu appear with the note option. Some participants were not able to type a note because they did not find the note option on the iPad. There was no indication that tapping on the text would make the note option appear.
Another problem encountered while making a note on the iPad was that participants were unsure of what to do after they typed in the note text. There was no save or submit key, so participants were confused on how to close the note. On the Nook there was a “Submit” button on the keypad and on the Kindle there was a “Save Note” button available onscreen, so participants completed the task after they clicked on either of these easy-to-locate buttons. Most participants tried the “return” key on the iPad, but this just made the cursor move down one line on the note. To close the note on the iPad, one merely needed to tap outside of the note, but this did not seem intuitive for the users. Perhaps either a “Save Note” or “Submit” button could be added to make submitting a note more obvious or add text that directs the users to tap outside of the note to submit it.
iPad: The bigger screen of the iPad, multi-touch capability, color display, and “flashy” graphics contributed to overall ease-of-use and higher preference for the iPad. Participants felt the iPad was “more natural” and easy to use.
Nook: Most participants did not like using the small touch screen menu on the Nook when compared to the iPad. Participants complained that it was hard to hit the correct button on the touch screen. Some participants said the Nook was “overly complex”. Also, many participants took a long time to turn pages because they did not notice the arrow buttons by the e-Ink screen to turn pages. Most participants tried turning pages using the side scrolling page locator with the touch screen. However, a few participants did like the menu structure of the Nook claiming it was easy to find functions within the menus.
Kindle: A few participants said they preferred the size of the Kindle over the iPad because they would be more likely to carry the smaller Kindle around. Some dislikes about the Kindle were the small buttons and the difficulty navigating with the joystick. Adding an “enter” key may help with some of the problems with the Kindle. The "enter" key would allow a user to search for text or a page number with one click instead of using the joystick to move the cursor over to the option to search the text.
It may be interesting to conduct a similar study with newer e-Readers. Nielsen (2011) found that the Kindle Fire provided a poor usability experience, which was in part due to a small screen size. However, Nielsen looked mostly at web browsing on the Kindle Fire and not the device’s e-Reading capabilities. Since many of the users in this study preferred the multi-touch screen capabilities of the iPad, comparing touch screen e-Readers may yield different results. It may also be interesting to see how the legibility of an e-Ink e-Reader, like the Kindle used in this study, compares to the Kindle Fire.
The authors would like to acknowledge and thank Amber Teschendorf, Karla Morton, and Tamell Jason-Hertman for their help in collecting data for this study.
Barnes & Noble (2011). Barnes & Noble reports fiscal 2011 third quarter financial results. Retrieved from: http://www.barnesandnobleinc.com/press_releases/2011_feb_22_3rd_quarter_financial_results.html..
Brooke, J. (1996). SUS: A quick and dirty usability scale. In P. Jordan, B. Thomas, B. Weerdmeester, & I. L. McClelland (Eds.), Usability evaluation in industry (pp. 189-194). London, UK: Taylor & Francis.
Glover, J. (2011). Rise of the Amazon Kindle as eBooks outsell paperbacks in 2010. Retrieved from: http://www.suite101.com/news/rise-of-the-amazon-kindle-as-ebooks-outsell-paperbacks-in-2010-a339232#ixzz1GR5ywPmz..
Kennedy, S. D. (2010). Internet waves: eBooks by the numbers. Information Today, 27 (9). Retrieved from: http://goliath.ecnext.com/coms2/gi_0199-13641152/Ebooks-by-the-numbers.html..
Nielsen, J. (2011). Kindle Fire usability findings. Alertbox, December 5, 2011. Retrieved from: http://www.useit.com/alertbox/kindle-fire-usability.html..