Usability News

November 2010, Vol. 12 Issue 2

Usability News is a free web newsletter that is produced by the Software Usability Research Laboratory (SURL) at Wichita State University. The SURL team specializes in software/website user interface design, usability testing, and research in human-computer interaction.
Barbara S. Chaparro, Editor


Article PDF Print Article E-mail the Usability News Article PreviousNext


Keyboard Performance: iPad versus Netbook

B. Chaparro, B. Nguyen, M. Phan, A. Smith, & J. Teves

Summary. This study evaluated performance and subjective perceptions of three keyboard styles: the iPad soft keyboard (in both landscape and portrait orientation), as well as a Acer netbook keyboard. Sixteen participants (8 touchscreen phone users, 8 physical keyboard phone users) provided measures of hand anthropometry and performed phrase typing tasks on each of the keyboards. Results indicated that, while participants had roughly 20 WPM faster typing speeds with the netbook, overall satisfaction (SUS) scores indicate a slight preference for the iPad in landscape orientation. Overall, fewer errors were made on the iPad in portrait orientation. There were no significant differences for any of the measures due to reported touchscreen use. Surprisingly, there were no significant differences between the iPad in Landscape or Portrait orientation in terms of WPM and AdjWPM.

INTRODUCTION

We live in a mobile society, and recently we have been given many more opportunities for accessing our digital lives while on the go. Mobile devices, such as the smartphone, netbook, and eReader, are exploding in popularity. In fact, netbooks and low-end notebooks are the fastest growing component of the computing market (Elmer-DeWitt, 2010a). While they are not a substitute for a desktop PC or traditional laptop, these devices are attractive for many reasons, including cost and convenience. They allow users to surf the web, read electronic documents, play games, e-mail and remain in-the-know within their social networks. Shortly after its premier, the Apple iPad was forecasted to encroach on this market space, potentially capturing 7% of the market by 2011 (Elmer-DeWitt, 2010a, 2010b). Currently selling at a rate of 4.5 million units per quarter, the iPad is becoming the “most quickly adopted non-phone electronic device ever” (Jacobsson Purewal, 2010).

Predicting the iPad as a solid competitor is not necessarily surprising, given iPad users report completing many of the same tasks with the device as they could with a netbook (Nguyen & Chaparro, 2010). One big difference between the iPad and a netbook is that the iPad’s smaller form factor sports a soft (onscreen), rather than a physical, keyboard. This allows for a lighter, more portable device, and allows the user task-specific flexibility, allowing the keyboard to appear only when needed with adjustable orientation. While this is an option on many other mobile devices, many questions have emerged regarding touchscreen keyboard target size, feedback, and layout.

Text entry using fingers as the primary input method bring to focus concerns of touch target size, touch target layout, and limited tactile feedback. Since target size and layout are often restricted by the screen size of the device, users may be slower to type on devices with smaller target sizes (~10 mm) and those lacking tactile feedback (Lee & Zhai, 2009). This begs the question whether landscape or portrait orientations of the iPad soft keyboard can achieve a level of performance comparable to a physical keyboard. Aside from informal studies seen on the internet (e.g., Cross, 2010), no empirical studies have been done to explore this issue.

Purpose

The purpose of this study was to assess and compare performance and subjective measures between three keyboard styles. We evaluated one physical keyboard (netbook) and two soft keyboards (iPad in landscape and portrait orientations).

METHOD

Participants

There were a total of 16 participants (8 males and 8 females) in the study. Participants’ ages ranged from 18-27 years old (M = 20.75; SD = 3.42) and were recruited from Wichita State University. All were screened for prior keyboard experience; 8 participants reported using devices with physical keyboards only (e.g., laptops, desktop computers), and the other 8 reported experience with a touchscreen device (e.g., touchscreen phone). Twelve of the participants were right-handed and 4 were left-handed. None of the participants reported having experience with using an iPad prior to the experiment.

Materials

This study evaluated three keyboards on two benchmarked devices, an Acer Aspire One netbook (model D150-1920) physical keyboard and the Apple iPad (model A1219) landscape and portrait soft keyboard orientations (see Figure 1). Autocorrect settings were disabled on each device. Keyboard dimensions are given in Table 1. Phrase sets from MacKenzie and Soukoreff (2003) were randomized within Microsoft Visual Basic 2008 Express and presented to participants on a Dell XPS M140 laptop computer. A stopwatch was used to record time for each trial. Hand anthropometric data was gathered using an anthropometer.

Figure 1.  The keyboards tested in this study (left to right): iPad landscape orientation, iPad portrait orientation, and Acer netbook.

Figure 1. The keyboards tested in this study (left to right): iPad landscape orientation, iPad portrait orientation, and Acer netbook.

Table 1. Dimensions of the Three tested Keyboards

Dimensions (mm) iPad Landscape iPad Portrait Netbook
Overall x,y 196 x 67 147 x 50 240 x 90
Character key surface 14 x 14 10.5 x 10.5 13 x 13*
Space bar key surface 84 x 14 63.5 x 10.5 67 x 13*
Enter key 27 x 14 20 x 10.5 25 x 13*
Vertical spacing between keys 2 1.5 3
Horizontal spacing between keys 2.5 2 5

*excluding bezel

Procedure

The participants were first asked to sign an informed consent. Then, participants filled out an online background questionnaire. An experimenter obtained anthropometric measurements of the participants’ left and right hands. Measurements consisted of: hand length, hand breadth (metacarpal), finger lengths and breadths. Participants were also asked to keep the devices on the table top as they typed, rather than holding the device or resting it on their lap. Participants were allowed to adjust the height of their chair to facilitate a comfortable typing posture, maintaining a wrist position roughly parallel to the table top. Participants were instructed to type a series of phrases on each keyboard as quickly and as accurately as possible without correcting their mistakes, without capitalization or punctuation.

Task

The benchmark test was done by having the participants perform typing tasks on the three keyboards. For each keyboard, the participants were given a chance to familiarize themselves by typing 5 practice phrases. After the practice phrases, the participants were given 3 blocks of 5 trials (total of 15 recorded trials for each keyboard). The order of presentation of the three keyboards was counterbalanced across participants to control for practice effects. The time to complete typing each phrase was recorded with a stopwatch. Observations of which fingers the participants used for each keyboard were also recorded.

Post-Task Questionnaires

Upon completion of the typing task for each keyboard, the participants were asked to complete a subjective mental workload assessment (NASA TLX) and an online satisfaction questionnaire (adapted SUS from Brooke, 1996). After completing the tasks and questionnaires for each keyboard, the participants filled out an Overall Preference questionnaire to obtain information on which keyboard they preferred the most and why.

Dependent Variables

The following dependent measures were collected for each keyboard:

  1. Performance: phrase completion time, and error frequencies by type. There were five error types counted. For each phrase, multiple errors of the same type were possible. From these measures, words per minute (WPM), adjusted words per minute (AdjWPM) and uncorrected error rate (U) were calculated.
    • substitution errors (typing a different character than what was intended)
    • insertion errors (typing an extra character)
    • omission errors (typing fewer characters than intended)
    • word space (adding or omitting a character space using the space bar)
    • line space errors (adding or omitting a line space using the return/enter key)
  2. Subjective Mental Workload: NASA Task Load Index (NASA TLX) ratings for 6 dimensions.
  3. Satisfaction: System Usability Scale (SUS).
  4. Preferred Keyboard: Subjective rank order preference of the three keyboards.

RESULTS

Performance

For each typed phrase, performance measures were recorded and analyzed as per MacKenzie and Tanaka-Ishii (2007). The measures were typing speed in words per minute (WPM = T – 1/S × 12, where T = time to type a phrase, and S = actual phrase characters and spaces) and frequencies of five different error types (substitution, omission, insertion, word space, and line space). From these measures, uncorrected error rate (U) and adjusted words per minute (AdjWPM = WPM × (1 – U)) were derived.

A series of independent t tests were conducted to compare reported touchscreen users and physical keyboard users across performance measures for each keyboard (WPM, AdjWPM, and frequency of error type). No statistically significant mean differences were found.

Bivariate correlations were computed between anthropometric measurements and WPM, AdjWPM, and error frequency (for both total errors and each specific error type) for each keyboard. The only statistically significant negative relationship was between thumb breadth and netbook word spacing errors r(14) = -.51, p < .05. This indicates that fewer word spacing errors were made by users with greater thumb breadth when using the netbook.

WPM. The netbook facilitated significantly faster typing speeds in words per minute (see Table 2). A one-way within-subjects ANOVA revealed a statistically significant keyboard effect, Wilks’s λ = .15, F(2, 14) = 40.59, p<.001, multivariate η2 = .85. Three follow-up paired-samples t tests were conducted to investigate these mean differences. There were statistically significant differences between the netbook and iPad portrait keyboards, t(15) = -9.19, p< .001, and between the netbook and iPad landscape keyboards t(15) = -7.83, p< .001.

AdjWPM. The netbook typing speeds were also faster than the other keyboards after adjusting words per minute to accommodate error rate. A one-way within-subjects ANOVA revealed a statistically significant keyboard effect, Wilks’ λ = .17, F(2, 14) = 34.64, p<.001, multivariate η2 = .83. Three follow-up paired-samples t tests showed mean differences between the netbook and iPad portrait keyboards, t(15) = -8.19, p< .001, as well as differences between the netbook and iPad landscape keyboards, t(15) = -7.55, p< .001. While these findings are not surprising given participants’ familiarity with using a physical keyboard, it is surprising that there were no significant differences in WPM or AdjWPM between the iPad portrait and landscape orientations.

Errors.Typing error rates were calculated for each keyboard, relative to total characters entered. Substitution and insertion errors were most commonly made across each keyboard, followed by omission, word space, and line space errors (see Figure 2). Frequency means for each error type were compared across the three keyboards using five one-way within-subjects ANOVAs. Insertion errors was the only error type which yielded statistically significant keyboard mean differences, Wilks’ λ = .64, F(2, 14) = 3.82, p<.05, multivariate η2 = .35. Three follow-up paired-samples t tests indicated that the iPad portrait keyboard had significantly fewer mean insertion errors than the netbook keyboard, t(15) = -2.81, p< .05.

Table 2. Mean (SD) Performance per Phrase for Each Keyboard

  iPad Landscape iPad Portrait Netbook
Avg WPM 45.1 (13.2) 43.6 (14.7) 61.4 (17)
Substitution Err 0.58(0.98) 0.56 (0.87) 0.48 (0.95)
Omission Err 0.12 (0.41) 0.18 (0.46) 0.25 (0.64)
Insertion Err 0.53 (1.06) 0.27 (0.55) 0.53 (0.86)
Word space Err 0.1 (0.38) 0.24 (0.56) 0.13 (0.5)
Line space Err 0.05 (0.26) 0.01 (0.09) 0.0 (0)
Total Err 1.33 (1.74) 1.24 (1.4) 1.38 (1.65)
Avg Adjusted WPM 42.7 (13.1) 41.5 (13.3) 58.2 (17)

 

Figure 2. Error rates per total characters entered for each keyboard (%).

Figure 2. Error rates per total characters entered for each keyboard (%).

Perceived Mental Workload

After completing 3 blocks on each of the three keyboards (landscape iPad, portrait iPad, netbook), participants rated their subjective workload on a 20-point scale for six dimensions of the NASA TLX dimension (mental demand, physical demand, temporal demand, performance, effort, and frustration). Figure 3 shows a summary of the participants’ perceived workload for each dimension and keyboard. Repeated-measures ANOVA of all six dimensions did not reveal any statistically significant differences among three keyboards.

A series of bivariate correlations revealed statistically significant relationships between mean TLX ratings for individual keyboards and some of the anthropometric measures. The iPad portrait keyboard showed a statistically significant relationship between pinky width and mean ratings on the mental dimension, r(14) = -.53, p < .05, indicating that users with smaller pinky widths tended to report higher levels of mental demand. A significant relationship was also found between index length and the temporal dimension, r(14) = -.54, p< .05, as well as middle finger length and the temporal dimension, r(14) = -.63, p< .01, indicating that users with shorter index and middle fingers tended to report feeling more hurried or rushed to complete the task while using the iPad portrait keyboard. The netbook keyboard condition showed a statistically significant relationship between index finger length and the performance dimension, r(14) = .54, p< .05, as well as middle finger length and the performance dimension, r(14) = -.56, p< .05, indicating that those with longer index and middle fingers tended to report lower levels of perceived success with the typing task. A significant relationship was also found between middle finger length and the frustration dimension, r(14) = -.50, p< .05, indicating that users with longer middle fingers tended to report higher levels of feeling frustration while using the netbook keyboard.

Figure 3. Mean perceived mental workload ratings for the three keyboards across six NASA TLX dimensions. Scores range from 1 to 21. The higher the number, the higher the perceived demand.

Figure 3. Mean perceived mental workload ratings for the three keyboards across six NASA TLX dimensions. Scores range from 1 to 21. The higher the number, the higher the perceived demand.

Satisfaction

An adapted version of the System Usability Scale satisfaction scores range from 0 to 100, with larger numbers indicating a greater level of satisfaction. Results showed that participants had highest satisfaction with the iPad landscape keyboard and the lowest satisfaction with the iPad portrait keyboard (see Figure4), however, results from a one-way repeated-measures ANOVA revealed that these differences were not statistically significant, Wilks’ λ = .81, F(2, 14) = 1.64, p>.05, multivariate η2 = .19.

Figure4. Mean satisfaction ratings (+ standard error) for each keyboard.

Figure 4. Mean satisfaction ratings (+ standard error) for each keyboard.

Preference

The netbook and the landscape keyboards were equally preferred among participants (see Figure 5). Two participants preferred the iPad portrait keyboard over the other two types of keyboards. Among those who ranked the netbook keyboard as the most preferred, only one participant ranked the iPad portrait keyboard as their second choice, while the remaining participants ranked the iPad landscape keyboard as their second choice.

Figure 5. Keyboard Preference Summary

Figure 5. Keyboard Preference Summary

A correlation analysis was conducted between the hand measurements and overall keyboard preference. There were no significant correlations. General comments about the iPad landscape keyboard were that the keyboard size was relatively similar to the physical keyboard size they typically used which made it easier to type. Also, participants liked the way the keys were spaced out on the landscape orientation, which they felt helped them to avoid hitting two keys at the same time when they typed. This was particularly useful for people with large hands. The general complaints people had about the landscape keyboard concerned the lack of feedback for their hand and finger positions. Some participants also reported that they disliked that there was less viewing space when in landscape orientation.

Participants who chose the netbook keyboard as the most preferred keyboard reported that this was because they enjoyed the tactile feedback when they typed. This feedback allowed them to freely reposition their hands and fingers on the keyboard. This keyboard was also the most familiar to them. Some participants commented that the small key size made it difficult to type with all 10 fingers.

Most participants did not list the iPad portrait keyboard as their most preferred keyboard. Justifications for this ranking focused on the small keyboard size; participants claimed that the keys were“crammed”which made it more difficult for them to type accurately, especially with all ten fingers. They reported that it was not comfortable to type on this keyboard and as a result, many resorted to “hunt-and-peck” style typing (Table 3).

Table 3. Selected Participant Comments by Keyboard

iPad Landscape iPad Portrait

Netbook

"Very responsive, nice spacing, able to type very quickly, only lacked some sort of non-visual cue for key position to allow very fast typing." "The keys were too small and close for me to effectively use this keyboard." "It felt the most natural due to my affinity for common keyboards and my relative inexperience with touch screens."
"It was closest to a regular size keyboard and the easiest to use. Buttons were large and easy to see; well separated." "I liked least due to it being not the normal keyboard set up." "The keys were not a touch screen, I could actually feel like I was pressing a key down."
"I liked how the device wasn't too hard to use and the keys were bigger so your finger doesn't hit more than one key at a time." "It was hard for me to find the letters." "I liked it the best because I can actually type normally on it. It is still little compared to normal keyboards. So there was still a lot more mistakes when typing."
"I feel given a week or two I could master this keyboard and enjoy it." "Used to typing this way on my iPhone." "Too small for me to fit my hands on comfortably."


DISCUSSION

Results from this study show that participants were slower when typing on the iPad touchscreen keyboard when compared to the netbook physical keyboard. These results were not surprising given the familiarity of the participants with physical keyboards. It was somewhat surprising that user performance was equal for the iPad portrait and landscape keyboards, given the difference in size and researchers’ observations that typing style among users, who complained that the portrait keyboard was “crammed”, was compromised.

Performance on both iPad keyboards was impressive for first-time users. Some participants said that they thought they would enjoy using the keyboard on the iPad more if they had additional time to practice. Overall, however, participants commented that they probably would not ever use the iPad keyboards for typing long documents (e.g., writing an essay paper), but rather limit its use to shorter passages of text (i.e., email) or to surf the Internet.

This study had several limitations that could be addressed in future research. First, this was a laboratory study where each keyboard was positioned on a stable surface of adequate height. More errors would be expected when using each device in a mobile environment (Hoggan et al., 2008). Second, the study examined first-time usage of the iPad for typing. It is not known how fast users may be able to type on the iPad with adequate practice and how this would compare to their typical speed with a physical keyboard. Third, the iPad keyboard provided no tactile feedback to users as they typed. There are applications available to provide vibratory feedback while typing. It is not known how the presence of this feedback may impact typing performance or user satisfaction. Some studies have suggested that this can be done, implementing synthetic feedback (Lee & Zhai, 2009).

 

REFERENCES

Brooke, J. (1996). SUS: A Quick and Dirty Usability Scale. In P. Jordan, B. Thomas, B. Weerdmeester, &
I. L. McClelland (Eds.), Usability evaluation in industry (pp. 189-194). London, UK: Taylor &
Francis.

Cross, J. (2010, April 3). iPad vs. netbook: It’s a close call [Web log message].Retrieved from
http://www.pcworld.com/article/193352/ipad_vs_netbook_its_a_close_call.html?loomia
ow=t0:s0:a38:g26:r4:c0.011504:b32577402:z0

Elmer-DeWitt, P. (2010a, February 1). Apple’s iPad vs. the netbooks [Web log message]. Retrieved from
http://tech.fortune.cnn.com/2010/02/01/apples-ipad-vs-the-netbooks/

Elmer-DeWitt, P. (2010b, May 6). How the iPad gobbles up netbook sales [Web log message]. Retrieved
from http://tech.fortune.cnn.com/2010/05/06/how-the-ipad-gobbles-up-netbook-
sales/

Jacobsson Purewal, S. (2010, October 5). iPad, fastest-selling electronic device…ever [Web log message].
Retrieved from http://www.pcworld.com/article/206953/ipad_fastestselling_electronic_
deviceever.html

Hoggan, E., Brewster, S. A., & Johnston, J. (2008). Investigating the effectiveness of tactile feedback for
mobile touchscreens. Proceedings of CHI (pp. 1573-1582). Florence, Italy: ACM Press.

Lee, S. and Zhai, S. (2009). The performance of touch screen soft buttons. Proceedings of CHI (pp. 309-
318). Boston, MA: ACM Press.

MacKenzie, I. S., & Soukoreff, R. W. (2003). Phrase sets for evaluating text entry techniques. Extended
Abstracts of the ACM Conference on Human Factors in Computing Systems – CHI 2003 (pp. 754–
755). New York: ACM. Retrieved from
http://www.yorku.ca/mack/chi03b.html

MacKenzie, I. S., & Tanaka-Ishii, K. (2007). Text entry systems: Mobility, accessibility, universality. New
York: Morgan Kauffman.

Nguyen, B., & Chaparro, B. S. (2010). Survey results: iPads great for reading, communicating, and
gaming. Usability News, 12(2).


Article PDF Print Article E-mail the Usability News Article PreviousNext