An Assessment Method and Toolkit to Evaluate Keyboard Design on Smartphones

Podsumowanie

The presented protocol integrates various evaluation methods and demonstrates a method to evaluate the keyboard design on smartphones. Pairs matched by English characters are proposed as the input material, and the transition time between two keys is used as the dependent variable.

Streszczenie

Keyboard input has played an essential role in human-computer interaction with a vast user base, and the keyboard design has always been one of the fundamental objects of studies on smart devices. With the development of screen technology, more precise data and indicators could be collected by smartphones to in-depth evaluate the keyboard design. The enlargement of the phone screen has led to unsatisfactory input experience and finger pain, especially for one-handed input. The input efficiency and comfort have attracted the attention of researchers and designers, and the curved keyboard with size-adjustable buttons, which roughly accorded with the physiological structure of thumbs, was proposed to optimize the one-handed usage on large-screen smartphones. However, its real effects remained ambiguous. Therefore, this protocol demonstrated a general and summarized method to evaluate the effect of curved QWERTY keyboard design on a 5-inch smartphone through a self-developed software with detailed variables, including objective behavioral data, subjective feedback, and the coordinate data of each touchpoint. There is sufficient existing literature on evaluating virtual keyboards; however, only a few of them systematically summarized and took reflection on the evaluation methods and processes. Therefore, this protocol fills in the gap and presents a process and method of the systematic evaluation of keyboard design with available codes for analysis and visualization. It needs no additional or expensive equipment and is easy to conduct and operate. In addition, the protocol also helps to get potential reasons for the disadvantages of the design and enlightens the optimization of designs. In conclusion, this protocol with the open-source resources could not only be an in-class demonstrative experiment to inspire the novice to start their studies but also contributes to improving the user experience and the revenue of input method editor companies.

Wprowadzenie

Keyboard input is the mainstream method of the human-smartphone interaction^1,2, and with the penetration of smartphones, keyboard input gets billions of users. In 2019, the global smartphone penetration rate had reached 41.5%³, while the United States, with the highest penetration, had come up to 79.1%⁴. Up to the first quarter of 2020, the Sogou mobile keyboard had about 480 million daily active users⁵. Up to May 6, 2020, the Google Gboard had been downloaded more than 1 billion times⁶.

Unsatisfactory keyboard input experience increases with the enlargement of the phone screen. Although the enlarged screen aimed to improve the viewing experience, it has changed the gravity, size, and weight of smartphones, causing users to change holding posture repeatedly to reach remote areas (e.g., button A and Q for right-handed users), thus leading to input inefficiency. The stretch of muscle may cause users to suffer from musculoskeletal disorders, hand pains, and different types of disease (e.g., carpal tunnel syndrome, thumb osteoarthritis, and thumb tenosynovitis^7,8,9,10). Users who prefer one-handed usage are under worse conditions^11,12.

Therefore, the evaluation and optimization of keyboard design have become hot topics of psychological, technical, and ergonomic research. Variable keyboard designs and concepts have constantly been proposed by input method editor (IME) companies and researchers to optimize input experience and efficiency, including layout-changed and character-reordered keyboards: Microsoft WordFlow Keyboard¹³, Functional Button Area in Glory of Kings¹⁴, IJQWERTY¹⁵, and Quasi-QWERTY¹⁶.

Existing evaluation methods of keyboard design vary from researcher to researcher except for several highly accepted indicators, and more accurate indicators are proposed. However, with a variety of indicators, there is not a summarized and systematic protocol provided to demonstrate the process of evaluating and analyzing the keyboard design. Fitts’ Law¹⁷ and its extended version FFitts Law¹⁸, which described human-computer interaction, were widely adopted to evaluate keyboard performance^19,20,21,22. Moreover, the functional area of the thumb was proposed to improve keyboard design, and it described a curved motion area for the thumb to comfortably complete the input task²³. Based on these theories, indicators including word per minute, word error rate, and subjective feedback (perceived usability, perceived performance, perceived speed, subjective workload, perceived exertion and pain, and intent to use, etc.), which were highly adopted, were partially used in previous studies^{24,25,26,27,28,29} except for modeling and simulation methods. In addition, the fitted ellipse of touchpoints on each button and its offset^30,31 were used in recent years to investigate the accurate performance of inputting events. Also, the galvanic skin response, heart rate, electromyographic activity, hand gesture, and body movement^32,33,34,35 were adopted to directly or indirectly evaluate muscle fatigue, comfort, and satisfaction of the users. However, these various methods lack reflection on the appropriateness of the indicators used, and a novice researcher may be confused to select the appropriate indicators for his or her research.

The research about keyboard design is also easy to be conducted, operated, and analyzed. With the boom of screen technology, more behavioral data could be easily collected to evaluate the keyboard design in-depth (e.g., the transition time between two keys and the coordinate data of each touchpoint). Based on the mentioned data, researchers could precisely explore the details of keyboard design and analyze its disadvantages and advantages. When compared with other human-computer interaction research, the research of keyboard design on portable smartphones also has high application value for its vast user base with no expensive equipment, complicated materials, or huge laboratory space needed. The questionnaires, scales, and Python script about the research are open-source and easy to access.

The purpose of this research is to summarize the previous methods to demonstrate a systematic, precise, and general protocol to evaluate and analyze the keyboard design on smartphones. The exemplar experiment and results aim to show whether the curved QWERTY keyboard with size-adjustable buttons could optimize the input experience of one-handed input on a 5-inch smartphone when compared with traditional QWERTY keyboard and share the visualization method and Python script of data analysis.

Access restricted. Please log in or start a trial to view this content.

Protokół

The study was conducted in accordance with the ethical principle and was approved by the Ethics Committee of Tsinghua University. Figure 1 shows the process of evaluating the keyboard design of smartphones.

Figure 1: General process of conducting a keyboard experiment and evaluating the keyboard design. Please click here to view a larger version of this figure.

1. Preparation

1. Experiment design
   1. Define the research issue and propose the hypothesis.
   2. Design the experiment according to the hypothesis and define the independent variables (e.g., keyboard layout, typing posture). Use the within-subject design in order to reduce confounding factors and variance caused by the difference among participants.
2. Dependent variables
   1. Use physical data, including the hand length, the length of input finger, and the circumference of input finger, which were measured by a tape measure, as shown in Figure 2.
   2. Use physiological data, including galvanic skin response (measured by the portable wireless physiological detector), heart rate (measured by the portable wireless physiological detector), electromyographic activity (measured by surface electromyography), etc.
   3. Use input performance: word per minute, word error rate, and transition time between two keys.
      1. Word per minute refers to the input speed of participants (i.e., the number of correct-inputted words per minute).
      2. Word error rate refers to the input accuracy of participants (i.e., the number of incorrect-inputted words divided by the total number of words under one condition). Corrected error rate, uncorrected error rate, and total error rate have also been used in previous studies³⁶.
      3. Transition time between two keys refers to the reaction time of participants between two touchpoints of a correct-inputted word²² (i.e., the start time of the second touchpoint minus the departure time of the first character).
   4. Use body-movement data such as hand gesture and body (finger) movement. They could be collected by the motion capture system³⁵.
   5. Use subjective data such as perceived usability, intent to use, perceived accuracy and speed, perceived exertion and pain, and subjective workload, etc. Subjective data can be obtained through existing scales and questionnaires, which are highly reliable as well as valid to better evaluate the subjective feedback of participants about the keyboard design.
      1. Use NASA-TLX, a 21-point scale that is used to measure subjective workload through mental, physical, time, performance, effort, and frustration dimensions. A high score indicates a high subjective workload²⁶.
      2. Use the System Usability Scale, a 5-point questionnaire with 10 items, and the responses of one participant will be calculated as a single score from 0 to 100. A high score indicates a high perceived usability²⁴.
      3. Use the Borg CR10 Scale, which is ranged from 0 to 10 to measure perceived pain and exertion. A high score indicates a high-level perceived pain and exertion²⁵.
      4. Use the Intent to Use Scale: a 10-point questionnaire that is used to measure the likelihood that participants would use the technology or products. A high score indicates a high-level likelihood²⁸.
      5. Perceived speed and perceived accuracy are all measured by 50-point scales, and a high score indicates a good perceived performance²⁸.
   6. Collect the coordinate data of each touchpoint and change it into the fitted ellipse (95% CI) of touchpoints on each button^30,31. Adopt the area of each fitted ellipse and the offset from the center of the fitted ellipse to the target center of each button as dependent variables.
      NOTE: The coordinate data can be precisely collected by the self-developed application on the smartphone. If it is hard to obtain the coordinate data, objective and subjective data are sufficient to roughly evaluate the keyboard design.

Figure 2: The measurement of the hand. Please click here to view a larger version of this figure.

3. Materials
   1. Choose the experiment smartphone. Take weight, resolution, and screen size into consideration.
   2. Design and develop the experiment software on smartphones (optional step).
      NOTE: Transition time between two keys can be recorded automatically by this software or motion capture sensors (i.e., the accelerometer sensor). It may be difficult to collect it manually (e.g., a clock or stopwatch).
   3. Select the input task from the following suggestions based on the hypothesis and revise it to match the research purpose.
      1. For the character pair input task, randomly pair 26 English letters into 676 pairs and averagely divide them into several groups based on the experiment design.
      2. For the phrase (sentence) input task, use phrases that are moderate in length, easy to remember, and representative of the target language. If the target language is English, extract 15–20 (or based on research purpose) phrases or words from a 500 phrases set³⁷.
4. Participant recruitment
   1. Use the G*Power software to calculate the sample size.
   2. Post questionnaires to recruit potential participants.
   3. Filter potential participants with wanted characteristics, e.g., age, health, vision, handedness, and input experience. Ensure that the input experience of participants is balanced.

2. Procedure

1. Read out the informed consent form of the experiment to participants, including the experiment procedure, task, and whether they will encounter any mental or physical injuries. If participants agree to participate, they need to sign the informed consent form. If not, they can immediately withdraw. According to the informed consent form, participants can also withdraw at any stage of the experiment.
2. Collect physical as well as demographic data. Use a tape measure to measure the hand of every single participant (Figure 2) in order to eliminate the effect of the hand size difference and also provide repeatable data for future research. Collect demographic data such as age, gender, precise input experience, and occupation.
3. Disinfect all devices and clean the body parts of the participant that will touch the devices.
   1. Ask participants to wash their hands and clean the screen of smartphones so that sensors of smartphones can be more sensitive.
   2. Ask participants to wear portable wireless physiological detectors or a motion capture system. Ask participants to wear the portable wireless physiological detection wristband on the non-dominant hand to record galvanic skin response and heart rate with the noise interference avoided.
      1. Place passive markers of the motion capture system on the fingernails, the proximal phalanx of the finger, cervical vertebrae (C3–C5), and arm, to collect the precise body and finger movement. Stick wireless electrodes to the skin of two arms and two forearms to detect the electromyographic activity (optional step).
   3. Calibrate all the devices used in the experiment.
4. Practice part
   1. Let participants complete the training task. The training task is used to improve participants' familiarity with input tasks and keyboards to reduce the effect of practice or the unfamiliarity on the experiment result. It is composed of 50 pairs or 20 words randomly selected from the 676 English pairs set or 500 phrases set. Only when their input accuracy reaches 80% or more in 150 seconds can they enter the formal trials. The exemplar research adopted inputting 50 pairs as the training task.
5. Main task
   1. Let participants complete formal trials under all experimental conditions. They need to ensure their accuracy as quickly as possible during the time of the input task. Formal trials are real input tasks that will be evaluated and analyzed in the research. Each pair, word, or sentence represents a trial, and different experimental designs produce different experimental conditions.
   2. Have participants complete the input task in random order or a balanced order. Methods of the division of input materials are as follows. First, 676 pairs can be randomly divided into each experimental condition (i.e., participants have entered all pairs when they complete all experimental conditions). Second, under each experimental condition, 676 pairs can be divided into several blocks randomly, and participants need to complete these blocks randomly. Third, for inputting words, participants need to complete around 20 trials under each condition. Fourth, for inputting sentences, participants need to complete about 10–15 trials under each condition. Researchers should ensure no significant difference between the number of characters and the number of words entered by the participant under each condition. The exemplar research adopted the first method and had four experimental conditions.
   3. After each condition, ask participants to complete all the questionnaires (scales assessing their subjective experience) at random and give them 1 min or more to rest.
6. At the end of the experiment, let each participant finish the comprehensive questionnaire (Q & A) to obtain subjective feedback.
7. Express appreciation to participants with monetary or material rewards.

3. Data analysis

1. Hypothesis testing by appropriate parametric or non-parametric tests
   1. Analyze the physical, physiological, and body-movement data to test whether the difference between participants would significantly influence the results and inexpressive input experience of users (optional step).
   2. Analyze the input performance of participants to test the input efficiency on the keyboard.
   3. Analyze subjective data to test the perceived usability and subjective feedback of the keyboard.
   4. Figure out whether the practice effect and fatigue effect significantly influence the result. For each condition, trials are divided into two parts according to the timestamp (i.e., the first half part and the second half part). Specifically, under each condition, examine the difference of input performance between the first half part and the second half part to test whether the practice effect or fatigue effect exist.
   5. Analyze the area of the fitted ellipse of touchpoints on each button as well as the offset from its center to the target center of each button (optional step).
      1. Collect all the touchpoints of each button with the software, and they roughly accord with the bivariate Gaussian distribution. The 95% confidence interval of each button in both x- and y-directions is derived through the coordinate data of each touchpoint in pixel, and the 95% confidence ellipses over a 1:1 outline of the button for each keyboard is fitted through Python scripts on pixel coordinate (see Coding File 2).
      2. Use fitted ellipses (95% CI) and their areas to demonstrate the dispersion of touchpoints on each button. In each button, the offset of fitted ellipse calculated by Python scripts is defined as the center point of the fitted ellipse to the target point of the button, and it could be represented from x- and y-directions (i.e., in X-axis and Y-axis, see Coding File 3).
2. Modeling and simulation
   1. Use the data-driven model as a function of keyboard location and orientation to predict the finger movement by Python scripts. All movements of fingers are divided into eight directions³⁸ (the top to the bottom, the bottom to the top, the left to the right, the right to the left, the left-top to the right-bottom, the right-bottom to the left-top, the left-bottom to the right-top, the right-top to the left-bottom). For each direction, the average transition time between two keys is calculated to represent the effectiveness of finger movement, which is used to evaluate the keyboard design (optional step).
   2. Use linear regression analysis to build an enhanced Fitts' Law (or its extended version, FFitts’ Law) model to predict the transition time between two keys using an integrated cognitive architecture³⁹ by Python scripts. The enhanced Fitts' Law model could provide a better prediction and evaluation on keyboard design based on its analyses on the location and effective width of keys, as well as the distance of two keys (optional step).

Access restricted. Please log in or start a trial to view this content.

Wyniki

The representative study is mainly following the mentioned protocol. The study adopts a 2 (Keyboard layout: Curved QWERTY vs. Traditional QWERTY) × 2 (Button size: large, 6.3 mm × 9 mm vs. small, 4.9 mm × 7 mm) within-subject design to evaluate whether the curved QWERTY could improve the input efficiency and comfort when compared with the traditional QWERTY in different sizes of buttons by the character pair input task through our self-developed software (Figure 3). This study...

Access restricted. Please log in or start a trial to view this content.

Dyskusje

In this study, based on the development of screen technology, we presented a summarized and general protocol of keyboard design evaluation to assess the keyboard design systematically and precisely. Existing indicators and methods from previous studies, pairs matched by English characters, and transition time between two keys are integrated and modified to generate an effective protocol.

Several critical points need to be noticed in this protocol. The selection of variables and indicators is e...

Access restricted. Please log in or start a trial to view this content.

Materiały

Name	Company	Catalog Number	Comments
Changxiang 6S smartphone	Huawei		Smartphone used in the examplar study
Curved QWERTY keyboard software	Tsinghua University		Developed by authors
SPSS software	IBM		Data analysis software
G*Power software	Heinrich-Heine-Universität Düsseldorf		Sample size calculation
E4 portable wireless wristband	Empatica		Recording galvanic skin response and heart rate
Arqus	Qualysis		Motion capture camera platform
Passive marker	Qualysis		Appropriate sizes: 2.5 mm, 4 mm, and 6.5 mm
Trigno sEMG	Delsys		Recording electromyographic activity
Visual Studio Code	Microsoft		Python editor