Project overview
Challenge
As part of the German government's digitalization initiative, the "Onlinezugangsgesetz" there is an opportunity to enhance the usability of forms when implemented digitally. In this context, the new form design principle "One Thing per Page" (OTPP) could play a important role. Thus, a study of the usability-related potential of the OTPP design principle is highly purposeful.
Objectives
-
Comprehensive revision of form-usability and analysis of the knowledge base regarding the OTPP form design principle according to Caroline Jarrett.
-
Formative (Qualitative) investigation using task-based Think-Aloud to achieve the ideal OTPP form layout.
-
Summative (Quantitative) online study in A/B test design
Problems
-
The forms of the German government often contain numerous questions.
-
Forms are generally perceived as disruptive.
-
The form design is tailored to DIN A4 sheets, even though digitized forms offer much more potential.
Project scope
-
Form-Usability
-
UUX-Research
-
Exploratory sequential mixed design
Duration
-
September 2022 - March 2023
Role
-
Solo Researcher
Tools
Approach
01.
Literature Review
02.
Qualitative study
03.
Quantitative experiment
04.
Results
05.
Interpretation
01. Literature Review
A comprehensive literature review before one's own research work is indispensable, as it allows for a deeper understanding of established methods, capturing the current state of research, and thereby enhancing the quality and relevance of one's own work.
Forms are often perceived as an obstacle because people want to complete the process of data transfer as easily, quickly, and securely as possible.
(Bargas-Avila, et al., 2011)
Especially in the interaction between the government and citizens, the form plays a central role.
(Klischewski, 2006; Axelsson & Ventura, 2007; Sourouni et al. 2008;
Becker et al. 2012)
Meaningful studies indicate that digital forms have a substantial impact on citizens' perception of the quality of eGovernment
(Papadomichelaki &
Mentzas 2012;
Veeramootoo et al. 2018)
.png)
02. Qualitative Study
The qualitative study aimed to transition the sample form, initially consisting of 136 individual questions on separate pages, into a OTPP-designed form through qualitative data collection.
This was achieved through a formative usability test conducted over multiple iteration rounds. The Think-aloud methodology was employed during task execution.
The key findings for the ideal division in terms of an OTPP form were:
Personal and easily retrievable information should be on the same page. This often applied to form fields such as 'Street and House Number,' 'City and Postal Code,' or 'First and Last Name.
whereas information that always needs to be looked up, such as the tax identification number or questions about keywords without direct substantive addition, such as nationality, should be placed independently on a separate page
It became particularly clear that the ideal form layout varies depending on the situation and user preferences. Therefore, a formative usability test is essential for designing an OTPP form. ​
The participants mostly rated the new OTPP layout positively:
It is much more manageable and compact because you have each individual question in view without being overwhelmed by numerous boxes.
The filling process is more relaxed than usual because there is never the feeling of having too much in front of you at once.
Other types of forms with large form sheets are off-putting or even appear stressful, which can then result in a feeling of frustration.
03. Quantitative Experiment
The study compares the OTPP form with the currently used form type in an online experiment. The comparison is based on individual usability metrics, as well as subjectively perceived workload. Data is collected through various performance metrics and specific questionnaires.
Independent variable
In my experiment, a two-stage independent variable was examined, focusing on the differentiation of form layouts. The systematic variation and randomized distribution to participants allowed for an analysis and evaluation of the effects of different design approaches
Dependent variables
1
Effectiveness (DIN EN ISO 9241-11) is the accuracy and completeness with which users achieve a specific goal, and in the study, it is measured using the abandonment as well as the errors rate.
2
Efficiency (DIN EN ISO 9241-11) is the ratio of the result to the resources employed. In the study, it is measured as time efficiency
.png)
3
Satisfaction (DIN EN ISO 9241-11) is the extent of users' physical, cognitive, and emotional reactions, and in the study, it is measured using the USE questionnaire
4
Workload (according to the Cognitive Load Theory by J. Sweller and P. Chandler) is the load on working memory and is measured in the study using the TLX questionnaire
A-Priori power calculation
The statistical power as a function of the number of participants for 2 groups, 4 measurements, a = 0.05, f = 0.25 revealed
n = 44 Persons
04. Results
The results of the multivariate analysis of variance (MANOVA) were significantly.
F(1,64) = 3,77; Wilks λ = .845; partial η² = .056; p = .015 *
Thus, they indicate an influence of the independent variable on the dependent variables.
The results of the post-hoc tests of the analysis of variance of the overall model show that:
-
the OTPP layout maintains the effectiveness and efficiency of other form layouts,
-
the subjective workload remains almost constant with both form types,
-
user satisfaction has been significantly increased with the OTPP layout
The presentation illustrates the results of significantly improved satisfaction based on the Satisfaction module of the USE questionnaire, organized by form type.
OTPP-Layout Previous form layout
Points of USE questionnaire
Beyond the research question, a positive correlation was found between prior experience and the perceived satisfaction with the OTPP form in the test.
Points of USE questionnaire
Previous experience (number of previously completed forms)
Data of the linear regression:
The resulting equation: y = -1.0307 + 0.0323x represents the increase in the satisfaction score (y) with the number of times (x) the form has been filled out.
05. Interpretation
The study has shown that the OTPP principle is a true alternative to the form design principle that has been used so far. The OTPP principle, with its advantage in user satisfaction, is particularly suitable for forms where the satisfaction of the users plays a central role. Examples include customer surveys, market research forms, feedback forms, and, of course, all government forms. As outlined at the beginning, there is potential of influencing the overall satisfaction with the state.
Next Steps
The study has also shown that when processing forms, there can be far more challenges than can be offset by a change in layout. Thus future research in terms of AI-based solutions for automated filling, is crucial. Because that would minimizes the effort instead of just making it more satisfying.