This paper details a usability evaluation of scrolling techniques on Web sites. The scrolling methods evaluated were normal scrolling (with default pagination), infinite scrolling, infinite scrolling with a load more button and infinite scrolling with pagination. The four scrolling types were evaluated in the context of tasks that involved either serendipitous type tasks or goal-oriented type tasks. The evaluation was principally about the ‘raw’ performance and participant perceptions. This is because it was felt that the greatest gap in knowledge concerned these aspects. The evaluation was done by means of an experiment and the data collected was statistically analysed. The results were mixed in nature, where no single scrolling method stood out as being the most usable.
2. Literature review
4. Discussion and conclusions
Navigating a Web site efficiently and pleasurably is an important aspect that all Web designers and businesses should be concerned about. Some research has been carried out on Web navigation and menu structures (e.g., Murano and Sander, 2016; Murano and Khan, 2015; Murano and Lomas, 2015; Murano and Oenga, 2012). However, another aspect of navigation that requires more attention is navigating within a Web page by means of scrolling.
Scrolling is typically performed with a mouse, often with a built-in scroll wheel on a desktop computer and a touch pad on a laptop computer. A finger or a stylus are used for scrolling on mobile devices. With the development of Web page design in the last few years, there have been a number of enhancements in scrolling design.
This study is focused on the evaluation of scrolling techniques on Web sites. Infinite scrolling along with normal scrolling techniques were investigated in this study. Before investigating the common issues found in infinite scrolling, it is important to define infinite scrolling and normal scrolling. According to Loranger (2014), ‘Infinite scrolling is a Web-design technique that loads content continuously as the user scrolls down the page, eliminating the need for pagination.’ ‘Normal’ scrolling is simply a slide movement, which can be done either horizontally or vertically.
In this study we were interested in following up on the discussion in Loranger (2014) and in other online examinations (e.g., Ahuvia, 2013) concerning scrolling types and their suitability for certain kinds of Web pages. However, we were particularly interested in the ‘raw’ performance and participant perceptions. We feel that the greatest gap in knowledge concerned these aspects, while some work has already been done in ascertaining the accessibility of infinite scrolling (See section 2.4 below).
We chose to adopt an experimental approach to evaluate four different scrolling types using a prototype Web site with four versions of scrolling types. The types of scrolling methods we evaluated were normal scrolling with default pagination, infinite scrolling, infinite scrolling with a load more button and infinite scrolling with pagination. The four scrolling types were evaluated in relation to a series of tasks that involved either serendipitous type tasks or goal-oriented type tasks . A serendipitous task tends to be of the kind that is engaged in by users essentially simply having a look around a web site with no specific or detailed aim in mind. However, a goal-oriented task is one where a user has a specific aim in mind, e.g. buying a red silk tie that costs less than a certain amount of money.
The rest of this paper will contain a section reviewing some of the research on scrolling. Then, an empirical experiment will be presented in detail along with the main results of a statistical analysis on data collected from participants. Finally, the paper will conclude with a discussion and analysis.
2. Literature review
This section summarises related recent work supporting this study.
In this research, the main aim is to evaluate the usability of infinite scrolling that is used in some Web sites, in terms of ‘raw’ performance and participant perceptions.
2.1. Overview of scrolling techniques in Web sites
As any Web user knows, scrolling is among the most frequently used interactions in interactive applications such as word processors, spreadsheets and Web applications (Neervoort, 2004). Scrolling is common because the full content of applications is typically larger than a display screen. To see all of the content, users scroll in order to examine details in documents and other digital materials.
Nielsen (2010) found that first-time users of a Web page did not like scrolling. Among the studied sample only 23 percent of users used scrolling while visiting a given Web site .
Scrolling is one of the most common techniques for moving within a Web page. A study conducted by Chartbeat analysed data from two billion visits and found that 66 percent of visits look at information below the ‘fold’, i.e., scrolling (Haile, 2014). A study by Fessenden (2018) using eye tracking technology showed that people use scrolling to move down, especially if the page is designed to encourage scrolling .
Hinckley, et al. (2002) conducted an experiment with 27 people, 12 of whom were male, 22–50 years of age, to evaluate interactive scrolling techniques. They used Fitts’ paradigm as a tool to study scrolling, an IBM ScrollPoint Pro mouse with an isometric joystick (“ScrollPoint”) and the Microsoft IntelliMouse Explorer with a scrolling wheel were used to vary the scrolling distance as well as the required tolerance of scrolling. They concluded that the efficiency of the wheel could be considerably enhanced by means of an acceleration algorithm. Their findings resulted in a practical and rigorous technique for evaluating scrolling methods. In addition, there were other performance and user acceptance aspects. These included device procurement times, visual diversion needed to use a graphical scroll bar, and the incorporation of scrolling into complicated functions, such as navigation and choice of goals with the mouse.
Frederick, et al. (2015) performed a study on the Parallax scrolling technique that created an illusion of depth on a Web page by making background images move slower than foreground images. In addition to parallax scrolling engaging users with a Web site, it was claimed that it improved the user experience. They hypothesised that parallax scrolling positively influenced each of five variables: usability, satisfaction, enjoyment, fun and visual appeal, and subsequently the overall user experience. They performed an experiment with 86 participants, 43 of which performed a test with a parallax Web site and the others with a non-parallax site. The results of the study found the parallax scrolling was significantly more fun. However, there was no significant difference with respect to perceived usability, enjoyment, satisfaction and visual appeal. Two participants suffered motion sickness and experienced significant usability issues while interacting with parallax scrolling. Their study suggested parallax scrolling could be useful in ‘play’.
Lasch and Kujala (2012) investigated the comparative impact on driver distraction when searching music albums using user interface features of a touch-based mobile music player. They designed a driving simulator experiment involving nine male and nine female volunteers between the ages of 21 to 38 years old. As the study focused on the comparison of three scrolling methods: buttons, swipe and kinetic, participants were asked to search for music tracks in a grid-style menu using these methods. Swipe refers to a technique that enables page-by-page scrolling by using a swiping gesture. Kinetic scrolling utilizes the same swiping gesture, but it accelerates the menu, which then stops automatically after deceleration. Tapping on the menu can also stop the movement, whereas the number of music tracks presented in a list-style format varied between three, five and seven items per page. Nine of the participants were told to use the music player in portrait mode and the remainder of the sample were told to use it in landscape mode. The study assumed that the swipe technique should incur fewer distraction effects than kinetic or button techniques due to systematic page by page scrolling and low levels of pointing accuracy needed for browsing. They expected that three items would enable more efficient visual sampling efficiency per page. However, visual demands were increased since more scrolling was required. Screen orientation was assumed to have no distraction effects. The research findings suggested that visual searching was better with a list-style menu structure with a reduced number of items per page than by a grid-style menu with 12 items; this even applied to a list with kinetic scrolling. The swipe scrolling method could be better than the kinetic or small buttons options for in-car menu browsing activities because lower visual precision was needed.
Wherry (2003) performed a study to compare a touchpad scroll ring to a mouse scroll wheel and touchpad scrolling zone using a Fitts’ Law testing approach. Twelve right-handed participants with prior scrolling experience, seven of whom were male, evaluated time, error, and subjective result. The scroll ring achieved the lowest error rates and performance times. Participants also preferred the scroll ring. It was suggested that future studies explore ‘how the touchpad scrolling zone performs when not integrated within the touchpad and how these scrolling devices perform in compound tasks when users are typing or navigating while also scrolling in an application.’ (Wherry, 2003).
2.2. Pagination in Web sites
Pagination is the process of splitting content into discrete pages. Most Web sites use pagination to present a set number of results in Web search results or demonstrating a set number of posts while seeing a discussion string. Pagination can be used as a part of some structure on each Web application to isolate returned information and show it on different pages (Furche, et al., 2012). Pagination additionally incorporates a rationale of planning and showing connections to different pages.
Scrolling, along with pagination, is a technique to view more content which cannot fit on the first page. Pagination enables users to view a part of the result on the current page, and then the user has to click on a ‘next’ button to see a new page with another part of the result. Also, scrolling is a method of traversing a Web page, wherein users either roll the scroll wheel on their mouse, or manually move the scroll bar located on the right side of the browser screen (U.S. Department of Health and Human Services, 2003).
Kim, et al. (2016) conducted a user experiment to investigate the effects of horizontal and vertical control types (pagination versus scrolling) on Web searches with mobile devices. They recruited 24 participants, 14 of whom were male, aged 22–41 years. Despite participants having greater familiarity with vertical scrolling during a search, their findings suggested that ‘... participants using pagination were more likely to find relevant documents, especially those over the fold; spent more time attending to relevant results; and were faster to click while spending less time on the search result pages overall’ (Kim, et al., 2016). They also found that scrolling action affected speed for searching. Considering the limitations regarding users’ individual differences and the lab conditions of their study, the research could not conclusively say that horizontal control was better than vertical scrolling. Rather, they suggested that it is worthwhile for search engines to provide both scrolling types to enhance user experiences on touch-enabled mobile devices.
Thung, et al. (2010) examined and compared a number of design patterns for general features in Web applications focusing on architectural and navigational design patterns. The case study focused on the web site of a school and proposed several designs to improve the site’s usability. They added a pagination pattern that was used to organize search results in a simplified and more structured manner for performance reasons. Pagination was suitable when users needed to view a long list of items that could not fit on one page. In terms of efficiency, the pagination option worked if it only displayed a few results on a page rather than showing all results on a single page. It supported unstructured information, such as Google search results, but it failed to support user preferences for structured information.
2.3. Why is infinite scrolling not easy for every Web site?
Infinite or continuous scrolling works well for content that streams constantly and has a relatively flat structure, where each unit of content belongs at the same level and has similar chances of being interesting to users (Loranger, 2014). However, endless scrolling is not recommended for goal-oriented tasks, such as those requiring users to locate specific content or compare options. Users who need specific types of information expect content to be grouped and layered according to relevance, by pages. Users do not mind clicking links if each click is meaningful and leads them closer to a desired goal (Loranger, 2014). In addition, the way text in which is presented can interact with learners and affect their goals (Sanchez and Wiley, 2009).
Infinite scrolling has been widely used on Web sites with content of equal importance, such as social media (Leeds, 2014). However, on a site where users need to locate desired information quickly, such as in an e-commerce site, infinite scrolling is appropriate because it might overwhelm users with content. When loading seemingly endless content, at some point information needs be removed, otherwise the browser will eventually crash (Karlsson and Larsson, 2016).
Infinite scrolling is a type of Web interaction where data is fetched asynchronously from a data store and inserted into a page as the user consumes information (Leeds, 2014). This method results in an illusion that information on a page is for all appearances infinite.
Since the technique is suited to sites with large sets of data, it is not suitable for every design. Infinite scrolling has its advantages as well as its shortcomings. Some of the advantages associated with infinite scrolling are that it is suited for touch devices and better for visual content as it captures the interest of users and keeps them captivated (Akin, 2018; Andrew, 2018; Babich, 2016; Schenker, 2014). According to Ahuvia (2013), while viewing Google searches, only six percent advanced to the next page. This suggests that infinite scrolling had better content exposure. Further, it requires fewer clicks from the user’s side and novice users can use a site seamlessly as content is automatically updated.
However, infinite scrolling has also some negative aspects. Users need to stay focused on their desired information (Akin, 2018; Andrew, 2018; Karlsson and Larsson, 2016; Leeds, 2014; Loranger, 2014; Schenker, 2014). In returning back from a link in the page, the user’s position is lost. Also, when returning to specific content, users cannot skip previous content. Users cannot reach footer content as the page keeps loading automatically. With infinite scrolling, search engine optimization (SEO) is affected and Web developers have to consider special measures.
Karlsson and Larsson (2016) conducted a quantitative study on how user behaviour differed between a Web site with pagination and infinite scrolling. The study indicated that infinite scrolling could be useful on an intranet when adapted according to usability guidelines. The researchers evaluated user behaviour on an intranet of a health care company regarding which pagination technique to choose (pagination or infinite scrolling). Three versions of a prototype Web site were evaluated simultaneously; an infinite scrolling site with a load more button, a version with pagination and a specially designed version with infinite scrolling adapted to the specific intranet. They designed an evaluation task that made the site partly goal-driven, by finding specific information, and another part not so goal-driven when scrolling through the feed. Finally, the following conclusions and recommendations were drawn from their study:
- Implement the back button functionality together with automatic loading;
- Infinite scrolling can be valuable on a goal-driven site. Infinite scrolling and the paginated version performed similarly. The only area where differences were seen were between the number of articles loaded per session;
- Use infinite scrolling, and implement support to retrieve the user’s position when returning to the feed.
Etsy has used infinite scrolling features on their Web site, at https://www.etsy.com/. However, they observed the feature had certain undesirable aspects. Some of these were fewer clicks on result items and less items being classed as ‘favourite’. Nguyen (2013) conducted a study looking into the reasons for the failure of infinite scrolling, presenting two assumptions behind the use of infinite scrolling. Users wanted more results per page as well as faster results.
Both assumptions were tested with particular users but their engagement level was not statistically significant. Nguyen did not know why infinite scrolling did not succeed for Etsy. There was not, as far as they could tell, a technical fault, such as infinite scrolling breaking on a specific browser. The most annoying issue for infinite scrolling was a lack of a footer, which probably the average Etsy user does not need when making purchasing or viewing decisions (Nguyen, 2013).
Holst (2016) conducted a usability study which tested three design patterns against usability factors in the context of loading products in an e-commerce Web site. The design patterns used on desktop and mobile devices were infinite scrolling, pagination and load more options. The results suggested that load more buttons combined with lazy loading were superior than other methods that provided a seamless user experience. Infinite scrolling was not favorable for mobile in terms of usability for search results. Infinite scrolling was ideal for quickly showing the breadth of an entire category. However, because users were not naturally halted when scrolling, they tended to scan more and focus less on individual products on a list. The study concluded that no single method was perfect for all instances.
2.4. Infinite scrolling and accessibility
Web sites with infinite scrolling can have accessibility issues. A number of authors (Lembree, 2015; Langmo, 2017; Leeds, 2014; Loranger, 2014; Parker, et al., 2017; Peri, 2018; Roselli, 2014) have found a series of accessibility problems with infinite scrolling. Some of these are:
- With infinite scrolling, it is not possible to skip chunks of irrelevant content;
- Focus is lost when navigating back. Keyboard-only users, including screen reader users, are forced to re-navigate through all of the content to locate an element activated when the focus is lost. The user has to reorient themselves on the page;
- With infinite scrolling, finding content that one viewed previously can be very difficult once a page is more than a few screens long;
- Footer content is inaccessible. With infinite scrolling, assistive technology users have an issue in navigating to the footer. The footer usually contains copyright and contact information;
- Quick navigation is difficult to achieve. If the Web site has a lot of headings and links within each feed, navigation becomes very difficult when it comes to loading feeds infinitely even though the site has proper heading elements and structure. Many contain news feeds with additional links, forcing users to navigate through the active elements for each feed;
- Hard to locate the content. Sighted keyboard-only users and screen reader users may have difficulty locating content using search feature or navigation techniques;
- With infinite scrolling it is difficult to see updates to the URL bar as additional pages are loaded;
- Not able to go to the end of the page. Keyboard-only users are not able to use Ctrl+End keyboard command to navigate to the end of the page.
Although the use of infinite scrolling seems to be on the increase, there is a lack of a larger body of evidence concerning scrolling types and their usability and accessibility. There appears to be some consensus that infinite scrolling is not the best option in many circumstances. This research aims to build on the current research on scrolling types. To that end, the next section will detail an empirical experiment which evaluated four scrolling types with real users.
Four identical prototype Web sites were developed. Each was implemented with one of four scrolling methods: normal scrolling with default pagination; infinite scrolling; infinite scrolling with a load more button; and infinite scrolling with pagination.
Each site had two pages, home and shop pages. The home page had news and blog information in different categories while the shop page had different kinds of e-commerce products and electronic gadgets. A screenshot of each scrolling method used in the prototype is shown below.
Figure 1: Screenshots of each scrolling method. Note: Larger version of Figure 1 available here.
A total of 16 participants, 12 males and four females, were included in this experiment. The participants were university students pursuing bachelor’s or Master’s degrees; some were employed in software development, design and quality assurance roles. Most of the participants were 26–35 years old with the other age ranges represented in a minor way. To make the experiment more efficient, a choice of locations were made available. These were either at a university project room or the participant’s own home. The experiment date and time were set up in advance by means of social media, text message and/or telephone call. Only the experimenter and participant were present in the quiet and private experiment location.
The experiment was carried out ethically, in line with Norwegian research expectations. All participants gave informed consent and no identifiable data was collected from them. No participant was exposed to any dangerous or unpleasant circumstances. All participants had the opportunity to ask questions related to their participation and the research and all had the option of backing out of the experiment at any time.
3.2. Experimental design
A within users experimental design was used in this experiment. The main reason this approach was chosen was so that each participant could make subjective preference comparisons on all four scrolling methods used.
The independent variables were four types of scrolling methods and specific tasks (see section 3.5 below). The four scrolling methods consisted of normal scrolling with default pagination; infinite scrolling; infinite scrolling with a load more button; and infinite scrolling with pagination. The tasks involved shopping activities on prototype Web pages.
The defined dependent variables were participant performance and subjective opinions. The dependent measures were task completion time and number of errors.
Errors were categorized in two ways. The first consisted of a user clicking on something that they thought was a link when it was not a link. The second involved users being asked to not click on the search box. If they did click on the search box, this was categorized as an error. All the errors were combined into a single score used in statistical analysis.
Participants’ opinions regarding the four scrolling types in relation to task types (see Task design section for a detailed description of tasks) were obtained by using a post-experiment questionnaire. A Likert (1932) type scale ranging from one to five was used for all questions, where for all questions a score of five was the highest possible positive score.
3.4. Apparatus and materials
For this experiment the following systems and materials were used to conduct the experiment:
- MacBook Pro with Mac OS Mojave, Intel Core i5 processor, 4GB RAM and 13” screen;
- HP laptop with Windows 10 OS, 4GB RAM, Intel Core i3 processor and 17” screen;
- A stopwatch;
- Chrome Web browser (Firefox/Safari depending on participant’s choice);
- Chrome browser extension screen recorder;
- Consent form;
- Pre-experiment questionnaire;
- Tasks document for the experiment;
- Post experiment questionnaire.
3.5. Task design
Four task groups were designed for this experiment. All the tasks were related to serendipitous discovery, exploration and goal-driven finding tasks. The tasks were designed around the concept found in the study by Loranger (2014), where users’ activities tend to be either time-killing with serendipitous discovery, goal-driven finding or both. A serendipitous task tends to be of the kind that is engaged in by users simply having a look around a Web site with no specific or detailed aim in mind. However, a goal-oriented task is one where a user has a specific aim in mind, finding some specific piece of information on some topic. With these explanations in mind, four task groups with four tasks were designed. These are detailed in Table 1:
Table 1: Specific tasks. Tasks Number Task type Description Task group A 1 Serendipitous Go to home page, discover sports news that you find really interesting or informative. 2 Serendipitous Go to shop page, explore an electronics product which is latest or fascinating. 3 Goal-driven Go to home page, find music news with title “10 long-awaited albums that will be the soundtrack of 2019”. 4 Goal-driven Go to shop page, Find a book with title “The climb: Tragic ambitions on Everest” and price “200kr”. Task group B 1 Serendipitous Go to home page, discover cars related news that you find really interesting or informative. 2 Serendipitous Go to shop page, explore shoes product which is latest or fascinating in shop page. 3 Goal-driven Go to Home page, find health news with title “Two compounds in coffee may team up to fight Parkinson’s”. 4 Goal-driven Go to shop page, find a kitchenware product with Title “Richardson 5pcs knife block magnet” which is on “sale”. Task group C 1 Serendipitous Go to home page, discover music news that you find really interesting. 2 Serendipitous Go to shop page, explore a book which is interesting or informative. 3 Goal-driven Go to home page, find sports news with title “Women-only motorsport series launched to find potential F-1 Stars”. 4 Goal-driven Go to shop page, find an electronics product with title “JBL Xtreme2” and price “1200kr”. Task group D 1 Serendipitous Go to home page, discover health news that you find really beneficial or informative. 2 Serendipitous Go to shop page, explore a kitchenware product which is interesting and useful in shop page. 3 Goal-driven Go to home page, find car-related news with title “Porsche Taycan Electric Car Teased, to debut in September”. 4 Goal-driven Go to shop page, Find shoes with Title “JORDAN 1 MID RETRO BASKETBALL SHOES” with price “700kr”.
In order to minimise learning effects, tasks were administered to participants in a ‘randomised’ manner as described in Table 2:
Table 2: Task randomisation procedure. Participant Experiment part 1 Experiment part 2 Experiment part 3 Experiment part 4 Web site Task group Web site Task group Web site Task group Web site Task group 1 W1 A W2 B W3 C W4 D 2 W1 B W2 C W3 D W4 A 3 W1 C W2 D W3 A W4 B 4 W1 D W2 A W3 B W4 C 5 W2 A W3 B W4 C W3 D 6 W2 B W3 C W4 D W3 A 7 W2 C W3 D W4 A W3 B 8 W2 D W3 A W4 B W3 C 9 W3 A W4 B W1 C W2 D 10 W3 B W4 C W1 D W2 A 11 W3 C W4 D W1 A W2 B 12 W3 D W4 A W1 B W2 C 13 W4 A W1 B W2 C W1 D 14 W4 B W1 C W2 D W1 A 15 W4 C W1 D W2 A W1 B 16 W4 D W1 A W2 B W1 C W1: Prototype Web site 1 with “Normal scrolling”
W2: Prototype Web site 2 with “Infinite scrolling”
W3: Prototype Web site 3 with “Infinite scrolling with load more”
W4: Prototype Web site 4 with “Infinite scrolling with pagination”
Task group (A, B, C, D): Tasks (2 serendipitous discovery tasks 2 goal-driven finding tasks)
The experiment was carried out in a series of systematic stages. Before the start of the actual experiment with selected participants, pilot testing was conducted with two different participants. The goal of the pilot study was to identify possible problems in advance of the actual experiment. After completion of the pilot experiment, the following results were obtained and changes were made in the actual experiment:
- Participants took longer than expected. There were some difficulties in using scrolling techniques at the beginning. To overcome this issue, a training component was added before the start of the real experiment.
- Participants appeared to learn from task experiences and performed better in the last two tasks. To minimize this issue, randomization was used in the real experiment (see Table 2).
The experiment was divided into four sections. Participants were initially greeted and brief information about the experimenter was provided, consisting of personal name and course of study. The experiment then was conducted in four stages comprising of the pre-experiment, training, actual experiment and post-experiment.
The pre-experiment stage involved participants being seated and given a consent form. They were asked to read it carefully and, if in agreement, sign it. The consent form had details about the research study, purpose of the study, experimental procedures and confidentiality information. Then participants were given a pre-experiment questionnaire. The questionnaire gathered background information about participants and also information regarding their experiences with computers, Web sites and different scrolling techniques use.
The training stage provided participants with a short session about different scrolling techniques used in the prototype Web sites. This session lasted between three and seven minutes, where basic functionality of scrolling techniques was covered. Then some trial tasks were given to each participant.
The actual experiment stage was carried out next with the prototype Web sites and tasks as shown noted in Table 1. A screen recorder was used to record screen interaction and a stopwatch was utilized to time tasks. If a participant asked any questions regarding issues while performing tasks, the experimenter provided some hints to complete tasks. Those hints were the same for every participant so as to reduce potential bias.
The post-experiment stage involved completing a post-experiment questionnaire which dealt with aspects of participant preferences and opinions about the four scrolling techniques, task design and Web site design. Some of the areas covered in the questionnaire were aspects of simplicity and ease of use, pleasantness, responsiveness, feelings of confidence in use and recommendations. At the end of the experiment, the participants were thanked for their participation.
The data collected for performance and participant opinions were analysed by means of a one-way repeated measure ANOVA using SPSS (2019). As discussed earlier, there were two kinds of tasks in this experiment. These were time-killing activities with serendipitous discovery and goal-driven finding tasks. Therefore, the analyses were done based on each type of task individually. Data collected met all the required assumptions (Mayers, 2013) for a one-way repeated measure ANOVA and was based on N = 16.
Task time and errors were measured for all tasks. Tasks were either serendipitous or goal-driven in nature. No statistically significant results were observed. Therefore, they will not be discussed further in this paper.
The post-experiment questionnaire asked several questions using a Likert-type scale for the responses. The questions used a Likert-type scale ranging from one to five. For each question five was the most positive score one could allocate. For the serendipitous tasks, in relation to the four scrolling types, the questions asked whether: it was simple and easy to use the scrolling techniques; the scrolling techniques were very responsive; the scrolling techniques were pleasant to use; and confidence was felt whilst using the scrolling techniques. For these four questions no statistically significant results were observed.
The fifth question concerned whether participants felt that on completion of the serendipitous tasks and using the four scrolling techniques they would recommend use of the technique in a Web site. This question yielded a statistically significantly different result across the four scrolling types: infinite scrolling Web site (Mean = 4.44; SD = 0.512), Infinite load more scrolling (Mean = 3.94; SD = 0.443), normal scrolling (Mean = 3.62; SD = 0.500) and infinite pagination scrolling (Mean = 3.50; SD = 0.966), F (3, 45) = 7.336, p < 0.001, ηp2 = 0.328.
A post hoc Bonferroni analysis indicated which pairs of scrolling techniques were significantly different from each other. It was found that the pairs of normal scrolling and infinite scrolling were more significant (p = 0.001) followed by infinite scrolling and infinite scrolling with pagination (p = 0.003), and infinite scrolling and infinite scrolling with load more (p = 0.039). Overall the infinite scrolling technique scored highest.
The same questions and scales described above were also asked in relation to the goal-driven tasks that participants carried out. The first question asked whether it was simple and easy to use the scrolling techniques. This question yielded a statistically significantly different result across the four scrolling types: normal scrolling (Mean = 4.19; SD = 0.655), infinite load more scrolling (Mean = 4.06; SD = 0.574), infinite pagination scrolling (Mean = 3.56; SD = 0.892) and infinite scrolling (Mean = 3.44; SD = 0.512), F (3, 45) = 5.909, p = 0.002, ηp2 = 0.283.
A post hoc Bonferroni analysis indicated which pairs of scrolling techniques were significantly different from each other. There was one significant difference between the pairs, i.e., normal scrolling and Infinite scrolling (p = 0.009). Overall the normal scrolling technique was scored highest by participants for ease and simplicity of use.
The second question asked whether the scrolling techniques were responsive. This question yielded a statistically significantly different result across the four scrolling types: normal scrolling (Mean = 4.13; SD = 0.500), infinite load more scrolling (Mean = 3.75; SD = 0.577), infinite pagination scrolling (Mean = 3.69; SD = 0.479) and infinite scrolling (Mean = 3.13; SD = 0.619), F (3, 45) = 8.408, p = 0.000, ηp2 = 0.359.
A post hoc Bonferroni analysis indicated which pairs of scrolling techniques were significantly different from each other. There was one significant difference between the pairs, i.e., normal scrolling and infinite scrolling (p < 0.001). Overall the normal scrolling technique was scored highest by participants for responsiveness.
The third question asked whether the scrolling techniques were pleasant to use. This question yielded a statistically significantly different result across the four scrolling types: infinite load more scrolling (Mean = 4.06; SD = 0.574), infinite pagination scrolling (Mean = 3.88; SD = 0.619), Normal scrolling (Mean = 3.56; SD = 0.892) and infinite scrolling (Mean = 3.38; SD = 0.500), F (3, 45) = 4.041, p = 0.013, ηp2 = 0.212.
A post hoc Bonferroni analysis indicated which pairs of scrolling techniques were significantly different from each other. Two pairs were significantly different, i.e., infinite scrolling and infinite load more scrolling (p < 0.001) and infinite scrolling and infinite pagination scrolling (p = 0.009). Overall the infinite load more scrolling type was scored highest by participants for being pleasant to use.
The fourth question asked whether the participants felt very confident in using the four scrolling techniques. This question yielded a statistically significantly different result across the four scrolling types: infinite pagination scrolling (Mean = 4.31; SD = 0.602), normal scrolling (Mean = 4.13; SD = 0.619), infinite load more scrolling (Mean = 3.69; SD = 0.946) and infinite scrolling (Mean = 3.38; SD = 0.806), F (3, 45) = 5.000, p = 0.004, ηp2 = 0.250.
A post hoc Bonferroni analysis indicated which pairs of scrolling techniques were significantly different from each other. There was one significant difference between the pairs, i.e., infinite pagination scrolling and infinite scrolling (p = 0.011). Overall the infinite pagination scrolling was scored highest by participants for eliciting feelings of confidence whilst scrolling.
The fifth question asked participants to score their recommendations for each scrolling type. This question yielded a statistically significantly different result across the four scrolling types: Infinite load more scrolling Web site (Mean = 4.25; SD = 0.775), normal scrolling (Mean =4.19; SD = 0.574), infinite pagination scrolling (Mean = 3.88; SD = 0.719) and infinite scrolling (Mean = 3.06; SD = 0.574), F (3, 45) = 9.899, p = 0.000, ηp2 = 0.398.
A post hoc Bonferroni analysis indicated which pairs of scrolling techniques were significantly different from each other. It was found that the pairs of infinite scrolling and infinite load more scrolling (p = 0.000001) were more significant. These were followed by normal scrolling and infinite scrolling (p = 0.000315) and infinite scrolling and infinite pagination scrolling (p = 0.017). Overall the infinite load more scrolling type was scored highest by participants in terms of their recommendations.
The participants were also asked some general questions concerning the experiment and the prototypes used in the experiment. The questions used a Likert-type scale ranging from one to five. The questions aimed to elicit attitudes concerning whether the tasks were understandable, interesting and involving practicality. Further, the questions asked about the functionality of the prototype Web site design and personal satisfaction over the design and content of prototypes.
For all questions, participants scored these either as four or five, indicating an overall unanimous series of very positive opinions concerning the experiment and prototypes used in the experiment.
4. Discussion and conclusions
This study aimed to add to the current knowledge about different scrolling types. Namely, this study investigated normal scrolling with default pagination, infinite scrolling, infinite scrolling with a load more button and infinite scrolling with pagination. The study was closely linked with the work of Loranger (2014) and the use of tasks that were either serendipitous or goal-driven in nature.
Our work suggests that when it comes to performance in using the different scrolling methods tested, task times and errors were not statistically significantly different from each other. This was found for tasks that were either serendipitous or goal-driven in nature.
These results suggest that it may not matter which scrolling method might be used. However, it is a strong part of usability engineering to be concerned with how a user experiences and perceives an interface or some element of it.
This research primarily dealt with user perceptions by means of a post-experiment questionnaire. The first four questions asked participants to rank their opinions on whether: it was simple and easy to use the scrolling techniques; scrolling techniques were very responsive; scrolling techniques were pleasant to use; and confidence was felt whilst using the scrolling techniques. For these four questions, in relation to the serendipitous tasks, no statistically significant results were obtained, suggesting that perceptions about each scrolling method and these four topics were not strongly affected. This series of results could be expected as the whole approach to a serendipitous task is that it involves something that is potentially a ‘time-killing’ activity.
The fifth question concerned participants ranking which scrolling method they would recommend. This question had the infinite scrolling method ranked the highest. This result is within normal expectations, given that the tasks were serendipitous in nature.
The same questions were then asked in relation to goal-driven tasks. For questions one (ease and simplicity) and two (responsiveness) the normal scrolling method was ranked highest. For question three (pleasantness) the infinite load more scrolling method was ranked highest. For question four (confidence in use) the infinite pagination scrolling method was ranked highest. For question five (recommendation) participants ranked the infinite load more scrolling method highest.
Overall, the picture concerning which scrolling method a designer should use is not categorically clear in terms of ‘raw’ performance and participant perceptions. Previous work seem to overall suggest that for ease of use and accessibility, the normal scrolling method is better than infinite scrolling. This may be the case, but further work needs to be done to achieve more categorical results for ‘raw’ performance and participant perceptions. We did not achieve overall categorical results pointing to the normal scrolling method as being the way forward for ‘raw’ performance and participant perceptions. However, we did not specifically test in the experiment for the negative accessibility issues found by other authors and presented earlier, as these seemed already clear and unequivocal.
In future work we would like to have a larger participant sample. Future work should also address the possibility that the prototypes and subsequent tasks used perhaps were too simple, thus potentially making it more difficult to achieve categorical results. Although we did try to have a prototype and tasks that had validity, they could have been improved by having more complex information and in turn more complex tasks that could have highlighted differences. Also, serendipitous tasks had to be controlled for being valid in an experiment and this in turn could have affected their realism. A final thought is the questionnaire itself. This could have been perhaps followed up with a semi-structured interview of the participants to get more in-depth information about their perceptions on what they had experienced.
Until more work is done in the context of ‘raw’ performance and participant perceptions, normal scrolling is still recommended for designers as being the more usable option. It still appears to be the more accessible option when compared to infinite scrolling. If a different scrolling method such as infinite scrolling is used the designers should ensure that best practice for accessibility is followed.
Given the current body of knowledge, infinite scrolling appears to create accessibility problems. Any interaction that increases the time or number of steps required to accomplish tasks should be avoided. An interaction that makes it more difficult for certain kinds of users, such as users using a screen reader, should always be avoided. Lastly, navigation must always be designed to be efficient, pleasant to use and usable by all users, irrespective of their skills or personal requirements.
About the authors
Sushil Sharma is a Master of Universal Design of ICT and a former Master student at Oslo Metropolitan University, Norway.
Dr. Pietro Murano is an Associate Professor of Computer Science at Oslo Metropolitan University, Norway. He is a Computer Scientist specializing in interaction, HCI, Usability and Universal Design of computer systems.
E-mail: Pietro [dot] Murano [at] oslomet [dot] no
1. In this paper we will be using the same kind of expressions as found in Loranger (2014) referring to tasks types, i.e., serendipitous and goal-oriented.
2. See also the follow-up study by Fessenden (2018).
3. See also Nielsen (2010).
Y. Ahuvia, 2013. “Infinite scrolling: Let’s get to the bottom of this,” Smashing Magazine (3 May), at https://www.smashingmagazine.com/2013/05/infinite-scrolling-lets-get-to-the-bottom-of-this, accessed 18 February 2020.
F.K. Akin, 2018. “Stop building Web sites with infinite scroll!” LogRocket (18 October), at https://logrocket.com/blog/infinite-scroll/, accessed 18 February 2020.
P. Andrew, 2018. “Infinite scrolling: Is it helping or hurting your business?” speckyboy (5 March), at https://speckyboy.com/infinite-scrolling/, accessed 18 February 2020.
N. Babich, 2016. “UX: Infinite scrolling vs. pagination,” UX Planet (1 May), at https://uxplanet.org/ux-infinite-scrolling-vs-pagination-1030d29376f1, accessed 18 February 2020.
D. Lembree, 2015. “Infinite scrolling and accessibility (It’s usually bad),” Web Axe (25 August), at http://www.webaxe.org/infinite-scrolling-and-accessibility/, accessed 18 February 2020.
T. Fessenden, 2018. “Scrolling and attention,” Nielsen Norman Group (15 April), at https://www.nngroup.com/articles/scrolling-and-attention/, accessed 18 February 2020.
D. Frederick, J. Mohler, M. Vorvoreanu and R. Glotzbach, 2015. “The effects of parallax scrolling on user experience in Web design,” Journal of Usability Studies, volume 10, number 2, pp. 87–95, and at https://uxpajournal.org/the-effects-of-parallax-scrolling-on-user-experience-in-web-design/, accessed 18 February 2020.
T. Furche, G. Grasso, A. Kravchenko and C. Schallhart, 2012. “Turn the page: Automated traversal of paginated Web sites,” In: M. Brambilla, T. Tokuda and R. Tolksdorf (editors). Web engineering>. Lecture Notes in Computer Science, volume 7387. Berlin: Springer, pp. 332–346.
doi: https://doi.org/10.1007/978-3-642-31753-8_27, accessed 18 February 2020.
T. Haile, 2014. “What you think you know about the Web is wrong,” Time (9 March), at https://time.com/12933/what-you-think-you-know-about-the-web-is-wrong/, accessed 18 February 2020.
K. Hinckley, E. Cutrell, S. Bathiche and T. Muss, 2002. “Quantitative analysis of scrolling techniques,” CHI ’02: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 65–72.
doi: https://doi.org/10.1145/503376.503389, accessed 18 February 2020.
C. Holst, 2016. “Infinite scrolling, pagination or ‘load more’ buttons? Usability findings in ecommerce,” Smashing Magazine (1 March), at https://www.smashingmagazine.com/2016/03/pagination-infinite-scrolling-load-more-buttons/, accessed 18 February 2020.
Infinite Scroll, 2019. “Infinite scroll — Automatically add next page,” at https://infinite-scroll.com/, accessed 18 February 2020.
J. Karlsson and M. Larsson, 2016. “Adapting infinite-scroll with the user experience in mind,” thesis, Linköping University (Sweden), at http://liu.diva-portal.org/, accessed 18 February 2020.
J. Kim, P. Thomas, R. Sankaranarayana, T. Gedeon and H.-J. Yoon, 2016. “Pagination versus scrolling in mobile Web search,” >CIKM ’16: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, pp. 751–760.
doi: https://doi.org/10.1145/2983323.2983720, accessed 18 February 2020.
O. Langmo, 2017. “Accessibility problems with infinite scrolling,” Accessible360 (11 May), at https://accessible360.com/accessible360-blog/accessibility-problems-infinite-scrolling/, accessed 18 February 2020.
A. Lasch and T. Kujala, 2012. “Designing browsing for in-car music player: Effects of touch screen scrolling techniques, items per page and screen orientation on driver distraction,” AutomotiveUI ’12: Proceedings of the Fourth International Conference on Automotive User Interfaces and Interactive Vehicular Applications, pp. 41–48.
doi: https://doi.org/10.1145/2390256.2390262, accessed 18 February 2020.
C. Leeds, 2014. “The UX of infinite scroll: The good, the bad, and the maybe,” SitePoint (4 November), at https://www.sitepoint.com/ux-infinite-scroll-good-bad-maybe/, accessed 18 February 2020.
D. Lembree, 2015. “Infinite scrolling and accessibility (It’s usually bad),” Web Axe (25 August), at http://www.webaxe.org/infinite-scrolling-and-accessibility/, accessed 18 February 2020.
R. Likert, 1932. A technique for the measurement of attitudes. Archives of psychology, number 140. New York: Science Press.
H. Loranger, 2014. “Infinite scrolling is not for every Web site,” Nielsen Norman Group (2 February), http://www.nngroup.com/articles/infinite-scrolling, accessed 18 February 2020.
A. Mayers, 2013. Introduction to statistics and SPSS in psychology. Harlow: Pearson.
P. Murano and M. Sander, 2016. “User interface menu design performance and user preferences: A review and ways forward,” International Journal of Advanced Computer Science and Applications, volume 7, number 4.
doi: https://doi.org/10.14569/IJACSA.2016.070447, accessed 18 February 2020.
P. Murano an d I.N. Khan, 2015. “Pie menus or linear menus, which is better?” Journal of Emerging Trends in Computing and Information Sciences, volume 6, number 9, pp. 476–481; version at http://pietromurano.org/Papers/Murano-Khan-Published-Version.pdf, accessed 18 February 2020.
P. Murano and T.J. Lomas, 2015. “Menu positioning on Web pages. Does it matter?” International Journal of Advanced Computer Science and Applications, volume 6, number 4.
doi: https://doi.org/10.14569/IJACSA.2015.060419, accessed 18 February 2020.
P. Murano and K.K. Oenga, 2012. “The impact on effectiveness and user satisfaction of menu positioning on Web pages,” International Journal of Advanced Computer Science and Applications, volume 3, number 9.
doi: https://doi.org/10.14569/IJACSA.2012.030931, accessed 18 February 2020.
P.C. Neervoort, 2004. “Method of scrolling through a document,” Google Patents, at https://patents.google.com/patent/US7689934B2/en, accessed 18 February 2020.
D. Nguyen, 2013. “Why did infinite scroll fail at Etsy?” danwin.com (6 January), at http://danwin.com/2013/01/infinite-scroll-fail-etsy/, accessed 18 February 2020.
J. Nielsen, 2010. “Scrolling and attention (Original research study),” Nielsen Norman Group (21 March), at https://www.nngroup.com/articles/scrolling-and-attention-original-research/, accessed 18 February 2020.
S. Parker, S. Odio and A. Mosseri, 2017. “Infinite scrolling,” Google Patents, at https://patents.google.com/patent/US20120011430, accessed 18 February 2020.
R.S. Peri, 2018. “Infinite scroll & accessibility! Is it any good?” (25 April), at https://www.digitala11y.com/infinite-scroll-accessibility-is-it-any-good/, accessed 18 February 2020.
A. Roselli, 2014. “So you think you’ve built a good infinite scroll” (31 May), at http://blog.adrianroselli.com/2014/05/so-you-think-you-built-good-infinite.html, accessed 18 February 2020.
C.A. Sanchez and J. Wiley, 2009. “To scroll or not to scroll: Scrolling, working memory capacity, and comprehending complex texts,” Human Factors, volume 51, number 5, pp. 730–738.
doi: https://doi.org/10.1177/0018720809352788, accessed 18 February 2020.
M. Schenker, 2014. “The advantages of an infinite scrolling Web site” (3 February), at https://www.webdesign.org/the-advantages-of-an-infinite-scrolling-website.22396.html, accessed 18 February 2020.
SPSS, 2019. “IBM SPSS statistics,” at https://www.ibm.com/no-en/marketplace/spss-statistics, accessed 18 February 2020.
P.L. Thung, C.J. Ng, S.J. Thung and S. Sulaiman, 2010. “Improving a Web application using design patterns: A case study,” 2010 International Symposium on Information Technology.
doi: https://doi.org/10.1109/ITSIM.2010.5561301, accessed 18 February 2020.
U.S. Department of Health and Human Services, 2003. “Research-based Web design and usability guidelines,” at https://www.usability.gov/sites/default/files/documents/guidelines_book.pdf, accessed 18 February 2020.
E. Wherry, 2003. “Scroll ring performance evaluation,” CHI EA ’03: CHI ’03 Extended Abstracts on Human Factors in Computing Systems, pp. 758–759.
doi: https://doi.org/10.1145/765891.765973, accessed 18 February 2020.
Received 14 October 2019; revised 16 December 2019; accepted 20 January 2020.
“A usability evaluation of Web user interface scrolling types” by Sushil Sharma and Dr. Pietro Murano is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
A usability evaluation of Web user interface scrolling types
by Sushil Sharma and Pietro Murano.
First Monday, Volume 25, Number 3 - 2 March 2020