1. Heuristics for developing and evaluating smartphone mobile websites
Vasileios Xanthopoulos, MSc
York University
ABSTRACT
Websites for smartphone use require different design and development approaches to desktop websites, taking into account the different physical designs, functionalities, and contexts of use, as well as the mental load of working with each platform. This study investigated usability problems with 7 smartphone websites via both iPhone and Android smartphones. 24 participants undertook tasks using a retrospective think aloud protocol. The usability problems identified were analyzed using a grounded theory approach where they were iteratively categorized with similar problems and factoring for frequency of occurrences and mean severity resulting in a final list of 4 categories and 16 problem subcategories. This categorization of problems was transformed into a set of 16 heuristics for the development and evaluation of websites for smartphone use. Comparing the mobile heuristics with well-established web heuristics showed high overlap but with a specialized view concerning the mobile web. The use of the new heuristics increase the usability and user experience of smartphone websites and help create a more trustworthy, profitable and hospitable mobile web.
Author Keywords
Heuristics; mobile usability; mobile phone; mobile web heuristics; smartphone;
INTRODUCTION
The mobile web has been an everyday commodity of the past few years with 25% and rising, of global web access being mobile [1]. Usability has surprisingly fallen behind causing user frustration and confusion. The purpose of this study is to develop heuristics for the design and development of websites viewed on smartphones.
Smartphones and mobile web access have transformed every aspect of human life. It was not always like this, I am fortunate enough to remember the phones with the round dial and the feedback sound they made when the dial returned to its initial position. I hated having to call my father from that phone because, back then, mobile phone numbers in Greece started with 0. 0 made the round dial go all the way around which took more time because I had to wait for the dial to return back to its initial position to dial the next number which made you think ‘how badly do I want to talk to this person to have to endure this?’. Another issue with land line phones was the fact that they were stationary. If you happened to live in a big house you had to run to answer that
phone before the caller got bored and just hung up rendering your sprint effort moot.
Nowadays mobile phones have reached 7billion subscribers worldwide with 30% of those being smartphones [1]. 30% of 7 billion people have the ability to access the web on their smartphones. Viewing websites on our mobile phones has been a much different experience than accessing the web on desktops/laptops, with this experience often being a tedious one because designers and developers, without taking into account the differences of mobile phones such as the smaller screen size, input and output functions and context of use, they adopted the same guidelines for designing and developing websites meant for conventional web to the mobile web. Why they do that? One reason could be that there are no mobile web heuristics to help them achieve in building a usable website for mobile use. The results of adopting heuristics for conventional web access can be found in everyday interaction with the mobile web. Websites look different, they are difficult to use and navigate resulting in poor user experience and task performance.
The approach followed to resolving these problems was, firstly, understanding the need for developing new heuristics by examining the differences between conventional and mobile web access that would provide the foundation for conducting a usability study to discover usability problems that would eventually contribute to the development of heuristics that would provide ‘rules of thumb’ that would help mobile web access a usable and seamless experience.
Sections to follow are review of the relevant literature, method section where the process and the equipment and materials used to conduct this study, results section, discussion section where the results will be discussed as well as comparisons with well-established heuristics, limitations and further work in the field and conclusions will be drawn in the conclusion section.
REVIEW OF RELEVANT LITERATURE
Mobile web access being in its infancy has a lot of obstacles to overcome and research has focused on the differences in accessing the web on a mobile phone rather than from a desktop pc or laptop (conventional web access). The differences can be identified by just looking at the two devices. Their physical design is one important factor to be considered, the contexts of use are, theoretically, countless for the mobile phone so accessing the web while on the move

2. is an important feature of mobile phones if not the most important but how good are humans in multitasking i.e walking and browsing, listening to the announcements on a train to avoid missing your stop while browsing those are just some of the questions that someone could ask even if he does not have any special kind of knowledge about mobile web access. These are reasonable questions that need to be answered and that is why this literature review is structured in a sequence to answer these questions as they were asked.
Small screen size
Screen size is the most obvious difference between accessing the web from a mobile phone and accessing the web using conventional methods.
[2] investigated the effect of small screen size and results showed that users interacted in a much higher level when reading from small screen displays rather than those of larger displays because they had to page forward and backwards much more in order to have a view of the text than those on conventional displays on desktop pcs. Based on that fact, he predicted that users of mobile web will make use of scrolling and paging much more than conventional display users.
[3] conducted an experiment to measure the effects of screen size on usability and perceived usability using 3 devices with representative screen sizes on which users would interact with an application. Results showed that perceived usability and effectiveness were not affected by the screen size but efficiency was significantly affected. [4] conducted a study to examine the impact of screen size on users while they try to achieve certain goals on mobile devices. Results showed that the group with the large display answered twice as many questions than the one with the small size display, also screen size affects task performance, small screen users used the search facilities twice as many time then large screen users and that users of small screen performed a lot more navigation actions such as scrolling or paging than their conventional counterparts, interestingly most of the scrolling was down and right. [2-4] showed not only the severe effects of screen size and but also, [4] promoted the importance of search functions on a webpage especially when accessing the web through a mobile phone.
Search functions on the mobile web
Realizing the importance of search facilities on the mobile web, [5] conducted a study on how to improve search on mobile devices and results showed that when users succeed in their search they do so quickly, in 2-3 minutes, and with a small number of interactions while failures took considerably more time. On average, users took twice as long to successfully complete a search and were 60% less successful than when using the conventional large screen interface.
Context of use of mobile web
With accessing the web from our smartphone being so common nowadays, the need for deeper understanding of this phenomenon was required for this study to move forward. More specifically where, when and how users access the Web using their phones. Usage and its context are very important pieces of understanding the needs users have since the mobile phone was built to be used in a wide variety of contexts.
[6] conducted an important study to identify the contexts under which mobile internet is used most frequently and what is the impact of context on the ease of use. Results showed that participants used the internet 61 minutes on average, the most frequent context of use was identified when participants had a hedonic goal, their emotional state was joyful, they were stationary, visual and auditory distractions were low, very few people were around them and the interactions were reported as low. 85.9% of 256 contexts identified participants were stationary when interacting with their mobile phone and 69.5% of total use was for hedonic purposes rather than utilitarian and one hand interaction was used 76.6% of the sessions. [6],[7] agreed on the fact that users access the web from a stationary position. Additionally, [7] shows very short sessions of usage. Why is this the case?
Cognitive aspects of mobile use in different contexts
[8] provided an explanation on why mobile usage while in different contexts is done in short sessions conduct in a field study were participants were asked to visit certain websites and the researchers would record their actions while a page was loading. Results showed that participant’s attention shifted away 46% for mobile situations such as the railway station, 70% for Metro platform while waiting for a train and 80% in a long quite street. The effect of context had a significant effect on the duration of continuous attention to the mobile device with bursts of attention of 8-16 seconds while in the lab and the café while, in the escalator or a busy street the bursts of attention were much shorter, below 6 seconds. [7],[8] showed consistency in their results that usage sessions are very short because the second showed that attentional resources are limited
Visual information density and navigation
How information is presented in small screen devices is of grave importance because displaying information suitable for a conventional screen would be inappropriate and unusable because it is widely known that the legibility and also the readability is hampered by increased density of text on the screen [9],[10].
How do users navigate through webpage after webpage of, admittedly, visually dense pages, what kind of problems they face and what causes these problems?
[11] investigated whether the influence from cognitive preview or visual density, affects the usability of small screen devices and observe the effects on navigational performance by manipulating text size, information density and cognitive preview in older users. Results shows that font size did not significantly affect performance but there was a meaningful interaction between font size and size of preview showing that the combination of the two contribute to performance with a stronger impact on the preview size. Also, font size did not affect navigation performance either but the size of the preview affected disorientation measures.

3. Best performance was observed in the 4th condition of large text/large preview and poorest performance was observed in large font/small preview.
Conventional vs mobile web access
After examining the effects of context of use and cognition in mobile web, we need to get to the chase by examining the differences between conventional and mobile methods of accessing the web. The questions to be answered is how easy it is for users to browse full websites on their mobile phone compared to the conventional web?
[12] evaluated mobile web browsing compared to desktop web browsing. Results showed that user’s performance was poor on mobile phones, the average completion time on mobile was 5.7 minutes while on desktop it was 1.41minutes and total average task completion time for all tasks on desktop browsers for all participants was under 6 minutes while the same average was 23 minutes for mobile phones. [13] reached the same conclusion with results again showing that on mobile optimized versions, participants were 30-40% faster but they were annoyed by the limited features of the optimized version. Setting aside the limited features issue, mobile optimized pages, although far from perfect, are clearly more usable and efficient so why not every company designs a mobile optimized version of its full website? A few companies decided to create mobile tailored websites suitable for mobile viewing. These tailored websites were designed to have fewer functions than the full website and their design was fit for viewing on small screen displays but far from perfect, with usability problems persisting. A staggering report came from Google Inc. reporting that in 2011, only 21% of its largest advertisers have mobile friendly websites [14].
This study was conducted to help close the gap presented in the literature that mobile web access and interaction is different from the conventional web access and there are no heuristics to be used to help design and develop mobile usable websites or evaluate the existing ones. This study presents a unique and easy to follow and understand heuristics and help designers and developers to finally, build and evaluate websites for mobile phones.
METHOD
Design
The development of heuristics for websites viewed and interacted with on a smartphone required extensive user testing. Websites and participants to perform tasks on those websites were identified. Usability problems identified during retrospective think aloud were categorized iteratively following a grounded theory approach. The mobile web has been around for a few years now and its rise and usage ratio does not justify the lack of design and development heuristics for the creation of websites for mobile use with significant usability and user experience issue raised by users. To solve this problem, a usability study was conducted to help designers and developers by producing heuristics for smartphone websites. 7 interactive websites were identified, a mixture of mobile optimized and non-optimized ones, or full website as they are called in relevant literature. It was decided that a homogenous sample was needed in terms of age and web experience. 24 participants were identified who fit certain criteria such as age, experience with mobile web and experience with mobiles in general, to establish a coherent group of users of the same general attitude and attribute to mobile web because age and different experience levels would be confound variables and they would affect our results. These users would participate in a between participant design usability study with 12 participants being Android users and 12 being iPhone users both user types would complete all tasks in all 7 websites and rate the usability problem on a scale (0-4)[18].
Participants
24 participants took part in this study. 7 women and 17 men between the ages of 20 and 30 years old with a mean average of 26.58 years (standard deviation= 3.175). On average these participants have owned a smartphone for 2.79 years (SD = .977). Their weekly web access via their smartphone estimated at 4.75 (SD = .532) on a 5 point Likert scale from ‘never’ to ‘everyday’ with 79.17% (19 of 24 participants) of them reporting everyday access to the web via their smartphones during the previous week, with a mean average of 2.667 hours (SD = 1.5156) of daily web access. Results showed a mean average of 3.33 (SD=.816) on that Likert scale with 50% of them, not surprisingly, choosing the middle choice/ground and 37.5% leaning toward the ‘expert’ side of the scale. A 5 point Likert scale from ‘Not Important’ to ‘Very important’ was used and a mean average of 3.75 (SD= 1.327) showed that participants considered accessing the web through their phones as ‘important’ but not ‘very important’. At the end of the session participants were offered coffee and cookies as a reward for their participation.
Equipment
A laptop was used, throughout user testing, which carried the software needed for recording video and audio during user testing. Each participant would use his own mobile phone. The video recording equipment for mobile phones was self- made. Borrowing Steve Krug’s idea, a Creative 720p resolution USB 2.0 webcam and a lightweight LED reading light were purchased and with the help of a lot of duct tape, the LED reading light’s flexible neck attached firmly on the mobile phone and the webcam was taped on it to focus on the mobile phone’s display firmly throughout the user testing. The webcam’s native software was used to record the first stage of the user testing where users perform tasks. Camtasia 8.0, a screen capturing software was used to capture the retrospective think aloud stage of the session and rendering the finalized file for each participant. Finally, IBM SPSS Statistics 20 was used for data analysis.
Materials
A pre-screening questionnaire divided in two sections was given to the participant. The first section was for demographic information and the second section consisted of Likert scales, open-ended questions, and closed check box questions. The questionnaire was given to the participant

4. prior to the main testing session after he/she had read and signed 2 consent forms devised for both audio voice recording and mobile display recording. During the test session, pieces of paper with the website URL and tasks to be performed on each website and a sheet with the severity rating definitions, were given to the participant.
Procedure
Each session lasted around 60 minutes depending on how much the user had to say during the retrospective think aloud portion of the session. The users were greeted with coffee and biscuits and were given the consent forms for video and audio recording to read and sign them. After signing the consent forms, the researcher explained to them the procedure that would follow.
The mobile testing webcam was equipped on the participant’s mobile phone and a brief test on the audio and video recording quality followed. The participant was handed a piece of paper with the website he needed to visit and the tasks to be completed on that website so he would not have to ask the facilitator again and again if he had forgotten the task or he did not know how to type the URL of the website, which might make him feel uncomfortable. After the completion of the two tasks of that particular website, the second website task paper was handed to them and so on until the 3rd website-task paper was handed to them. At that point the task session was paused and the retrospective think aloud portion followed for the 3 first websites. For each participant who took part in this study the order of websites was reversed to accommodate for the participants becoming tired and bored close to the end of the process. During the retrospective think aloud portion, participants would go through the replay of their interaction, fast-forwarding in IDLE periods for example when pages were loading, with the first 3 websites and talk about problems they encountered as well as any good features they encountered for each website separately.
If participants proved reluctant to talk they were kindly prompted by the researcher on particular parts of the replay video where the researcher detected uncertainty in their (inter)actions, such as repeated scrolling left and right on the same section of the website indicating that the user is looking for something, or any prolonged pauses during the task that could mean that the user is lost or cannot find something important to continue with the task. Also, few participants were reluctant to talk because they were shy and/or because of their character. Those participants were prompted on the homepage of each website to answer questions such as ‘what do you see here’, ‘do you detect any problems or something good you would like to mention’ and ‘what are your thoughts of what you see on your display?’ If the user identified a problem, the process was paused and the participant was asked to rate the problem for its severity on a 4 point scale where 1 = cosmetic, 2=minor, 3= major and 4 = catastrophic. After the retrospective portion of testing was completed, the user resumed the task portion with the 4 remaining websites. At the end, the researcher thanked the users for their participation in the experiment.
Data Analysis
A grounded theory approach was followed by the researchers in the sense that the categories emerged from the data itself. We proceeded with identifying patterns and recurring themes. The first iteration of this process was the grouping of usability problems of the same subject/theme and a title was given to each group accordingly. The next iteration included the creation of subcategories within these categories and the merging of categories into more abstract categories if necessary. Subcategories were identified and each subcategory was then further analyzed for further placement into one of the categories or as a higher level category in itself. The third and last iteration of this data analysis process included finalizing the abstract high level categories, merging stand-alone categories into higher level categories based on how and where the user identified the problem. The completion of the third iteration resulted in the first list of categorized problems. Those problems were then further analyzed for frequency of occurrences and mean severity ratings (1 - 4) to decide which of those would be included in the final subcategory list. Categories with lower than 3 frequency of occurrences were omitted or merged into other subcategories. A second coder took a random sample of approximately 10% of the problems identified by the first coder and coded them independently into the initial set of categories. The inter-coder reliability between the two sets of coding was 82%. This inter-coder reliability was considered adequate, so the first coder’s categorizations were used.
RESULTS
138 distinct problems were identified by the participants during user testing. Emerging categorization of those problems after the iterative categorization resulted in a total of 4 categories and 32 subcategories. Categories identified were ‘Presentation’, ‘Content’, ‘Information Architecture’ and ‘Interaction’. Frequency of occurrences and mean severity ratings were calculated. It became apparent that further categorization and merging of categories were to follow. Since [20], with 30 participants, omitted categories with lower than 5 frequency of occurrences, we decided that categories with less than 3 frequency of occurrences would be omitted since this study had 24 participants because 32 is a large number of problem subcategories.
Merging those subcategories with less than 3 occurrences into other similar categories if appropriate. If it was deemed inappropriate to merge into other subcategories, they would be omitted from the final set of problem categories. This process resulted in 16 problems being omitted along with their subcategories, resulting in a new total of 122 problems, 4 categories and 16 subcategories reduced from 32 subcategories.

5. ‘Presentation’ category had 3 subcategories, ‘Content’ had 3, ‘Information architecture’ had 3 and ‘Interaction’ had 7 subcategories. Interaction category was the category with the highest frequency of occurrences with 45 occurrences, followed by ‘Content’ with 34 occurrences, ‘Presentation’ with 26 occurrences and ‘Information Architecture’ with 17 occurrences (figure 1).
Results from examining each category individually showed that the most frequently occurring subcategory for Interaction was ‘Broken interaction consistency/conventions not followed’ with 12 occurrences, the most frequently occurring subcategory for ‘Content’ was ‘Too much content/pictures/featurism’ with 22 occurrences which was the most frequently occurring problem overall, ‘Text/interactive elements not large/clear/distinct enough’ for ‘Presentation’ category with 15 occurrences and last but not least, ‘Content is not properly categorized/grouped’ was the most frequently occurring subcategory for ‘Information Architecture’ category with 10 occurrences.
‘Too much content/pictures/featurism’ was the most frequent problem identified in this study but which one of the 4 categories was rated as the most severe one, thus, identifying which category proved to be the most problematic for users. Results showed that ‘information architecture’ was the most problematic with a mean average of 3.3 followed by ‘Interaction’ with 2.99, ‘Presentation’ with 2.94 and ‘Content’ with 2.54 mean severity.
Finally, negative problem subcategory names were transformed into positive heuristics (table 1).
Figure 1: Graph depicting frequency of occurrences per category

6. The analysis led to the identification of the most severe as well as the most frequent categories and subcategories. Problems with high severity should be addressed but also problems appearing frequently cannot be ignored because the cumulative difficulty and frustration they cause could still severely hinder user performance and experience. One example could be the problem subcategory ‘Too much content/pictures/featurism’ of the ‘Content’ category which has the highest frequency of occurrences of all the subcategories. In this spirit, subcategories with severity mean of over 3 in the 0-4 scale. 10 problem subcategories were identified as very severe and should be prioritized when addressing usability problems but, again, the need to address
problems with high frequency of occurrence cannot be overstated. [15] reported user frustration has a time factor embedded in so if the user faced a problem once but he overcame it fairly quickly and the same problem persisted requiring workarounds, even short ones, would be a problem of increased severity according to [16]. The ‘8 or more’ frequency criterion was decided considering [20] criterion for the same frequency measure. They had identified 907 problems and they set the criterion for high frequency at 10 occurrences thus, the decision for setting the criterion at 8 or more. 7 subcategories were identified as occurring frequently based on 8 occurrences or more criterion. Problem subcategories were identified as being both of high severity
Table 1: VX heuristics

7. and high frequency. They can be identified as the severest usability problems that must be fixed as soon as possible on existing websites and must be avoided at all costs when building a website for mobile use.
DISCUSSION
Overview and rationale
The mobile web, even today, offers a mediocre user experience with the majority of websites having low usability, making users prefer the conventional way of accessing the web for what they deem as ‘serious’ tasks. Using the same design guidelines for the design and development of mobile websites proves unsuitable for mobile web access because they do not consider the purpose, physical design and context of use of mobile phones as seen in the literature review. Mobile phones’ screen size, context of use and cognitive requirements are very different from those of a desktop or a laptop computer. Although, users may be expecting the interaction to be as easy and straight forward as the interaction with conventional desktop/laptop web, the interaction is different and users prefer the conventional ways than the mobile web.
The smaller screen size affects efficiency, task completion, the cognitive workload required for interaction in different contexts and the amount of interactions needed by the user. Mobile phones are most often used indoors, for hedonic purposes, when the user is stationary and there are not a lot of people around. When mobile phones are used on the move, the interaction is done in short bursts of less than 6 seconds because attentional resources are limited and interaction with the mobile and sampling the environment challenge the brain’s attentional capacity.
138 usability problems, for both full and mobile-optimized websites, were identified by this usability study which focused on producing usability heuristics for the mobile web. First, the identified usability problems went through an iterative grounded categorization process with 3 iterations to be categorized into problem categories and subcategories resulting in 4 major categories namely, Presentation, Content, Information Architecture and Interaction and each category had its own problem subcategories labeled appropriately to represent the emerged problem. These categories went through another iteration of categorization where frequency and severity were measured and the subcategories with lower than 3 frequency of occurrence were omitted from the final problem table if they could not be merged with other subcategories to form a new subcategory with more than 3 occurrences while others were merged into one category.
The results of this last iteration produced a finalized list of 16 evidence-based problem subcategories grouped into 4 major categories. Presentation had 3 heuristics, Content had 3, Information architecture had 3 and finally, Interaction had 7. One explanation for the majority of usability problems being grouped in the Interaction category is that websites being interactive is a given or at least they try to make them interactive, leading to increased interaction problems identified. These 16 problem subcategories were turned into heuristics by transforming the negative problem subcategory titles to 16 positive heuristic titles.
Interpretation and Analysis
Results showed that the most frequent usability problem was identified as being ‘Too much content/pictures and featurism’ which was also researched by [11], presented during the literature review and it is not surprising. The advances in e-marketing requiring an ever rising portion of a page and the ever increasing features and functions fighting for their own portion of the website can be compared to a high value real estate where everyone wants a piece of. If that was true for the conventional web, it is especially true and important for the mobile web where that real estate is a hut in terms of size. Also, Information Architecture is the category with the highest mean severity of the four categories with all of its 3 subcategories being rated as of high severity. Structure, placement and grouping of information are very important for the user to find his way towards the completion of a task.
If information is not grouped or placed appropriately, user has to search more than he wants to and should have to, prolonging the task duration and increasing the interactions he has to perform on that device. That device being a mobile phone which, as seen in literature, inherently requires a lot more interactions than the conventional web, leading to an increase in effort needed, workload, cumulative frustration and time. That is why it is not surprising that users rated problems related to ‘Information Architecture’ so highly.
The highest severity subcategory from ‘Interaction’ category is none other than ‘Broken consistency and convention not followed.’ Anyone who has performed usability evaluations knows that this problem comes up a lot and there is very good reason why. Conventions are practices concerning structure, placement, design and behavior of elements of the website that have been in place for so long, they became norms. The majority of websites try to keep conventions in the design because users expect those conventions to be in place. Inability to follow conventions leads to a phenomenon that can be compared to ‘change blindness’, the inability of human beings to identify changes in their visual periphery, in the sense that if the user expects something to be placed on the right side of the website and with a particular label, it will take a lot of time for him/her to identify if he/ she ever does, the same element if it is on the left side no matter how big it is. This phenomenon happened numerous times during user testing providing this study with a subcategory of high frequency of occurrence and severity.
An interesting and unexpected problem came up during user testing which led to a problem subcategory, based on its frequency and eventually made it to the final list of heuristics. The ‘Choose language type based on the context and website’s target users’ heuristic and how it came to be a problem subcategory is worth discussing. During user testing

8. users were asked to find and enable Facebook’s option to ‘review tags before they are posted on their timeline’. Most users had a big problem with finding that option because Facebook’s website was in Greek and the majority of users did not know what the Greek translation of ‘tag’ was. This problem led to the realization that even if they were Greeks and they preferred the website in Greek, they had never used the Greek word for it because ‘tag’ is a universal word when it comes to Facebook. The interesting thing about this particular usability problem they identified is that, firstly, that usability problem would not have come up in an expert evaluation if it was performed by the researcher of this study because it had never occurred that something like that would happen. Secondly, this problem illustrates how context relevant language use supersedes the need of merely using native language.
Comparison of study’s heuristics to conventional web heuristics
An important point in the discussion of this study is how the heuristics proposed by this study fit in with the heuristics for conventional web. Molich and Nielsen’s heuristics [17,18,19] are the most popular heuristics, used for design and evaluation of websites for years and Petrie and Power’s heuristics [20], published in 2012 provide the most modern and empirically sound heuristics for interactive websites. Molich-Nielsen’s heuristics were compared to our new heuristics (VX heuristics). This comparison proved problematic because the labels of those heuristics are too abstract and the discrepancy between the label and its description in the type of language used makes them very hard to remember.
Only 4 out of 10 of Nielsen’s heuristics are represented in VX heuristics. 5 out of 7 of Interaction heuristics from VX heuristics are not covered by any of the Nielsen’s heuristics and this could be because Nielsen’s developed these heuristics in 1990 and revised them in 1995. Back then, websites lacked one important ingredient, interactivity. For the same reason, only 1 out of 3 ‘Presentation’ heuristics of VX heuristics were covered by Nielsen’s heuristics and that heuristic was navigation’s design leaving out presentation aspects of interactive elements, again highlighting the lack of interactivity on Nielsen’s heuristics.
Comparison continues with the VX heuristics compared to Petrie and Power’s web heuristics published in 2012. Petrie and Powers’ heuristics [20], cover 87.5% of the problems identified by this study with overlap of 14 out of 16 heuristics of VX heuristics, again, VX heuristics lack error related heuristics because incidentally, users made mistakes or slips that did not result in errors. It was expected to have overlapping heuristics with Petrie and Power’s heuristics because both VX heuristics and Petrie and Power’s heuristics investigate website usability problems. A central claim of this study is that using web heuristics for the conventional web is a mistake and leads to usability and user experience issues. This overlap might make this claim seem rejected. This overlap consists of two categories of overlap. The first category is for general heuristics where the overlap is 100% for each pair in terms of principle, in other words the first category addresses heuristics that actually mean the same thing and they are about the same problem such as: These heuristics are the same and they describe the same problems and the same general principle. The fact that VX heuristics have such a high overlap with well-known heuristics is very important and adds to its external validity. The second category though addresses heuristics where the label is similar and the general principle is the same but VX identify heuristics specialized for the mobile web. The same general principals apply stemming from testing websites, there are major differences though, and those can only be seen when reading the descriptions and examples provided by our newly proposed heuristics. Examples follow:
 # 3 VX heuristic and #1 Petrie and Power’s heuristic were presented to overlap but the description of the first paints a specialized picture about the mobile web based on user data. It describes the fact that text and interactive elements are expected to be small on a full website viewed on a mobile phone but users need to be able to recognize the text before they zoom in because they, first, look for content and then they zoom in to read/select.
 # 4 VX heuristic and #6 Petrie and Power’s heuristic overlapped based on the general principle of avoiding having too much content on a page. That is true for both mobile and conventional web but the issue for the mobile web is much more intense because of the screen size and the inability users have anyway to not be able to perceive the whole page that makes them using scrolling, zooming and paging functions a lot more, as seen in the review of literature. The same goes for #5 VX heuristic
 Another overlapping heuristic is #7 VX heuristic which overlaps with #8 Petrie and Power’s heuristic where content on mobile phones must be categorized and grouped properly because large amounts of content make for a lot of interactions required by the users to go through it all. Content must be organized in a way that users can find what they want without having to read irrelevant to them information or having to scroll large amounts of content to get to where they want. Especially if they know what they are after.
 An important overlapping heuristic is #9 VX heuristic with #4, #13 Petrie and Power’s heuristics. Interaction indicators must be proper and salient enough, in others words the user needs to be informed whenever something changes.
 When users of this study used search filters on the left side bar, they were automatically zoomed in close to the filter they selected. The problem was that the rest of the page was not visible so they did not know if selecting

9. this filter actually changed the results on the screen because that part of the page was out of sight on the mobile screen. Another reason why this usability problem came up frequently is that the loading indicator was out of sight too because the designers had placed it in the middle of the results page but when designing on a desktop or a laptop pc.
The above stated, demonstrate the fact that heuristics are similar in their general principles but the differences between the mobile web and the conventional web makes them specialized to the mobile web’s restrictions and that is what the heuristic descriptions are explaining including examples from user data collected during user testing.
Limitations
The comparison between our new heuristics and Nielsen’s heuristics helped identify weaknesses of our heuristics. 3 of Nielsen’s heuristics are concerned with errors and documentation which none of VX heuristics covers due to the lack of errors appearing during user testing. That can be attributed to the fact that the tasks to be performed by the users were error free but not mistake/slip free. In other words users made mistakes but those mistakes did not result in any kind of error. The forms, an error causing feature of the web, had auto-complete embedded, calendars and lists for date input and radio buttons. In fact, calendars and the auto- complete function of input fields were identified as some of the good features that made their interaction much easier and efficient. Another limitation some could identify about this study is the fact that the usability study took place in a laboratory environment, isolated by any kind of visual or auditory distractions which is the opposite of how the mobile phones are supposed to be used. Instead a field study would increase result validity. Those comments are considered perfectly reasonable and might be correct but the review of relevant literature and especially literature on context of use discovered that mobile phones are primarily used indoors and when not too many people are around and more specifically, mobile web use is most common when sitting on the couch of one’s own home because the couch is a comfortable place for users to access the mobile web and there is no computer in that room. The above stated facts could suggest that conducting studies for the mobile web in a lab might not be invalid. Other limitations could be the amount and quality of tasks to be performed by the user. Tasks were simple and short, albeit very common for users visiting these kinds of websites which were identified by the users themselves. Or the fact the homogeneity of participant sample and especially all users being of Greek nationality and the role culture plays in usability evaluation.
Future work
Further work is needed to focus on examining whether these heuristics are more effective in designing and developing of mobile websites. Also, future work should focus on the limitations mentioned in the previous section and examine how effective and efficient field usability studies are compared to laboratory studies when testing for mobile web usability. Evaluations must be conducted using these new heuristics and the results must be compared to results from other heuristics. The limitation of those heuristics to identify and evaluate error resulting interactions and feature is something that needs to be addressed.
Benefits and implication
This study will greatly improve the mobile web because this study and its products are based on solid research foundations, deep understanding of the literature surrounding mobile phones and the web as individual entities and together, forming the mobile web.
The improvements directly resulting from adopting these heuristics will not only be usable websites but also seamless interaction with the web leading to improved user experience. This improved user experience could lead to a greater mobile web market penetration and the percentage of web access through mobile could increase because users would enjoy going online and browsing for goods, information and services. The percentage of utilitarian mobile web access could be increased, thus allowing more users to trust the mobile web for their utilitarian tasks and they would be less dependent on stationary means such as desktop/laptop computers thus, making mobile web access, really mobile. Speaking about trust, some of the participants of this study mentioned that they would perform account setting changed and pay online from their PC rather than their mobile phone. Improved usability and user experience could lead to user trusting their mobile phone to perform tasks they would not perform otherwise because they did not trust the mobile web mainly because of its design and the increased number of usability problems they identified and had to deal with.
Also these heuristics, being reasonable in number makes them fairly easy to remember. Being able to remember the heuristics when evaluating a website greatly increases efficiency and performance because evaluators and designers would not have to go back and forth reading revisiting the table of heuristics and their descriptions. Additionally, following these heuristics could solve another issue discussed in the literature review, information density. Information density has proven to be a major problem, especially for the mobile web. Those heuristics cover this problem by mean of a ‘Content’ category heuristic ‘Provide the user with sufficient content but not excessive’ which was derived by the most frequent usability problem occurring during this study’s evaluation sessions. Avoiding high information density allows for clarity, making the important, for the user and the client, functions clearer and in conjunction with heuristics on structure (Information architecture) and design (presentation) make these functions more visible and readily distinguishable.
Adoption of these heuristics for the design and development of mobile websites could lead to websites of higher usability and user experience, finally, making the mobile web a place

10. where the user would be able to perform most of the tasks he used to perform on the web via conventional means.
CONCLUSIONS
This study showed mobile phone’s differences to the conventional web access must be taken into consideration when designing, developing and evaluating websites on a mobile phone. Screen size, mental load, I/O functions and context of use make for a mobile web in need of specialized heuristics that adhere to the attributes and restrictions of the mobile web. While heuristics for conventional web access and mobile web adhere to the same basic design and interaction principles, the mobile web heuristics proposed by this study suggest a specialized approach. These heuristics take consider the same basic design and interaction principles of the conventional web and factoring the attributes of mobile phones and their differences from conventional web access, would help design and develop mobile website with increased usability and user experience that could turn mobile web into a more hospitable ‘place’. Users would be able to perform both utilitarian and hedonic tasks and decrease dependency to the conventional web to those who previously used it for what they deem as ‘serious’ tasks due to mobile web’s usability problems and low trustworthiness.
REFERENCES
[1] M. Meeker.(2014, May). "Internet Trends 2014" [Online]. Available: http://www.kpcb.com/internet-trends.
[2] A. Dillon et al. “The effect of display size and text splitting on reading lengthy text from the screen”. Behaviour and Information Technology 9 (3) 215–227, 1990.
[3] D. Raptis, et al. "Does size matter? Investigating the impact of mobile phone screen size on users’ perceived usability, effectiveness and efficiency.," Proc. 15th Int. Conf. Human-Comput. Interaction With Mobile Devices Services. 2003.
[4] M. Jones et al. "Improving Web interaction on small displays," Comput. Networks: Int. J. Computer Telecommunications Networking, vol. 31, pp. 1129–1137, 1999.
[5] M. Jones et al. "Improving web search on small screen devices," Interacting With Computers, vol. 15, p. 479– 495,2013.
[6] H. Kim et al. "An Empirical Study of the Use Contexts and Usability Problems in Mobile Internet," Proc. 35th Hawaii Int. Conf. Syst. Sciences, 2002.
[7] Y. Cui & V. Roto, "How People Use the Web on Mobile Devices," Www 2008 / Alternate Track: Industrial Practice and Experience, 2008.
[8] A. Oulasvirta. et al. “Interaction in 4-second bursts: The fragmented nature of attentional resources in mobile HCI”. In Proceedings of SIGCHI Conference on Human Factors in Computing Systems, ACM Press, New York, pp. 919-928, 2005.
[9] Norman, K.L., 1991. “The Psychology of Menu Selection”. Ablex, Norwood, NJ.
[10] M. Ziefle. “Instruction format and navigation aids in mobile devices”. In: Holzinger, A. (Ed.), Usability and Human Computer Interaction for Education and Work. LNCS 5298. Springer, , Berlin, Heidelberg, pp. 339-358, 2008.
[11] M. Ziefle, "Information presentation in small screen devices: The trade-off between visual density and menu foresight," Applied Ergonomics, vol. 41, pp. 719–730, 2010.
[12] S. Shrestha, "Mobile Web Browsing: Usability Study," Mobility '07 Proc. 4th Int. Conf. Mobile Technology, Applications, Syst., pp. 187–194, 2007.
[13] G. Schmiedl et al. "Mobile Phone Web Browsing – A Study on Usage and Usability Of The Mobile Web," Proc. 11th Int. Conf. Human-Comput. Interaction With Mobile Devices Services, no. 70, 2009.
[14] A. Ha: Google Pushing Advertisers to Build for Mobile Search. Adweek (2011)
[15] J. Lazar. "Severity and Impact of Computer User Frustration: A Comparison of Student and Workplace Users," Interacting With Computers, vol. 18, pp. 187–207, 2006.
[16] J. Nielsen. “Severity Ratings” [Online]. Available: http://people.cs.uct.ac.za/~gaz/teach/hons/papers/Severity% 20Ratings%20for%20Usability%20Problems.html.
[17] R. Molich and J. Nielsen. Improving a human computer dialogue. Communications of the ACM, 33(3), 338 – 348, 1990.
[18] J. Nielsen. “Usability engineering”. San Diego, CA: Morgan Kaufmann, 1993.
[19] J. Nielsen. and R. Molich. Heuristic evaluation of user interfaces. Proc. CHI 1990, ACM Press (1990), 249-256, 1990.
[20] H. Petrie and C. Power, "What do users really care about?: a comparison of usability problems found by users and experts on highly interactive websites," CHI '12 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2107-2116, 2012.