28th June 2015. My presentation at Mobile Services 2015 (http://www.themobileservices.org) about our study on end users' perception of hybrid mobile Apps in the Google Play Store. Mobile Services 2015 is the 4th IEEE International Conference on Mobile Services, and it has been held in June 27 - July 2, 2015, New York, USA. Accompanying paper: http://www.ivanomalavolta.com/files/papers/MS_2015.pdf Abstract: Today millions of mobile apps are downloaded and used all over the world. Mobile apps are distributed via different app stores, such as the Google Play Store, the Apple App Store, the Windows Phone Store. One of the most intriguing challenges in mobile apps development is its fragmentation with respect to mobile platforms (e.g., Android, Apple iOS, Windows Phone). Recently, companies like IBM and Adobe and a growing com- munity of developers advocate hybrid mobile apps development as a possible solution to mobile platforms fragmentation. Hybrid mobile apps are consistent across platforms and built on web standards. In this paper, we present an empirical investigation into mobile hybrid apps. Our goal is to identify and analyse the traits and distinctions of publicly available hybrid mobile apps from end users’ perspective. The study has been conducted by mining 11,917 free apps and 3,041,315 reviews from the Google Play Store, and analyzing them from the end users’ perception perspective. The results of this study build an objective and reproducible snapshot about how hybrid mobile development is performing “in the wild” in real projects, thus establishing a base for future methods and techniques for developing hybrid mobile apps.

1. Ivano Malavolta Gran Sasso Science Institute
Stefano Ruberto Gran Sasso Science Institute
Tommaso Soru University of Leipzig
Valerio Terragni Hong Kong University of Science and Technology
End Users’ Perception of Hybrid Mobile
Apps in the Google Play Store
New York, 28th June 2015

2. FRAGMENTATION à a native mobile app is written from scratch for
each platform
Mobile platforms today
Objective
-C
code
Swift
code
XCode
Java
code
C++
code
Eclipse
C#
code
C++
code
Visual Studio
JS
code

3. A possible solution to mobile platforms fragmentation
Recurrent architecture:
– apps are developed using standard web technologies
– on top of a hybrid development framework
• providing a native wrapper and a generic JavaScript API that bridges all the
service requests to the corresponding platform API
Web-based hybrid mobile apps
Single code base

4. Pros
• cross-platform portability
• reuse of existing knowledge of
web developers
• simpler and less expensive
development processes
Pros and cons
Cons
• restricted access to hardware
features
• decrease in performance
• variations on user experience
As of today, limited empirical investigations
have been performed on hybrid mobile apps
Strong debate about benefits and drawbacks

5. Research goal
What is the difference between hybrid and
native mobile apps as perceived by end users?
Perceived value
Perceived performance
Perceived bugginess
Initial download overhead
Developer
End users
creates
download
& use
App
Previous
work[1]
FOCUS
OF THIS
PAPER
RQ1
RQ2
RQ3
RQ4

6. We analysed hybrid mobile apps
• in their actual context of use
• with a reproducible empirical strategy
– well-defined empirical protocol
– dataset comprising 11,917 real apps
and 3,041,315 user reviews*
– dedicated analysis process and tool**
Design of the study
* complete replication package: http://cs.gssi.infn.it/ms_2015
** analysis tool from [1]: http://github.com/GabMar/ApkCategoryChecker

7. Data extraction
Classified apps
(hybrid vs native)
Hybrid apps
classifier*
Reviews
analyzer
top-500 most popular free apps for
each category of the Google Play Store
~11k app binaries
50 pages (~255) of
reviews for each app
~3M user
reviews
apps scores
Apps and
reviews
mining
perceived value: 0.5
users sentiment: 0.6
#reviews: 243
performance: 0.6
bugginess: 0.1
size: 3,456 kb
* analysis tool from [1]: http://github.com/GabMar/
ApkCategoryChecker

8. Reviews analysis
Stopwords
removal
manually performed
by 2 domain experts
Single review
Single review
score
polaritypos: 0.8 performancepos: 0.6
polarityneg: 0.1 performanceneg: 0.05
bugginess: 0.2
300 random
reviews
Keywords
extraction
Relevant keywords
Lemmatization
Tf-idf based
vectors similarity
computation

9. Results
Data-intensive
mobile apps[2]
Apps with closer interaction
with the Android platform
Winners, in line with
informal claims[3,4,5]

10. Results – value (RQ1)
Average of the ratings as provided by end users
3.35 3.75
Rating = real number in [1, 5]
Certain balance, with neglectable differences

11. Results – value (RQ1)
Polarity of sentiment of end users
where posa = #reviews with positive sentiment
nega = #reviews with negative sentiment
Balance between hybrid and native apps, with
some exceptions
Non data-intensive or
requiring multimedia
capabilities

12. Results – value (RQ1)
Average review count
whre Ra ∈ ℕ
Native apps have been reviewed in average
6.5 times more than hybrid mobile apps
Possible interpretation:
hybrid mobile apps are neither perceived as too
satisfying nor dissatisfying w.r.t. native ones [6]

13. Results – performance (RQ2)
where posa = #reviews with positive sentiment w.r.t. performance of the app
nega = #reviews with negative sentiment w.r.t. performance of the app
Balance between hybrid and native apps,
with some exceptions

14. Results –bugginess (RQ3)
where buga = #reviews signalling the presence of bugs or failures
reviewsa = total number of reviews of the app
The highest unbalance between the two
development strategies in our study
bugginessa = buga / reviewsa
Possible interpretation:
absence of full-fledged testing frameworks for
hybrid apps, such as those provided by native
apps IDEs like Eclipse and Android Studio

15. Results – initial download size (RQ4)
6,586 kb4,625 kb
In line with the average size of
Android apps [7]
sizea = file size in kilobytes of the app APK file

16. A possible solution to mobile platforms fragmentation
Recurrent architecture:
– apps are developed using standard web technologies
– on top of a hybrid development framework
• providing a native wrapper and a generic JavaScript API that bridges all the
service requests to the corresponding platform API
Hybrid mobile apps
Single code base
Conclusions
Data extraction
Classified apps
(hybrid vs native)
Hybrid apps
classifiers
Reviews
analyzer
top-500 most popular free apps for
each category of the Google Play Store
~11k app binaries
50 pages (~255) of
reviews for each app
~3M app
reviews
apps scores
Apps and
reviews
mining
perceived value: 0.5
users sentiment: 0.6
#reviews: 243
performance: 0.6
bugginess: 0.1
size: 3,456 kb
End users value hybrid and native apps similarly
Hybrid may be good for data-intensive apps, whereas it performs poorly when
dealing with low-level, platform-specific features
In some categories, native apps are perceived as better with respect to
performance and bugginess
Reviews analysis
Stopwords
removal
manually performed
by 2 domain experts
Single review
Single review
score
polaritypos: 0.8 performancepos: 0.6
polarityneg: 0.1 performanceneg: 0.05
bugginess: 0.2
300 random
reviews
Keywords
extraction
Relevant keywords
Lemmatization
Tf-idf based
vectors similarity
computation

17. References
[1] Ivano Malavolta, Stefano Ruberto, Valerio Terragni, Tommaso Soru, Hybrid Mobile Apps in
the Google Play Store: an Exploratory Investigation. International Conference on Mobile
Software Engineering and Systems (MOBILESoft), ACM, 2015.
[2] Mirco Franzago, Henry Muccini, and Ivano Malavolta. Towards a collaborative framework
for the design and development of data-intensive mobile applications. International
Conference on Mobile Software Engineering and Systems (MOBILESoft), pages, 58-61,
ACM, 2014.
[3] Emiliano Masi, Giovanni Cantone, Manuel Mastrofini, Giuseppe Calavaro, and Paolo
Subiaco. Mobile apps development: A framework for technology decision making. In Mobile
Computing, Applications, and Services, pages 64–79. Springer, 2013.
[4] Julian Ohrt and Volker Turau. Cross-platform development tools for smartphone
applications. Computer, (9):72–79, 2012.
[5] Luis Corral, Alberto Sillitti, and Giancarlo Succi. Mobile multiplatform development: An
experiment for performance analysis. Procedia Computer Science, 10:736–743, 2012.
[6] Nan Hu, Jie Zhang, and Paul A Pavlou. Overcoming the j-shaped distribution of product
reviews. Communications of the ACM, 52(10):144–147, 2009.
[7] Aapo Markkanen. Findings from Mobile Application File-size Research, 2012. ABI
Research market report. Code: IN- 1014787.

18. Contact
Ivano Malavolta |
Post-doc researcher
Gran Sasso Science Institute
iivanoo
ivano.malavolta@gssi.infn.it
www.ivanomalavolta.com