SlideShare ist ein Scribd-Unternehmen logo
1 von 44
Downloaden Sie, um offline zu lesen
1
Inside Android's UI
ADC IV
Karim Yaghmour
@karimyaghmour
karim.yaghmour@opersys.com
2
These slides are made available to you under a Creative Commons Share-
Alike 3.0 license. The full terms of this license are here:
https://creativecommons.org/licenses/by-sa/3.0/
Attribution requirements and misc., PLEASE READ:
● This slide must remain as-is in this specific location (slide #2), everything
else you are free to change; including the logo :-)
● Use of figures in other documents must feature the below “Originals at”
URL immediately under that figure and the below copyright notice where
appropriate.
● You are free to fill in the “Delivered and/or customized by” space on the
right as you see fit.
● You are FORBIDEN from using the default “About” slide as-is or any of its
contents.
●
You are FORBIDEN from using any content provided by 3rd
parties without
the EXPLICIT consent from those parties.
(C) Copyright 2012, Opersys inc.
These slides created by: Karim Yaghmour
Originals at: www.opersys.com/community/docs
Delivered and/or customized by
3
About
● Author of:
● Introduced Linux Trace Toolkit in 1999
● Originated Adeos and relayfs (kernel/relay.c)
● Training, Custom Dev, Consulting, ...
4
Agenda
● Android's UI, what's that?
● Architecture Basics
● Display Core
● OpenGL
● Input Layer
● Relevant Apps and Services
● System Startup
● References and Pointers
5
1. Android's UI, what's that?
●SoC / GPU
●Touch input
●LCD
●Keyboard
???
6
1.1. What I'm NOT covering
● Media layer
● StageFright
● Video playback
● Camera
● DRM
● Etc.
7
2. Architecture Basics
● Hardware used to run Android
● AOSP
● Binder
● System Services
● HAL
8
9
10
11
12
13
/frameworks/base/services/java/...
/frameworks/base/services/jni/
/hardware/libhardware/
/device/[MANUF.]/[DEVICE]
/sdk/emulator/
Kernel or module
/frameworks/base/core/...
AOSP-provided
ASL
Manuf.-provided
Manuf. license
Manuf.-provided
GPL-license
14
3. Display Core
● Display Hardware
● Classic Linux display stack
● Display stack in Android
● Kernel driver
● HAL definition
● HAL module
● Surface Flinger
● Window Manager
● Walkthrough
15
3.1. Display Hardware
MMU
IOMMU
16
3.2. Classic Linux display stack
17
3.3. Display stack in Android
18
19
3.4. Kernel driver
●
Video memory management
●
Mode setting
● Checking of parameters
● Motorola Xoom:
●
/dev/nvhdcp1
● /dev/nvhost-ctrl
● /dev/nvhost-display
● /dev/nvhost-dsi
●
/dev/nvhost-gr2d
● /dev/nvhost-gr3d
●
/dev/nvhost-isp
●
/dev/nvhost-mpe
● /dev/nvhost-vi
● /dev/nvmap
● /dev/tegra-crypto
●
/dev/tegra_avp
●
/dev/tegra_camera
●
/dev/tegra_fuse
● /dev/tegra_rpc
● /dev/tegra_sema
● ... whatever hides in hwcomposer HAL module
20
3.5. HAL Definition
● /hardware/libhardware/include/hardware/hwcomposer.h
● struct hwc_procs:
● invalidate()
● vsync()
● struct hwc_composer_device:
● prepare()
● set()
● dump()
● registerProcs()
● query()
● *()
21
3.6. HAL module
● Skeleton /hardware/libhardware/modules/hwcomposer.cpp
● /system/lib/hw/hwcomposer.BOARD.so
● /system/lib/hw/gralloc.BOARD.so
● Ex. - Mot Xoom:
● hwcomposer.tegra.so
● gralloc.tegra.so
● Surface Flinger hook:
● /frameworks/native/services/surfaceflinger/DisplayHardware
– HWComposer.cpp
– Provides fake vsync if none is provided in HW
22
3.7. Surface Flinger
● Actual server:
● /frameworks/native/services/surfaceflinger
● Client side:
● /frameworks/native/libs/gui
● Client / Server interface:
● ISurfaceComposerClient.cpp
● ISurfaceComposer.cpp
● This is NOT an aidl'ed service
● All communication is manually
marshalled/unmarshalled
23
3.8. Window Manager
● Server side:
● /frameworks/base/services/java/com/android/server/wm/
– WindowManagerService.java
– Session.java
● Client side:
● /frameworks/base/core/java/android/view/
– WindowManager.java
– WindowManagerImpl.java
– ViewRootImpl.java
● Interfaces:
● IWindowManager.aidl
● IWindowSession.aidl
● Parameters (incl. z-order):
● See WindowManager.java
24
3.9. Walkthrough
● Activity Manager relies on Activity Thread
● AT calls on attach() and makeVisible()
● makeVisible does wm.addView()
● wm.addView() - this also called by StatusBar to display itself
● Creates a new ViewRootImpl
● Call on its setView()
● setView() calls on mWindowSession.addToDisplay(...)
● This results in call to WM's addWindow()
● ViewRootImpl's performTraversals()
● Calls on relayoutWindow()
● Calls to WM session's relayout()
● Call to WM's relayoutWindow()
● Call to createSurfaceLocked()
● new Surface(...)
25
4. OpenGL
● What's OpenGL?
● What's in a modern-day GPU?
● Software layers involved
● Kernel driver
● EGL libs
● Native interface
● Java interface
● Software GL implementation
26
4.1. What's OpenGL?
● It's just an API ... nothing but an API ...
● Check out Wikipedia
● Multiple versions out
● “ES” versions for embedded use
● Up to ES 3.0
● Android support up to ES 2.0
27
4.2. What's in a modern-day GPU?
● A tremendous amount of parallel processing units
● “SIMD”-like instruction set
● Video decoding/encoding capabilities
● ...
28
4.3. Software layers involved
● Kernel driver
● GL libraries
● Native GL API
● Java GL API
29
4.4. Kernel driver
PROPRIETARY
30
4.5. EGL libs
● /frameworks/base/native/opengl/libs
● Entry point: /system/lib/libEGL.so
●
Looks for /system/lib/egl/egl.cfg
● /system/lib/egl - Mot Xoom:
● egl.cfg
● libEGL_perfhud.so
● libEGL_tegra.so
● libGLES_android.so
● libGLESv1_CM_perfhud.so
● libGLESv1_CM_tegra.so
● libGLESv2_perfhud.so
● libGLESv2_tegra.so
●
elg.cfg:
0 0 tegra
31
4.6. Native interface
● /frameworks/native/opengl/include
● EGL
● ETC1
● GLES
● GLES2
● KHR
32
4.7. Java interface
● GL libs required by libandroid_runtime.so
● /frameworks/base/opengl/java/android/opengl:
● ...
33
4.8. Software GL implementation
● /frameworks/native/opengl/libagl
34
5. Input Layer
● Kernel side - “std” Linux input layer:
● /dev/input/*
● No HAL use
● Native lib:
● libinput
● /frameworks/base/services/input
● Input Manager Service:
● /frameworks/base/services/java/com/android/server/input
● Started and directly tied to Window Manager
● Specific config files (see source.android.com)
● Soft keyboard:
● /frameworks/base/core/java/android/inputmethodservice
● Input methods:
● /packages/inputmehods
● http://developer.android.com/guide/topics/text/creating-input-method.html
35
6. Relevant Apps and Services
● Launcher
● StatusBar
● Wallpaper Manager Service
● Notification Service
● App Widgets
36
6.1. Launcher
● An app like any other
● See /packages/app/Launcher2
37
6.2. StatusBar
● A unique app
● See /frameworks/base/packages/SystemUI
● Connects to Status Bar Manager and gives an
interface it can use to call back into Status Bar
● Can use setIcon() to display icons on the right
● Provides a CPU usage add-on that renders
straight on rest of display using higher z-order
38
6.3. Wallpaper Manager Service
● See
/frameworks/base/services/java/com/android/se
rver/WallpaperManagerService.java
39
6.4. Notification Service
● Toasts
● Status bar notifications
● Gets handle to Status Bar Service at
instantiation
● Uses handle to communicate with Status Bar
40
6.5. App Widgets
● See
/frameworks/base/services/java/com/android/se
rver/AppWidgetService.java
41
7. System Startup
● Kernel
● Init
● Boot animation
● Launcher
42
7.1. Boot animation
● Started by Surface Flinger
● “bootanim” binary
● /frameworks/base/cmds/bootanimation
● Relies on bootanimation.zip w/ PNGs (nothing but)
● See
https://github.com/CyanogenMod/android_vendor_cm/tree/jellybean/pre
built/common/bootanimatino
● Must contain a desc.txt:
<width> <height> <fps>
p <count> <pause> <path>
p <count> <pause> <path>
43
8. References and Pointers
● “Use the source, Luke”
● Jim Huang's “Android Graphics”
● Benjamin Zores' “Linux Magazine / France” articles
● MIPS article on graphics internals:
● http://developer.mips.com/2012/04/11/learning-about-
android-graphics-subsystem/
● Stéphane Marchesin's “Linux Graphics Drivers: an
Introduction”
● http://source.android.com/tech/input/index.html
44
Thank you ...
karim.yaghmour@opersys.com

Weitere ähnliche Inhalte

Was ist angesagt?

Customizing Android's UI
Customizing Android's UICustomizing Android's UI
Customizing Android's UIOpersys inc.
 
Android Platform Debugging and Development
Android Platform Debugging and DevelopmentAndroid Platform Debugging and Development
Android Platform Debugging and DevelopmentOpersys inc.
 
Android Platform Debugging and Development
Android Platform Debugging and DevelopmentAndroid Platform Debugging and Development
Android Platform Debugging and DevelopmentOpersys inc.
 
Headless Android at AnDevCon3
Headless Android at AnDevCon3Headless Android at AnDevCon3
Headless Android at AnDevCon3Opersys inc.
 
Android Security Internals
Android Security InternalsAndroid Security Internals
Android Security InternalsOpersys inc.
 
Extending Android's Platform Toolsuite
Extending Android's Platform ToolsuiteExtending Android's Platform Toolsuite
Extending Android's Platform ToolsuiteOpersys inc.
 
Android Platform Debugging and Development
Android Platform Debugging and DevelopmentAndroid Platform Debugging and Development
Android Platform Debugging and DevelopmentOpersys inc.
 
Android Platform Debugging and Development
Android Platform Debugging and DevelopmentAndroid Platform Debugging and Development
Android Platform Debugging and DevelopmentOpersys inc.
 
Developing Android Platform Tools
Developing Android Platform ToolsDeveloping Android Platform Tools
Developing Android Platform ToolsOpersys inc.
 
Brillo / Weave Internals
Brillo / Weave InternalsBrillo / Weave Internals
Brillo / Weave InternalsOpersys inc.
 
Android Platform Debugging and Development
Android Platform Debugging and DevelopmentAndroid Platform Debugging and Development
Android Platform Debugging and DevelopmentKarim Yaghmour
 
Android Jumpstart ESC SV 2012 Part I
Android Jumpstart ESC SV 2012 Part IAndroid Jumpstart ESC SV 2012 Part I
Android Jumpstart ESC SV 2012 Part IOpersys inc.
 
Android's HIDL: Treble in the HAL
Android's HIDL: Treble in the HALAndroid's HIDL: Treble in the HAL
Android's HIDL: Treble in the HALOpersys inc.
 
Brillo / Weave Internals
Brillo / Weave InternalsBrillo / Weave Internals
Brillo / Weave InternalsOpersys inc.
 
Memory Management in Android
Memory Management in AndroidMemory Management in Android
Memory Management in AndroidOpersys inc.
 
Scheduling in Android
Scheduling in AndroidScheduling in Android
Scheduling in AndroidOpersys inc.
 
Embedded Android Workshop with Marshmallow
Embedded Android Workshop with MarshmallowEmbedded Android Workshop with Marshmallow
Embedded Android Workshop with MarshmallowKarim Yaghmour
 
Android Things Internals
Android Things InternalsAndroid Things Internals
Android Things InternalsOpersys inc.
 
Android Platform Debugging and Development
Android Platform Debugging and DevelopmentAndroid Platform Debugging and Development
Android Platform Debugging and DevelopmentOpersys inc.
 
Embedded Android Workshop with Marshmallow
Embedded Android Workshop with MarshmallowEmbedded Android Workshop with Marshmallow
Embedded Android Workshop with MarshmallowOpersys inc.
 

Was ist angesagt? (20)

Customizing Android's UI
Customizing Android's UICustomizing Android's UI
Customizing Android's UI
 
Android Platform Debugging and Development
Android Platform Debugging and DevelopmentAndroid Platform Debugging and Development
Android Platform Debugging and Development
 
Android Platform Debugging and Development
Android Platform Debugging and DevelopmentAndroid Platform Debugging and Development
Android Platform Debugging and Development
 
Headless Android at AnDevCon3
Headless Android at AnDevCon3Headless Android at AnDevCon3
Headless Android at AnDevCon3
 
Android Security Internals
Android Security InternalsAndroid Security Internals
Android Security Internals
 
Extending Android's Platform Toolsuite
Extending Android's Platform ToolsuiteExtending Android's Platform Toolsuite
Extending Android's Platform Toolsuite
 
Android Platform Debugging and Development
Android Platform Debugging and DevelopmentAndroid Platform Debugging and Development
Android Platform Debugging and Development
 
Android Platform Debugging and Development
Android Platform Debugging and DevelopmentAndroid Platform Debugging and Development
Android Platform Debugging and Development
 
Developing Android Platform Tools
Developing Android Platform ToolsDeveloping Android Platform Tools
Developing Android Platform Tools
 
Brillo / Weave Internals
Brillo / Weave InternalsBrillo / Weave Internals
Brillo / Weave Internals
 
Android Platform Debugging and Development
Android Platform Debugging and DevelopmentAndroid Platform Debugging and Development
Android Platform Debugging and Development
 
Android Jumpstart ESC SV 2012 Part I
Android Jumpstart ESC SV 2012 Part IAndroid Jumpstart ESC SV 2012 Part I
Android Jumpstart ESC SV 2012 Part I
 
Android's HIDL: Treble in the HAL
Android's HIDL: Treble in the HALAndroid's HIDL: Treble in the HAL
Android's HIDL: Treble in the HAL
 
Brillo / Weave Internals
Brillo / Weave InternalsBrillo / Weave Internals
Brillo / Weave Internals
 
Memory Management in Android
Memory Management in AndroidMemory Management in Android
Memory Management in Android
 
Scheduling in Android
Scheduling in AndroidScheduling in Android
Scheduling in Android
 
Embedded Android Workshop with Marshmallow
Embedded Android Workshop with MarshmallowEmbedded Android Workshop with Marshmallow
Embedded Android Workshop with Marshmallow
 
Android Things Internals
Android Things InternalsAndroid Things Internals
Android Things Internals
 
Android Platform Debugging and Development
Android Platform Debugging and DevelopmentAndroid Platform Debugging and Development
Android Platform Debugging and Development
 
Embedded Android Workshop with Marshmallow
Embedded Android Workshop with MarshmallowEmbedded Android Workshop with Marshmallow
Embedded Android Workshop with Marshmallow
 

Ähnlich wie Inside Android's UI at AnDevCon IV

Android's Multimedia Framework
Android's Multimedia FrameworkAndroid's Multimedia Framework
Android's Multimedia FrameworkOpersys inc.
 
Inside Android's UI
Inside Android's UIInside Android's UI
Inside Android's UIOpersys inc.
 
Customizing Android's UI
Customizing Android's UICustomizing Android's UI
Customizing Android's UIOpersys inc.
 
Inside Android's UI at AnDevCon V
Inside Android's UI at AnDevCon VInside Android's UI at AnDevCon V
Inside Android's UI at AnDevCon VOpersys inc.
 
Android Platform Debugging and Development
Android Platform Debugging and DevelopmentAndroid Platform Debugging and Development
Android Platform Debugging and DevelopmentOpersys inc.
 
Android Platform Debugging and Development
Android Platform Debugging and DevelopmentAndroid Platform Debugging and Development
Android Platform Debugging and DevelopmentOpersys inc.
 
Android Platform Debugging and Development at ABS 2014
Android Platform Debugging and Development at ABS 2014Android Platform Debugging and Development at ABS 2014
Android Platform Debugging and Development at ABS 2014Opersys inc.
 
Embedded Android Workshop part I ESC SV 2012
Embedded Android Workshop part I ESC SV 2012Embedded Android Workshop part I ESC SV 2012
Embedded Android Workshop part I ESC SV 2012Opersys inc.
 
Leveraging Android's Linux Heritage at AnDevCon V
Leveraging Android's Linux Heritage at AnDevCon VLeveraging Android's Linux Heritage at AnDevCon V
Leveraging Android's Linux Heritage at AnDevCon VOpersys inc.
 
Android Platform Debugging and Development at ELCE 2013
Android Platform Debugging and Development at ELCE 2013Android Platform Debugging and Development at ELCE 2013
Android Platform Debugging and Development at ELCE 2013Opersys inc.
 
Android Security, From the Ground Up
Android Security, From the Ground UpAndroid Security, From the Ground Up
Android Security, From the Ground UpOpersys inc.
 
Running Code in the Android Stack at ELCE 2013
Running Code in the Android Stack at ELCE 2013Running Code in the Android Stack at ELCE 2013
Running Code in the Android Stack at ELCE 2013Opersys inc.
 
Leveraging Android's Linux Heritage at AnDevCon IV
Leveraging Android's Linux Heritage at AnDevCon IVLeveraging Android's Linux Heritage at AnDevCon IV
Leveraging Android's Linux Heritage at AnDevCon IVOpersys inc.
 
Working with the AOSP - Linaro Connect Asia 2013
Working with the AOSP - Linaro Connect Asia 2013Working with the AOSP - Linaro Connect Asia 2013
Working with the AOSP - Linaro Connect Asia 2013Opersys inc.
 
Android for Embedded Linux Developers
Android for Embedded Linux DevelopersAndroid for Embedded Linux Developers
Android for Embedded Linux DevelopersOpersys inc.
 
Is Android the New King of Embedded OSes at Embedded World 2014
Is Android the New King of Embedded OSes at Embedded World 2014Is Android the New King of Embedded OSes at Embedded World 2014
Is Android the New King of Embedded OSes at Embedded World 2014Opersys inc.
 
Android Things: Android for IoT
Android Things: Android for IoTAndroid Things: Android for IoT
Android Things: Android for IoTOpersys inc.
 
Android Things Internals
Android Things InternalsAndroid Things Internals
Android Things InternalsOpersys inc.
 
Is Android the New Embedded Linux? at AnDevCon VI
Is Android the New Embedded Linux? at AnDevCon VIIs Android the New Embedded Linux? at AnDevCon VI
Is Android the New Embedded Linux? at AnDevCon VIOpersys inc.
 

Ähnlich wie Inside Android's UI at AnDevCon IV (20)

Android's Multimedia Framework
Android's Multimedia FrameworkAndroid's Multimedia Framework
Android's Multimedia Framework
 
Inside Android's UI
Inside Android's UIInside Android's UI
Inside Android's UI
 
Android Platform Debugging & Development
Android Platform Debugging & Development Android Platform Debugging & Development
Android Platform Debugging & Development
 
Customizing Android's UI
Customizing Android's UICustomizing Android's UI
Customizing Android's UI
 
Inside Android's UI at AnDevCon V
Inside Android's UI at AnDevCon VInside Android's UI at AnDevCon V
Inside Android's UI at AnDevCon V
 
Android Platform Debugging and Development
Android Platform Debugging and DevelopmentAndroid Platform Debugging and Development
Android Platform Debugging and Development
 
Android Platform Debugging and Development
Android Platform Debugging and DevelopmentAndroid Platform Debugging and Development
Android Platform Debugging and Development
 
Android Platform Debugging and Development at ABS 2014
Android Platform Debugging and Development at ABS 2014Android Platform Debugging and Development at ABS 2014
Android Platform Debugging and Development at ABS 2014
 
Embedded Android Workshop part I ESC SV 2012
Embedded Android Workshop part I ESC SV 2012Embedded Android Workshop part I ESC SV 2012
Embedded Android Workshop part I ESC SV 2012
 
Leveraging Android's Linux Heritage at AnDevCon V
Leveraging Android's Linux Heritage at AnDevCon VLeveraging Android's Linux Heritage at AnDevCon V
Leveraging Android's Linux Heritage at AnDevCon V
 
Android Platform Debugging and Development at ELCE 2013
Android Platform Debugging and Development at ELCE 2013Android Platform Debugging and Development at ELCE 2013
Android Platform Debugging and Development at ELCE 2013
 
Android Security, From the Ground Up
Android Security, From the Ground UpAndroid Security, From the Ground Up
Android Security, From the Ground Up
 
Running Code in the Android Stack at ELCE 2013
Running Code in the Android Stack at ELCE 2013Running Code in the Android Stack at ELCE 2013
Running Code in the Android Stack at ELCE 2013
 
Leveraging Android's Linux Heritage at AnDevCon IV
Leveraging Android's Linux Heritage at AnDevCon IVLeveraging Android's Linux Heritage at AnDevCon IV
Leveraging Android's Linux Heritage at AnDevCon IV
 
Working with the AOSP - Linaro Connect Asia 2013
Working with the AOSP - Linaro Connect Asia 2013Working with the AOSP - Linaro Connect Asia 2013
Working with the AOSP - Linaro Connect Asia 2013
 
Android for Embedded Linux Developers
Android for Embedded Linux DevelopersAndroid for Embedded Linux Developers
Android for Embedded Linux Developers
 
Is Android the New King of Embedded OSes at Embedded World 2014
Is Android the New King of Embedded OSes at Embedded World 2014Is Android the New King of Embedded OSes at Embedded World 2014
Is Android the New King of Embedded OSes at Embedded World 2014
 
Android Things: Android for IoT
Android Things: Android for IoTAndroid Things: Android for IoT
Android Things: Android for IoT
 
Android Things Internals
Android Things InternalsAndroid Things Internals
Android Things Internals
 
Is Android the New Embedded Linux? at AnDevCon VI
Is Android the New Embedded Linux? at AnDevCon VIIs Android the New Embedded Linux? at AnDevCon VI
Is Android the New Embedded Linux? at AnDevCon VI
 

Mehr von Opersys inc.

Android Automotive
Android AutomotiveAndroid Automotive
Android AutomotiveOpersys inc.
 
Android 10 Internals Update
Android 10 Internals UpdateAndroid 10 Internals Update
Android 10 Internals UpdateOpersys inc.
 
Embedded Android Workshop with Pie
Embedded Android Workshop with PieEmbedded Android Workshop with Pie
Embedded Android Workshop with PieOpersys inc.
 
Embedded Android Workshop with Oreo
Embedded Android Workshop with OreoEmbedded Android Workshop with Oreo
Embedded Android Workshop with OreoOpersys inc.
 
Scheduling in Android
Scheduling in AndroidScheduling in Android
Scheduling in AndroidOpersys inc.
 
Embedded Android Workshop with Nougat
Embedded Android Workshop with NougatEmbedded Android Workshop with Nougat
Embedded Android Workshop with NougatOpersys inc.
 
Embedded Android Workshop with Nougat
Embedded Android Workshop with NougatEmbedded Android Workshop with Nougat
Embedded Android Workshop with NougatOpersys inc.
 
Memory Management in Android
Memory Management in AndroidMemory Management in Android
Memory Management in AndroidOpersys inc.
 
Embedded Android Workshop with Nougat
Embedded Android Workshop with NougatEmbedded Android Workshop with Nougat
Embedded Android Workshop with NougatOpersys inc.
 
Brillo/Weave Internals
Brillo/Weave InternalsBrillo/Weave Internals
Brillo/Weave InternalsOpersys inc.
 
Embedded Android Workshop with Marshmallow
Embedded Android Workshop with MarshmallowEmbedded Android Workshop with Marshmallow
Embedded Android Workshop with MarshmallowOpersys inc.
 
Memory Management in Android
Memory Management in AndroidMemory Management in Android
Memory Management in AndroidOpersys inc.
 

Mehr von Opersys inc. (14)

Android Automotive
Android AutomotiveAndroid Automotive
Android Automotive
 
Android 10 Internals Update
Android 10 Internals UpdateAndroid 10 Internals Update
Android 10 Internals Update
 
Embedded Android Workshop with Pie
Embedded Android Workshop with PieEmbedded Android Workshop with Pie
Embedded Android Workshop with Pie
 
Embedded Android Workshop with Oreo
Embedded Android Workshop with OreoEmbedded Android Workshop with Oreo
Embedded Android Workshop with Oreo
 
Scheduling in Android
Scheduling in AndroidScheduling in Android
Scheduling in Android
 
Embedded Android Workshop with Nougat
Embedded Android Workshop with NougatEmbedded Android Workshop with Nougat
Embedded Android Workshop with Nougat
 
Embedded Android Workshop with Nougat
Embedded Android Workshop with NougatEmbedded Android Workshop with Nougat
Embedded Android Workshop with Nougat
 
Memory Management in Android
Memory Management in AndroidMemory Management in Android
Memory Management in Android
 
Embedded Android Workshop with Nougat
Embedded Android Workshop with NougatEmbedded Android Workshop with Nougat
Embedded Android Workshop with Nougat
 
Project Ara
Project AraProject Ara
Project Ara
 
Brillo/Weave Internals
Brillo/Weave InternalsBrillo/Weave Internals
Brillo/Weave Internals
 
Embedded Android Workshop with Marshmallow
Embedded Android Workshop with MarshmallowEmbedded Android Workshop with Marshmallow
Embedded Android Workshop with Marshmallow
 
Memory Management in Android
Memory Management in AndroidMemory Management in Android
Memory Management in Android
 
Project Ara
Project AraProject Ara
Project Ara
 

Kürzlich hochgeladen

Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsTop 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsRoshan Dwivedi
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...Martijn de Jong
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoffsammart93
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processorsdebabhi2
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century educationjfdjdjcjdnsjd
 
Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024SynarionITSolutions
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyKhushali Kathiriya
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...apidays
 
Manulife - Insurer Innovation Award 2024
Manulife - Insurer Innovation Award 2024Manulife - Insurer Innovation Award 2024
Manulife - Insurer Innovation Award 2024The Digital Insurer
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businesspanagenda
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUK Journal
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?Igalia
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingEdi Saputra
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdflior mazor
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfsudhanshuwaghmare1
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MIND CTI
 

Kürzlich hochgeladen (20)

Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsTop 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Manulife - Insurer Innovation Award 2024
Manulife - Insurer Innovation Award 2024Manulife - Insurer Innovation Award 2024
Manulife - Insurer Innovation Award 2024
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 

Inside Android's UI at AnDevCon IV

  • 1. 1 Inside Android's UI ADC IV Karim Yaghmour @karimyaghmour karim.yaghmour@opersys.com
  • 2. 2 These slides are made available to you under a Creative Commons Share- Alike 3.0 license. The full terms of this license are here: https://creativecommons.org/licenses/by-sa/3.0/ Attribution requirements and misc., PLEASE READ: ● This slide must remain as-is in this specific location (slide #2), everything else you are free to change; including the logo :-) ● Use of figures in other documents must feature the below “Originals at” URL immediately under that figure and the below copyright notice where appropriate. ● You are free to fill in the “Delivered and/or customized by” space on the right as you see fit. ● You are FORBIDEN from using the default “About” slide as-is or any of its contents. ● You are FORBIDEN from using any content provided by 3rd parties without the EXPLICIT consent from those parties. (C) Copyright 2012, Opersys inc. These slides created by: Karim Yaghmour Originals at: www.opersys.com/community/docs Delivered and/or customized by
  • 3. 3 About ● Author of: ● Introduced Linux Trace Toolkit in 1999 ● Originated Adeos and relayfs (kernel/relay.c) ● Training, Custom Dev, Consulting, ...
  • 4. 4 Agenda ● Android's UI, what's that? ● Architecture Basics ● Display Core ● OpenGL ● Input Layer ● Relevant Apps and Services ● System Startup ● References and Pointers
  • 5. 5 1. Android's UI, what's that? ●SoC / GPU ●Touch input ●LCD ●Keyboard ???
  • 6. 6 1.1. What I'm NOT covering ● Media layer ● StageFright ● Video playback ● Camera ● DRM ● Etc.
  • 7. 7 2. Architecture Basics ● Hardware used to run Android ● AOSP ● Binder ● System Services ● HAL
  • 8. 8
  • 9. 9
  • 10. 10
  • 11. 11
  • 12. 12
  • 14. 14 3. Display Core ● Display Hardware ● Classic Linux display stack ● Display stack in Android ● Kernel driver ● HAL definition ● HAL module ● Surface Flinger ● Window Manager ● Walkthrough
  • 16. 16 3.2. Classic Linux display stack
  • 17. 17 3.3. Display stack in Android
  • 18. 18
  • 19. 19 3.4. Kernel driver ● Video memory management ● Mode setting ● Checking of parameters ● Motorola Xoom: ● /dev/nvhdcp1 ● /dev/nvhost-ctrl ● /dev/nvhost-display ● /dev/nvhost-dsi ● /dev/nvhost-gr2d ● /dev/nvhost-gr3d ● /dev/nvhost-isp ● /dev/nvhost-mpe ● /dev/nvhost-vi ● /dev/nvmap ● /dev/tegra-crypto ● /dev/tegra_avp ● /dev/tegra_camera ● /dev/tegra_fuse ● /dev/tegra_rpc ● /dev/tegra_sema ● ... whatever hides in hwcomposer HAL module
  • 20. 20 3.5. HAL Definition ● /hardware/libhardware/include/hardware/hwcomposer.h ● struct hwc_procs: ● invalidate() ● vsync() ● struct hwc_composer_device: ● prepare() ● set() ● dump() ● registerProcs() ● query() ● *()
  • 21. 21 3.6. HAL module ● Skeleton /hardware/libhardware/modules/hwcomposer.cpp ● /system/lib/hw/hwcomposer.BOARD.so ● /system/lib/hw/gralloc.BOARD.so ● Ex. - Mot Xoom: ● hwcomposer.tegra.so ● gralloc.tegra.so ● Surface Flinger hook: ● /frameworks/native/services/surfaceflinger/DisplayHardware – HWComposer.cpp – Provides fake vsync if none is provided in HW
  • 22. 22 3.7. Surface Flinger ● Actual server: ● /frameworks/native/services/surfaceflinger ● Client side: ● /frameworks/native/libs/gui ● Client / Server interface: ● ISurfaceComposerClient.cpp ● ISurfaceComposer.cpp ● This is NOT an aidl'ed service ● All communication is manually marshalled/unmarshalled
  • 23. 23 3.8. Window Manager ● Server side: ● /frameworks/base/services/java/com/android/server/wm/ – WindowManagerService.java – Session.java ● Client side: ● /frameworks/base/core/java/android/view/ – WindowManager.java – WindowManagerImpl.java – ViewRootImpl.java ● Interfaces: ● IWindowManager.aidl ● IWindowSession.aidl ● Parameters (incl. z-order): ● See WindowManager.java
  • 24. 24 3.9. Walkthrough ● Activity Manager relies on Activity Thread ● AT calls on attach() and makeVisible() ● makeVisible does wm.addView() ● wm.addView() - this also called by StatusBar to display itself ● Creates a new ViewRootImpl ● Call on its setView() ● setView() calls on mWindowSession.addToDisplay(...) ● This results in call to WM's addWindow() ● ViewRootImpl's performTraversals() ● Calls on relayoutWindow() ● Calls to WM session's relayout() ● Call to WM's relayoutWindow() ● Call to createSurfaceLocked() ● new Surface(...)
  • 25. 25 4. OpenGL ● What's OpenGL? ● What's in a modern-day GPU? ● Software layers involved ● Kernel driver ● EGL libs ● Native interface ● Java interface ● Software GL implementation
  • 26. 26 4.1. What's OpenGL? ● It's just an API ... nothing but an API ... ● Check out Wikipedia ● Multiple versions out ● “ES” versions for embedded use ● Up to ES 3.0 ● Android support up to ES 2.0
  • 27. 27 4.2. What's in a modern-day GPU? ● A tremendous amount of parallel processing units ● “SIMD”-like instruction set ● Video decoding/encoding capabilities ● ...
  • 28. 28 4.3. Software layers involved ● Kernel driver ● GL libraries ● Native GL API ● Java GL API
  • 30. 30 4.5. EGL libs ● /frameworks/base/native/opengl/libs ● Entry point: /system/lib/libEGL.so ● Looks for /system/lib/egl/egl.cfg ● /system/lib/egl - Mot Xoom: ● egl.cfg ● libEGL_perfhud.so ● libEGL_tegra.so ● libGLES_android.so ● libGLESv1_CM_perfhud.so ● libGLESv1_CM_tegra.so ● libGLESv2_perfhud.so ● libGLESv2_tegra.so ● elg.cfg: 0 0 tegra
  • 31. 31 4.6. Native interface ● /frameworks/native/opengl/include ● EGL ● ETC1 ● GLES ● GLES2 ● KHR
  • 32. 32 4.7. Java interface ● GL libs required by libandroid_runtime.so ● /frameworks/base/opengl/java/android/opengl: ● ...
  • 33. 33 4.8. Software GL implementation ● /frameworks/native/opengl/libagl
  • 34. 34 5. Input Layer ● Kernel side - “std” Linux input layer: ● /dev/input/* ● No HAL use ● Native lib: ● libinput ● /frameworks/base/services/input ● Input Manager Service: ● /frameworks/base/services/java/com/android/server/input ● Started and directly tied to Window Manager ● Specific config files (see source.android.com) ● Soft keyboard: ● /frameworks/base/core/java/android/inputmethodservice ● Input methods: ● /packages/inputmehods ● http://developer.android.com/guide/topics/text/creating-input-method.html
  • 35. 35 6. Relevant Apps and Services ● Launcher ● StatusBar ● Wallpaper Manager Service ● Notification Service ● App Widgets
  • 36. 36 6.1. Launcher ● An app like any other ● See /packages/app/Launcher2
  • 37. 37 6.2. StatusBar ● A unique app ● See /frameworks/base/packages/SystemUI ● Connects to Status Bar Manager and gives an interface it can use to call back into Status Bar ● Can use setIcon() to display icons on the right ● Provides a CPU usage add-on that renders straight on rest of display using higher z-order
  • 38. 38 6.3. Wallpaper Manager Service ● See /frameworks/base/services/java/com/android/se rver/WallpaperManagerService.java
  • 39. 39 6.4. Notification Service ● Toasts ● Status bar notifications ● Gets handle to Status Bar Service at instantiation ● Uses handle to communicate with Status Bar
  • 40. 40 6.5. App Widgets ● See /frameworks/base/services/java/com/android/se rver/AppWidgetService.java
  • 41. 41 7. System Startup ● Kernel ● Init ● Boot animation ● Launcher
  • 42. 42 7.1. Boot animation ● Started by Surface Flinger ● “bootanim” binary ● /frameworks/base/cmds/bootanimation ● Relies on bootanimation.zip w/ PNGs (nothing but) ● See https://github.com/CyanogenMod/android_vendor_cm/tree/jellybean/pre built/common/bootanimatino ● Must contain a desc.txt: <width> <height> <fps> p <count> <pause> <path> p <count> <pause> <path>
  • 43. 43 8. References and Pointers ● “Use the source, Luke” ● Jim Huang's “Android Graphics” ● Benjamin Zores' “Linux Magazine / France” articles ● MIPS article on graphics internals: ● http://developer.mips.com/2012/04/11/learning-about- android-graphics-subsystem/ ● Stéphane Marchesin's “Linux Graphics Drivers: an Introduction” ● http://source.android.com/tech/input/index.html