Android Internals - A Confectioner's CookBook
Volume I : The Power User's View
Major rewrite for Android 11/12 - 440 pages, complete (now with Config chapter, too)
Changes/additions with respect to 1st edition shown in yellow
The original (ISBN: 978-0-9910555-2-4) remains free on this website (Thank you, CIA).
The book underwent a complete rewrite (effectively, a 2nd edition). Updates to align with Android 11, and latest SD865, Exynos, MediaTek and Kirin devices
- About this book
- Overview and Reading Suggestion
- The AOSP
- Experiments
- Tools
- Conventions Used in this Book
- The Companion WebSite
- Introduction and evolution of the Android Architecture
A technical overview of the Android architecture, evolution of its features, and forked derivatives
- Android Versions - From Cupcake to
MarshmallowR (11.0) - The Android Architecture
- Android vs. Linux: Notable differences
- Not just another Linux distribution
- And then came Android
- Commonalities and Divergences from Linux
- Obtaining and compiling Android
- The sources of Android Code
- AOSP
- AOSP external
- The Linux kernel
- Platform/BSP (vendor)
- ODM
- Carrier
- A high level view of the Android Architecture
- Applications
- The Android Frameworks
- Dalvik/Android Runtime
- JNI
- Native Binaries
- Native Libraries
- Bionic
- Omissions
- Additions
- Porting Challenges
- The Hardware Abstraction Layer (HAL)
- The Traditional HAL (2.2-8.0)
- The Treble HAL (8.0+)
- The Linux kernel
- Linux kernel features
- Androidisms
- Drivers and Modules
- Generic Kernel Image (GKI)
- Android Derivatives
- FireOS, FireTV
- Android Wear
- Android Auto & Automotive
Google Glass- Headless Android
- Pondering the way ahead
- Android Versions - From Cupcake to
- Hardware
A new chapter providing a detailed introduction to the hardware of Android devices, with an emphasis on hardware abstraction and Project Treble compliance
- The ARM architecture
- Aarch32 and Aarch64
- ARM architecture revisions
- Devices
- System on Chip (SoC) overview
- SoC vendors
- Qualcomm (Snapdragon)
- Samsung (Exynos)
- Huawei (Kirin)
- MediaTek (MTK)
- The Device Tree
- Firmware images
- The ARM architecture
- Partitions & Filesystems
Examining Android storage types and partitions,
as well a detailed breakdown of directories and files in./system and/data - Partitioning scheme
- The Need for Separate Partitions
Flash Storage Systems- GUID Partitioning (GPT)
- A/B[/C] Slotted devices
- Dynamic partitioning (super.img)
lpdumpd
(Android 10+)
- Android Device Partitions
- Mountable Partitions
- System-as-root
- Supported filesystem types
/vendor ,/odm ,/product - Division of responsibility
- Non-Mountable Partitions
- boot, vendor_boot and recovery
- dtbo
- frp
- misc
- vbmeta
- Mountable Partitions
- Chipset-specific Partitions
- Qualcomm
- cdt
- devinfo
- splash
- Samsung
- Huawei
- MediaTek
- Qualcomm
- Linux Pseudo-Filesystems
- bpf (/sys/fs/bpf)
- cgroupfs
- configfs
- debugfs
- functionfs (/dev/usb-ffs/adb)
- FUSE
- incrementalfs (11.0)
- overlayfs
- procfs (/proc)
- pstore (/sys/fs/pstore)
- sdcardfs/esdfs
- securityfs (/sys/fs/selinux)
- sysfs (/sys)
- tmpfs
- tracefs (/sys/kernel/debug/tracing))
- Partitioning scheme
- Files & Directories
A detailed breakdown of directories and files Android, up to and including Android 11 (Spun off from first edition's discussion in Partitions & Filesystems chapter, and greatly expanded).
- The Root Filesystem
- /system
- /system/bin
- /system/lib[64]
- Core Libraries
- Other system libraries
- Framework support libraries
- External native libraries
- /system/etc
- /vendor
- /vendor/bin
- Qualcomm specific binaries
- Huawei specific binaries
- Samsung specific binaries
- MediaTek specific binaries
- /vendor/bin
- /data (Excerpt)
- /data/data
- /data/misc
- /data/misc_ce and /data/misc_de
- /data/system
- /data/system_ce and /data/system_de
- /data/vendor
- /cache
- Storage Management
Refactoring vold and OBB/ASEC, as well as adding new content on Storage* and APEX
- Mounting
- Mount options
- Loop mounting
- Bind mounting
- Mount namespaces
- fs_mgr
- The
fstab files - External Storage
- Portable Storage
- Adoptable Storage
- Scoped Storage (Android 10)
- Incremental FS (Android 11)
- Daemons
- vold
- StorageManager
- storaged
- storagestats
- devicestoragemonitor
- Protected filesystems
- Obb - Opaque Binary Blobs
- ASec - Android Secure Storage
- APEX - Android Pony EXpress (Android 10)
- apexd
- Execution Flow
- Additional Command Line Arguments
- The AIDL interface
- APEX and the linker configuration
- Android 11.0 modifications
- Mounting
- Android System Images & Updates
Split from the older Chapter III, and greatly expanded
- Factory Images and OTA updates
- Factory Images
- OTA packages
- Samsung OTA
- Standardized Payload Formats
- Android Sparse Images
- super[_empty].img
- Block Based Updates (transfer.[dat|list])
- Mounting Filesystem Images
- Android boot.img
- Vendor boot.img (11.0, GKI)
- Updates
- Fastboot
- Samsung: ODIN
- Updates via recovery
- The
update_binary
- Updates on slotted (A/B) devices
- update_engine
- Generic System Images (Android 9+)
- gsid (Android 10+
- Dynamic System Update (DSU)
- The
dynamic_system
service
- Factory Images and OTA updates
- The Android boot process
Generalizing the Android Boot process amongst vendors, and then focusing on vendor specifics
- The Boot ROM/PBL
- Second Stage/eXtensible Boot Loader
- Qualcomm (SD835+) UEFI Loader
- Samsung S-BOOT
- MediaTek Preloader
- The Android Boot Loader
- Little Kernel (32-bit, ARMv7 and ARMv8 non Qualcomm UEFI)
- (Generalized) LK execution flow
- LinuxLoader (Qualcomm UEFI)
- Boot loader locking
- The Linux Kernel
- Kernel Boot
- The RAM Disk (initramfs)
- The Boot Control HAL
- User mode startup - init and Zygote
- Init
- as watchdogd
- mounting filesystems
- System Properties
- Accessing properties
- Special namespace prefixes
- Property files
- PropertyInit()
- The property store
- The property_service
- The rc files
- Triggers, actions, and services
- init.rc syntax and command set (updated for 11.0)
- Command syntax
- Service option syntax
- Keychords
- Putting it all together
- Zygote
- Design Rationale
- Zygote32, Zygote64 and webview_zygote
- UnSpecialized Application Processes (USAPS, Android 10)
- Android Daemons, at a glance
The Android Runtime services: Native Services chapter (formerly Chapter 5), has been removed, as now all daemons are covered, but each within its context
- Init
- The Android Service Architecture
- The Service call pattern
- Binder (an overview)
- A little history
- So what, exactly, is Binder?
- Using Binder
- 8.0+: The
vndbinder
andhwbinder
- Tracing Binder (bindump, etc)
- Service Manager, revisited
- The system_server architecture
- Handling services
- Startup and Flow
- A bird's eye view of Android's services
- Configuration & Management
New chapter dealing with users, settings and more
- User Management
- The
user
service
- The
- Account Management
- The
accounts
database - The
account
service
- The
- Configuration Settings
config.xml and other files- Overlays
- The
device_config
service - Server Configurable Flags (10.0)
- The
settings
service - The
etc/sysconfig directories - The
system_config
service (11.0)
Mobile Device Management- Moved to Volume III- Work profiles
- The
device_policy
service - The
restrictions
service
- User Management
- Android Applications through Linux Lens
Monitoring and viewing Applications through the Linux command line
- Application during runtime (with /proc/task/..)
- Processes and threads in Android (and Linux)
- User mode memory management
- USS, PSS, RSS, VSS, etc
- procrank, librank, and /proc/../smaps
- Native binaries, libraries and ELF Tools
- Optimizations in Android native and Dalvik apps
- Logging, Statistics & Monitoring
- Android Logging
- logd
- Statistics
- statsd
- statscompanion
- The IStats HIDL
- Lesser Statistics Services
- Incident Reporting
- Vendor Diagnostics
- Qualcomm's Diag (/dev/diag)
- Debugging
- Monitoring
- inotify
ptrace(2)
-based toolsstrace
jtrace
- Using eBPF for tracing
- Android Logging
- Power Management
- Native APIs
- The
PowerManagerService
and Friends - Battery Monitoring
- Low-level CPU Control
- MultiCore
- Interrupt Affinity
- Governors
- Heterogeneous Multi-Processing (HMP) Scheduling
- Energy Aware Scheduling (EAS)
- Thermal Monitoring
- Linux kernel support
- Android support
hardware_properties
servicethermalservice
- The thermal HAL
- Vendor thermal support
- Qualcomm
- Samsung
- Huawei
- MediaTek
- Case study: Google Pixel
- The Power HAL interface
- Power Management Statistics
Volume II: The Developer View - Available ! 360 pages
- Building Android from the source
- The AOSP
- Androidxref.com, android.googlesource.com, cs.android.com
- A whirlwind tour of Android projects
- The NDK
- Android.mk and Android.bp (soong)
- Cross compiling with custom Makefiles
- The AOSP
- Android at a Native Level
- Bionic, in depth
- Native Level debugging, core dumps and tombstones
- Package Maintenance
- APK Components
- AndroidManifest.xml
- classes.dex
- resources.arsc
- Digital signatures on apps
- Runtime Resource Overlay (RRO)
- Package Installation
- Behind the scenes
- installd
- The package database
- Monitoring Packages
- Package statistics
- The
PackageManagerService
- APK snapshots & rollback
- APK Components
- Anatomy of an an Android Application
Break down and detail of APK and application components
- Application Components
- Activities
- Services
- Broadcast Receivers
- Content Providers
- JNI
- The need for native code
- Compiling JNI code
- Houdini - Briding the ARM/Intel divide
- Decompiling applications
- Application Internals
Break down and detail the nooks of crannies of starting an app, and maintaining its lifecycle
- Runtime Primitives
- Looper
- Handler
- MessageQueue
- Zygote explained
- Accessing services from native code
- Activity Manager in depth
- Behind the scenes of the application lifecycle
- Runtime Primitives
- Application Components
- Application Services
- Dalvik Internals
The inner workings of Android's Virtual Machine and bytecode format
- Dalvik vs. Java
- DEX, demystified
- Running DEX apps
- Dalvik's JNI implementation
- Alternative Dalvik Implementations
- Android RunTime Internals
- ART - An alternative to Dalvik
- ART Components (The com.android.art APEX)
- The files: OAT, ART, CDEX, VDEX, etc
- The runtime
- Setup and initialization
- Support threads
- Compilation
- JIT
- Profiling/Tracing
- Memory allocators and Garbage Collection
- Binder, in depth
- A Brief Overview of Binder
- The App Developer's Perspective - AIDL
- AIDL Syntax
- AIDL code generation
- The Parcel object
- The Parcel wire format
- The Framework Perspective -
android.os.Binder
Binder.java - References
- Default Transactions
- 11.0: Extensions
- Death Notifications
- The JNI Layer
- The native code Perspective -
libbinder.so
RefBase
: Strong and Weak Pointers-
The
BpBinder
andBBinder
- The native proxy and stub interfaces
ProcessState
andIPCThreadState
- Detailed case study: A Binder service in native code
- The Binder kernel interface
- The Binder character devices
- 11.0; BinderFS
- The ioctl(2) command set
- The
BINDER_WRITE_READ ioctl(2)
code - Transactions
- Flattened Objects
- The Binder character devices
- Binder Driver Internals
- Module initialization
- Device open
- Transactions in-kernel processing
- Kernel Driver State
- Thread Management
- Death Notifications
- Tracing & Debugging
Volume III: The [Hacker/Security Analyst]'s View
This was chapter 8 in the 1st edition - but that was a long time ago, before TrustZone, Titan, AVB ... and a host of Android exploits & APTs..5/31/2023: so far O(250) pages. Lots of work remains, but this, too, shall pass :-) Almost final TOC below - some chapters might be broken further
- Software-implemented security
- Linux Native Permissions
- AID ranges
- Treble and the return of passwd/group files
- AID ranges
- SELinux
- SECCOMP-BPF
- Android Runtime permissions
- Appops
- Linux Native Permissions
- Hardware-backed security
- TrustZone
- Theory & Design
- Vendor Implementations:
- Qualcomm: QSEE/QHEE
- MTK/Older Samsung: Mobicore
- Samsung: TEEGRIS
- Google: Trusty
- Beyond Trustzone: Hardware Security Modules
- Titan M/M2
- Qualcomm SPU
- TrustZone
- Authentication subsystems
- The Lock Screen (
lock_settings
service) - The
auth
service - The
biometric
service - Face authentication (The
face
service)
- The Lock Screen (
- Encryption facilities
- DM-Crypt
- Ext4Crypt
- Keystore
- Linux keyrings
- Gatekeeper
- Integrity & Attestation
- Android Verified Boot
- AVB 1.0
- AVB 2.0
- AVBMeta tool
- DM-verity
- 11: App Integrity, File Interity (fs-verity)
- Samsung TIMA & Knox
- Google SafetyNet
- Android Verified Boot
- Introduction/Threat Modeling Android
Lorem ipsum
- Threat Modeling
- Attack classes
- ..
- ...
- Android Security Model
- Rooting
Rooting Android using boot-to-root methods
- Prerequisite: OEM unlocking
- Android IOEMUnlock interface
- ...
- Case Study: Magisk
- Malware Case Study: Intellexa's "Alien"
- Vulnerability/Exploit case studies:
(Jury's still out on which of those I'll use - comments/suggestions welcome)
- Linux Kernel: CVE-2021-1048 (epoll) or CVE-2022-0847 (Dirty Pipe)
- AOSP Linux Kernel: Bad Binder (CVE-2019-2215) and/or num_not_so_valid CVE-2020-0041
- Vendor: Pixel 6 - Samsung's MFC
- TrustZone: likely Trusty
- AOSP: (still looking for something nice here)
- Vendor: MTK-su and/or Boot chain vulnerability?
- Baseband: Samsung Exynos (Shannon) VoLTE/SIP vulns
- Appendices:
- Android App Hardening Guide
- Android System Hardening Guide
Volume IV: The Implementer's View
- HAL & Treble
- The Hardware Abstraction Layer (pre-Treble)
- hwservicemanager
- HIDL and Binderized HAL
- sensors/CHRE
- Oslo/Soli as a case study
- GPS
- Implementing a custom HAL module
- The Android Input Architecture
- Android Media
- The Audio Architecture
- Audio at the Linux Kernel level
- The Audio HAL
- Audio modules
- Audio policy modules
- Audio Effects
- AudioFlinger
- Media Player
- The Dalvik APIs
- Video
- Digital Rights Management
- The Audio Architecture
- The Android Graphics Architecture
- Android Graphics Architecture
- Graphics at the Linux Kernel Level
- Graphics at the Native Level
- SKIA
- OpenGLES
- RenderScript
- MinUI
- SurfaceFlinger
- StageFright
- Connectivity
- BlueTooth
- Android Beam & NFC
- Wi-Fi & Wi-Fi Direct
- wpa_supplicant
- WifiMonitor
- WifiNative
- WifiConfigStore
- WifiStateMachine
- VPN (Racoon and MTPd)
- Tethering and Mobile Access Point
- Kernel Layer: NetFilter
- User Mode support
- hostapd
- dnsmasq
- Detecting Network State
- Monitoring Data Usage
- Telephony
- Radio interface layer (phone)
- Location
- Android and USB
- Android as a USB Target
- Framework USB Target Support
- The Linux Gadget Driver
- ADB
- Authenticated ADB
- MTP/PTP
- Mass Storage Device
- RNDIS (USB Tethering)
- Android as a USB host
- Framework USB Host Support
- Android as a USB Target