Note: This file is a sample chapter from the full book - "Android Internals: A confectioner's cookbook" - which can be found on http://NewAndroidBook.com/. The chapter was made available for free as a preview of the book (think of it like Amazon's "Look Inside" :-). I encourage you to check out Technologeeks.com Android Internals training, which builds on the book and expands it further with Instructor Led Training.

You can also preorder the book by emailing preorder@NewAndroidBook.com.

Note some links (to other chapters in the book) will not work in this file (since it is partial), but external links will. Feedback, questions and requests are always welcome.

Chapter XII - The Android Input Architecture

Most users interact with their device through its touch screen. Android, however, can handle input from multiple input sources, of which the touch screen is only one. Additional sources may be a physical keyboard, a mouse, joystick or other controller, and virtually any other HID (Human Interface Devices), or even device sensors. Android provides a surprisingly elaborate stack structure, which receives input from the lower level kernel driver, translates the raw input data into a more manageable higher level event (key press, gesture, and such), and then propagates that to the active view in the foreground activity, by (eventually) invoking the callback associated with the view. On the way, Android uses both native level and Java level components. The high level view is shown in figure 11-1:

Figure 11-1:The Android Input Stack
Android: Dalvik Android: Native Linux Kernel Input Device Driver Interrupt Linux Input Stack /dev/input/eventXX Application View WindowInputEventReceiver android.view.InputChannel InputChannel com.android.server.wm.WindowManagerService com.android.server.input.InputManagerService NativeInputManager InputDispatcherPolicy InputReaderPolicy InputManager EventHub InputReader InputDispatcher

This chapter explores the Android input stack. We'll take a bottom-up approach, starting with the lower layer - provided by the device driver and the Linux kernel, through the runtime frameworks, conversion to an event, and finding the correct handler for that event. Be warned: The journey up the stack is long and arduous. But understanding it will hopefully make you gain new appreciation to just what happens when you next touch your device.

 

Input Handling at the Linux Kernel Layer

Interrupt Handling

An input event begins with an interrupt, which is generated when the device (touch screen, physical button, headphone jack, etc) detects the physical event. At the lowest level, this event causes electrical current, which is converted to an interrupt, which is delivered via the interrupt controller to the CPU, on one of the many interrupt request lines (IRQs).

Linux provides a default interrupt handler in the form of do_IRQ. This function checks if any device drivers have "claimed" the interrupt (by calling request_irq()). Usually, each driver will claim the IRQ line corresponding to its device, though it is not uncommon to see interrupt lines shared between two or more drivers. A driver claiming an IRQ also supplies a callback - often referred to as an ISR (Interrupt Service Routine), which will be invoked by do_IRQ when an interrupt is received on the claimed IRQ.

You can view the interrupt lines claimed by drivers if you look at /proc/interrupts. This highly architecture dependent file will show you the interrupt count for interrupts which have been registered (a different file, /proc/stat, will show all interrupt counts). Each interrupt line is shown with the count, controller type, and registered driver name. The driver names are often quite cryptic, but in some cases (such as the Galaxy S series) are well defined. The Galaxy S4 also shows a good example of multiple interrupt controllers, as shown in the following output, demonstrating some of the more interesting interrupts on the device:

shell@s4:/ $ cat /proc/interrupts
           CPU0       
 17:          0       GIC  dg_timer
 18:    9245245       GIC  gp_timer           # Timer interrupt 
	...
 34:          1       GIC  MSM_L2
 36:     243472       GIC  synaptics_rmi4_i2c # Touch screen
 47:          0       GIC  pm8821_sec_irq
 48:     981035       GIC  msmgpio
  ...

134:    4960502       GIC  msm-sdcc (cmd), msm-sdcc (pio)  #
135:     950126       GIC  msm-sdcc (cmd), msm-sdcc (pio)  # Flash storage
136:    4359946       GIC  msm-sdcc (cmd), msm-sdcc (pio)  #

            ...
343:        107   msmgpio  max77693-irq
351:        517   msmgpio  bluetooth hostwake      # Bluetooth devices
353:     503146   msmgpio  bcmsdh_sdmmc
362:     453332   msmgpio  pm8xxx_usr_irq
365:          0   msmgpio  sii8240
369:          0   msmgpio  sec_headset_detect      # headset jack
371:          0   msmgpio  msm_hsl_wakeup
474:          0    pm8xxx  pm8921_overtemp_irq
479:       3896    pm8xxx  pm8xxx_rtc_alarm
486:          0    pm8xxx  pm8921_batt_alarm_irq
489:          0    pm8xxx  msm_otg
490:         63    pm8xxx  pmic8xxx_pwrkey_release # Power down
491:         63    pm8xxx  pmic8xxx_pwrkey_press   # Power up
             ...

661:        579    pm8xxx  home_key                 # home button
662:       1338    pm8xxx  cypress_touchkey         # menu, back
664:          0    pm8xxx  msm-sdcc (slot)
666:         28    pm8xxx  volume_up_key            # Volume UP  
668:         56    pm8xxx  volume_down_key          # Volume DOWN
                 ...

# Inter processor (core) interrupts: Note four columns for quad-core
IPI0:          0      13962      13870      11649  CPU start interrupts
IPI1:          0          0          0          0  Timer broadcast interrupts
IPI2:    3947099    5539984    2449051    1427641  Rescheduling interrupts
IPI3:       1369      12875      12846      12533  Function call interrupts
IPI4:        402     126466     109265      91618  Single function call interrupts
IPI5:          0          0          0          0  CPU stop interrupts
IPI6:          0          0          0          0  CPU backtrace
Err:          0
 

In other cases, however, like the HTC One M8, the physical keys are all grouped under one interrupt, gpio_keys. Another interesting feature is that on multi-core CPUs, you will actually see multiple interrupt count columns when additional cores are active. You can further control which core responds to which interrupt by writing a hexadecimal mask value to the /proc/irq/##/smp_affinity. Advanced boards may also show "IPI" (inter-processor interrupts) as well.

You can empirically determine which IRQs are associated with which devices by viewing /proc/interrupts before and after triggering an interrupt from a device (for example, touching the screen, or pressing a button). A physical button press will usually result in 2 or more interrupts (corresponding to the button down and button up events), whereas swipes and other gestures may very well result in dozens or more, the same as mouse motion would on a desktop system.

Interrupts are effectively handled at the highest possible priority. With rare exceptions, an interrupt will preempt whichever thread is executing on the CPU, as it demands the kernel's immediate attention. Desktop systems make use of this fact in some cases (for example, with the SysRQ mechanism), though a more immediate application is that, when a device is hanging, most users intuitively press buttons, tap, swipe, or otherwise try to "wake up" the device. In practice, this can, in theory, be detrimental, as more events trigger more interrupts, which can increase the system load by preempting what the CPU was doing, in favor of handling otherwise meaningless gestures. In practice, however, interrupt handling is so quick the load they introduce is fairly minimal.

Well behaved drivers obey the strict requirements of handling IRQs - they never block, and take as little time as possible, deferring work toa software IRQ (the kernel's ksoftirq threads). Interrupts therefore usually have minimal effects on the system. In addition some devices can use tricks such as interrupt coalescing (firing one interrupt instead of several), and devices will not fire interrupts when the device is sleeping (for example, the touch screen, when the display is turned off).

 

The Linux Input Driver Model

Android uses the standard Linux input driver model, introduced back in 2.4 and standardized in 2.6 and later. The Linux kernel documentation contains a subdirectory documenting the model and usage of its programming interfaces (Documentation/input/, and in particular input.txt). The interested reader is encouraged to consult the documentation for more detail, though we provide an overview in this section.

A device driver responsible for an input device is required to allocate an input_dev structure and populate its capabilties. This structure is defined in <linux/input.h> and shown in figure figInputDev, along with the ioctl(2) codes which can be used to retrieve selected fields from user mode:

fieldcontainsioctl(2) code
namedevice display name EVIOCGNAME
physdevice physical path in /sys EVIOCGPHYS
uniqunique code, if anyEVIOCGUNIQ
idstruct input_id
propbitdevice properties and quirksEVIOCGPROP
evbitEV_ event types supported by device
keybitkeys/buttons this device hasEVIOCGBIT(EV_KEY..)
relbitrelative axes for the deviceEVIOCGBIT(EV_REL..)
absbitabsolute axes for the deviceEVIOCGBIT(EV_ABS..)
mscbitmiscellaneous events supported by deviceEVIOCGBIT(EV_MSC..)
ledbitLEDs present on the deviceEVIOCGBIT(EV_LED..)
sndbitsound effects supported by deviceEVIOCGBIT(EV_SND..)
ffbitsupported force feedback effects, if anyEVIOCGBIT(EV_FF..)
swbitswitches present on the deviceEVIOCGBIT(EV_SW..)
hint_events_per_packetaverage # of events generated by device
keycodemax size of keycode table
keycodesize size of elements in keycode table
keycodemap of scancodes to keycodes for device
getkeycode(legacy) retrieve current keymap.
ffForce-Feedback, if any
repeat_keyLast pressed key, for auto-repeat
timerauto-repeat timer
repauto-repeat parameters
mtstruct input_mt holding Multitouch state
absinfoAbsolute axes coordinate information
keycurrent state of device keys/buttonsEVIOCGKEY
ledcurrent state of device LEDs, if anyEVIOCGLED
swcurrent state of device switches, if anyEVIOCGSW
opencallback for open(2) on device
closecallback for close(2) on device
flushflush device events,e.g. force-feedback
eventhandler for events sent to device
Figure figInputDev: The struct input_dev (from <linux/input.h>)
 

The input driver registers its input_devs with the kernel input manager by a call to input_device_register. Doing so will automatically create a sysfs entries (symbolic links) for the device in /sys/class/input, and character device entries, which in turn will be picked up by Linux's udevd (or Android's ueventd) to create corresponding /dev/input/event## entries. The character device node is registered under the major of the input manager, with a minor corresponding to its order of registration + 64 (e.g. event0 would have minor 64, event1 would have minor 65, etc). Operations on the created device node will be handled by an evdev_fops file_operations structure (in <drivers/input/evdev.c>), with generic blocking implementations for read, write, poll, etc.

As shown in the figure, the key fields of the input_dev structure are accessible via standard ioctl(2) calls with specific EVIOC* constants. Additionally, the capabilties and properties of a device are exported to user mode via entries in sysfs, under /sys/class/input/event##/device/capabilities/ and /sys/class/input/event##/device/properties.

The Linux device drivers respond to interrupts, generated by the respective devices. The drivers then report the events using the input_report_[key/...] functions, and events are then queued onto the /dev/input/event## device as structs containing the timestamp, event type, associated code and value. User mode applications use the standard system calls (that is read(2), select(2)/poll(2) and the like) to retrieve events (always an integer multiple of sizeof(input_event)) from the device. The supported event types are defined in <input/event.h> and shown in table 11-evttypes:

#Event codeSpecifies
0x00EV_SYNSeparate/synchronize other events (e.g. SYN_REPORT/SYN_MT_REPORT), or report events lost (SYN_DROPPED)
0x01EV_KEYKey press (KEY_*) or touch (BTN_TOUCH)
0x02EV_RELRelative changes to a property. Changes relayed through REL_[XYZ] values.
0x03EV_ABSAbsolute coordinates for an event. Values are usually ABS_[XYZ], or ABS_MT for multi-touch
0x04EV_MSCMiscellaneous codes
0x05EV_SWBinary switches. E.g. SW_JACK_PHYSICAL_INSERT for headphone insertion
0x11EV_LEDUsed for device LEDs, if any
0x12EV_SNDUsed for sound devices
0x14EV_REPUsed for auto-repeating events
0x15EV_FFUsed for force-feedback capable devices (e.g. joysticks). An EVIOCSFF ioctl may be used to upload force feedback effects
0x16EV_PWRReserved for power events. Largely unused
0x17EV_FF_STATUSUsed for force-feedback capable devices.

Naturally, not all input devices support all event classes; The input_dev structure maps the various events a particular device supports by its bitmaps, and the device's sysfs entry makes those bitmaps visible in user mode as /sys/class/input/inputXX/capabilities. The input event source of each device is in /sys/class/input/inputXX/eventXX, and also conveniently symlinked directly from /sys/class/input.

Note, that even though the Linux input model was designed for HID type devices, it can be used on virtually any devices, including sensors. This is shown in the following experiment.

 
Experiment: Looking at low-level input events

You can examine input events yourself through the shell. First, look at the files in /dev/input. On the emulator, you should see something like this:


root@generic:/ # ls -l /dev/input
crw-rw---- root     input     13,  64 2013-11-15 18:24 event0
crw-rw---- root     input     13,  63 2013-11-15 18:24 mice
crw-rw---- root     input     13,  32 2013-11-15 18:24 mouse0

Output on a real device will likely be different, as these have many more input channels, mapped to its various sensors; The Samsung S3, for example, has event0 through event13, and no "mouse0". The major and minor numbers, however, should be the same, and the major (13) is associated with the kernel input driver subsystems (as can be verified with grep input /proc/devices).

The system keeps track of all devices quite conveniently in /proc/bus/input/devices:


shell@htc_m8wl:/ $ cat /proc/bus/input/devices 
I: Bus=0000 Vendor=0000 Product=0000 Version=0000
N: Name="h2w headset"
P: Phys=
S: Sysfs=/devices/virtual/input/input6
U: Uniq=
H: Handlers=kbd event6 keychord 
B: PROP=0
B: EV=3
B: KEY=84 10000 138 0 e0800 0 0 0
..

To see the properties and capabilities of a given device, say /dev/input/event0, examine its sysfs entry:

## File sizes are reported as 4k (pagesize) arbitrarily - all virtual anyway
shell@generic:/sys/class/input/event0$ ls -l
-r--r--r-- root  root 4096  2013-11-15 18:24 dev
lrwxrwxrwx root  root       2013-11-15 18:24 device -> ../../input0
drwxr-xr-x root  root       2013-11-15 18:24 power
lrwxrwxrwx root  root       2013-11-15 18:24 subsystem -> ../../../../../class/input
-rw-r--r-- root  root 4096  2013-11-15 18:24 uevent
root@generic:/sys/class/input/event0 # cat uevent
MAJOR=13
MINOR=64
DEVNAME=input/event0

By accessing the device nodes, you can treat the input sources as files, and by using a command such as "cat" on them, dump raw events directly to standard output, though those would be appear to be nothing more than synchronized garbage. A better way would be to use the getevent command, which is part of toolbox. The source for this tool is at system/core/toolbox/getevent.c.

When used as getevent, toolbox invokes getevent_main(), which uses Linux's inotify to enumerate the files in /dev/input/event. It then opens each detected device, and uses the ioctls from the previous table to obtain the device information. Finally, it adds the device to an array of file descriptors which it polls continuously. As each device (event source) signals input, getevent reads the event records and dumps it to stdout.

getevent has several particularly useful switches:

  • -i: Show HID codes and other information
  • -l: Show numbers in human readable constant form (e.g. "0001" as KEY_ESC)
  • -p: Show device capabilities (as per sysfs capabilities file) and exit
  • -t: Show timestamps
  • -h: Show help on all switches (-n, -t, -s, -S, -v, -d, -p, -i, -l, -q, -c and -r) then exit

 
Experiment: Looking at low-level input events (cont.)

Armed with this information, you can conduct this simple experiment: Make sure your display isn't sleeping, then run getevent -l via adb on your device (or emulator), and then touch the screen. On the emulator, you should see output similar to the following:


shell@generic:/$ getevent -l
could not get driver version for /dev/input/mouse0, Not a typewriter
add device 1: /dev/input/event0
  name:     "qwerty2"
could not get driver version for /dev/input/mice, Not a typewriter
/dev/input/event0: EV_ABS  ABS_X                00000083 # X position of touch
/dev/input/event0: EV_ABS  ABS_Y                00000129 # Y position of touch
/dev/input/event0: EV_KEY  BTN_TOUCH            DOWN     # Touch start       
/dev/input/event0: EV_SYN  SYN_REPORT           00000000 # Event "terminator"
/dev/input/event0: EV_KEY  BTN_TOUCH            UP       # Touch end
/dev/input/event0: EV_SYN  SYN_REPORT           00000000 # Event "terminator" 

On a real device (for example, the Samsung S3) you'll see much more output, owing to the myriad input sources it has. The msm8960 sensors, for example, are what enables headphone vendors to control music playing and call answering through the headphone jack.

shell@s3:/$ getevent -l
## getevent will display all enumerated devices:
add device 1: /dev/input/event6   name:     "sec_touchscreen"
add device 2: /dev/input/event8   name:     "barometer_sensor"
add device 3: /dev/input/event4   name:     "sec_jack"
add device 4: /dev/input/event3   name:     "light_sensor"
add device 5: /dev/input/event2   name:     "proximity_sensor"
add device 6: /dev/input/event0   name:     "sii9234_rcp"
add device 7: /dev/input/event12  name:     "msm8960-snd-card Headset Jack"
add device 8: /dev/input/event11  name:     "msm8960-snd-card Button Jack"
add device 9: /dev/input/event10  name:     "msm8960-snd-card Volumeup Jack"
add device 10: /dev/input/event9  name:     "msm8960-snd-card Volumedown Jack"
add device 11: /dev/input/event13 name:     "sec_touchkey"
add device 12: /dev/input/event1  name:     "fsa9485"       
add device 13: /dev/input/event5  name:     "sec_keys"         # physical keys
add device 14: /dev/input/event7  name:     "sec_powerkey"     # power button - KEY_POWER
# These are sent regularly from the light sensor, as it's highly sensitive
/dev/input/event3: EV_REL  REL_X                00000020            
/dev/input/event3: EV_REL  REL_Y                00000021            
/dev/input/event3: EV_REL  REL_Z                00000015            
/dev/input/event3: EV_REL  REL_MISC             00000024            
/dev/input/event3: EV_SYN  SYN_REPORT           00000000            
## Touch event: Notice ABS_MT is used here, rather than BTN_TOUCH
/dev/input/event6: EV_ABS  ABS_MT_TRACKING_ID   00000043 # unique ID for this touch           
/dev/input/event6: EV_ABS  ABS_MT_WIDTH_MAJOR   0000000a # major axis of approaching ellipse           
/dev/input/event6: EV_ABS  ABS_MT_POSITION_X    0000011f # center x touch position          
/dev/input/event6: EV_ABS  ABS_MT_POSITION_Y    0000022d # center y touch position
/dev/input/event6: EV_ABS  ABS_MT_TOUCH_MAJOR   00000010 # major axis of touch ellipse
/dev/input/event6: EV_ABS  ABS_MT_TOUCH_MINOR   0000000c # minor axis of touch ellipse
/dev/input/event6: EV_ABS  ABS_MT_TOOL_X        ffffffc4 # center y tool position
## Physical buttons: HOME and VOLUMEDOWN together (notice /dev/input/event5)
/dev/input/event5: EV_KEY  KEY_VOLUMEDOWN       DOWN         
/dev/input/event5: EV_SYN  SYN_REPORT           00000000       
/dev/input/event5: EV_KEY  KEY_HOMEPAGE         DOWN                
/dev/input/event5: EV_SYN  SYN_REPORT           00000000       
/dev/input/event5: EV_KEY  KEY_VOLUMEDOWN       UP         
/dev/input/event5: EV_SYN  SYN_REPORT           00000000       
/dev/input/event5: EV_KEY  KEY_HOMEPAGE         UP
/dev/input/event5: EV_SYN  SYN_REPORT           00000000       

 
Experiment: Simulating low-level events with sendevent

As it so happens, toolbox also has a sendevent tool which you can use to simulate events at the lowest level, by writing directly to the /dev/input/eventXX device nodes. The code itself is straightforward, constructing an input_event from the command line, and calling write(2) to send it to the device node. The caller needs write access to the device node - which the shell provides for you even on non-rooted device thanks to its group membership in the input group.

Using this tool alongside getevent brings out the real usefulness of low-level UI Automation: With getevent, you can record input events - touch, swipe, and physical button events on most devices, but even sensor readings like orientation, proximity and light (on those devices which support it, like the Samsung S3, above). You can then replay them with sendevent. There are only two caveats to remember:

  1. While getevent spits out the events in hexadecimal, sendevent will expect decimal input. So you will have to do the conversion yourself (or use a simple script to do so)
  2. Most events, though atomic to the user, are broken up into multiple components, and require the SYN_REPORT synthetic event to act as a delimiter. It's therefore important to send that event at the end of the sequence.
The following example shows a simulation of pressing the home button on an S3. since the EV_KEY constants are the same across Linux (and Android) versions, the only adaptation required for other devices would be the figuring out which /dev/input/eventXX node to use. Continuing the previous output, we would have something like:

# simulate EV_KEY KEYHOMEPAGE DOWN followed by REPORT
shell@s3$ sendevent /dev/input/event5 1 172 1; sendevent /dev/input5 0 0 0
# To simulate home button hold, delay the following line, simulating the UP/REPORT
shell@s3$ sendevent /dev/input/event5 1 172 0; sendevent /dev/input5 0 0 0 

As we continue traversing up the input stack, the next experiment you encounter will introduce you to yet another method of simulating input events.

 

Input Handling at the Android Native Layer

... (not part of this preview)...