Input compatibility for Chromebooks

Users on laptops often use different input devices than on phones.

Running on Chromebooks, Android apps find themselves in an environment they were not designed for, displayed inside a window on a laptop with mouse, touchpad and keyboard. While Android does have support for those devices, they are rarely properly implemented in apps. In order to make these apps work, Chromebooks use a compatibility mode that is applied to all apps by default.

Compatibility mode

In compatibility mode, your app will receive events differently from how they are provided by Android APIs. This mode is useful for developers that do not want to implement Chromebook-specific features, but want their touch-centric app to just work.

The compatibility mode introduces the following behaviors:

• Touchpad scrolling may emulate touch screen scrolling with 2 ‘fingers’.
• Mouse wheel scrolling may do the same.
• All measurements and events behave as if the app window is displayed full screen. Methods like getRawX and getRawY may not return display-space coordinates but window-space coordinates.

Depending on the API level your app is targeting, compatibility mode provides more or less of these compatibility treatments.

Use android.hardware.type.pc

If you want to optimize your app for running on Chromebooks and to make good use of the input devices, declare the following in your app manifest:

<uses-feature
    android:name="android.hardware.type.pc"
    android:required="false" />

This tells Android that you are targeting PC devices that are typically used with keyboard, mouse, and touchpad. It disables the compatibility mode and allows you to develop custom behavior for mouse and touchpad.

Beware the DecorCaptionView

In free-form window mode, the apps caption bar is part of your view hierarchy and under your control. You generally do not have to be aware of this, but there are cases where you have to be careful:

• Do not fiddle with Window.getDecorView(). If you want to add top-level views, add them to the view you have set as Activity.setContentView().
• Do not expect your Activity.setContentView() to be at (0, 0) of your app. That’s where the caption bar is.
• If possible, avoid using MotionEvent.getRawX() or MotionEvent.getRawY(). If you do use them, use them in conjunction with View.getLocationOnScreen() to transform coordinates to view-space coordinates.

Input device support

Once you are targeting PC devices via the android.hardware.type.pc flag, you can expect the following events:

Mouse and Touchpad

Mouse and touchpad both generate MotionEvents just like touch events. Check MotionEvent.getSource() to distinguish SOURCE_MOUSE, SOURCE_TOUCHPAD and SOURCE_TOUCHSCREEN.

• Mouse/touchpad movement. Generates ACTION_HOVER_MOVE events. Those are handled in View.onGenericMotionEvent().
• Mouse/touchpad buttons. Sends ACTION_BUTTON_PRESS and ACTION_BUTTON_RELEASE events to View.onGenericMotionEvent(). You can also check for pressed buttons in all mouse/touchpad events by using getButtonState().
• Touchpad scrolling. Scrolling using two fingers on the touchpad is reported similar to touchscreen drag events to allow for smooth and kinetic scrolling. Use getSource() if you want touchscreen drag and touchpad scrolling to behave differently.
• Mouse wheel scrolling. Mouse wheel scrolling is reported as ACTION_SCROLL events to View.onGenericMotionEvent().
• Mouse/touchpad click and drag. Click and drag events are very similar to touchscreen drag events. Use getButtonState() to distinguish click and drag from a touchpad scroll event. Click and drag will always be accompanied by a button, while touchpad scrolling does not.

Keyboard

Keyboard input is important on desktop and laptop devices. It is also highly required for accessibility.

To get proper input focus activation, you should follow the normal resource additions as outlined in the default Android API:

• If you want to handle keyboard input yourself, you can use the default functions via KeyEvent.callback. There is no need to handle keyboard input inside a TextEdit element.
• If you want to edit text yourself, you should use onKeyDown, onKeyLongPress, onKeyUp but not onKeyPreIME events unless you want to implement the full breadth of IME—which is not recommended.

Stylus

A stylus reports events similar to a touchscreen via View.onTouchEvent(). However, stylus events carry more information that should be accounted for:

• MotionEvent.getToolType(). The tool type allows you to distinguish TOOL_TYPE_FINGER from TOOL_TYPE_STYLUS events. It also allows you to recognize the usage of the eraser side of a pen if the user has one via TOOL_TYPE_ERASER.
• MotionEvent.getPressure(). Represents the physical pressure applied to the stylus pen if supported.
• MotionEvent.AXIS_TILT/AXIS_ORIENTATION. Can be used with MotionEvent.getAxisValue() to read the physical tilt and orientation of the stylus if supported.

Historical Points

Android batches input events to be delivered once per frame. A stylus pen, however, can report at much higher frequencies (200Hz being quite common). When creating drawing apps, it is important to use of points acquired from the historical API:

• MotionEvent.getHistoricalX()
• MotionEvent.getHistoricalY()
• MotionEvent.getHistoricalPressure()
• MotionEvent.getHistoricalAxisValue()

Palm Rejection

Chrome OS attempts to recognize resting palms so they are never reported to the app. However this is not always possible: sometimes a touch may be reported before the OS recognizes it as a palm. In that case, touches will be cancelled by reporting an ACTION_CANCEL event.

This event tells the app that all touches are invalid and it should undo all interactions caused by the touches. For example, in case of drawing apps, the app will temporarily draw new lines and commit them to the canvas only once the touch series is finished cleanly. If the touch is cancelled, that temporary line can be easily removed.

Note-Taking Intent

Chrome OS displays a list of apps that are registered to handle intents containing an org.chromium.arc.intent.action.CREATE_NOTE action and the Intent.CATEGORY_DEFAULT (i.e. android.intent.category.DEFAULT) category as shown in the following code snippet:

  <intent-filter>
    <action android:name="org.chromium.arc.intent.action.CREATE_NOTE" />
    <category android:name="android.intent.category.DEFAULT" />
  </intent-filter>

The user will be able to select an app, and that app will later be launched when a note-taking app is requested.

When the user requests creating a new note, the app will be launched using an intent containing just the aforementioned action and category. The app should create an empty note in a mode where the user can write using a stylus.

When the user requests annotating an image (e.g. a screenshot or downloaded image) context, the app launches using an intent with the aforementioned action and category, including ClipData containing one or more items with content:// URIs. The app should create a note that uses the first attached image as a background image in a mode where the user can draw on it using a stylus.

Testing without a stylus

1. Switch to dev mode and make the device writable.
2. Press Ctrl+Alt+F2 (forward-arrow) to open the shell.
3. Run the command sudo vi /etc/chrome_dev.conf.
4. Add --ash-enable-palette to a new line at the end of the file.
5. Press Ctrl+Alt+F1 (back-arrow) to return to the UI.
6. Log out and back in.

You will now see the stylus entry points: - In the shelf, you can tap the stylus button and choose “New note”. This should open a blank drawing note in your application. - If you take a screenshot (from shelf: stylus button > Capture screen) or download an image, you should see the option “Annotate image” in the notification. This should open your app with the image ready to be annotated.

Gamepads

Chromebooks support up to 4 gamepads and follows standard Android APIs for reporting them. Unfortunately it is quite common that gamepads not designed for Android show the right button mapping.