HTML5: Mobile web dev notes for games & fancy interfaces

Note: Use vendor prefixes for CSS3 styles mentioned here.

• Bind to touchend events rather than click events for far more responsive click/tap handling. This is one of the most elementary steps to make a mobile site feel more app-like. Mobile boilerplate has a nice way to do this (see the fastButton methods).
• Lock your window in place by canceling a touchmove event on document:

  var lockTouchScreen = function( locks ) {
    if( locks == true ) {
      document.ontouchmove = function( event ) {
        event.preventDefault();
      };
    } else {
      document.ontouchmove = null;
    }
  };
  

• Hide the browser address bar to gain more screen real estate. There's another nice method for this in the Mobile boilerplate project (see hideUrlBar). An important note about this working properly: your content has to be tall enough to convince the browser it needs to hide the address bar. This may require manually setting the height property of an outer container to $(window).height() + 50 before calling hideUrlBar().
• Use transform: translate3d(0,0,0) to hardware accelerate the movement and CSS animation of individual HTML elements. This is true for all version of iOS, and Android 4.0 and newer. However, there are some potential side-effects described below.
• Using the above hardware acceleration trick will accelerate any CSS animations that are added to the same element, such as opacity, background-color, width, etc.
• Sometimes z-index is ignored if you've used transform: translate3d(0,0,0) and placed a non-accelerated button above an accelerated element. The accelerated element can block clicks or touch events. (iOS)
• Adding this acceleration to sub-elements inside an accelerated container can further improve performance. For example, a series of img elements inside a larger element that's being positioned with translate3d (iOS)
• Swapping a background image with a class toggle on an element that's been hardware-accelerated can lead to the element disappearing. (Intermittent on iOS)
• Android ignores z-position in translate3d positioning as of this posting
• Android ignores background-size in 2.x versions, and possibly later versions, though Android 4.x seems to fix the problem.
• iOS can get really chunky when it's loading images. For less chunky loading, use containers with smaller background images, and use webkit scaling to size them up to the right size.
• Make sure you're loading @2x images for retina screens, or larger screens if you need to fill up a larger area. Also make sure you're not double-loading by using max and min pixel density media queries:
  (-webkit-min-device-pixel-ratio: 1.5)
  (-webkit-max-device-pixel-ratio: 1.5)
• It really can't hurt to turn off backface-visibility: none - this is a common optimization in 3D programming, though I would expect browsers to handle it by default?
• Holding a touch down on some versions of Android will reduce the image quality of images on the page, thus increasing framerate significantly for elements that are being animated or updated each frame. This fact could be exploited for gaming purposes...
• Using scale and translate3d at the same time on an element will break on Android 2.x and maybe others. For better cross-platform positioning/sclaing, use the technique described in my previous post.
• This Android transform bug is not only on the same element - if a container that's been scaled has a child element with a transform, animating transition or backface-visibility defined, the outer container will lose its scale. This is a MAJOR bug in Android 2.2 and 2.3, and there are probably other properties that will break transform: scale().
• Detect pinch gesture capabilities with isEventSupported('gesturestart'), using the event support detection method by Kangax. Then use my GestureCallback class to perform the pinch detection, or fall back to another input style.
• The default iOS menu that shows when an iOS user taps & holds an img element can be blocked if the img element is within a container that has a transform: translate3d(0,0,0); applied to it. Though, much like the button-blocking note above, by adding transform: translate3d(0,0,0); to the img, the issue is solved.
• Likewise on Android, this same menu that lets you save or copy an img can be blocked when using the touchstart event-canceling trick that's needed to provide a draggable interface. I'm not sure what a good fix would be for this if you want to use the image menu inside a draggable element.
• When detecting accelerometer support, it's not enough to check existence of the devicemotion event. You have to read the properties of the incoming event and check for real numbers. Something like: if( event.acceleration != undefined && event.acceleration.x != undefined ) var hasAccel = true;
• iOS caches the previous page you've visited, so if you press the browser back button, you can end up in a broken state if your page requires a re-initialization that would've happened on $(document).ready(). In my case, I was using document.location = '/new-location';. To force a reload if the user comes back via the browser back button, I used the following code:

  document.location = '/new-location';
  // reload page if we end up back at a cached page
  setTimeout(function(){
      window.location.reload();
  },3000);

  The browser will execute the timeout the user comes back, and problem solved.

More to come...

Android: fix the Android 2.x CSS transform bug with translate3d() and scale()

There's a serious bug with the use of the CSS3 transform property in Android's webkit browser, specifically in Android versions 2.2 and 2.3. The problem is that the scale() property is discarded in the presence of translate3d() (and other applicable transform properties). I can confirm it doesn't happen in 2.1, and I'm not seeing it consistently in all versions/devices with 2.3.x. The bug is officially documented here. There is a workaround, unfortunate as the bug is, which is to separately apply the translate3d() property to an outer element, and scale() to an inner element, like so:

Markup:

<div class="outer">
  <div class="inner"></div>
</div>

CSS:

.outer {
  transform: translate3d(100px, 100px, 0px)
  -webkit-transform: translate3d(100px, 100px, 0px)
  /* add other vendor prefixes so all browsers handle your hack */
}

.inner {
  transform: scale(5)
  /* add vendor-prefixed property again */
}

This doesn't work with scale() on the outer element, and translate3d() on the inner element, for what it's worth.

CSS: Rotation animation with CSS3

Sometimes you want to continuously spin something on your web page. In my case I was spinning a .png image for a smooth loading indicator. The following CSS will spin any element:

/* rotation animation */
@-webkit-keyframes rotate {
  from { -webkit-transform:rotate(0deg); }
  to { -webkit-transform:rotate(360deg); }
}

@-moz-keyframes rotate {
  from { -moz-transform:rotate(0deg); }
  to { -moz-transform:rotate(360deg); }
}

@-ms-keyframes rotate {
  from { -ms-transform:rotate(0deg); }
  to { -ms-transform:rotate(360deg); }
}

@-o-keyframes rotate {
  from { -o-transform:rotate(0deg); }
  to { -o-transform:rotate(360deg); }
}

.rotating {
  -webkit-transform-origin: 50% 50%;
  -webkit-animation-name: rotate;
  -webkit-animation-duration: 1.5s;
  -webkit-animation-iteration-count: infinite;
  -webkit-animation-timing-function: linear;
  -moz-transform-origin: 50% 50%;
  -moz-animation-name: rotate;
  -moz-animation-duration: 1.5s;
  -moz-animation-iteration-count: infinite;
  -moz-animation-timing-function: linear;
  -ms-transform-origin: 50% 50%;
  -ms-animation-name: rotate;
  -ms-animation-duration: 1.5s;
  -ms-animation-iteration-count: infinite;
  -ms-animation-timing-function: linear;
  -o-transform-origin: 50% 50%;
  -o-animation-name: rotate;
  -o-animation-duration: 1.5s;
  -o-animation-iteration-count: infinite;
  -o-animation-timing-function: linear;
}

Just add/remove the .rotating class to an element to start/stop the animation.

Android + Phonegap: Scale the WebView to fit the device

I was porting an iPad app to Android for the new Samsung tablet, and I had some trouble getting my web view to scale to the size of the device screen so that I wouldn't have to resize any of my assets. Obviously this is a questionable tactic, but I was experimenting and wanted to see how it would look :)

Here's the meat of my main App.java class:

public class App extends DroidGap {
 
 // declare the original size of the iPad app
 protected float ORIG_APP_W = 768;
 protected float ORIG_APP_H = 1004;
 
    /** Called when the activity is first created. */
    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        super.loadUrl("file:///android_asset/www/index.html");
        
     // set some defaults
     this.appView.setBackgroundColor(0x000000);
     this.appView.setHorizontalScrollBarEnabled(false);
     this.appView.setHorizontalScrollbarOverlay(false);
     this.appView.setVerticalScrollBarEnabled(false);
     this.appView.setVerticalScrollbarOverlay(false);
     
     // get actual screen size
     Display display = ((WindowManager) getSystemService(Context.WINDOW_SERVICE)).getDefaultDisplay();
     int width = display.getWidth(); 
     int height = display.getHeight(); 
     
     // calculate target scale (only dealing with portrait orientation)
     double globalScale = Math.ceil( ( width / ORIG_APP_W ) * 100 );
     
     // make sure we're all good
     Log.v( "ORIG_APP_W", " = " + ORIG_APP_W );
     Log.v( "ORIG_APP_H", " = " + ORIG_APP_H );
     Log.v( "width", " = " + width );
     Log.v( "this.appView.getMeasuredHeight()", " = " + height );
     Log.v( "globalScale", " = " + globalScale );
     Log.v( "this.appView.getScale()", "index=" + this.appView.getScale() );
    
     // set some defaults on the web view
     this.appView.getSettings().setBuiltInZoomControls( false );
     this.appView.getSettings().setSupportZoom( false );
     this.appView.getSettings().setGeolocationEnabled( true );
     this.appView.getSettings().setLightTouchEnabled( true );
     this.appView.getSettings().setRenderPriority( RenderPriority.HIGH );
     
     // set the scale
     this.appView.setInitialScale( (int)globalScale );
   }
}

I also updated the AndroidManifest.xml file to lock the app into portrait orientation, work on tablet-sized devices, have a nice app name, and give the device access to the Internet and geolocation:

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
      package="com.phonegap.testapp"
      android:versionCode="1"
      android:versionName="1.0">     
     
    <application android:icon="@drawable/icon" 
        android:label="@string/app_name"
        android:debuggable="true">
        <activity android:name=".App" 
                  android:label="Test App" 
                  android:configChanges="orientation|keyboardHidden"
                  android:noHistory="true" 
                  android:stateNotNeeded="true" 
                  android:screenOrientation="portrait">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />
                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
    </application>

    <!-- allows access to phonegap hardware features -->
 <uses-permission android:name="android.permission.INTERNET" />
 <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
 <!--<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />-->
 <uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" /> 
 <uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />
 <uses-permission android:name="android.permission.ACCESS_LOCATION_EXTRA_COMMANDS" />
    <uses-permission android:name="android.permission.READ_PHONE_STATE" />


   <supports-screens
     android:largeScreens="true"
     android:normalScreens="false"
     android:smallScreens="false"
     android:resizeable="true"
     android:anyDensity="true"
     />

</manifest> 

And finally my res/layout/main.xml file, though I'm not sure if this is different from the Phonegap default:

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:orientation="vertical"
    android:layout_width="fill_parent"
    android:layout_height="fill_parent"
    >    
            <WebView android:id="@+id/appView"
            android:layout_height="fill_parent"
            android:layout_width="fill_parent"
            /> 
</LinearLayout>

I hope this helps someone port their hybrid html5 iPad app to Android tablets.

Android: touchmove event bug

touchmove events in Android web browsers have a really serious bug. If you don't include the following code, the touchmove event will fire once, but not again until you're done moving your touch, which utterly kills the usefulness of the touchmove event. It's a weird one, and may very well break more advanced touch logic that works on iOS. But if you preventDefault() on the touchstart event, your touchmove will function as expected.

element.addEventListener( "touchstart", function(e){ onStart(e); }, false );
function onStart ( touchEvent ) {
  if( navigator.userAgent.match(/Android/i) ) {
    touchEvent.preventDefault();
  }
}

This bug is documented here:
http://code.google.com/p/android/issues/detail?id=5491

and is probably not entirely unrelated to my previous post about weird Android Touch event behavior:
http://uihacker.blogspot.com/2010/10/android-bug-miss-drag-as-we-are-waiting.html

Android: Phonegap issue with HTML5 video

We've been trying to implement HTML5 video inside a WebView in an Android Phonegap app, and there's a big difference between the embedded web browser and the native web browser on the Samsung Galaxy Tab (and likely other Android devices). In the native browser, an HTML5 <video> player will pop the video into the native media player, and should play fine if you've used an appropriate codec, and jumped through the right hoops. But, in an embedded browser in a Phonegap app, the video won't play at all. We resorted to using an <a> link with a _blank target to pop you completely out of the app. This was the only solution we could come up with. It's a pretty sad story dealing with the "standards" of HTML5 video across all the platforms and browsers that supposedly support it. I'm not sure there's a good way to deal with all of the different platforms/scenarios...

Android: Phonegap 0.9.2 doesn't work with Prototype.js 1.6.1

This morning I upgraded to Phonegap 0.9.2 on an HTML-based native app for Android, in order to use the new notification.confirm() function. After moving my project to this new version, I started getting this error:

Error initializing PhoneGap: JSON error

I was using Prototype.js 1.6.1, and after a bunch of investigation, I determined that it was Prototype that was causing the error. I replaced 1.6.1 with 1.7_rc3, and magically, everything works. I looked into why this might be happening, but decided to move on with my life and just go with 1.7_rc3 :)

Hopefully this post saves someone a little madness.

Android bug: "Miss a drag as we are waiting for WebCore's response for touch down."

I'm using Phonegap to port an HTML5 iPad app over to the Android platform. On my development HTC Incredible phone, every time I swipe my finger far enough, my app would freeze, and the adb debugging console would give me this error: "Miss a drag as we are waiting for WebCore's response for touch down." I researched a bunch and didn't find any solutions. I did find the Java code that logs this error in the core Android WebView.java class, but it didn't give me any clues to fix it.

I searched and hacked, and removed all my touch event listening code, and it would still break. Luckily the company work for has the resources to invest in development, and we went out and got a Samsung Galaxy S phone. I set the device up, published the app, and this phone did not have the cryptic issue! It did, however, show a range of other issues with fonts and the <canvas> object, which made me sad, as it's clear that building an HTML5-based app for Android isn't as easy as I hoped. The fragmentation of the Android platform is definitely an issue if you're attempting complex UI design and interaction with HTML/Javascript. I recommend keeping your HTML5 app very simple if you're targeting multiple Android platforms. Even though all new Android devices use Webkit, there are plenty of small, ugly differences.

[UPDATE]: The following code, when removed from my project, got rid of this weird error:

document.ontouchmove = function(event) {
    event.preventDefault();
};

Another update... Check out this post, and try out the demo code to get a bit more idea about how to handle preventDefault() on touch events in the Android browser: http://code.google.com/p/android/issues/detail?id=4549. It still crashes on my HTC device, but works great on the Samsung device.

Android browser bug: -webkit-transform scaling discrepency

[UPDATE]: this only happens on my HTC Incredible device, but not my Samsung Galaxy S device. yeesh.

I'm porting an HTML5 app we built for the iPad over to Android for the upcoming Samsung tablet. With the small difference in aspect ratio, I'm scaling down the entire site to avoid rebuilding everything. It turns out, that if you scale a container with CSS like so:

-webkit-transform-origin: 0 0;
-webkit-transform : scale(0.78125);

That works fine. However, I wanted to scale it dynamically with javascript, based on the device size, like so:

var globalScale = window.innerWidth / 768;
element.style.webkitTransformOrigin = '0 0';
element.style.webkitTransform = 'scale(' + globalScale + ')';

The result looks the same, but now clicking on anything in the scaled container is completely broken. :(

HTML/CSS on Android: rotation CSS difference between Android 2.1 and 2.2

Well, the splinternet is getting more interesting. As many developers settle on html as the most cross-functional platform, we're faced with ever more browsers and small differences between them. One that I just found is a difference in CSS positioning and rotation between the browsers on Android 2.1 and Android 2.2.

On my current project, I have a fancy UI that has an element constantly changing rotation and position using webkit transform CSS built with javascript. On Android 2.1, it worked fine as a 1-liner:

element.style.webkitTransform = "translate3d(" + xPos + "px, " + yPos + "px, 0px) rotate(" + rotation + "deg)";

But, on Android 2.2, the rotation stopped working. It seems that you can't have the translate3d and the rotate properties all set in the style.webkitTransform property. To fix the issue, I positioned using traditional absolute coordinates with the top and left CSS properties, and then used the webkitTransform property to do the rotation. There were a ton of special browser cases in my project to handle different things. Check out my platform detection class below to see how I handled a lot of special cases in one place.

PlatformHelper = function ()
{
    this.webkit_css_enabled = false;
    this.animations_enabled = false;
    this.is_android = false;
    this.is_android21 = false;
    this.is_android22 = false;
    this.is_idevice = false;
    this.is_touchscreen = false;
    this.is_msie = false;
    this.is_msie6 = false;
    this.is_msie8 = false;
    this.is_firefox = false;
    return this;
};

PlatformHelper.prototype.init = function ()
{
    // check for webkit positioning capability
    if( navigator.userAgent.match(/iPhone/i) ) this.webkit_css_enabled = true;
    else if( navigator.userAgent.match(/iPod/i) ) this.webkit_css_enabled = true;
    else if( navigator.userAgent.match(/iPad/i) ) this.webkit_css_enabled = true;
    else if( navigator.userAgent.match(/Chrome/i) ) this.webkit_css_enabled = true;
    else if( navigator.userAgent.match(/Safari/i) ) this.webkit_css_enabled = true;
    
    // check for certain platforms
    if( navigator.userAgent.match(/Android/i) ) this.is_android = true;
    if( navigator.userAgent.match(/Android 2.1/i) ) this.is_android21 = true;
    if( navigator.userAgent.match(/Android 2.2/i) ) this.is_android22 = true;
    if( navigator.userAgent.match(/MSIE/i) ) this.is_msie = true;
    if( navigator.userAgent.match(/MSIE 6/i) ) this.is_msie6 = true;
    if( navigator.userAgent.match(/MSIE 8/i) ) this.is_msie8 = true;
    if( navigator.userAgent.match(/iPhone/i) || navigator.userAgent.match(/iPod/i) || navigator.userAgent.match(/iPad/i) ) this.is_idevice = true;
    if( navigator.userAgent.match(/Firefox/i) ) this.is_firefox = true;
    
    // special cases for touchscreens
    if( this.is_android == true || this.is_idevice == true ) this.is_touchscreen = true;
    
    // decide who sees animations
    if( this.is_msie == true ) this.animations_enabled = false;
    else this.animations_enabled = true;
};

PlatformHelper.prototype.updatePosition = function ( element, xPos, yPos, rotation )
{
    if( !this.webkit_css_enabled || this.is_android22 )
    {
        element.style.left = xPos + 'px';
        element.style.top = yPos + 'px';
        element.style.MozTransform = 'rotate(' + rotation + 'deg)';
        element.style.webkitTransform = 'rotate(' + rotation + 'deg)';
    }
    else
    {
        var new_transform = "translate3d(" + xPos + "px, " + yPos + "px, 0px) rotate(" + rotation + "deg)";
        if( element.style.webkitTransform != new_transform )    // only apply style if not already in position
         element.style.webkitTransform = new_transform;
    }
};

Android browser bug: pinch/zoom kills setTimeout()

I'm working on some cross-platform/mobile touch/mouse code for a fancy html/js UI, and everything's been working great, but when I pinch/zoom the web page in an Android browser, my setTimeout() calls stop running. To be safe, I recommend using setInterval() instead.

// before:
setTimeout( function() { runTimer(); } , 1000/30 );
function runTimer() {
    // update graphics here
    setTimeout( function() { runTimer(); } , 1000/30 );
}

// after:
setInterval( function(){ runTimer(); }, 1000/30 );
function runTimer() {
    // update graphics here
}

I initially thought that my touch events (touchstart, touchmove, touchend) were randomly failing after zooming, because my custom motion code would completely break after running at a solid 30+ fps. It appears that this is a known bug in pre-2.2 (Froyo) Android web browsers: http://code.google.com/p/android/issues/detail?id=8566