Wednesday, 10 August 2011

Seaside with HTML5 for iPad & iPhone

We did a demo recently on a iPad showing how, with HTML and some javascript, we could make a Seaside client web site look native-ish. It's impressive what a minimalist layout, large fonts and big buttons can do.

This is new system development: the client had already seen a simple Seaside 'portal' web site which gave employees access to their shifts, with selected shift details and client information (each shift is a service to a specific client). All done as one integrated system for which they have a VW fat client.

For the iPad and iPhone we wanted to be aware of the screen real estate, include some device orientation sensitivity and to make sure that all selections were fat figure friendly (I'm a good tester for the 'fat finger' part).

For device orientation, you can use @media rule in CSS, or javascript, which is what I did. I set up a landscape and portrait DIVs and then used javascipt to hide and show the appropriate one. The layout was simple: in landscape I have a 75% / 25% horizontal split. In portrait right 25% area is rendered below in a wide layout. You can do some cool stuff with orientation. In most cases you don't what the display to flip around too much. But you can set up a image, like a logo, that responds to each small tablet movement on any axle. I didn't use that in the demo, but it looks cool.

This is the orientation script I used.

if (window.DeviceOrientationEvent) {
window.addEventListener('deviceorientation', function(eventData) {
}, false);
} else {
function deviceOrientationHandler() {
if ( orientation == 0 ) {
else if ( orientation == 90 ) {
else if ( orientation == -90 ) {
else if ( orientation == 180 ) {
function makePortrait() {
document.getElementById("tabletLandscape").style.display = 'none';
document.getElementById("tabletPortrait").style.display = 'block';
function makeLanscape() {
document.getElementById("tabletLandscape").style.display = 'block';
document.getElementById("tabletPortrait").style.display = 'none';

And this does the funky real time image orientation, with an image with id "imgLogo".
if (window.DeviceOrientationEvent) {
  window.addEventListener('deviceorientation', function(eventData) {
        var LR = eventData.gamma;
        var FB = eventData.beta;
        var DIR = eventData.alpha;
        deviceOrientationHandler(LR, FB, DIR);
}, false);
} else {
alert("Not supported on your device or browser.  Sorry.");
function deviceOrientationHandler(LR, FB, DIR) {
   //for webkit browser
   document.getElementById("imgLogo").style.webkitTransform = "rotate("+ LR +"deg) rotate3d(1,0,0, "+ (FB*-1)+"deg)";
   //for HTML5 standard-compliance
   document.getElementById("imgLogo").style.transform = "rotate("+ LR +"deg) rotate3d(1,0,0, "+ (FB*-1)+"deg)";

To get the screen size, I dug up some code that Lukas had posted on how to get screen size from the browser. It's one of those code snippets that really helped me understand how Seaside gets the browser to trigger callback blocks on the server.

renderScreenSizeOn: html
self screenX isNil ifTrue: [
self screenX: 0.
self screenY: 0.
script: ('window.location.href="' , html context actionUrl asString ,
'&' , (html callbacks registerCallback: [:v | self screenX: v asNumber]) , '=" + screen.width + "'
, '&' , (html callbacks registerCallback: [:v | self screenY: v asNumber]) , '=" + screen.height') ].
html text: self screenX printString, ' x ', self screenY printString

...which generates the script...

window.location.href="/Test?_s=sOMeODiqoU20GBt4&_k=pbWTZltjW35uekru&1=" + screen.width + "&2=" + screen.height

So, I triggered href="/Test?_s=sOMeODiqoU20GBt4&_k=pbWTZltjW35uekru&1=4288&2=1143 which Seaside reads and sends '4288' as the #value: to block [:v | self screenX: v asNumber] and '1143' to  [:v | self screenY: v asNumber].

If I edit the URL and change the values in the same session...
 ?_s=sOMeODiqoU20GBt4&_k=pbWTZltjW35uekru&1=1234&2=4567 screen will render the view values: 1234 x 4567. Neat.

Anyway, it was simpler to just use the request 'user agent' information to redirect to different applications, in the same image, for each device. This also made it easy to test the tablet and phone URLs on a full browser by using the appropriate URL.

renderContentOn: html | string |  string := self requestContext request userAgent.
(string includesSubString: 'iPad') ifTrue: [^self requestContext redirectTo: self tabletURL].
(string includesSubString: 'iPhone') ifTrue: [^self requestContext redirectTo: self phoneURL].
(string includesSubString: 'webOS') ifTrue: [^self requestContext redirectTo: self tabletURL].
^self requestContext redirectTo: self portalURL

I subclasses most of my components with *Tablet and *Phone suffixed classes, which made for easy code reuse and kept all of the device specific code in Seaside.

The portal required some diagram annotation, so I used javascript to detect figure movement and added more javascript for mouse movement. This way images could be annotated from any device. I started with a canvas ...

html div id: 'container'; with: [
html canvas id: 'sketchpad'; width: 350; height: 350; style: 'border: 1px solid black; background: white';
with: [html strong: 'Your browser does not support canvas.']].

...and then used this script for the iPad & iPhone figure drawing...

// get the canvas element and its context
var sketchpadCanvas = document.getElementById('sketchpad');
var context = sketchpadCanvas.getContext('2d');
context.fillStyle = 'red'; // red
context.strokeStyle = 'red'; // red
context.lineWidth = 4;
//Load the image object in JS, then apply to canvas onload
var myImage = new Image();
myImage.onload = function() {
context.drawImage(myImage, 0, 0, 350, 350);
// create a drawer which tracks touch movements
var drawer = {
  isDrawing: false,
  touchstart: function(coors){
     context.moveTo(coors.x, coors.y);
     this.isDrawing = true;
  touchmove: function(coors){
     if (this.isDrawing) {
        context.lineTo(coors.x, coors.y);
  touchend: function(coors){
     if (this.isDrawing) {
        this.isDrawing = false;
// create a function to pass touch events and coordinates to drawer
function draw(event){
  // get the touch coordinates
  var coors = {
     x: event.targetTouches[0].pageX,
     y: event.targetTouches[0].pageY
  // pass the coordinates to the appropriate handler
// attach the touchstart, touchmove, touchend event listeners.
sketchpadCanvas.addEventListener('touchstart', draw, false);
sketchpadCanvas.addEventListener('touchmove', draw, false);
sketchpadCanvas.addEventListener('touchend', draw, false);
// prevent elastic scrolling
},false); // end body:touchmove

To work on a non-touch browser, I loaded scripts to do the annotation with a mouse based on this tutorial.

Setting a background image, which also cleared the annotation, was done with a script
myImage.src = "/files/TSwaFileLibrary/body.png";
Saving the image was done with #onClick: script from a button.... the 'toDataURL()' is the interesting bit.
html jQuery ajax
callback: [:value | self saveImageFile: value]
value: (Javascript.JSStream on: 'sketchpadCanvas.toDataURL()')

...and... (removing the first 22 characters is a hack to strip out the ''data:image/png;base64,'' MIME declaration). The code is in VW.
saveImageFile: anImageString
| writestream string | writestream := (self newImageFilename asFilename withEncoding:  #binary) writeStream.
string := anImageString copyFrom: 23 to: anImageString size.
[writestream nextPutAll: (Seaside.GRPlatform current base64Decode: string) asByteArray] ensure: [writestream close].
The demo went well. I'm always surprised how making something pretty makes people think it's better. In this case I just added a new facade on a web site, and it was considered a big thing, yet it was only a few days work. It's like my buddy in marketing says: ya gotta sell the sizzle, not the steak. There's a lesson in there somewhere for Smalltalk.

(here is a good summary of iPod HTML5 javascript development)

Simple things should be simple. Complex things should be possible.