Digging in the dirt

VW Code Coverage

2012-01-31T08:16:00.002-05:00

The past few days I've been working with the code coverage extensions to SUnitToo in VW (SUnitToo(verage) and SUnitToo(lsoverage). It is an impressive tool. And the metrics accurately reflected the way I developed some packages. Our in-house issue tracker and patch manager had tests added after the fact, with a resulting code coverage of 30%. The Report4PDF package, the PDF4Smalltalk based report writer I'm working on, was developed from the start with SUnit tests, with a code coverage of 65%.

At first I thought that 65% seemed a bit low, since all the code has associated tests. But the code coverage is smart: it does not just measure method hits, but which paths within a method are exercised. In many cases these were exception and error message branches; the paths less taken. But some cases should have been tested. These were very narrow special cases that I had simply not thought of. The code looks OK, and I'm confident that it will work, but I won't release it without an SUnit test.

If feels like I've been given a flashlight to see the dark nooks and crannies of my code.

I also like the idea of using the path count (the denominator in the code coverage number) as an application complexity metric. It could work well when used with number of classes, number of methods, and average method size. I wonder if 'number of paths per method' would be useful.

It is an impressive tool. Thanks to all who made it available.

Simple things should be simple. Complex things should be possible.

PDF Report and the Law of Demeter

2012-01-08T19:05:00.004-05:00

I'm finishing a small project which uses Christian Haider's pdf4smalltalk to build report output using a Seaside influenced coding style. A report with a header, text and footer would be coded as...

| report |
report := PRwReport new.
report portrait.
report page: [:page |
page header string: 'This is a header'.
page text string: self someText.
page footer string: 'This is a footer'].
report saveAndShowAs: 'TestText.pdf'.

The tool supports the usual report output options, like fonts, alignment, tables, images, bullets and so on.

I've built a couple of other Smalltalk report frameworks over the years. One used Crystal Reports for the layout with configurable data gathering, and another (much better tool) that used Totally Object's Visibility (I did a presentation on that one at Smalltalk Solutions 2004). Both of those used a data + layout spec model, which, with the benefit of hindsight, was not be best choice. It was a challenge to keep the code and layout in sync. Maintenance was painful. For PDF Report I opted for Seaside's 'paint the content on a canvas' pattern. It is working nicely (I'll be presenting the details at Smalltalk Industry Conference in Biloxi).

Here's the part that got me thinking about how nice objects are and the Law of Demeter... when building a report output, you have to deal with coordinating the size and position of layout components on the page. Do you give the responsibility to the page, or do you have the layout objects find their own place? I opted for a 'builder': it knows how much space is available on a page and which layout objects need to be processed.

The interesting part was in deciding how much the builder needed to know about each layout. The first few iterations were rudimentary: each layout had a calculated height (word wrapped text with a selected font) and the builder would output as much as would fit on one page, then trigger a page break and continue on to the next page.

But that did not work with tables, since each row could have some cells that spanned pages. The builder could not blindly trigger a page break on a tall cell, since the next cell would be on the previous page. The table, row and cell had to communicate layout information to the builder, with the cell width and height dependent on neighbouring cells. And, to make things especially interesting, tables can be nested and cells can span rows and columns, like this...

...and this...

Each time I added a new layout mix to the SUnit tests I had to rethink what each object knew. After several iterations a pattern emerged: the less the builder knew about the layout objects, the better. And as the builder got dumber, its code got simpler and new layout mixes just worked. A tricky part was in sequencing the layout calculation for nested objects: a cell's height is dependent on the row's height, but the row's height is the maximum of it's cell heights.

Once the calculation sequence was correct, each layout object was able to answer it's layout values: position, margin and padding. The builder could ask if a layout could fit in the remaining space on a page without knowing what the layout objects was (text, table, bullet, image or line) and could create a new physical page without knowing how a layout object would be split. Now each refactoring cycle starts with me asking myself: how can I reduce what the builder needs to know? The latest version is much cleaner than the first. It's nice to apply well known object design rules and see real results.

Still a lot of work to do, but I'm looking forward to showing it at the conference. And, if it's good enough, it will be added to the VW public store (long term plans are to port to other Smalltalk dialects).

Simple things should be simple. Complex things should be possible.

Smalltalk head space

2011-11-15T10:39:00.001-05:00

At the last Toronto Smalltalk User Group meeting we had a basic introduction to Seaside. It had been requested by some of the members, especially by the two profs that sponsor our group at Ryerson University. They teach an OO course with a Smalltalk component and felt it would be of interest to their students. A few did show up and the feedback was very positive. A bit surprising, given how basic the demo was (here is a link to the PDF), but it got me thinking about how easy it is to get out of touch with how other people see things. There is something to be said about baby steps.

After the talk we spent a while talking about the sad sate of computer science education; how it's become a C syntax programming training school. Way back in my day (early 80's) there was no dominant computer language. Intro to comp. sci. was taught in FORTAN. We learned BASIC, PL/I, COBOL, SNOBOL, Lisp, 360 Assembler, Prolog and Smalltalk. My OS course was taught as a history lesson, explaining why and how each OS concept was introduced, and why some were abandoned. There was a big focus on 'why', not just 'how'. That does not seem to be the case today.

During the conversation, I talked a bit about how selling Smalltalk can be a challenge. Smalltalk is different. If all you know is C and Java, it's really different. Why learn something different if it will not help you get a job? I made the case that learning Smalltalk helps you learn how to think in objects. A perspective that may be helpful when dealing with the other object hybrid languages (it would also be good to learn Lisp, Prolog and assembler). You can't help thinking that a Java-centric education gives students only one tool: a hammer, and they then view every task as a nail.

But how do you communicate the value of a "Smalltalk head space"? A world where everything is a live object that you can see and change. A place where delegating behaviour to Integer and String makes perfect sense; where the language is minimalist and the libraries add the complexity; where working with a debugger feels natural. And how do you explain how sending messages is different than calling a function? The mechanics are simple enough, but the mental model is harder.

I see functions as a handing off of total control of the world to something else. Within a function you change whatever data you need to. The world is a large, porous bit universe that you manipulate. Whereas sending a message is asking another object to do something for you. You don't think about tickling the bits of another object. You do your job, send messages to other objects, and maybe answer something. You work in a local scope, not the whole world.

Every professional Smalltalk developer I know can tell talk about their "aha" moment, the point where object-think made sense. In my case it was a course at The Object People where Paul White was explaining how to model a transfer between a chequing and savings account: asking either account to do the transfer smelled wrong, so instead you could model the transaction. Really? You can do that? A 'transction' can be an object? Cool. A more whimsical example is modelling the milking of a cow. Do you ask the cow to milk itself? Do you ask the milk to un-cow itself? No. You model a farmer to do the milking 'transaction'.

The problem is that those "aha" moments take time to learn. And who has time for that? As Dave Thomas said in his recent SPLASH video interview, the complexity that we've added to the development tools is not always a benefit to someone building a simple application. Do you really need to know about objects if a BASIC program will do? Is learning about classes and instances to big a hurdle? Would a prototype object model be easier to learn? I don't know, but we really have to find a better way to communicate the up side of using Smalltalk. Waiting for each student to experience their own "aha" moment is not good enough.

Simple things should be simple. Complex things should be possible.

XML RESTful Seaside interface to legacy system

2011-09-12T11:48:00.000-04:00

I'm mandated with creating web interfaces to two legacy frameworks, one written in VA and the other in VW.

For the VW framework I started with a full application implementation; a Seaside interface that duplicated the entire application interface by parsing VW window specs and building Seaside components. But for both the VA and VW systems there is also a need for a 'portal' web site, one that exposes a limited view of the application but is intended for a larger user base.

Both systems were designed over a dozen years ago, using GemStone and a fat client. Grafting a multi-user Seaside interface to the fat client designed for a singe user was not going to work. That's why the VW 'full user' implementation uses one 200MB VW image per Seaside session, a deployment model that was not an option for the portal: too many users (a resource issue), too much exposed (a security concern), too integrated (a maintenance problem).

To work around these constraints I've implemented an XML based domain interface, where a Seaside image can gather the data it needs to render using a RESTful interface to GemStone. There is no domain model in the Seaside image, just components that know what part of the domain they represent. With GemStone this works particularly well since it eliminates the need to fault objects into the client. It's a common technique with GS, popularised by James Foster, to build strings on GS, fault them to the client, then parse and display the content. Some displays, like lists of objects, can improve their performance from tens of seconds to sub-second.

Each Seaside component knows which dedicated server method to use. Since the portals have a limited scope there are not that many XML data methods. I would not use this approach if I expected the portal application scope to increase significantly.

All of this was made easier by thinking of the final rendering as a limited set of display patterns: lists, tables, trees, image and a single domain object. The XML has tags that identify the display pattern: <list> <table> <tree> <image> <domain>. These are used to build 'XmlObject' subclasses for each pattern. This makes accessing and debugging data easier than referencing the parsed XML code directly in the Seaside components.

Parsing XML is very different between VA and VW. Wrapping the results in my own XML node objects made it easy to port my code between dialects. I just needed to abstract out the XML parser references. Predefined tags, like 'oop, objectclass, text' are stored in instance variables. Any other tags are stored in a dictionary. The Seaside component would expect certain tags to be present and will trigger an error otherwise.

Some examples...

domain
- only attributes the Seaside component needs are included
- used in the list, table and tree patterns with minimal attributes; if selected the oop is used to get whatever else is needed
<domain>

<oop>461206257</oop>
<objectclass>AlaItemWorksheet</objectclass>
<text>ABTP Lab Results - D - DEN</text>
<icon>AlaDataEntry.ico</icon>
<label>ABTP Lab Results - D - DEN (TAB [Final Effluent])</label>
</domain>

list
- stored as named lists in the collection of attributes for a domain node
- each element of a list is another domain node, with enough information to be displayed (label, icon) and with the domain object oop in case it gets selected
<list type="actions">

<domain>
<oop>291900165</oop>
...
<domain>
<oop>291913433</oop>
...

table

- added tags for header and styles. The style contents is encoded as key, value pairs so that the server code can include style information that a generic table component will render
<table>

<header>
<data>
<value>ABTP Lab Results - D - DEN</value>
</data>

tree
- each tree node knows how to get its sub-tree
- sub-trees are retrieved and cached when a tree node is expanded
<list type="productSet">

<domain>
<oop>199226369</oop>
<objectClass>INproductSet</objectClass>
<text>By Function</text>
<hasSubtree>true</hasSubtree>
<hasProducts>false</hasProducts>
</domain>

A Seaside component is created for each tree node which reads the 'hasSubtree' attribute and, based on a configuration, gets the nested list of domain nodes when expanded.

image
- for the VA implementation, images are stored as byte arrays on GS
- for the VW, the images are stored as server paths
- in each case, the image node needs to answer a byte array that can then be rendered
<domain>

<oop>249729481</oop>
<objectClass>AlaSiteMapImage</objectClass>
...
<image>
<oop>249728229</oop>
<mimeType>image/gif</mimeType>
</image>

Building the XML on the server is done with a 'canvas' object which wraps tags around indented nested content. Formatting of the XML content is a debugging convenience.

To add a domain object...

aCanvas
addDomain: anItem
text: anItem product id
with: [
aCanvas add: 'description' put: anItem product description.

...

To add a list...

aCanvas
addListNode: productSet
type: 'productSet'
with: [
collection do: [:each | self buildXmlProductSet: each on: aCanvas]].

For both apps a user identifier, in the form of a GS object oop, is included in each data request. Seaside stores the user object oop in the WASession subclass instance variable. Communication between the Seaside image and GS is behind a firewall, so I'm not that concerned about it being monitored.

RESTful communication works well with my Smalltalk message passing sensibilities. Objects passing messages feels so natural, and debugging is easy since I can record and replay any message. And I don't care what gets changed on the server, as long as the Seaside XML methods answer the same way.

Although the code is written in both a VW and VA client, it's dialect agnostic; they could be swapped. I stuck with the two Smalltalks because I was replacing a domain model Seaside implementation in each, and there was enough sunk cost to leave that code as is.

Ideally I would like to host the Seaside sessions from GS with a GLASS deployment. It's a long term option with the VW application, but the VA application has to be hosted on a Windows server, which limits us to 32 bit GS and no GLASS.

So far testing is going well. The next step is to test this under load and see how many concurrent Seaside sessions one image can handle. We already have a multi-Seaside image dispatcher implementation working with Apache that supports session affinity, so I'm confident that scaling will not be a problem.

Simple things should be simple. Complex things should be possible.

Seaside with HTML5 for iPad & iPhone

2011-08-10T10:42:00.001-04:00

We did a demo recently on a iPad showing how, with HTML and some javascript, we could make a Seaside client web site look native-ish. It's impressive what a minimalist layout, large fonts and big buttons can do.

This is new system development: the client had already seen a simple Seaside 'portal' web site which gave employees access to their shifts, with selected shift details and client information (each shift is a service to a specific client). All done as one integrated system for which they have a VW fat client.

For the iPad and iPhone we wanted to be aware of the screen real estate, include some device orientation sensitivity and to make sure that all selections were fat figure friendly (I'm a good tester for the 'fat finger' part).

For device orientation, you can use @media rule in CSS, or javascript, which is what I did. I set up a landscape and portrait DIVs and then used javascipt to hide and show the appropriate one. The layout was simple: in landscape I have a 75% / 25% horizontal split. In portrait right 25% area is rendered below in a wide layout. You can do some cool stuff with orientation. In most cases you don't what the display to flip around too much. But you can set up a image, like a logo, that responds to each small tablet movement on any axle. I didn't use that in the demo, but it looks cool.

This is the orientation script I used.

if (window.DeviceOrientationEvent) {
window.addEventListener('deviceorientation', function(eventData) {
deviceOrientationHandler();
}, false);
} else {
makeLanscape();
}
function deviceOrientationHandler() {
if ( orientation == 0 ) {
makePortrait();
}
else if ( orientation == 90 ) {
makeLanscape();
}
else if ( orientation == -90 ) {
makeLanscape();
}
else if ( orientation == 180 ) {
makePortrait();
}
}
function makePortrait() {
document.getElementById("tabletLandscape").style.display = 'none';
document.getElementById("tabletPortrait").style.display = 'block';
}
function makeLanscape() {
document.getElementById("tabletLandscape").style.display = 'block';
document.getElementById("tabletPortrait").style.display = 'none';
}

And this does the funky real time image orientation, with an image with id "imgLogo".

if (window.DeviceOrientationEvent) {
window.addEventListener('deviceorientation', function(eventData) {
var LR = eventData.gamma;
var FB = eventData.beta;
var DIR = eventData.alpha;
deviceOrientationHandler(LR, FB, DIR);
}, false);
} else {
alert("Not supported on your device or browser. Sorry.");
}
function deviceOrientationHandler(LR, FB, DIR) {
//for webkit browser
document.getElementById("imgLogo").style.webkitTransform = "rotate("+ LR +"deg) rotate3d(1,0,0, "+ (FB*-1)+"deg)";
//for HTML5 standard-compliance
document.getElementById("imgLogo").style.transform = "rotate("+ LR +"deg) rotate3d(1,0,0, "+ (FB*-1)+"deg)";
}

To get the screen size, I dug up some code that Lukas had posted on how to get screen size from the browser. It's one of those code snippets that really helped me understand how Seaside gets the browser to trigger callback blocks on the server.

renderScreenSizeOn: html
self screenX isNil ifTrue: [
self screenX: 0.
self screenY: 0.
html
script: ('window.location.href="' , html context actionUrl asString ,
'&' , (html callbacks registerCallback: [:v | self screenX: v asNumber]) , '=" + screen.width + "'
, '&' , (html callbacks registerCallback: [:v | self screenY: v asNumber]) , '=" + screen.height') ].
html text: self screenX printString, ' x ', self screenY printString

...which generates the script...

window.location.href="/Test?_s=sOMeODiqoU20GBt4&_k=pbWTZltjW35uekru&1=" + screen.width + "&2=" + screen.height

So, I triggered href="/Test?_s=sOMeODiqoU20GBt4&_k=pbWTZltjW35uekru&1=4288&2=1143 which Seaside reads and sends '4288' as the #value: to block [:v | self screenX: v asNumber] and '1143' to [:v | self screenY: v asNumber].

If I edit the URL and change the values in the same session...

?_s=sOMeODiqoU20GBt4&_k=pbWTZltjW35uekru&1=1234&2=4567

..my screen will render the view values: 1234 x 4567. Neat.

Anyway, it was simpler to just use the request 'user agent' information to redirect to different applications, in the same image, for each device. This also made it easy to test the tablet and phone URLs on a full browser by using the appropriate URL.

renderContentOn: html | string | string := self requestContext request userAgent.
(string includesSubString: 'iPad') ifTrue: [^self requestContext redirectTo: self tabletURL].
(string includesSubString: 'iPhone') ifTrue: [^self requestContext redirectTo: self phoneURL].
(string includesSubString: 'webOS') ifTrue: [^self requestContext redirectTo: self tabletURL].
^self requestContext redirectTo: self portalURL

I subclasses most of my components with *Tablet and *Phone suffixed classes, which made for easy code reuse and kept all of the device specific code in Seaside.

The portal required some diagram annotation, so I used javascript to detect figure movement and added more javascript for mouse movement. This way images could be annotated from any device. I started with a canvas ...

html div id: 'container'; with: [
html canvas id: 'sketchpad'; width: 350; height: 350; style: 'border: 1px solid black; background: white';
with: [html strong: 'Your browser does not support canvas.']].

...and then used this script for the iPad & iPhone figure drawing...

// get the canvas element and its context
var sketchpadCanvas = document.getElementById('sketchpad');
var context = sketchpadCanvas.getContext('2d');
context.fillStyle = 'red'; // red
context.strokeStyle = 'red'; // red
context.lineWidth = 4;
//Load the image object in JS, then apply to canvas onload
var myImage = new Image();
myImage.onload = function() {
context.drawImage(myImage, 0, 0, 350, 350);
};
// create a drawer which tracks touch movements
var drawer = {
isDrawing: false,
touchstart: function(coors){
context.beginPath();
context.moveTo(coors.x, coors.y);
this.isDrawing = true;
},
touchmove: function(coors){
if (this.isDrawing) {
context.lineTo(coors.x, coors.y);
context.stroke();
}
},
touchend: function(coors){
if (this.isDrawing) {
this.touchmove(coors);
this.isDrawing = false;
}
}
};
// create a function to pass touch events and coordinates to drawer
function draw(event){
// get the touch coordinates
var coors = {
x: event.targetTouches[0].pageX,
y: event.targetTouches[0].pageY
};
// pass the coordinates to the appropriate handler
drawer[event.type](coors);
}
// attach the touchstart, touchmove, touchend event listeners.
sketchpadCanvas.addEventListener('touchstart', draw, false);
sketchpadCanvas.addEventListener('touchmove', draw, false);
sketchpadCanvas.addEventListener('touchend', draw, false);
// prevent elastic scrolling
document.body.addEventListener('touchmove',function(event){
event.preventDefault();
},false); // end body:touchmove

To work on a non-touch browser, I loaded scripts to do the annotation with a mouse based on this tutorial.

Setting a background image, which also cleared the annotation, was done with a script

myImage.src = "/files/TSwaFileLibrary/body.png";

Saving the image was done with #onClick: script from a button.... the 'toDataURL()' is the interesting bit.

html jQuery ajax
serializeForm;
callback: [:value | self saveImageFile: value]
value: (Javascript.JSStream on: 'sketchpadCanvas.toDataURL()')

...and... (removing the first 22 characters is a hack to strip out the ''data:image/png;base64,'' MIME declaration). The code is in VW.

saveImageFile: anImageString
| writestream string | writestream := (self newImageFilename asFilename withEncoding: #binary) writeStream.
string := anImageString copyFrom: 23 to: anImageString size.
[writestream nextPutAll: (Seaside.GRPlatform current base64Decode: string) asByteArray] ensure: [writestream close].

The demo went well. I'm always surprised how making something pretty makes people think it's better. In this case I just added a new facade on a web site, and it was considered a big thing, yet it was only a few days work. It's like my buddy in marketing says: ya gotta sell the sizzle, not the steak. There's a lesson in there somewhere for Smalltalk.

(here is a good summary of iPod HTML5 javascript development)

Simple things should be simple. Complex things should be possible.

Simple jQuery contact application

2011-07-03T11:06:00.001-04:00

I continue to be impressed with how easy it is to build nice, useful applications with Seaside using the jQuery support. We have an old Unix character based contact application that those who don't want to change (and you know how you are) continue to use. I gave up on character based interfaces the late 80's, so I created a simple Seaside app to access the contact data.
The contact data is dumped daily into a special character delimited file, with 40 fields per row. Trivial to parse in Smalltalk...

array := aString subStrings: 1 asCharacter.

...where aString is one row of the data file.

Accessing the fields is just an array access, like...

postalCode
^self dataArray at: 16

The collection of contacts (about 8500) is stored in a contact library object, in three collections, each sorted by a key stored on the object as a lower case string (by first name, by last name and by company). At first I tried using some fancy nested tree structure, then I tried large complex key dictionaries, but doing a match on 8500 instances takes less than 10 ms. Trying to optimize that just added complexity. Storing the search keys as lower case strings rather than translating them on each iteration was helpful. Search times dropped from 60 ms. The search method also has a limit to keep the auto-complete list is to a reasonable size. A '*' is appended to the string to support the general 'match' case.

string := aString asLowercase.
string last = $* ifFalse: [string := string copyWith: $*].
list := aCollection select: [:each | string match: each nameKey].
list size > limit ifTrue: [^list copyFrom: 1 to: limit].
^list

We match on #nameKey, but the displayed list uses #displayFullName, the mixed case version of the string.

OK, so the domain in as simple as it gets. The fun part was the Seaside code. JQAutocomplete>>search:labels:callback: made it easy (props to its author). The method wraps a lot of complexity into a tight, useful package. In my case, the code looks like this...

html textInput
id: anId;
style: 'width: 300px; font-size: 14pt';
script: (html jQuery this autocomplete
search: [:string | self buildSearchResponse: aSearchBlock with: string]
labels: aLabelBlock
callback: [:value :script | self redirectToContact: value script: script])

#buildSearchResponseh:with: is a hack to show error messages in the auto-complete list. An exception answers an error object which is a subclass of the contact class, which is then displayed in the drop-down list.

^[aBlock value: aString]
on: Error
do: [:ex | Array with: (TSroloError newForErrorMessage: ex displayString)]

#aLabelBlock uses one of the three display methods for the drop down list (#displayFullName #displaySurname #displayCompany).

#redirectToContact:script: shows a nice URL with a ?contact=nnnn suffix.

| urlString |
urlString := aScript requestContext request url startingURL ,
'?contact=',
aContact displayContactIndex.
aScript goto: urlString.
self session unregister

In #renderContactOn: I check for the 'contact' parameter and render the contact details if found.

Here is what it looks like...

...typing 'bob n' into the full name fields shows...

...selecting 'Bob Nemec' then redirects to '?contact=3784', which shows the contact details...

...the '+' toggles a display of all the contact fields, using the jQuery toggle...

| moreId lessId |
moreId := html nextId.
lessId := html nextId.
html anchor
id: moreId;
onClick: (
html jQuery ajax script: [:s |
s << (html jQuery id: aComponentId) toggle: 0.3 seconds.
s << (html jQuery id: moreId) hide.
s << (html jQuery id: lessId) show]);
title: aMoreTitle;
with: [html image url: TSwaFileLibrary / #morePng].
html anchor
id: lessId;
style: 'display: none;';
onClick: (
html jQuery ajax script: [:s |
s << (html jQuery id: aComponentId) toggle: 0.5 seconds.
s << (html jQuery id: lessId) hide.
s << (html jQuery id: moreId) show]);
title: aLessTitle;
with: [html image url: TSwaFileLibrary / #lessPng]

Finally, reloading the list of contacts is done by adding a ?load parameter, which loads the contacts from a predefined file location. This way the contact load action does not need to be done by the application, but can be done by an OS scheduled task.

And again, a big thanks for the Seaside jQuery code which makes writing simple apps like this easy.

A bad day in [] is better than a good day in {}

Low-tech Seaside graphs

2011-06-01T20:25:00.000-04:00

I really like the basic 'how to' tutorials people post, whether they are about Seaside, C#, VisualStudio or motorcycle maintenance (that pretty much covers where my head has been lately).

So here are a couple of things I've done in Seaside that I think are simple yet proved handy.

The first example is from a home financial application that I wrote a few years ago as a way to learn Seaside. My wife and I use it to track our budget and daily transactions, which it imports from a scheduled download done by an iMacro script.

I wanted a bar graph to show how much of the budget for a category was spent for the month. And I wanted parts of the graph to support drilling down for more details. Here is what I ended up with (with some random data)...

...each square is one day of the month. The narrow grey line is to the right of today. If spending in a category is running ahead of the budget, it's red.

The graph is created as a two column table, with the title and the progress bar. The bar is created by rendering same sized (16x16) image buttons, one for each day, and a narrow divider button (3x16) for 'today'.

html imageButton
callback: [self openTransactions];
title: 'Show transactions';
url: TxFileLibrary / #progressredPng

The images were created with Paint.net, a handy tool for simple graphics.

I used a similar approach to create a Kanban graph for an issue tracking system. In this case, a table with fixed sized cells is filled with image buttons representing an issue, to a maximum of five rows. Selecting a cell shows the list of issues for that cell, and the title of each cell shows the display string of the corresponding issue.

I think it's cool what you can due with some dynamic HTML table generation and a few PNG files.

User group marketing

2011-05-20T08:08:00.000-04:00

The Toronto Smalltalk User Group has been active for 20 years. At our peak we had about 45 people attending meetings. Lately we've averaged about a dozen.

What can we do to get more people to attend?

I suspect we already know all Smalltalkers in the Greater Toronto Area, if not personally then by just one level of separation. Our best bet is to target new people that may have an interest. We've had some success with Ryerson students. The two professors that sponsor our group encourage the students that take their OO course to attend. It has some Smalltalk content (Pharo) and a couple of students have come out.

Some of the TSUG crowd have attended XP Toronto events. They have regular meetings, pair programming pub nights and they organize well run Agile conferences. Those that know about Smalltalk respect it because of XP's roots, but they tend to be surprised that it's still an active language.

Yanni Chiu has been something of a TSUG ambassador, representing Smalltalk at things like the "Dynamic Languages Smack Down". He and Chris Cunnington signed up for Toronto Startup Weekend and plan to show some Seaside app. I had some tentative plans to talk to the Durham Ruby group (something I need to follow up on).

Whenever it's appropriate I hand out TSUG pamphlets. They list all the Smalltalk dialects, web frameworks and projects I know of. I print off a new batch for each TSUG meeting, with updated meeting schedules and conference information (ESUG in in the current one). It's an MS Publisher document which I print off on heavier stock paper which has a light blue tint. Here is what part of it looks like...

I'd love to get suggestions on how to promote the group.

Malcolm Gladwell in his book Outliers makes the point that people with very high IQs are not more likely to be successful than people with just 'good' IQs (120, evidently). Once you're smart enough, other factors like people skills and communication skills make the difference. Smalltalk has an IQ above 180, the other tools are probably in the 120 range. Smalltalk needs to work on its people and communication skills.

Pushing Smalltalk

2011-05-03T17:09:00.000-04:00

The next meeting of the Toronto Smalltalk User Group is May 9 with Don MacQueen talking about JWARS.
See the web site for details.

I've been involved with the Toronto Smalltalk User Group for 20 years now, the past dozen or so as the primary organizer. Lately we've had a few people show up that were new to Smalltalk and wanted to learn more. We talk to them about the simple syntax, show them Pharo and Seaside, tell them about the other dialects, and try whatever we can to get them enthused.

But there is one core strength that I value which I cannot easily demo: the lack of brick walls.

When working with tools like MS's Visual Studio, I'm constantly frustrated by the lack of universal object inspection and 'down to primitive' code tracking. Coding something new and with unfamiliar APIs gets a bit messy. You don't always know what you need until you trip over its absence. I find myself adding diagnostic code to show me stuff, which in Smalltalk I'd just debug and inspect. And sometimes in VS you just can't get what you want; you hit a brick wall. If it were not for Google and people posting esoteric workarounds, I don't know how I'd get anything done. And I love the examples that say "don't forget the comma, or else it won't work"... no error message, no warning, just no output.

Thing is, that's not the kind of thing you appreciate until you try something complex, which does not happen to a new user. I wonder if we would benefit from having code examples with pre-defined bugs, written in various languages, and then using them to show how you'd debug the problem. Show just how few barriers we have in Smalltalk to understanding what is going on in our code (and, more importantly, in other people's code).

I used to tell people that I saw a lot of similarity between programming in MVS 360 assembler and Smalltalk. For me, transparency was the big stick that I could use to whack the problem. From what I've seen of other IDEs (an admittedly short list), Smalltalk still does that the best.

Smalltalk trivia

2011-04-22T07:39:00.000-04:00

I miss the days of The Smalltalk Report and discussions on comp.lang.smalltalk. It was a time of lively debate about syntax, naming conventions, coding patterns, dialects ... all the signs of an energetic community. It's not like that any more. I think it's due to the fragmentation of the Smalltalk forums. The threads are interesting, but specialised.

My coding is strongly influenced by Kent Beck and his coding patterns. I remember reading his column in The Smalltalk Report and seeing the evolving consensus of how to code Smalltalk. My iteration block still have :each or :each<something> as the binding variable. In his "Fundamentals of Smalltalk Programming Technique", Andres Valloud makes the point that code formatting is a pointless debate and that we should be comfortable reading any Smalltalk code. And he's right. But I'd still like to know what others find more readable. It's like learning to write gooder English.

For example, I trip over the extra white spaces inside blocks that most of the Seaside code has. Where I'd write a method like...

aBoolean
ifTrue: [self doThis]
ifFalse: [self doThat].
aList do: [:each | self doSomethingWith: each]

...I find the Seaside code is formatted as...

aBoolean

ifTrue: [ self doThis ]

ifFalse: [ self doThat ].

aList do: [ :each | self doSomethingWith: each ]

...I wonder why that convention was adopted. In my eyes the block is just not hugging the code enough. I don't find that style in other Smalltalk code. And it really does not matter; I'd just like to understand the though behind it.

Another example: In our VW application, the framework code uses abbreviations for local and parameter variables, so I see a lot of code like printOn: aStrm and | errMsg idx t v m s | ... drives me nuts. But it was adopted because of how easy it used to be to name a local variable the same as an instance variable. We do refactor code this kind of code now as we maintain it, but it's easier to accept when the thought behind it is understood.

And here's a pattern question: when is it better to double-dispatch? In our VW code was have an #echo method which writes a printString of the receiver to the Transcript, like the newish VW #out method (unfortunate name conflict with an Opentalk method). We also have #echo: which writes both the receiver and the argument to the Transcript, so...

'Time' echo: Time now

...shows in the Transcript as...

Time 10:10:54 AM

I added the same code to our VA application. So what's the best way to write this method? We don't want strings to be printed with single quotes, like...

Transcript cr; show: 'this string' printString
'this string'
vs.

Transcript cr; show: 'this string'

this string

...so we can implement #echo on Object as Transcript cr; show: self printString and on String as Transcript cr; show: self. But what about #echo: ? We could implement on Object as...

Object>>echo: anObject
Transcript cr; show: self printString, ': ', anObject printString

...(I prefer the : delimiter to just a space) and we'd need...

String>>echo: anObject
Transcript cr; show: self , ': ', anObject printString

...but what if anObject is a String? You would end up with superfluous single quotes. You could add a new method like #asEchoString with an Object and String implementation and send that instead of #printString, which is easy to read, or you could try double dispatching, which I decided to do (mostly out of curiosity). In this case I don't think it's a better pattern, but it's interesting (and very useful for more complex problems). And I like that they are all one line method.

Here is what my implementation (no value returned so that a := b echo answers b)...

Object>>echo
self printString echo

String>>echo
Transcript cr; show: self.

Object>>echo: anObject
anObject echoAfter: self printString

String>>echo: anObject
anObject echoAfter: self

Object>>echoAfter: aString
aString echoString: self printString

String>>echoAfter: aString

aString echoString: self

String>>echoString: aString

Transcript cr; show: self , ': ', aString

'test' echo: 123 >> 'test: 123''test' echo: 'this' >> 'test: this'123 echo: 'this' >> '123: this'123 echo: 456 >> '123: 456'

VXML server isolation

2011-04-19T10:14:00.000-04:00

Our IVR VXML Seaside server needs to handle up to 700 calls per hour. Normally that's not a problem, but we do get a few timout errors per week, which is why we're moving to a three image load balanced deployment.

Recently we had another issue with our current setup: the server hosting the file share, to which call files are written, failed. Calls were still processed, but the final write failed. Write are done in a forked block with an exception handler, so the image dealt with the errors cleanly. And the call data was all written to a log, so no data was lost.

However, there was a performance hit on the server image and we started to get timeout errors on more than 10% of the calls. So we turned off the Seaside server which triggered an automatic failover to our AWS hosted server. It works, but it's a bit slow. Callers could wait several seconds for the call prompts (we'll be upgrading it soon).

This was the first time we had an error with the file share and it pointed out a dependency that we'd prefer to avoid. Ideally, the VXML servers would continue to process calls even if other servers are down; they should as isolated as possible.

Our strategy is to use the VW Opentalk image to image interface and have a data server gather call information from the three VXML servers. It will also send them updated validation files. This way, the VXML servers have no folder share dependencies, the data server can show status across all three VXML server, and the VXML servers can function even if the entire back office is down. And we can host the data & VXML servers with different service providers.

We've been using Opentalk in our client patch system for a couple of years now. It's familiar and reliable. And ya gotta love the deployment flexibility you get with Smalltalk.

IVR VXML from Seaside

2011-04-15T10:21:00.000-04:00

We use Seaside to serve anything that takes data from an http call. One of these services is an IVR system hosted by Voxeo. It requires a VXML file that defines the audio script and records answers. We have a sublcass of WARequestHandler that answers static VXML content for the initial request:

handleRequest: aRequestContext
| response |
response := aRequestContext response.
response
contentType: self contentType;
nextPutAll: self document requestVXML.
aRequestContext respond.

...and builds contents for the response, which contains a simple 'Thank you' if all the data is valid...

handleResponse: aRequestContext
| response result request |
request := aRequestContext request.
result := self processCall: request.
response := aRequestContext response.
response
contentType: self contentType;
nextPutAll: result.
aRequestContext respond.

Our production system, which our client uses to record at-home service calls, peaks at about 600 calls per hour. And the same image that serves the VXML file also has a conventional Seaside interface for monitoring the system and for configuration. Very handy. For example, we have a Google graph to show one day's call distribution.

Recently we added more validation to the VXML script, so that shifts and employee numbers could be checked during the IVR dialog. The new interaction places more demands on the Seaside image, so we're testing a deployment with three images and simple round-robin load balancing (the current single image deployment gets about two timeout errors per week). Each VXML call is RESTful so we have no need for session affinity. Testing all went well, until something interesting came up.

Testing was all done on a Windows server and deployment is on a Linux box. No big deal for VW. But when we tried the new deployment, the IVR system said the VXML content had errors. Looking at the VXML file on our Seaside display showed extra blank lines, but the content was correct. Inspecting the file content as read showed that the <cr><lf>s were all replaced with <cr><cr> (you gotta love all the delimiter differences between Windows, Linux and Smalltalk). I didn't think that IVR server would have an issue with that, but to be sure I replaced the file read from...
^file contentsOfEntireFile
...to...
^file contentsOfEntireBinaryFile asString
...and it turns out that fixed the problem.

Makes we wonder what people do who work with tools that don't allow the deep diving that Smalltalk does. I suspect that when it all works their productivity is high, but there must be more frustration when things break.

VA Smalltalk and Seaside

2011-04-14T07:39:00.000-04:00

I support a VA Smalltalk app for which we're adding a web interface, using Seaside. The first feature we've ported is a 'Site Map' view, a hierarchy of GIF images with configured 'hot point' rectangles for navigation. Users click to drill down to more other images or to display data in a table.

In VA the display is updated by replacing the 'image area' #labelPixmap and adjusting the window shell width and height. 'Hotpoints' are managed by mapping 'Pointer Motion' and 'Button Press' callbacks to a collection of objects that define the rectangle, which shows an action specific mouse pointer, and and triggers the action when the mouse button is pressed. All of this is user configurable using a view where they add image, draw rectangles and define actions.

A few years ago we added a feature which optionally displayed the current value of a measurement in the displayed image. It eliminated the need to click on a data 'hot point' and was handy for images with several data points. We've now added the same capability to the Seaside view.

In VA, the data is displayed by first defining instances of CwLabel in the imageArea (a WkImageWidget) at the correct position. The #labelString and #backgroundColor are then set based on user parameters, like the display date. Background color is used to show the 'quality' of value, so a grey background indicates incomplete data. To hide the data, the CwLabel's #visible property is set to false.

This is what it looks like in VA...

To do this in Seaside, we use a style 'position:absolute; left:10px; top:140px; z-index:-1;' for the GIF and then render the displayed data with...
'position: absolute; top: %1px; left: %2px; z-index:2; border: 2px solid black; background-color: %3'
...using #bindWith:with:with: to set the top, left and background-color. The result looks like this...

I continue to be impressed with how easy it is to do interesting web work in Seaside.

Dynamic Wiki

2011-04-11T15:48:00.000-04:00

We use an internal MediaWiki web site for document and status reporting. Combining our static wiki text with dynamic output from our issue library and patch manager has worked very well for us. We use a MediaWiki webservice extension to add graphs, tables and issue lists.

For example, this code will get a list of my issues...

{{#webservice:<our internal web server>/Issues?wiki=user:Bob_N.| %//div%}}

...the output can be tested by using the URL directly in a browser, which answers wiki formatted output...
==Bob N. - current==
{| ! ID ! System ! Client ! Title ! Stage ! Text ! P. ! Assigned ! Date ! Type |- | [http://<our internal web server>/Issues?issue=3353 3353] || BID || CS || HTS Customer Portal: changes to project display || [[image:Blueplay.png]] current || '' '' || 3 || Bob N. || 2011-04-11 || Modification |- |...etc.

...and, when parsed by MediaWiki, shows as a list of issues...

...with links back to the issue library. We combine the web service extension with Google charts to add live charts to our wiki, using an entry like...

{{#webservice:<our internal web server>/Issues?Issues?wikichart=backlog| %//div%}}

...which generates wiki formatted text that contains a Google Chart API call...

<websiteFrame> website=http://chart.apis.google.com/chart?&cht=bvo&chs=800x350&chg=0,10,4,0&chtt=Working+Issue+Backlog&chts=19293D,20&chbh=24,30&chco=00FFFF,00CCFF,0099FF,0066FF,0033FF,0000CC,000066&chdl=a: week|b: 2 weeks|c: month|d: 2 months|e: quarter|f: 2 quarters|g: 6 months&chd=t:0,0,0,0,0,0,0,0,0,0,0,0,795|0,0,0,0,0,0,0,0,0,0,0,766,767|0,0,0,0,0,0,0,0,0,0,0,730,699|0,0,0,0,0,0,0,0,0,0,660,661,639|0,0,0,0,0,0,0,0,0,540,543,491,474|0,0,0,0,0,0,290,465,456,398,381,344,335|141,182,214,254,282,289,256,235,76,72,69,54,54&chds=0,795&chxt=x,y&chxl=0:|Apr 2010|May 2010|Jun 2010|Jul 2010|Aug 2010|Sep 2010|Oct 2010|Nov 2010|Dec 2010|Jan 2011|Feb 2011|Mar 2011|Apr 2011|1:|0|79|158|237|316|395|474|553|632|711|790 name=Working Issue Backlog align=middle height=380 width=800 border=0 scroll=no longdescription=chart </websiteFrame>

...again, this can be tested directly in a browse to show the expected charted (the data shows a ramp up of using the issue library)...

Patch Manager

2011-04-08T10:14:00.000-04:00

We use a VW / Seaside based Patch Manager to send patches to our clients. Some of the design ideas may be of interest.

We cannot access our clients machines directly, but we can get permission for outbound ports through their firewalls. So our patch management process is based on polling: the remote patch clients connect to a patch server every minute, get a list of patch names and timestamps, and then request updates or send changes (for patches that are filed out at a client site). Commands, status, logs, exception walkback files are also passed along.

User access is though a Seaside interface to the patch server. Developers select patches to send to a client from a table of published patches, filtered by applications and versions. Remotely filed-out patches can also be retrieved and published. If a patch is updated, the developer can choose to see the difference and can then send the updated patch. Our code file-out methods add a user and timestamp comment, which is extracted by the patch server.

In this example, = shows a patch that is published, > < shows an updated patch (current for one version, different for another), pressing an arrow either retrieves the updated patch or sends the older version (grey button for older). And + is for patches that have not been sent, pressing + sends the patch. Each patch is linked to an issue. Some developers prefer to prefix their patches with issues numbers, but it's not required.

Communication between the client and server images is by VW's Opentalk. As of version 7.7 it has been rock solid (version 7.6 has issues with brokers timing out).

Every ten minutes a 'retrieve client state' command is put on a 'pending commands' queue. At the next polling action, the command is retrieved by the patch client and a 'state file', generated by the client application, is send back to the server. If any of the client states indicate a problem an email is sent to the support team and a red notification icon is displayed on both the patch manager and issue library web page, as well as our in-house wiki. Pressing the icon shows a formatted view of the state file with errors in red. We are similarly notified if communication with the client fails, which is detected by an absence of polling activity.

A similar mechanism is used to retrieve client walkback files. The client image already emails the walkback file, so this is a backup process. It retrieves walkback files that were not able to be sent, and it builds a useful history of client errors (which, fortunately, continue to decline).

Not having to manage the patch files manually has proven to be a big productivity gain.

Graphing with Google Charts

2011-04-07T14:00:00.000-04:00

We use Google charts to show various issue metrics. As is typical of a Smalltalk developer, I built my own class to manage the API calls. When I first looked into Google charts (two years ago, I think) I could not find community sourced options and building my own proved educational. Props to those that do build general purpose Seaside components, it's not trivial. My stuff is narrowly focused, and even that took some work.
My latest update added 'age' colours to an issue backlog bar graph. Something that may look like this...

Simple enough to do, but I tripped over an interesting API quirk. Originally, if you wanted to show a stacked bar graph, each value needed to have the lower value added to it (using cht=bvo, for chart type bar, vertical, overlap). So, if you wanted to show a bar with 10, 20, 15, 12 as stacked colours, the series was defined as 10, 30, 45, 57. Works great and it's easy to write the code to adjust the numbers. But, if one of the numbers is zero, the simple adjustment fails. If we want to graph 10, 20, 0, 12, the adjustment would be 10, 20, 20, 32. But, that second 20 ends up overlaying the colour of the first 20. What you really want is 10, 20, 0, 32.

At least that was true back then. Using the Chart Wizard shows that you can use...

chxt=y
chbh=a
chs=300x225
cht=bvs
chco=00CCFF,0099FF,0066FF,0000FF
chd=t:10,10|20,20|15,0|12,12
chtt=Vertical+bar+chart
...to get as stacked chart like this in the Live Chart Playground...

It's stuff like this that makes the general purpose Seaside graph components impressive. My original charts used cht=bvo instead of cht=bvs. Maybe it was supported, but I didn't see it (Actually, I still use the cht=bvo in order to keep the legend order the same as the graph colours). It takes time to keep up on all this, while still doing your regular job. Thanks to those that do.

Issue Library - how it started

2011-04-06T13:32:00.000-04:00

A To Do list grows up.

Some background on one of our projects: our in-house Issue Library, a Seaside application to track workflow.

The Issue Library started off as a simple online 'to do' list, a way to track my own workload and to experiment with Seaside code. Others started to use it, and eventually it became a real application. A nice bit of guerilla marketing; a home grown issue tracking project would never have been approved. We did look at some open source options, but in order to make an issue tracker useful to a wide audience, it seems that you need to have a lot of options, with the app eventually getting really big and complicated. Gjaller http://www.gjallar.se/ looked promising: I really liked the idea of working with a configurable Seaside code base. But I had no luck getting it to work.

The design goals were to make it web based, simple, easy to update and more than a toy. It needed to be real enough to learn the solid Seaside development details. Data is stored in XML files, one per issue. Files are read at image startup and written when changed. Since the Seaside image is the only thing that updates the XML files, the data in memory and on disk is consistent (and trivial to check). Object are always in memory which give us excellent performance. Writes are fast, about 10ms. I've used this technique with my home 'Quicken' type app. As long as the image size is reasonable, it works great. Right now the image, which also runs our Patch Manager, is about 225MB. As it grows, old issues will be archived.

Objects which map to XML files are subclassed from our an abstract class with all the XML support code. Read / write is done with #loadFile + #parseXML and #saveFile + #buildXML.

Parsing is done with VW's XMLParser

[xml := [XML.XMLParser new

validate: false;

parseElements: stream]

on: XML.BadCharacterSignal

do: [:ex | ex resume]]

Values are then extracted by interating over the XML elements. Each element is sent as an argument to #parseElement: which subclasses implement. As each element is found, the corresponding value is set and the next element processed.

A utility method is used to find the tags.

parseElement: anElement tag: aString doText: aBlock

anElement tag isNil ifTrue: [^false].

anElement tag type = aString ifFalse: [^false].

anElement elements isEmpty ifTrue: [^true].

aBlock value: anElement elements first text.

^true

For example, extracting the 'title' element is done with...

(self parseElement: anElement tag: 'title' doText: [:value | self title: value]) ifTrue: [^self].

...title is initilized to a new String, so if none in found in the XML file, or if it's empty, the title will stay as initialized. Other string attributes are set to '<none>' so show that they were never set.

Collection attributes are extracted by iterating over the nested collection, using another utility method...

parseElement: anElement tag: aString doEach: aBlock

anElement tag isNil ifTrue: [^false].

anElement tag type = aString ifFalse: [^false].

anElement elements isEmpty ifTrue: [^true].

anElement elements do: [:each | aBlock value: each].

^true

To extract the list of comments, we use...

(self parseElement: anElement tag: 'comments' doEach: [:value | self parseCommentElement: value]) ifTrue: [^self].

...we create a new instance of a comment object and pass on the XML elements for it to parse...

parseCommentElement: anElement

| comment |

((anElement tag type) = 'issuecomment') ifFalse: [^self error: 'XML parse error... expected ''issuecomment'' tag'].

comment := TSissueComment newForElement: anElement issueFile: self.

self comments add: comment.

Building the XML is done by adding XML nodes to an XML document. This could probably be made cleaner with some utility methods, but it's a very static method.

buildXML

| document node |

document := XML.Document new.

document addNode: (XML.PI name: 'xml' text: 'version="1.0"').

document addNode: (node := XML.Element tag: 'issue').

node addNode: (XML.Element tag: 'id' elements: (Array with: (XML.Text text: self id printString))).

node addNode: (XML.Element tag: 'title' elements: (Array with: (XML.Text text: self title asOneByteString))).

...

#asOneByteString is a hack to eliminate any characters over an integer value of 255. We have Mac users that are able to paste in characters that the XML.Document will not tolerate.

Collections are just nested nodes...

buildCommentsXML: aNode

"allow for comments as strings, for initial conversion to comment objects"

| listNode |

self comments notEmpty ifTrue: [

aNode addNode: (listNode := XML.Element tag: 'comments').

self comments do: [:each |

listNode addNode: (each buildCommentsXML)]].

Where #buildCommentsXML adds text content to a new node...

node := XML.Element tag: 'issuecomment'.

Simple, low tech and robust enough to handle our needs.

Getting started

2011-04-06T13:25:00.000-04:00

Hello.

Maybe it's time to start writing this stuff down. Over the years I've benefited from the public postings of others, and I've done a few of experience reports at conferences, usually with positive feedback. I'd like to do more, hence this blog. Let's see how this goes.

I work in Smalltalk. Learned it in a York University night school AI class in 1990 and switched to it professionally in 1993. Something of an orthogonal move from my work as an MVS systems programmer at EDS. Lately I've spent a lot of time with Seaside. I really like where browser technology is going: smart remotely connected clients, obviating the need to install remote code. And on the Internet, no one knows you're writing Smalltalk. I buy in to Avi Briant's view of Seaside 4.0: lots of Javascript talking JSON to the server (check out his talk at Smalltalk Solutions 2011 (hopefully the videos will be available soon)

Some of the stuff I've tripped over my be of value to others. I'll write about various Seaside projects, my work with automating SUnit tests and running the Toronto Smalltalk User Group. All Smalltalk; no sports, travel or home renovation. As with anything, I believe this will get better with experience. Eventually this may even by useful.

For those in the Toronto area, check out the Toronto Smalltalk User Group at www.smalltalk.ca (or www.smalltalk.toronto.on.ca)