Agile Ditch Digging

I'm a strong believer in agile software development. Every project I've worked on evolved into something every different from what it started as. Agile believes in trapping that type of change; it believes in understanding that you will know more as you build; agile deals with "managing ignorance".

On our current project we are migrating from a VW fat client to a Seaside application with short GS transactions that uses the legacy meta model, UI layouts and data structures. Our design isolates the UI layer in the VW Seaside server and the domain layer in GS.  We parse the VW window spec into Seaside components and communicate with GS using RESTful data calls (nested arrays of strings and oop numbers), and get XML back.

All of that works well: it's quick, looks nice, scales and is much easier to wrap SUnit tests around. And it was built using agile development techniques. Mostly.

I find that agile works best in the 'construction' side of the work, where you can define the user stories and measure the pace of delivery (the ditch digging). There is, however, another flavour of software development, the R&D or 'creative' side, like designing the framework and tools that the application code rests on. It's not something the user sees; it's just part of the application's fabric.

Recently we thought about how we were dealing with widget level feedback. That's where you enter 'abc' in the 'name' field and get feedback when you move on to another field. If 'abc' is an invalid value, it would be useful to see that right then ('on blur'; when the widget loses focus), instead of waiting until the 'save' button is pressed. The same goes for updating depending values: if the 'comment' defaults to 'name', having it change when 'name' is entered is useful.

Our original approach was to do this behaviour in the web code, since it had access to the display components. We soon realized that is was more important that the code have access to the domain and be able to reuse the legacy validation code, so we moved the logic to GS. Seemed simple enough.

Turned out that shifting that one responsibility triggered a lot of framework redesign. Originally, the web component built up the set of changes, and passed them to GS on 'save'. It was simple and worked. With the field level code on GS, each 'on blur' event had to trigger a GS call and, more importantly, had to package the full view update state into call so the domain code would see the current displayed state . Performance is not an issue, since each call takes from 10 to 50 ms, but the code change was more complex than it originally seemed.

I could find no good way to communicate the status of this 'big bang' change. The problem was complex, then things got delayed due to other work, and we made some critical design changes as we understood the technical issues better. None of that was well communicated. From the outside looking in, the project just stalled. Precisely the kind of optics you don't want, and the kind of problem that agile techniques are supposed to deal with.

I simply do not know how to measure 'thinking time', especially my own.  Finding a solution may take me five minutes, or five hours.

It was interesting to feel the pendulum swing from 'creative' to 'construction' as the work progressed; the construction phase is so much easier. Easier to do, easier to measure and easier to manage. Everyone is more at ease when you can show that you're 80% done, vs. just telling them that "you're close".

Digging a ditch is easy, assuming you know how.


Simple things should be simple. Complex things should be possible.