Time's up, put down your pastry bags!

I'm having high school flashbacks right now. The sabbatical ends tomorrow morning, and I'm up at quarter past 1 "cramming". So much for "coming back refreshed".

So I've got things mostly done. Some small functional holes still exist, but I really need some sleep. I imagine I'll be sneaking in the remaining features when i have pockets of time.

There was a lot to this GUI - and the things I'd pegged as being easy were the things that bit me the hardest. I have so much appreciation for the Information Architects (or Information Architecters) of the world.

No longer making sense. . must get sleep.

I had Raisin Bran for dinner. Not even real Raisin Bran, I had
the Malt-O-Meal knock off.

Duty Calls. . .

As it turns out, the Process Designer thing I'm building is missing an important feature. That feature is what we in "the biz" call "Save". Its used to take important data that's teetering on the edge of existence, sustained only by the good fortune that electricity has managed to flow constantly through the millions of circuits that compose a stick of RAM - and securely deliver it to a much safer place, like a hard drive, whose promise of data integrity can only be defeated by bumps, magnets, static electricity, liquid, dust, above-normal temperatures or any other condition that will probably happen soon.

So anyway - Nitram's been waiting for me to get this feature finished so that he can use the tool to help him test and exercise the engine. Dragging and dropping sure beats hand-editing XML, right? Fair enough. So, like a good partner, I fired up Photoshop and worked on the UI.

I cut some new graphics for the different node types, tweaked some CSS, and upgraded the look a bit, and added some bells and whistles. I also discovered that my home-brewed method for detecting the center of a Bezier curve wasn't always working. If you remember, I'm adding a button to the middle of each of the connecting lines - well, in some cases, the button was rendering nowhere near the line. So, i had to roll up the sleeves and do this right. I found de Casteljau's algorithm which is perfect for doing just this. I'm not a heavy math guy, but i was able to come up with a cheap implementation in js:

Transition.midpoint = function(e1,cp1,cp2,e2){
   var top = [_mp_line(e1,cp1),_mp_line(cp1,cp2),_mp_line(cp2,e2)];
   var mid = [_mp_line(top[0],top[1]),_mp_line(top[1],top[2])];
   return _mp_line(mid[0],mid[1]);
 
   function _mp_line(p1,p2){
       return {
       x: (Math.abs(p1.x-p2.x)/2) +Math.min(p1.x,p2.x),
       y: (Math.abs(p1.y-p2.y)/2) +Math.min(p1.y,p2.y)}
   };
};

All this really does is take the end points and control points that define a curve, and find the midpoints between each adjacent pair. Then it finds the midpoints between each of those midpoints, then again, until there is only one point left. Thats our guy. I think this could've been implemented more slickly using recursion, but i think it would actually make it a longer piece of code. Any way here's the jing of the algorithm, and some of the new UI in action. (Please don't mind the pimpley-looking drag circles and the misplaced text)

In all honesty, I spent a good bit of time today working out the details of the "save" feature. There actually are some interesting challenges in it. Since I've decided not to auto-save or make calls to the server at each event in the gui, There are potentially many changes that will need to be carried out by the time the user saves their work. Within one save request there may be tens of actions to perform, some nodes may get updated, others deleted, others created - some will have new Actions associated with them, some Transitions will be deleted, created, or moved. So, to get ready for this I wrote it out in pseudo-code to make sure that i'll be doing things in the correct order. I'm really almost done, Nitram, I swear. Maybe if I didn't blog so windily, I'd be done with it.

Oh yeah, I had Chix (hippie chicken) patty burgers, sauteed kale and baked beans for dinner whoooooo hoooo!

Rails forms...not so hot

Before running through the activities from the last two days, a word on rails forms. While this is not my first time using rails, I still fall into a state of confusion and bewilderment whenever using the forms on anything more complicated than the ideal one-object-to-one-form mapping. So far I am really displeased with form handling in the rails template language, it just doesn't seem to scale well to more complex objects.

That said, over the course of this project I've developed a pretty good approach for myself to keep my sanity. It basically amounts to two guidelines: keep it simple by not overusing helpers, think of everything as it will exist when passed into the params hash. In retrospect, these rules are fairly obvious, but they still work well as a mantra. I love the params hash, when used in conjunction the model helpers it is a very clean, JSON-like, solution. It feels intuitive to me. The rendered html, with its bracket notation (...input name="sibling[126067293][take]"...) is also very intuitive, you can "see" how it translates into the params hash. It's just the code that generates this html that sucks...hey, maybe I could just code straight html? Or make my own helpers? Need to think more about this. I wish I had more of an argument, but it's more of a feeling than anything else right now. More to come on this topic...

Yesterday and today:

• Solved JSON issue (from_json was not robust enough) with dave
• Played some foos!
  
• Process engine now passes first 8 of 20 workflow patterns
• Fixed process definition XML serialization bug
• Process engine now defaults all spawned tokens to wait-state on any fork, event code must "continue" the tokens, helper method created to facilitate this
  
• Controller integration
• Added wait_for_user_input helper method to process_engine
  
• Now supports custom view callbacks for controller interaction
  
• Default view handlers are in place for forks and end-of-process
  
• Created routing to handle process engine requests seamlessly and globally

Hey, JSON, WTF?

Ok, figured out the annoying JSON bug I was having yesterday. . . thanks to my use of virtual properties to traverse the model relationships and produce the json - my sub-arrays were getting double-JSON encoded. So instead of:


{name:'test node',
id: 234323,
type: 'decision'
transitions: [
{to_id: 432343, arrival:"top", departure:"left"},
{to_id: 343233, arrival:"right", departure:"bottom"}
]
}

I was getting:

{name:'test node',
id: 234323,
type: 'decision'
transitions: [
"{to_id: 432343, arrival:\"top\", departure:\"left\"},"
"{to_id: 343233, arrival:\"right\", departure:\"bottom\"}"
]
}


Notice the difference? Neither did I.

I actually got a number of items off of my list today. One of them, "Multiple hotspot/dragpoints per node" means that creating a transition between two nodes is more than defining a "from" and a "to". . . You might want to connect the bottom of one node to the left side of the other. Of course there's no functional difference, but users would become annoyed and diagrams messy without this kind of control. jQuery and my caveman-level CSS knowledge made it pretty trivial to dynamically add to each node, North, South, East, and West drag zones and drop zones. Now why am i calling them top, bottom, left, and right in the code? North, South, East and West sounds so much cooler. I wish i had thought of this earlier. Perhaps we can save these changes and promote version 2.0 with "Improved Variable Names! Upgrade Now!" or "Code base features even more consistent indentation!"

It hit me (while taking a power drill to my laptop (in order to justify an upgrade to the MacBook Air (Thanks Jared!) ) ) that users of our system will need to be able to select transitions. Up to this point, transitions have been a purely visual thing, simply reacting to the location and size of their related nodes. Selecting could be complex being that 1) nobody wants to try to click on a line that's curvy and only 2 pixels wide. While it is possible to detect whether or not the click was along the curve, this type of user experience is likely to make all the Davie recipients weep. 2) these Canvas elements (1 per curve, remember) overlap each other like the f'endas on a waitress - so the highest z-index canvas steals all the clicks. I quickly IM'ed Nitram to see if he had any ideas. He said, "how about a circle in the middle of each transition? -And you click it"
"Well Mr. Smarty-Pants, How's it feel to always have things like 'Answers' and 'Ideas'? I hope that works out for you!" So, i took his weak stump of an idea and reworked it tirelessly and I came up with this: Each transition will have a circle in the middle of it. And you can click it. This action will bring up the properties for that transition. This didn't solve the overlapping canvas problem - so what i've done is this: Mid-points for all transitions are determined and mapped to absolute page coordinates. Clickable DIVs are placed at these locations at the highest z-index. Seems to work so far. I can't wait to put a real skin on this thing. I hope i have time for that.

Unfortunately, each line I strike off, has me adding a line or two. My list currently looks like this. it may not make sense to you but, i mean, its MY list.

build/integrate graphics
finish: make transitions selectable/visually selected
finish: building property editing
finish: saving nodes.
panning / zoom
add more quality
self-transitioning nodes.
stop scratching it

Ok. I've gotta go, my laptop still works.

PS - I inadvertently got "fancy" tonight and ate a green curry w/ chicken out of a real coconut @ a local restaurant tonight. It wasn't what I ordered, but its what they brought me and what i had to pay for.

Another day...

Today was mostly research along with several disturbances. I looked through the rails controller api, messed with routes.rb, and learned more about the ruby language, which was great. I was sidetracked by watching the Macworld 2008 keynote. Then, I saw this article in the nytimes which led to this paper which led to me wondering who in physics is actually performing scientific experimentation anymore...and on and on until I came to the fundamental meta-conclusion that the chief danger of our information saturated culture is tangential thought...and that I probably have severe ADD.

Update:

• Shaved nascent sabbatical beard this weekend. Fiancee Melissa gave it a thumbs down for our anniversary date. Not that I blame her, it was looking pretty bad and was itchy as hell.
  
• Cleaned up the model by adding a few polymorphic associations.
• Started work on process-engine + controller interaction.
  

getting there.

Well, I've been chugging through my side of things albeit somewhat slowly. Spent most of the day working with Nitram talking architecture and planning our last steps.

We made one particular change that took a little time. For each Node in our model, we have a corresponding Layout object. It contains the visual information needed to represent the node in the designer. The Node and the Layout have a one-to-one relationship. Each of the transitions in our model have a corresponding TransitionLayout object. I have been generating JSON to represent the Layout portion of the PDL for transmission to the client. Layouts used to have a direct relationship with TransitionLayout objects so building the json for this portion of the model was simple. "to_json" allows me to recursively traverse the object model pulling exactly what I want along the way. But now, We've updated the model like this: Instead of having direct relationship, A Layout and its TransitionLayouts are now only related through their corresponding Nodes/Transitions. In other words, a Layout belongs to a Node which "has_many" Transitions, and that transition, "has_one" LayoutTransition. This I believe is a better model, but does not work as nicely with the rails to_json method. Eat butter in moderation. So now I'm using "virtual properties" to do my traversing and having to_json pick them up as methods. This seems to be working out alright, although I've got to work out some oddities in the resulting JSON thats killing my JS.


The front end is coming along nicely and is pulling up processes from the DB and displaying them. A couple of small steps until we're able to update and save. Then, I think I'll be giving it some real graphics and get it looking smoov.

Had leftover fajitas - wasn't in the mood, so i wrapped 'em up and pretended they were burritos.

Time for bed.

Pier process engine is alpha...and an assortment of prophetic remarks

I am excited to announce that the Pier process engine for Ruby on Rails is alpha. I am defining alpha as a "functionally whole" system, in OS jargon: we have a kernel. Now that the process engine is alpha, we can begin end-to-end testing using the process patterns outlined here, and start building feature enhancements. In its primordial alpha state the rails process engine can (and you may want to sit down for this):

• Read and instantiate a serialized process definition.
• Create and load a token with contextual application data.
• Launch the token into a process flow
  
• Execute the flow: creating forks and merges, executing embedded logic
  

Wow, pretty amazing stuff, no? Well it is! The great part about reaching this stage is despite any the future headaches, strange bugs, and the inevitable its-just-too-slow refactoring, we have a foundation to work off of and a number of proven concepts that are past the point of no return. It's alive.

Great, so what's next? The goals for next week are to build out the engine and integrate more closely with the rails framework. First, in keeping with the test driven approach, I will build and execute the process pattern tests mentioned at the beginning of this post. Running through these detailed tests and exercising the system will likely reveal missing features and a host of bugs. Fun!

Next, one major requirement that I already know the tests will reveal is the engine's ability to collect user input at runtime. In order to facilitate this, I will need to extend the rails controller with process engine power. In terms of MVC architecture, and in the context of a single application, the process engine enhances the controller. Certain processes may need user input to continue, so my goal is to create a seamless and intuitive connection between the process engine, rails controller, and application views.

As a final note, I'd like to set aside the process engine sub-project and take a look at the big picture. Pier is a BPM solution, not just a process engine. Dave is doing incredible work on the process designer side and I'm really excited to see the two sub-projects working together.

Once these sub-projects are fully integrated and tested, Pier will be capable of bringing agility to the enterprise application development process. Our goal is to enhance business software planning and development by providing Business Analysts, Information Architects and Techs an intuitive "living" picture of the system as it grows from a couple napkin doodles to a production ready application.

On a larger scale, I think Pier will be an essential piece of the emergent group of light weight, service oriented solutions built off the Rails framework. These solutions will promote and enable the development of multi-application systems. As everyone already knows, web-applications are talking to each-other. SOA was developed in the business world to define and encapsulate this phenomena. Public facing sites are quickly standardizing and catching up. One of the "third-arms" in the evolution of multi-application systems will be applications that do nothing more than orchestrate other applications ( Yahoo Pipes is an example of this concept). Pier is being built with these SOA concepts in mind.