Mu

Improved Sandbar Forms

2010-12-12T20:54:00.000-05:00

There has been a flurry of improvements made to the Clojure web stackduring the past year. Compojure has matured, there's some non-trivialcode available (for example, see Brian Carper'scow-blog) and the newSandbar library whichbrings a higher-level of abstraction on top of Compojure and Ring. Fornow, it provides a stateful session mechanism, authorization +authentication and a clever way of defining forms layout, processing andvalidation. Also there's much more to come, you can look at the detailsin the followingroadmap.

For today, I'll discuss the recent changes to the formsnamespace. There has been a lot of work done on that part during thepast week, but the changes to the API are minimal. I'll go over eachoptions of the defform macro, but first lets talk about the codebehind. As Stuart Halloway stated in hisbook, thefirst rule of the Macro Club is: "Don't Write Macros." So the mostsignificant alteration is the rewrite of the defform macro. Previously,it was a big piece of code (134 lines) generating a bunch of definitionsof all sorts. It was quite hard to modify and had the usual constraintsof using macros that way. It now has been replaced by a much more simplemacro that call the new make-form function.

Lets see a sample form written for the current (0.3) version of Sandbar.

(forms/defform group-form "/group/edit"
  :fields [(forms/textfield :name)
           (forms/select :region
                         (db/all-region)
                         {:id :name :prompt {"" "Select a Region"}})
           (forms/textarea :description)]
  :load #(db/fetch-group %)
  :on-cancel "/groups"
  :on-success #(do (db/store-group %)
                   (session/flash-put! :user-message
                                       "Group has been saved.")
                   "/groups")
  :properties {:name "Group's name:"
               :description "Description:"})

(defroutes group-form-routes
  (group-form (fn [request form] (views/layout form))))

It was nice but had a severe limitation: you were forced to use a fixedset of routes. When creating a record, "/new" was appended to the givenURI else the "id" key was used. Also, the fields option tended tobecome cluttered with the data source bindings. These are the two mainpoint I'll address here.

Firstly, lets rewrite the fields option, it's quite similar intaking a vector of field descriptions. The difference is in the fielddescription functions that are taking the name of the field (as akeyword) followed by a list of optional key/value pairs.

  :fields [(forms/textfield :name
                            :label "Group's name:")
           (forms/select :region
                         :prompt {"" "Select a Region"})
           (forms/textarea  :description
                            :label "Description:")]

Each field functions can take a label option, then most have anoptional boolean required option which auto-generate thecorresponding validator for that field and finally there's theprompt option for the select field. Any other options will beadded to the field's HTML attribute.

Secondly, there are the new options for managing the form action andmethod attributes. Each are prefixed by create or updatefollowed by -action or -method whether it's for an action ormethod. They can take parametrized routes which will get theirparameters replaced by the matching route parameters from the incomingrequest.

  :create-action "/groups"
  :update-action "/groups/:id"
  :update-method :put

Thirdly, here's the new bindings option which take a map of fieldnames followed by their respective binding information in a map.

  :bindings {:region {:value :id
                      :visible :name
                      :source (constantly (db/all-regions))
                      :data :id}}

The source option needs a function that fetch the relevant data,the visible option determines what field to show on the page andthe value and data options represent the actual value to usein the source data map and the form data map respectively.

Finally here's the whole code for this example using the new formsfeatures using RESTful routes.

(forms/defform group-form
  "Form handler for Group entity."
  :fields [(forms/textfield :name
                            :label "Group's name:")
           (forms/select :region
                         :prompt {"" "Select a Region"})
           (forms/textarea  :description
                            :label "Description:")]
  :load #(db/fetch-group %)
  :on-cancel "/groups"
  :on-success #(do (db/store-group %)
                   (session/flash-put! :user-message
                                       "Group has been saved.")
                   "/groups")
  :create-action "/groups"
  :update-action "/groups/:id"
  :update-method :put
  :bindings {:region {:value :id
                      :visible :name
                      :source (constantly (db/all-regions))
                      :data :id}})

(defroutes group-form-routes
  (GET  "/groups/new"      request (group-form request))
  (POST "/groups"          request (group-form request))
  (GET  "/groups/:id/edit" request (group-form request))
  (PUT  "/groups/:id"      request (group-form request)))

Furthermore, other modifications include that most defform optionscan take a function of the request instead of just a value and thatredirection URIs in the on-success and on-cancel options canbe parametrized.

The forms namespace is still a work in progress and is still beingimproved. Particularly there is talk concerning the way forms getrendered, which currently lack flexibility as pointed out by David Nolenin thisthread.Nothing has been done yet in this regard, so it's the right time foranyone interested to chime in this discussion.

Getting Back on Rails

2010-07-23T18:00:00.002-04:00

I've been pretty busy in the past months and didn't felt inspired enoughto write. Today is time to get back to writing in a human languageinstead of a computer one.

Coming Back Home

I've been involved in web development for more than ten years, but for along time I could only be labeled a designer. A little more than fouryears ago I began developing web sites using that awesome littleframework named Ruby on Rails. After toying with it for less than ayear, I gave it up to go on the dangerous path of trying to reallyunderstand what was going on behind all that magic. After learning a lotwhile dabbling into some lesser known frameworks likePylons, Helma andthen Compojure, I now gotthe chance to code in a more professional setting using Ruby on Rails.

Good Old Rails Has Changed

For the better! Since four years ago, Rails has visibly matured and itis now used in a lot of very successful web sites.All this popularity pushed the framework to evolve and it has becomemuch more capable with the help of an army of plugins. Some simplefeatures were kicked out of the main codebase to be replaced byplugins. For example, the built-in pagination was removed from Rails 2.0and turned into a plugin called classic_pagination. It has fallenout of favor for another plugin,will_paginate, which isreally much better.

Back then there was a generator called login to build basicauthentication and user management features into your application. Thisgenerator has been deprecated and now there'smultiple pluginsavailable. I've only triedAuthlogic by now and I'mpleased by its simplicity and customizability so far.

And there's a lot more! More conventions, more syntactic sugar, moreready to use plugins. One notable change is that Rails is now RESTful,meaning that it uses the HTTP protocol intelligently for routingrequests. For example, if a DELETE request with only the name of thecontroller is sent, by convention, Rails will call the destroyaction. One last thing to note is that ERb template files have given upthe .rhtml file extension for a more versatile .erb appendedafter the output file extension.

Development Environment

Well, I'm a staunch Emacs user, so I ended up using it for developingRails applications. Nonetheless, I was curious about the degree ofsupport offered by some IDEs, especially from the latest release ofNetBeans which some people say is getting pretty good. I've tried it fora couple of hours to find out that it was really not the kind ofenvironment I'd like to work in.

First, the wizard wasn't able to create a database in Derby because ittried using the inexistent root user. Then it was the included GlassFishserver that didn't want to use another port than 8080, completelyignoring the provided configuration file (which is WEBrick specific itappears). Those are all minor glitch, but it continued like that for anhour or two and soon I felt the need for this trusty Emacs texteditor. The most glaring bug that really pushed me off NetBeans wasthe configuration screen for syntax highlighting, it was at some pointusing 90% of both cores on my machine! No wonder everybody saysMicrosoft have the lead in the IDE world, even I must say that it's adecent development environment after manually setting Emacs likeshortcuts (the included ones just don't cut it).

Emacs Configuration

There's two mode for Rails on Emacs, namely emacs-rails andRinari. The first one is older and isquite outdated. It has been replaced by a new mode namedemacs-rails-reloadedby the same author. It provides navigation between the multitude offiles in a Rails project, make it easy to call rake or generators and alot more. Rinari is a lightweight alternative that I haven't tried yetbut it looks quite capable too.

For editing the ERb templates, you can use thenXhtml mode,which is already included in the Windows port of Emacs. I had somecompatibility issues between that version andemacs-rails-reloaded and had to install thelatest version.

Deployment

I had the opportunity of deploying an application and find out thatthere were great improvements made on this side.

First and foremost, is that incredible tool calledCapistrano. It makedeploying Rails application in a consistent and reversible manner abreeze. With very little configuration, it will copy your files in adate/time tagged directory, help you easily start/stop your applicationserver(s) and get back online previously deployed versions easily. It'seven more than just a tool for RoR projects, you can understand it asbeing a sort of Makefile for remotely controlling multiple servers.

Finally, back in the days, most hosting provider offered only fastCGI ormod_rails on top of Apache for running your applications. Today there'smore options, but I won't go through all of them as I've still got muchto learn about them. For now I've learned thatPassenger (previously mod_rails ormod_rack) is the recommended way of deploying a Rails app, it can useApache or Nginx as its web server.

Cygwin Tip #1

2010-04-18T12:41:00.000-04:00

Calling Java Under Cygwin

Cygwin is a fantastic piece of software for those who, like me, can'tlive without a Linux-like command line and environment. But for anyperson who have tried to blend two completely different cultures, weknow it's not always easy to make them play well together. In this postI'll explain how to make Java's Windows version feel more like a Cygwinapplication.

The main problem comes from the way a file path differs between Windowsand the POSIX standard. The most obvious difference is the pathseparator issue, in Windows world it's \ while for the rest of theplanet it's /. Another one is the way the root path(s) are beingrepresented, in Windows each drive/partition have it's own letters withits own root path. Cygwin being POSIX-based has a single root path(/) and maps the Windows letters in a directory called/cygdrive. Those problems are easily circumvented using a thecygpath command:

$ cygpath -aw '/cygdrive/c/Program Files/Java/jdk1.6.0_18/bin/java'
C:\Program Files\Java\jdk1.6.0_18\bin\java

That's Not Enough

It wouldn't be hard to call this command on the classpath argument whencalling java from cygwin, but there's another hindrance. The classpathusually contains more than one path, these paths being separated by acolon (:) in POSIX systems or by a semicolon (;) under Windows. It wouldbe also quite tiresome edit java calls manually so I came up with alittle Ruby script that can do the job for us.

#!/bin/ruby
# Slightly obfuscated cygwin + windows java wrapper, automate cygpath

cpi = ARGV.index("-cp") + 1
cp = ARGV[cpi] if cpi

XBCP = "-Xbootclasspath/a:"

xbcpi = ARGV.index{|i|i=~/^#{XBCP}.*/}
xbcp = ARGV[xbcpi] if xbcpi

if cp or xbcpi
  def convert_paths(paths)
    paths = paths.gsub(':', ';').split(';')
    paths.map{|p|`cygpath -aw #{p}`.strip}.join ';'
  end
  ARGV[cpi] = convert_paths(cp) if cp
  ARGV[xbcpi] = XBCP + convert_paths(xbcp.sub(XBCP, '')) if xbcp
end

java = '/cygdrive/c/Program Files/Java/jdk1.6.0_18/bin/java'
cmd = [java].concat ARGV

def e(s); "\"#{s.strip.gsub('"','\"')}\""; end

exec(cmd.map{|a|e a}.join(' '))

This is more of a hack than a solid solution, but it's enough for mycurrent usage and hopefully yours. Now the Windows java commandcan take POSIX paths without complaining under Windows, isn't thatmarvellous!

Gödel, Escher, Bach and Smartphones

2010-04-15T12:46:00.000-04:00

After many many years, I'm still (slowly) readingGEB. That's because I'm wasting tomuch time reading the (whole) Internet to bother with printed books!Yet, I like to finish what I start (eventually) and yesterday I found areally interesting quote in that book. Mind you, it has been writtenthirty one years ago, at a time when personal computers were a noveltythat only a few lucky ones could afford. Most computer usage(Punched Cards!) was being done inUniversities laboratories and these machines weren't fast enough to domost of what we take for granted today.

Dogs, Garbage and Light Bulbs...

The world have changed a lot since then and computers are incrediblymore useful now a day's. While discussing the Church-Turing thesis inchapter seventeen, Mr. Hofstadter goes on the representation ofknowledge about the real world and say:

Because of the complexity of the world , it is hard to imagine a little pocket calculator that can answer questions put to it when you press a few buttons bearing labels such as "dog", "garbage", "light bulb", and so forth. In fact, so far it has proven to be extremely complicated to have a full-size high-speed computer answer questions about what appear to be rather simple subdomains of the real world.

Hard to imagine back then, but today it's quite commonplace to seepeople asking such questions to their smartphones. Which bring me to thepoint of this post, is the Web can be considered a form of artificialintelligence? I'd say no, it's just a hack, yet a pretty useful one. Theraw data that can be found here isn't that different than theinformation that could be gathered in any other format. It is the searchcapabilities offered by various engines that truly makes the greatnessof the World Wide Web. Still, these facilities aren't quite smartenough, we still have to choose carefully the wording of our queries andclick around.

The Semantic Web

This is the next step, and a big one also. A lot of have been written onthe subject, a lot of code too and thus far that concept hasn't yielded(easily) usable tools. I think that most problems concerning semanticalrepresentation of data are intrinsic to the nature of information. AsOleg once said: "Without a reader, a book has no meaning." It's allabout asking the right question. And here is the missing link, doesnatural languages are good enough? Where's the real burden located, informulating a query or in understanding it?

Redesigning ClojureQL's Frontend

2010-02-22T17:02:00.000-05:00

I've not updated this blog in a while as I've been pretty busy for thepast few weeks. I've got many projects on the table and one of them isstarting to get really interesting. I'm obviously talking about thesubject of this post,ClojureQL. I've mainly beenworking on the backend since the end of last year. In the meantime,Meikel has started areworkof the frontend to provide a cleaner API, free from the magicalartifacts introduced by using macros like in the current (0.9.7)version. This rework have been triggered by apostcarefully explaining the issue, courtesy of Zef.

I won't go further than a linkfest so here are the important links forthose interested in the future of ClojureQL:

We're also ready to receive your suggestions on the brand newclojureql group. Please,visit the Wiki pages and tell us about what you think.

Google AI Challenge in Clojure

2010-02-07T16:47:00.001-05:00

These days, the web have been noisy about the brand new Google sponsoredAI Challenge organized by theUniversity of Waterloo Computer Science Club. I find that it's a greatinitiative and is a good occasion for amateur AI researchers like meto do something else than web development or database relatedcoding. Being fond of Clojure, I've taken on the task of making astarter package for this language and making it available on a GitHubrepository. It'snot an official package yet, but it shouldn't be a problem as theorganizers seems really open.

The code has been translated from the Java starter package. I've triedto keep the map code close to the original, so it may not be the mostidiomatic Clojure code. There's four global refs containing the state ofthe game: width, height, walls and players. Thetwo first are simple integers, walls is an hash map keyed withpoints (which are vectors) and the last one a vector of points. There'ssome convenient functions to access that data, wall? to tell if awall is found at the given coordinates and two others to get eachplayers position, you and them. To move your bike you mustcall make-move with one of :north, :east, :southor :west. The map code end up being nearly half the size of theJava version.

With this code we can rewrite the sample random bot in Clojure, I'llspare you any more explanation and only show the code.

(load "map")

(defn valid-moves [x y]
  [(when-not (wall? x (- y 1)) :north)
   (when-not (wall? (+ x 1) y) :east)
   (when-not (wall? x (+ y 1)) :south)
   (when-not (wall? (- x 1) y) :west)])

(defn compact [coll]
  (filter identity coll))

(defn choose-at-random [coll]
  (let [size (count coll)]
    (when (< 0 size)
      (nth coll (rand-int size)))))

(defn next-move []
  (choose-at-random
   (compact
    (apply valid-moves (you)))))

(defn game-loop []
  (loop []
    (initialize)
    (make-move (next-move))
    (recur)))

(game-loop)

Hack and be mery!

Update: I've updated the GitHub repository with some improvementsand instructions on how to compile your entry. While discussing with oneof the contest organizer over their forum, I realized that even thoughClojure is more than fast enough for this kind of task, the Clojure jarloading time take a good chunk of thefirst turn time,so be careful. By the way, it's now an official package.

Closure Library Tutorial: Tasks (part 2)

2010-01-31T22:30:00.000-05:00

It's been nearly two months since I've written about JavaScript andnow is the time to follow up on theprevious partof thistutorial. Sincethen, there have been some changes to the Closure library, nothingmajor, mostly improvements to the documentation and bug fixes.

Sliders

Adding a Slider is unsurprisingly as easy to do as any othercomponents. Contrary to those we already used though, it doesn't comewith a nice interface with pre-made CSS and images. Closure let youdecide how your sliders will look like, which is comprehensible as thiscontrol is really versatile and it would be hard to come up with a goodgeneric design for it. A slider element is composed of two DIVs, onerepresenting the slider itself, the other being the thumb. Both have aclass name to let you control their appearance. For the slider elementthis name is composed of a prefix found in the Slider'sCSS_CLASS_PREFIX static property followed by either of"-horizontal" or "-vertical" depending on the orientation. The thumb hasonly a single class name determined by the THUMB_CSS_CLASSproperty, note that as the thumb is contained in the slider element,it can be styled depending on orientation too.

There's some rules to obey while styling a slider:

a slider cannot be inlined, but you can use inline-block;
a slider cannot be floated;
the thumb position style must be set to absolute or relative.

Note that it's preferable to make both elements hide their overflowingcontents to prevent scrolling bars from appearing.

Using Sliders

As sliders cannot be floated, we have a problem with the existingcode. If you remember the way tasks were constructed in the previoustutorial, the action buttons were added to the summary DIV directly withtheir float styles set to right. This isn't the correct manner of doingthings anyway, so we'll fix that first. We'll group the task controlsinto a single floating element.

    this.summaryControlsDiv = goog.dom.createDom('div', { 'class': 'controls' });
    this.summaryDiv = goog.dom.createDom('div', { 'class': 'summary' }, this.summary, this.summaryControlsDiv);

This break some ugly part of our code, the makeActionButtonfunction is being passed the task DIV and renders the button beingcreated by selecting the appropriate child node. We'll just do a quickfix and correct this issue in the next section.

    button.render(element.childNodes[0].childNodes[1]);

Another more subtle change we need to make is in makeButtons, wemust change the order in which the action buttons are created. We mustalso change some styles and add an entry for the new CSS class.

      .task .summary .controls { float: right; }
      .task .description { background: #eee; border-top: solid 1px #333; }
      
      .taskButton { margin-top: -6px; margin-left: 7px; }
      .editorButton { margin: 0.2em 0.3em 0.5em; }

The following code create a slider, we'll stop event propagation thesame way it's done for tasks' action buttons.

mu.tutorial.tasks.Task.prototype.makeSlider = function(task, element) {
    if (task.parent.id == 'taskList') {
        var slider = new goog.ui.Slider;
        slider.createDom();
        slider.render(element.childNodes[0].childNodes[1]);

        slider.addEventListener(
            goog.ui.Component.EventType.CHANGE,
            function() { /* TODO: something... */ });

        goog.events.listen(
            slider.getContentElement(),
            goog.events.EventType.CLICK,
            function(e) { e.stopPropagation(); });
    }
}

We need to call this function from makeDom just before creatingthe task buttons. To make the page actually show the sliders, we'll addthe styles below.

      .goog-slider-horizontal, .goog-slider-thumb {
        overflow: hidden;
        height: 20px;
      }
      
      .goog-slider-horizontal {
        background-color: #ccc;
        display: inline-block;
        position: relative;
        width: 200px;
      }
 
      .goog-slider-thumb {
        background-color: #777;
        position: absolute;
        width: 20px;
      }

Cleaning Up

Earlier, we talked about how bad it was for makeActionButton tohave to know where to render the button. We'll take the time to cleanthat up. If we look back at the code, one thing is obvious, we pass lotsof arguments around. That make the code overly functional and this isn'ta good approach here. Another thing is that we carefully initializedprototype members for every elements contained in a task and we end upnot using them.

For this section I'll skip the code, so use your imagination. Firstthing to do is to remove the element argument from functions havingit. For the controls, we'll use the object's properties,summaryControlsDiv for action buttons and sliders, andeditorContainer for editor buttons. Finally, we need to replacethe taskDiv local variable in makeDom by a property thatwe'll name element. We can't use it directly in our callbackfunction as it is a closure and this doesn't point to the taskobject. We can work around that simply by using a temporary variable,we'll rewrite that code later anyway.

Making the Sliders Do Something

To put the sliders to good use, we'll make them control the priority ofa task. We'll first make the tasks display their priorities, to be able totest things manually.

    this.priorityDiv =  goog.dom.createDom('div', { 'class': 'priority' }, this.priority.toString());
    this.summaryDiv = goog.dom.createDom('div', { 'class': 'summary' },
                                         this.summary,
                                         this.summaryControlsDiv,
                                         this.priorityDiv);

We'll also have to add styling for the new priority DIV, it must floatto the right and have some padding on the right side. Now, lets add twothings to the makeSlider function. Firstly, setting the slidervalue to the priority of the task currently being created. Secondly,replacing our TODO comment by changing the priority property as well asthe content of the priority DIV.

        slider.setValue(this.priority);

        var task = this;
        slider.addEventListener(
            goog.ui.Component.EventType.CHANGE,
            function() {
                task.priority = slider.getValue();
                task.priorityDiv.innerHTML = task.priority;
            });

There's a small bug with this code, can you find it?

If you delete or mark a task as done, then undo that action, you'll seethat the resurrected tasks' slider thumb is set at its lowest value. Ifyou insert an alert to display the sliders' value when the task isrecreated, you'll see that it's value is correct. Also the slider behave~~as if the thumb was at the correct position~~strangely,it can even go into infinite loop. This all points out toward the factthat when we recreate our tasks, they're being added to an invisibleelement. We'll reorganize our code in the next section to circumventthis issue.

Sorting Tasks

The sliders are now working (with some issues) that's good but itdoesn't add much to our little application feature-wise. The obvious useof tasks priority would be to sort them. We'll need to do some majormodification to our code to do this though. Currently the tasks addthemselves to their containers and appear in whatever order they'rebeing appended. We'll need to create a new object to represent tasklists. I won't be showing all the changes made, just the most importantones, you can always refer to thefinal codeif you need more details. First, we'll need to provide a new class namefor our TaskList prototype.

goog.provide('mu.tutorial.tasks.TaskList');

mu.tutorial.tasks.TaskList = function(name, data, container) {
    this.name = name;
    this.container = container;
    this.tasks = [];

    for (var i = 0; i < data.length; i++) {
        this.tasks.push(new mu.tutorial.tasks.Task(data[i], this));
    }

    this.sort()
    this.render();
}

It's followed by its constructor which creates, sorts and renders thegiven tasks, this greatly simplify the makeTasks function. Withthis modification, we introduced some unwanted duplication ofinformation, our tasks objects already have a reference to theircontainers. We'll just change that reference for another pointed at thelist it's associated to. We'll also change the way we're using thetaskLists namespace variable, it will now contains TaskListobjects. Now lets see how the render function works.

mu.tutorial.tasks.TaskList.prototype.noTasks = function() {
    this.container.appendChild(
        goog.dom.createDom('h2', { 'class': 'empty' }, 'This list is empty!'));
}

mu.tutorial.tasks.TaskList.prototype.render = function() {
    goog.dom.removeChildren(this.container);

    if (this.tasks.length == 0)
        this.noTasks();
    else
        for (var i = 0; i < this.tasks.length; i++) {
            this.tasks[i].makeDom();
        }
}

It's basically the same code that was previously in makeTasks withthe only difference that it clean up its container before adding newelements. We also moved noTasks into the TaskList object forconvenience.

Next, some new code. We've got an array of tasks and we'll sort it usingthe default JavaScript sort function. Google has thought aboutadding helper functions for arrays including one for sorting, itsadvantage over the default JavaScript one is that it sorts numberscorrectly. Here this sort function wouldn't be helping us as we'llwrite our own compare function.

mu.tutorial.tasks.defaultCompare = function(t1, t2) {
    return goog.array.defaultCompare(
        t2.priority,
        t1.priority);
};

mu.tutorial.tasks.TaskList.prototype.sort = function() {
    this.tasks.sort(mu.tutorial.tasks.defaultCompare);
}

We'll need to insert and remove tasks from our task lists. As they'realways sorted, the Closure Library has two functions to help us:binaryInsert and binaryRemove. These allow for fast manipulation of sorted arrays by using a binary search algorithm.

mu.tutorial.tasks.TaskList.prototype.add = function(task) {
    task.list = this;
    
    goog.array.binaryInsert(
        this.tasks,
        task,
        mu.tutorial.tasks.defaultCompare);
}

mu.tutorial.tasks.TaskList.prototype.remove = function(task) {
    this.container.removeChild(task.element);
    
    goog.array.binaryRemove(
        this.tasks,
        task,
        mu.tutorial.tasks.defaultCompare);

    if (this.tasks.length == 0)
        this.render();
}

The add method is very simple, it just set the current task listas the given task parent and insert it into the array. For theremove method though, it's actually in charge of cleaning up theelement of the task being removed. It also ensures to call noTaskif there's no task left in that task list.

Next, we'll replace the clickActionButton listener code by a callto a new Task method called moveTo. This method will be incharge of removing the task from its parent list and inserting it intothe specified one using the above methods.

mu.tutorial.tasks.Task.prototype.moveTo = function(target) {
    this.list.remove(this);
    mu.tutorial.tasks.taskLists[target].add(this);
}

Yet another change that merit a mention is the switchPanelfunction which takes a new argument to know what TaskList torender. This could be made cleaner, but we'll keep it that way for thispart of the tutorial.

When a user change the priority of a task, we better have themsorted. We'll create a new Task method that will be called whensomeone stop using the slider. It make use of a new property of theTask object to remember the previous value so as not to reload atask list if the priority didn't changed.

mu.tutorial.tasks.Task.prototype.reload = function() {
    if (this.priority != this.previous_priority) {
        this.list.sort();
        this.list.render();
        this.previous_priority = this.priority;
    }
}

But now we have a problem. When shall we call this function? I'vethought about it for some time and came up with acomplex and convoluted solution.The correct solution, that is to use a timer and accumulate events,being too much bothersome for the scope of this tutorial, I've decidedto settle on a small hack I've found that should do the job.

By the principles of YAGNI, Ifigured out that the sliders are to heavy to handle. Why would we needfour different ways of modifying a task priority? So, from now on, we'llonly listen for MOUSEOUT and KEYUP events.

        goog.events.listen(
            slider.getContentElement(),
            [goog.events.EventType.MOUSEOUT,
             goog.events.EventType.KEYUP],
            function() { task.reload(); });

There's still a problem with dragging the sliders, if only we could makethat issue disappear.

Invisible Sliders

Sometimes the best style is no styles at all. We must face it, thesliders are ugly. Making them look awesome can take some time for anon-designer like me. While thinking about this problem, I've came upwith a solution that would also simplify sliders event handling code. Wecould make the sliders invisible and put them on top of the prioritynumbers shown. That doesn't actually fix our issue, we only preventusers from thinking about dragging the slider thumb. This is just a hackand should only be done if you're really short on time. Talking of time,this part is much more longer than I expected. So I'll let youlook atthe resultrather than the code as it's a very simple modification, mostly new DOMelements and CSS.

Less Rendering and More Stability

One last thing, our code is working like a charm and is more than fastenough for the small data set we're testing it with. But in real worldsituations, there's some bottlenecks that could potentially slow ourapplication down. I'll let the testing for another part, but we'llremove one of those bottleneck.

The most processor hungry function in our code is certainly therender one. Presently, we're calling it every time the priority ofa task change. But if the order of tasks don't change, we don't need torender them again. Moreover, the JavaScript sort function isn'tstable, two tasks having the same priority could be reordered. So, tokill two birds with one stone a second time, we'll make our own stablesort function. Well, in fact, we'll just take the code fromgoog.array.stableSort and alter it to return true if the arraychanged.

mu.tutorial.tasks.TaskList.prototype.sort = function() {
    var arr = this.tasks
    for (var i = 0; i < arr.length; i++) {
        arr[i] = {index: i, value: arr[i]};
    }

    function stableCompareFn(obj1, obj2) {
        return mu.tutorial.tasks.defaultCompare(obj1.value, obj2.value) ||
            obj1.index - obj2.index;
    };
    
    goog.array.sort(arr, stableCompareFn);

    var changed = false;
    for (var i = 0; i < arr.length; i++) {
        if (i != arr[i].index)
            changed = true;
        arr[i] = arr[i].value;
    }
    return changed;
}

Then, it's only a matter of calling render when the sortcall returns true in the reload method.

mu.tutorial.tasks.Task.prototype.reload = function() {
    if (this.priority != this.previous_priority) {
        if (this.list.sort())
            this.list.render();
        this.previous_priority = this.priority;
    }
}

Phew, that was a big post, I hope you enjoyed it! I'm still not quitesure about what will be the subject of the next part, if you have anysuggestion, drop by a comment.

Emacs Tip #1

2010-01-24T17:51:00.001-05:00

Emacs is a great editor, I won't elaborate on this as there's countlessblog posts already celebrating its countless features. After some yearsof usage, anyone using Emacs end up with a bloated .emacs filethat may contains some good tips. I'll be explaining some of the tricksI'm using in a series of posts of which this one is the first.

Web Search Shortcuts

It's a known fact (in the sense that it's a commonIntertubes meme) that some people usetheir computer exclusively through Emacs. I'm not that fond of thiseditor and prefer to do things like navigating the web witha real browser. Emacs as a way to send URLs to one,using the browse-urlfunction. Feeding an unencoded URL to it could yield a bad requestthough. To fix that, we can use url-hexify-string from the urllibrary. Here's a simple example of putting these functions to good use.

(require 'url)

(defun google ()
  (interactive)
  (let ((url (if (and transient-mark-mode mark-active)
               (buffer-substring-no-properties
                 (region-beginning)
                 (region-end))
               (thing-at-point 'symbol))))
    (browse-url
      (concat "http://www.google.com/search?"
        (format "q=%s" (url-hexify-string url))))))

This interactive function let us do a simple search with the text underthe currently active region or with the symbol found under the cursor ifthere's no active region. This can be useful, you're no longer forced tocopy-paste text from Emacs to your browser.

Next, we could try to send more complex queries to our preferred searchengine, which in this case is Google. Standard library documentationsuch as Java's API specification generally have URLs that are easilyqueryable. With Google's allinurl keyword, we can make a functionto search for a specific class symbol.

(defun search-all-in-url (query site inurl)
  (concat "http://www.google.com/"
    (format "search?q=allinurl:%s+site:%s+inurl:%s&btnI"
      (url-hexify-string query)
      (url-hexify-string site)
      (url-hexify-string inurl))))

(defun web-help (site inurl)
  (browse-url (search-all-in-url (thing-at-point 'symbol) site inurl)))

With web-help, we can now define custom interactivefunctions. Currently in my .emacs file, I've got two suchfunctions, one for Java and the other for Ruby.

(defun java-help ()
  (interactive)
  (web-help "java.sun.com" "/javase/6/docs/api/"))

(defun ruby-help ()
  (interactive)
  (web-help "www.ruby-doc.org" "/core/"))

Then, it's a simple matter of adding key bindings for our newinteractive functions. Personally, I like to have them bound to thelower F* keys.

(global-set-key [(f1)] 'google)
(global-set-key [(f2)] 'java-help)
(global-set-key [(f3)] 'ruby-help)

You might already use these shortcuts for other things (I think that F3serves to start recording macros) but you certainly still have someunbound key combinations!

Update: I've corrected the second code example that I messed up while writing this post.

A Recursive Walk on a String

2010-01-19T18:30:00.001-05:00

Sometimes you don't pay attention to some details and this lead youright into the wrong direction. That's exactly what happened to me theother day. I had a specific problem I intended to solve for a while,nothing particularly difficult, but an interesting one. After hackingaround for an hour, the situation was getting weird and I decided to askfor help on theClojure Google Group.This post is an account of how that situation arose and what have beendone to solve it.

Walking Recursively

The task is nearly as simple as explaining it. I needed a way to applya function to each strings contained in a tree of nested data structuresin Clojure. Not that easy to explain as you see, well you didn't reallysaw me rewrite the previous sentence a thousand times but you canprobably picture it. My first attempt is really naive and leverages theclojure.walk library written by Stuart Sierra. It's a small library thatprovides functions to perform non-recursive walking of generic Clojuredata structures. Lets first have a look at the walk functiondocumentation.

user> (doc clojure.walk/walk)
-------------------------
clojure.walk/walk
([inner outer form])
  Traverses form, an arbitrary data structure.  inner and outer are
  functions.  Applies inner to each element of form, building up a
  data structure of the same type, then applies outer to the result.
  Recognizes all Clojure data structures except sorted-map-by.
  Consumes seqs as with doall. 
nil

It isn't hard to take this function and use it in a recursive one to dothe job. Our function to transform strings could go in the innerargument, it's also where we'll add the recursive call. We don't needthe outer one, so we will just use the identity function.

(use 'clojure.walk)

(defn recursive-string-walk [f form]
  (walk #(if (string? %) (f %) (recursive-string-walk f %))
    identity form))

It works well but there's an issue with this way of doing things,recursion will end up filling up the stack which will ultimatelyoverflow. For the use I intend to make of this function, it isn't reallya big problem, but I always like to take these kind of challenges.

A Buggy Test

The first thing that came to my mind to test the above issue was tocreate a nested list of arbitrary depth. We can take the iteratefunction to create an infinite sequence of nested function calls tolist, then take and last to fetch the pit we wantto test with.

user> (def pit (iterate list "bottom!"))
#'user/pit
user> (take 6 pit)
("bottom!" ("bottom!") (("bottom!")) ((("bottom!"))) (((("bottom!")))) ((((("bottom!"))))))
user> (last (take 6 pit))
((((("bottom!")))))
user> (recursive-string-walk identity (last (take 6 pit)))
((((("bottom!")))))
user> (recursive-string-walk #(str "reached " %) (last (take 6 pit)))
((((("reached bottom!")))))
user> (recursive-string-walk #(str "reached " %) (last (take 1000 pit)))
; Evaluation aborted.
user> (recursive-string-walk #(str "reached " %) (last (take 100 pit)))
((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((("reached bottom!")))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))

Work great! Now lets create an informal test function. This functionshould take a range and a recursive walk function then will test itusing pits starting with the shallowest. This way we can find exactly atwhich point the overflow happen.

(defn test-walk [walker shallowest deepest]
  (doseq [depth (range shallowest deepest)]
    (println (walker #(str depth " reached " %)
               (last (take depth pit))))))

We'll use it to make sure it works and see which depth ourrecursive-string-walk can reach.

user> (test-walk recursive-string-walk 400 500)
...
((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((("486 reached bottom!")))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
(...(; Evaluation aborted.

On my computer, this function cannot go deeper than 486 (I wonder how itwould fare on a 486?) which isn't that much. It shouldn't be all thatdifficult to make a lazy version of that function. It boils down to lookfor sequences and wrap their recursive calls with lazy-seq.

(defn lazy-recursive-string-walk [f form]
  (walk #(cond
           (string? %) (f %)
           (seq? %)    (lazy-seq (lazy-recursive-string-walk f %))
           :default    (lazy-recursive-string-walk f %))
    identity form))

From my understanding of lazy data structures and the walk library, thisversion should be able to handle data structures of infinite depth giveninfinite resources. However the test function didn't seems to be inagreement with me as it was crashing at 828. That's about the time Iasked for help and stopped thinking about this problem for a while.

Lazy Zippers

A week-end passed by and in the meantime, some people from the grouptried to help me, yet nobody found what was the real problem. That wouldhave been miraculous given the actual source of the bug. Furthermore, Iwas now more convinced than before than there was no bug in the lazyversion. Tom Hicks provided a lazy version of walk, which wasbehaving in the same way. Laurent Petit also suggested that I try towrite a version using zippers.

It was an intriguing suggestion, I never used zippers but heard oftenabout them. Additionally, that day, MarkCC from Good Math, Bad Mathjust postedan article about them.So I learned what they are and how to use them. I must say this is avery neat functional programming technique, really worth the time tolearn. It's not that hard, Mark has done a great job explainingit. I won't go into the details, but this version is quite easy tounderstand when you know that the next function do a depth-firsttraversal automatically.

(require '[clojure.zip :as z])

(defn lazy-recursive-string-walk-with-zipper [f form]
  (loop [loc (z/seq-zip form)]
    (if (z/end? loc)
      (z/root loc)
      (recur (z/next
               (if (string? (z/node loc))
                 (z/replace loc (f (z/node loc)))
                 loc))))))

To my surprise, the test function still overflowed the stack using thisversion. And it could simply not throw the StackOverflowError as itexecutes in a loop. Maybe the error wasn't coming from the lazy walkfunction after all.

So Where's The Bug?

I've been negligent about one thing, I didn't look closely enough at thestack traces. The one thrown by recursive-string-walk happen to becompletely different than the one thrown bylazy-recursive-string-walk. Here's both of them one after theother.

No message.
  [Thrown class java.lang.StackOverflowError]

Restarts:
 0: [ABORT] Return to SLIME's top level.

Backtrace:
  0: clojure.lang.RT.boundedLength(RT.java:1128)
  1: clojure.lang.RestFn.applyTo(RestFn.java:135)
  2: clojure.core$apply__4370.invoke(core.clj:436)
  3: clojure.walk$walk__7942.invoke(walk.clj:43)
  4: recursive_string_walk$recursive_string_walk__4137.invoke(NO_SOURCE_FILE:1)
  5: recursive_string_walk$recursive_string_walk__4137$fn__4139.invoke(NO_SOURCE_FILE:1)
---
No message.
  [Thrown class java.lang.StackOverflowError]

Restarts:
 0: [ABORT] Return to SLIME's top level.

Backtrace:
  0: clojure.lang.RT.assoc(RT.java:666)
  1: clojure.core$assoc__4259.invoke(core.clj:146)
  2: clojure.lang.AFn.applyToHelper(AFn.java:179)
  3: clojure.lang.RestFn.applyTo(RestFn.java:137)
  4: clojure.lang.Ref.alter(Ref.java:174)
  5: clojure.core$alter__4907.doInvoke(core.clj:1542)

The actual source of the exception is the print function that isrecursive on nested sequences. This make sense, why would someone wantto print such monsters. Because of that inattention I lost timesearching for an imaginary bug. Only a small modification is needed tomake our test function working.

(defn get-bottom [p]
  (loop [p (first p)]
    (if (seq? p) (recur (first p)) p)))

(defn test-walk [walker shallowest deepest]
  (doseq [depth (range shallowest deepest)]
    (println (get-bottom
               (walker #(str depth " reached " %)
                 (lazy-seq (last (take depth pit))))))))

We simply search for the bottom item and print it, this will also makethe test output clearer. We can now see what our original walk functioncan handle.

user> (test-walk lazy-recursive-string-walk 100000 100001)
...
; Evaluation aborted.
---
Java heap space
  [Thrown class java.lang.OutOfMemoryError]

Well, the heap too has its limit! We would need to augment the JVM heapmemory size to be able to run the previous example. Lets reduce ourambitions and do some benchmarking instead.

user> (time (test-walk lazy-recursive-string-walk 10000 10001))
10000 reached bottom!
"Elapsed time: 25.138697 msecs"
nil
user> (time (test-walk lazy-recursive-string-walk-with-zipper 10000 10001))
10000 reached bottom!
"Elapsed time: 122.691972 msecs"
nil

As you see, the version using clojure.walk is much faster. That'sperfectly understandable considering the more complex concept of zipperswhich have multiple uses aside walking trees. As both versions are fullylazy, the first one is the clear winner here.

What Can We Use This For

After all these trials and tribulations, you may wonder what use I mayhave for such a function. It's to help create functions or macros thatperform string replacement on all strings in a Clojure form. This can beuseful with Compojure while deploying your application to a differentroot path than "/" for example. The idea come from my experience withASP.NET in which URLs can contain a character (~) to denote theweb site root path.

(use 'clojure.contrib.str-utils)

(defn expand* [expansions string]
  (reduce (fn [s [regex replacement]]
            (re-gsub regex replacement s))
    string (partition 2 expansions)))

(defn expand-form [url form]
  (recursive-string-walk (partial expand* [#"^~" url]) form))

Having the same functionality in a macro would be nice too.

(defmacro with-expansions [expansions & body]
  `(do ~@(recursive-string-walk (partial expand* expansions) body)))

(defmacro expand [url & body]
  `(with-expansions [#"^~" ~(str url)] ~@body))

To put this to use I've taken a function written by James Reeves afew months ago duringa discussionon the Compojure group. A function with the same name(with-context) already exists in Compojure though, so we'll renameit to with-root-path. The way it work is by binding the givencontext to a var and then we can use a function called url to addthe context so that internal links still work once the application isdeployed elsewhere. Using the expand-form function on html'sarguments can reduce repetition and make the code less cluttered ifthere's a lot of such URLs.

(use 'compojure)

(declare *context*)

(defn with-root-path
  [ctx & route-seq]
  (let [handler (apply routes route-seq)
         pattern (re-pattern (str "^" ctx "(/.*)?"))]
    (fn [request]
      (if-let [[_ uri] (re-matches pattern (:uri request))]
        (binding [*context* ctx]
          (handler (assoc request :uri uri)))))))

(defn url [path]
  (str *context* path))

(defn html-expand [& trees]
  (apply html (expand-form *context* trees)))

(defroutes test-routes
  (with-root-path "/foo"
    (GET "/"    (html-expand [:h1 "test"] [:p "~/"]))
    (GET "/bar" (html [:h1 "test"] [:p (url "/bar")]))
    (ANY "*" (page-not-found))))

(defn start-test-server []
  (run-server {:port 8000} "/*" (servlet test-routes)))

The first route use a special version of the html function, it'sonly a slight improvement but I really like that way of doingthings. There's plenty of other possible use cases though, but thatshould be enough for this post.

P.S.: I know the title isn't correct, but it sounds way betterthan A Recursive Walk on Strings Contained in Arbitrary Nested DataStructures.

Documenting Clojure Code

2010-01-12T21:04:00.001-05:00

Clojure is a truly amazing language and still a very young one. There isalready a substantial quantity of documentation available for Clojureand some of its libraries. Yet, when talking about documentation, someis not enough. Sure, there's the REPL with all its bells and whistles,but sometimes I prefer to navigate through a well formatted API documentusing a browser. There's already some solutions for this, one on which Iworked for the defunct Compojure website (which was based on similarcode for Clojure) and another more serious attempt in clojure-contrib,gen-html-docsby Craig Andera. This library only output HTML and there arise a small(but annoying) problem: not all projects have their documentationformatted in HTML. For example, the new Compojure website runs onDokuWiki which have its own syntax andGitorious integrated Wiki, used for ClojureQL, is usingMarkdown. If only everyWiki engine would understand Creole!

clj-doc

To be able to solve the above problem, I decided to start a new projectcalled clj-doc. For now it's pretty much just a weekend coding session,but I'll continue to work on it so that one day it will be the best toolavailable to generate documentation for Clojure code. The basic usage isdone through the gen-doc macro.

user> (use 'clj-doc)
nil
user> (gen-doc clj-doc.utils)
("<!DOCTYPE html ...Documentation for clj-doc.utils...")

This gives us a sequence of strings being the rendered output of eachnamespaces given using the default markup language. Namespaces couldalso be grouped using nested sequences, for now clj-doc support only onelevel of nesting. Another notable feature is that namespace groups canbe specified using regular expressions. Here's an example demonstratingall these methods at once.

user> (use 'clojure.contrib.pprint)
nil
user> (pprint (gen-doc clj-doc
                       (clj-doc.markups clj-doc.generator clj-doc.utils)
                       #"clj-doc\.markups\."))
("<!DOCTYPE html ...Documentation for clj-doc..."
 "<!DOCTYPE html ...Documentation for clj-doc.markups, clj-doc.generator, clj-doc.utils..."
 "<!DOCTYPE html ...Documentation for clj-doc.markups.creole, clj-doc.markups.dokuwiki, ...")

Writing to Files

Usually you would want to write the documentation for your library to afile. I've added basic support for this with the gen-doc-to-filemacro. It behaves exactly as gen-doc but writes each of the resultingdocumentation strings to a file.

user> (gen-doc-to-file "~/test.html" clj-doc)
nil

If you use more than one group of namespaces, this macro will write theminto multiple files that will be numbered starting with zero byappending a number before the last dot if there's one, at the endotherwise. I intend to make this argument a format string where you canspecify more complex filenames in a next version. So, if you got anyideas about how to design or implement such feature, I encourage you toadd a comment to this post.

Supported Markups

Up until now, we've only been using the default output markup. The mainpoint of this library being generating something else than HTML, webetter have a way to use another markup language. Before talking abouthow to do that, lets look under the hood.

(ns clj-doc.markups.creole
  "Creole markup."
  (use clj-doc.markups))

(defmarkup
  #^{:doc "Creole markup."}
  creole
  :title        #(str "\n= "     %     " =")
  :namespace    #(str "\n== "    %    " ==")
  :section      #(str "\n=== "   %1  " ===\n" %2)
  :var-name     #(str "\n==== "  %  " ====")
  :var-arglist  #(str "\n**" % "**\\\\")
  :var-doc      #(str "\n\n" %))

This is the current implementation for the Creole Wiki markupgenerator. All markups are defined in the same way, they're basicallyonly struct maps, for more details see the clj-doc.markupsnamespace. The library currently provides four output methods. We'vealready seen two of them, here's the complete list:

creole
dokuwiki
html-simple
markdown

To define your own markups, you can take one of the implementations,modify it and see how it work out, the keywords should be explicitenough. The library only look for markups into namespaces starting with"clj-doc.markups." though, but that's just a temporary limitation,eventually you'll be able to pass a markup struct map as an option.

The Option Map

The option map is an optional argument that must be specified before anynamespaces. It presently only support two keywords, :markup and:separated-by. The first letting you choose the output method andthe second used to change the default page separation behavior.

user> (gen-doc {:markup creole} clj-doc.markups.creole)
("\n= Documentation for clj-doc.markups.creole =\n== clj-doc.markups.creole namespace ==\n=== others ===\n\n==== creole ====\n\nCreole markup.")

The :separated-by option can only take one value, namespace,which makes the macros separate pages between namespaces instead ofgroups of them.

user> (pprint (gen-doc {:markup markdown :separated-by namespace} #"clj-doc\.markups\."))
("\n#Documentation for clj-doc.markups.creole..."
 "\n#Documentation for clj-doc.markups.dokuwiki..."
 "\n#Documentation for clj-doc.markups.html-simple..."
 "\n#Documentation for clj-doc.markups.markdown...")
nil

The Future

This is a very early release done the worse is better way, so it'sstill a pretty bare-bone tool. I'll be taking a pause this week to workon more (hopefully) profitable projects, so in the meantime I'll bepleased to ear about any suggestions to improve the way clj-docworks.

Git Tip #1

2010-01-11T18:34:00.004-05:00

Changing Filename Case Under Cygwin

When using Git on Windows XP, you're confronted with a basic problem,case changes aren't accounted for while renaming files as Windowsinternals doesn't have a clue about the difference between lower andupper case. You can actually change the casing, but you can't have twofiles using the same name with different casing for example. Thisbehavior obviously affect Git, which cannot detect filename casechanges.

$ git init Initialized empty Git repository in /home/budu/tmp/.git/

$ touch foo

$ git add .

$ git commit -m 'test' [master (root-commit) 5a1fbb3] test 0 files
changed, 0 insertions(+), 0 deletions(-) create mode 100644 fo

$ mv foo FOO

$ l total 0 drwxr-xr-x+ 1 budu None 0 Jan 11 18:15 .git/ -rw-r--r-- 1
budu None 0 Jan 11 17:59 FOO

$ git status # On branch master nothing to commit (working directory
clean)

This is an unfortunate situation which can be easily resolved by using asimple trick. The first thing you need to do is to copy the file youwant to modify to a different temporary name. Then you must usegit rm on the original file and finally rename the temporary fileto the new name with changed casing.

$ cp FOO bar

$ git rm foo rm 'foo'

$ mv bar FOO

$ git add .

$ git status # On branch master # Changes to be committed: # (use "git
reset HEAD ..." to unstage) # # renamed: foo -> FOO #

One thing to note here is that when calling git rm you have to usethe original file name if its casing is already changed.

Testing ClojureQL

2009-12-31T20:10:00.002-05:00

After working on ClojureQL for some time, I've became tired of runningmanual tests all the time, even with thedebug macro.So I began working on a test framework for this project. It's acombination of the test-is library and the concept of demos alreadybuilt-in. For now we only have them, but they aren't very comprehensiveand require a working connection for each backend you want totest. Furthermore, there's practically no validation of the results, weonly know that the database accept the expression that have been sent.Now that we're approaching the release of version 1.0 we need somethingmore flexible and exhaustive to keep the code stable.

The Plan

The idea of demo is not bad in itself, so I'll just complement it withtests that could be run without connection. It's all we need for day today development on ClojureQL, the actual execution of the generated SQLcode can be done only once in a while. To not break the DRY principle,I'll make them both use the same code, demos will be written using aspecial macro and tests will be generated automatically from them. Thedemos should also be improved by validating the results coming out ofthe tested databases.

The Prototype

Implementing this scheme directly into ClojureQL is not a simple taskthough, so I'll make a prototype to show how it works at a smallerscale. We'll use a very simple database written in Clojure and aClojureQL-like DSL that compile to pseudo-SQL statements. No backends,no intermediate forms and only two kind of statements: insertsand selects. We'll walk through this prototype below, it's merely ahundred lines long.

The Database

All data is kept in a ref, *db*, which is a map of maps withstrings as the sole data type for keys as well as for values. The onlyfunction we really need here is exec, which takes a compiledstatement and returns a result if no exception is raised. I've alsoincluded a simple helper function, reset, to empty the database.

(def *db* (ref {}))

(defn reset [] (dosync (ref-set *db* {})))

(defn exec-insert [stmt]
  (let [tokens (drop 2 (seq (.split stmt " ")))
        table  (first tokens)
        values (apply hash-map (rest (rest tokens)))]
    (dosync (alter *db* assoc table
              (merge (@*db* table) values)))))

(defn exec-select [stmt]
  (let [tokens (drop 1 (seq (.split stmt " ")))
        table  (last tokens)
        keys   (take (- (count tokens) 2) tokens)]
    (map #((@*db* table) %) keys)))

(defn exec [stmt]
  (condp #(.startsWith %2 %1) stmt
    "INSERT INTO " (exec-insert stmt)
    "SELECT "      (exec-select stmt)
    (throw (SQLException. "error: unrecognized statement."))))

(defn insert [name & key-val-pairs]
  (apply str "INSERT INTO " name " VALUES " (interpose " " key-val-pairs)))

(defn select [name & keys]
  (str "SELECT " (apply str (interpose " " keys)) " FROM " name))

Our DSL is composed of the insert and select functions,which output the compiled pseudo-SQL code that the database canexecute. It's a really dumb system, but we don't need more for thepurpose of this experiment.

The Demos

The demos will remain the heart of the ClojureQL testing framework. Themain objectives of the following code is to make it easy to define demosand to make their output human-readable. The trick is to also make themreusable, so demos will be defined as data, not code.

(defmacro defdemo [demo & stmts-results]
  `(def ~demo
     ~(vec (map #(vector (list 'quote (first %))
                         (list 'quote (second %)))
             (doall (partition 2 stmts-results))))))

(defdemo demo1
  (insert 'test1 'foo 1) {"test1" {"foo" "1"}}
  (select 'test1 'foo)   ("1"))

(defdemo demo2
  (insert 'test2 'foo 1 'bar 2) {"test2" {"bar" "2", "foo" "1"}}
  (select 'test2 'foo 'bar)     ("1" "2"))

(defn run-demo [name]
  (reset)
  (println "\n===" name "===\n")
  (doall
    (map (fn [[stmt expected-result]]
           (println "statement: " stmt)
           (let [compiled-stmt (eval stmt)
                 result        (exec compiled-stmt)]
             (println "compiled:  " compiled-stmt)
             (println "result:    " result "\n")
             (is (= result expected-result)))) (eval (symbol name)))))

(def demos ["demo1" "demo2"])

(defn run-demos []
  (every? identity
    (apply concat (map run-demo demos))))

The run-demo function test everything in a demo and provide a niceoutput. It returns a boolean that is the result of the assertion made atthe end.

The Tests

We can now use the above to write some tests. The demos ensure us thatthe compiled statements are all valid. After that it's simply a matterof generating Clojure code, to define tests, and writing it to afile. The gen-test function will take care of the first part andwrite-tests the second one.

(defn gen-test [name]
  (let [demo (eval (symbol name))]
    `(deftest ~(symbol (.replace name "demo" "test"))
       ~@(map (fn [[stmt _]]
                `(is (= ~stmt ~(eval stmt)))) demo))))

(defn write-tests [filename]
  (when (run-demos)
    (with-open [writer (java.io.FileWriter. filename)]
      (binding [*out* writer]
        (newline)
        (doseq [demo demos]
          (pprint (gen-test demo))
          (newline))))))

(write-tests "tests.clj")

(load-file "tests.clj")

Once the tests file written, we can load it to run the tests itcontains. So now we can use run-demos to test everything andrun-tests to test compilation only.

Conclusion

There's still many details to figure out before implementing thisframework. The namespace layout to be used for example or how tointegrate the test-is library depending on what Clojure version we end uptargeting. This is a work in progress so I'll keep this post updateduntil it's done. In the meantime, any comments or suggestions arewelcomed.

Happy New Year!

What the Hell is Happening at ATI?

2009-12-21T01:52:00.001-05:00

I got an ATI card for building my last box, would have prefered a Nvidia one, but got a good deal on this card. Overall it's been a good investment, but the situation is slipping down. There have been more and more driver updates lately and now it's getting desperate. I rarely play these days, but all the time I tried in the past months, lots of games were crashing in various ways. Hoping to get these problems resolved, today I tried to install the new Catalyst 9.12 driver, but the installer fails, it simply vanish while using 10% of the processor! To figure out that one, I googled to see if I wasn't alone and there happen to be quite a lot of forum posts discussing the issue. Seeing their content wasn't really comforting either, lots of steps to fix the problem, too much for my current motivation. I'd rather get an Nvidia card instead, but can't currently fit this in my (ramen) budget, even tho entry-level cards are now really powerful. So no gaming for the future ahead, programming is much more fun anyway.

Writing a Help Macro.

2009-12-21T00:12:00.002-05:00

Emacs may seems like a pretty barebone tool for the uninitiated, butpack a lot of things under the hood. Especially when you're coding witha Lisp, Emacs really shines thanks to its renownedslime mode. Yet, most of thepower of using a Lisp-like language with any editors lies in the use ofthe REPL. Clojure has a great set of functions and macros available tohelp you code using it, but these are dispersed around many librariescontained in clojure-contrib. As there's lots of such helpers, you haveto remember a bunch of names and refer to documentation often beforegetting used to all of them. Having these functionalities packed into asingle command could be very useful for beginners as well as forpractitioners. We'll first pass trough the code and review it, thenwe'll show how to use the help macro.

Help Macro

We'll proceed in a top-bottom manner, starting with the namespacedefinition. We'll need some Java classes from clojure.lang package andalso add some aliases for the clojure-contrib libraries we'll be using.

(ns help-macro
  "The help macro regroups clojure-contrib most useful help functions and
  macros into a single call."
  (import [clojure.lang IFn PersistentList Symbol])
  (require
    [clojure.contrib.classpath  :as classpath]
    [clojure.contrib.repl-utils :as repl-utils]
    [clojure.contrib.str-utils  :as str-utils]
    [clojure.contrib.ns-utils   :as ns-utils]))

Then comes the definition of *help-usage*, a simple var containinga string explaining how to use the help macro. That could have beenincluded in the docstring, but I chose to emulate the way command linescripts work instead.

(def *help-usage*
  #^{:doc "Help macro usage text."}
  (str
    "Usage: (help pwd)\n"
    "       (help classpath)\n"
    "       (help dir <ns>)\n"
    "       (help docs <ns>)\n"
    "       (help vars <ns>)\n"
    "       (help source <symbol>)\n"
    "       (help <class> [<n>])\n"
    "       (help <expr>)\n"
    "       (help <string>)\n"
    "       (help ? <query-type>)"))

As you see, we'll support nine kinds of help queries, six of which usea command symbol while the three others are dispatched on the class oftheir first argument. Lets review each types of queries:

pwd - Returns the current working directory.
classpath - Prints the current classpath.
dir - Prints a sorted directory of public vars in a namespace.
docs - Prints documentation for the public vars in a namespace.
vars - Returns a sorted seq of symbols naming public vars in a namespace.
source - Returns a string of the source code for the given symbol, if it can find it.
<class> - Get help on class members, like clojure.contrib.repl-utils/show.
<expr> - Get help on the result of an expression, like clojure.contrib.repl-utils/expression-info.
<string> - Like find-doc, but easier to type.

The help usage message is nice, but it would be great to have morespecific ones for each types of queries, like the above list. To dothis, we'll create a help-usage function that will print variousmessages depending on the symbol given as first argument.

(defn help-usage
  "Returns documentation on help macro usage."
  [query-type & args]
  (condp = query-type
    'class     (doc clojure.contrib.repl-utils/show)
    'expr      (doc clojure.contrib.repl-utils/expression-info)
    'string    (doc find-doc)
    (println
      (condp = query-type
        'pwd       "Returns the current working directory."
        'classpath "Prints the current classpath."
        'dir       "Prints a sorted directory of public vars in a namespace."
        'docs      "Prints documentation for the public vars in a namespace."
        'vars      "Returns a sorted seq of symbols naming public vars in a namespace."
        'source    "Returns a string of the source code for the given symbol, if it can find it."
        "This type of help query is not recognized."))))

Each symbols are separated in two categories. For the simpler queries,we only display a description of what they do. For more complex onesthough, we show the documentation for the original function or macrousing the doc macro. The function accept other arguments only toprevent exceptions in case of unintentional input.

We'll now look at the generic-help multimethod, which will dispatchcalls on the class of its first argument. We'll only respond to threeclasses: Class, Clojure's PersistentList and String.

(defmulti generic-help
  "Makes the help macro generic on its first argument if no command found."
  {:arglists '([query args])}
  (fn [query _] (class query)))

(defmethod generic-help Class
  [query args]
  (apply repl-utils/show (cons query args)))

(defmethod generic-help PersistentList
  [query args]
  (repl-utils/expression-info (second query)))

(defmethod generic-help String
  [query args]
  (find-doc query))

A thing to note here is the reason why we take the second argument ofthe list in the PersistentList method. This is so because the helpmacro quote all arguments it receives, but the expression query must bealready quoted.

We'll also add a default dispatch method which displays a warningmessage followed by the help macro usage.

(defmethod generic-help :default
  [query _]
  (println "No help available for object of type " (class query))
  (help-usage))

All simple commands are contained in the *help-command*map, which is used by the help* function following it.

(def *help-commands*
  #^{:doc "This is a map containing all the commands for the help macro."}
  { 'pwd       #(.getCanonicalPath (java.io.File. "."))
    'classpath #(println (str-utils/str-join "\n" (classpath/classpath)))
    'dir       ns-utils/print-dir
    'docs      ns-utils/print-docs
    'vars      ns-utils/ns-vars
    'source    (comp println repl-utils/get-source)})

(defn help*
  "Driver for the help macro."
  [query args]
  (if-let [sc (get *help-commands* query)]
    (apply sc args)
    (generic-help (if (symbol? query)
                    (resolve query)
                    query) args)))

This function look into the *help-command* map to see if itcontains the given query symbol. If found, it calls the associatedrunnable, else it calls generic-help. Before calling themultimethod, we first try to resolve the query quoted expressionin case it's a symbol. This is because this function receives only thesymbol of a class when help is called with a class name.

Finally, here's the help macro in all it's glory.

(defmacro help
  "Get help for various kind of expressions, use without arguments for
  detailed usage."
  ([] `(println *help-usage*))
  ([query & args]
    (let [quoted-args (map #(list 'quote %) args)]
      (if (= '? query)
        `(help-usage ~@quoted-args)
        `(help* '~query (list ~@quoted-args))))))

Usage

Here's some examples of using the help macro:

user> (help pwd)
"C:\\"
user> (help classpath)
g:\libraries\java\swank-clojure-1.0-SNAPSHOT.jar
g:\libraries\java\servlet-api-2.5-20081211.jar
...
nil
user> (help dir help-macro)
*help-commands*
*help-usage*
generic-help
help
help*
help-usage
nil
user> (help docs help-macro)
-------------------------
help-macro/*help-commands*
nil
  nil
...
nil
user> (help vars help-macro)
(*help-commands* *help-usage* generic-help help help* help-usage)
user> (help source +)
(defn +
  "Returns the sum of nums. (+) returns 0."
  {:inline (fn [x y] `(. clojure.lang.Numbers (add ~x ~y)))
   :inline-arities #{2}}
  ([] 0)
  ([x] (cast Number x))
  ([x y] (. clojure.lang.Numbers (add x y)))
  ([x y & more]
   (reduce + (+ x y) more)))
nil
user> (help String)
===  public final java.lang.String  ===
[ 0] static CASE_INSENSITIVE_ORDER : Comparator
[ 1] static copyValueOf : String (char[])
...
nil
user> (help String 1)
#<Method public static java.lang.String java.lang.String.copyValueOf(char[])>
user> (help '1)
{:class java.lang.Integer, :primitive? false}
user> (help '(int 1))
{:class int, :primitive? true}
user> (help "help")
-------------------------
clojure.main/help-opt
([_ _])
  Print help text for main
-------------------------
clojure.main/main
([& args])
...

Installing

You can find the complete codehere. To use it,place that file in your classpath and create an user script to be calledwhen stating a REPL. From there, you simply have to add the help-macronamespace with the use function if you want to be able to justtype "(help...". To have it around in every namespace, you could add thefollowing function in the same file than the macro and use it instead ofin-ns.

(defn to-ns [n]
  (in-ns n)
  (use 'help-macro))

Any comments, suggestions, improvements, insults?

Reading The F*ck!ng Manual

2009-12-15T19:41:00.001-05:00

While contributing code to ClojureQL, I made a huge mistake. That is,commiting changes to the master branch directly instead of using adevelopment branch. I had made a dozen commits, but in the meantime Laumerged the dev branch on ClojureQL's main repository. The end result wasthat the pull request I made after was incredibly huge as it includedall changes in the merge.

I'll walk you through all steps I've taken to fix this problem. We'llfirst create a dev branch, it should be a copy of the current masterbranch, which is the default behavior.

$ git branch dev

This will create a new branch named dev that will be a copy of thecurrent one, master. We'll then search for the first commit we'vemade using git log.

commit 07a516b03287b606d6a40b2e073cd3e3a1a74244
Author: Nicolas Buduroi 
Date:   Wed Dec 9 18:52:35 2009 -0500

    Removed global connection support for MySql demo as it breaks the build process when MySql is not installed.

commit cc8f234fb61739fca36e982f68eef005b3d4389c
Author: Lau_of_DK 
Date:   Tue Dec 8 23:18:03 2009 +0100

:

We must return the master branch to the state it was before commitingour first change. We can achieve this by using the reset command.

$ git reset --hard cc8f234fb61739fca36e982f68eef005b3d4389c
.gitignore: locally modified
build.xml: locally modified
ivy.xml: locally modified
ivysettings.xml: locally modified
...

Notice that you should use the --hard flag to make sure Git cleanup the working repository. We can then update our master branch with themain repository. This assume you've already set up a remote Gitrepository named main-repo, checkthis pageto see how to make one.

$ git pull main-repo master
From git://gitorious.org/clojureql/clojureql
 * branch            master     -> FETCH_HEAD
Updating cc8f234..b85d55b
Fast forward
 build.gradle                                       |   64 +++++++
 build.xml                                          |  182 --------------------
 ivy.xml                                            |   31 ----
 ivysettings.xml                                    |    9 -
 src/{dk/bestinclass => }/clojureql.clj             |    4 +-
...

Now, we need to switch to our dev branch with git checkout dev,and there, issue the rebase command.

$ git rebase master
First, rewinding head to replay your work on top of it...
Applying: Removed global connection support for MySql demo as it breaks the build process when MySql is not installed.
error: src/dk/bestinclass/clojureql/demos/mysql.clj: does not exist in index
Using index info to reconstruct a base tree...
Falling back to patching base and 3-way merge...
Renaming src/dk/bestinclass/clojureql/demos/mysql.clj => src/clojureql/demos/mysql.clj
Auto-merging src/clojureql/demos/mysql.clj
Applying: Made demos code formatting more uniform.
error: src/dk/bestinclass/clojureql/demos/derby.clj: does not exist in index
error: src/dk/bestinclass/clojureql/demos/mysql.clj: does not exist in index
error: src/dk/bestinclass/clojureql/demos/postgres.clj: does not exist in index
Using index info to reconstruct a base tree...
Falling back to patching base and 3-way merge...
Renaming src/dk/bestinclass/clojureql/demos/derby.clj => src/clojureql/demos/derby.clj
Auto-merging src/clojureql/demos/derby.clj
CONFLICT (rename/modify): Merge conflict in src/clojureql/demos/derby.clj
Renaming src/dk/bestinclass/clojureql/demos/mysql.clj => src/clojureql/demos/mysql.clj
Auto-merging src/clojureql/demos/mysql.clj
CONFLICT (rename/modify): Merge conflict in src/clojureql/demos/mysql.clj
Renaming src/dk/bestinclass/clojureql/demos/postgres.clj => src/clojureql/demos/postgres.clj
Auto-merging src/clojureql/demos/postgres.clj
CONFLICT (rename/modify): Merge conflict in src/clojureql/demos/postgres.clj
Failed to merge in the changes.
Patch failed at 0002.

When you have resolved this problem run "git rebase --continue".
If you would prefer to skip this patch, instead run "git rebase --skip".
To restore the original branch and stop rebasing run "git rebase --abort".

OK, that doesn't look good: error, conflict, failed... Not reassuring,but looking more closely, this message concerns only the first patch,which I have to revert anyway. Lets see how to resolve this. You canfind the conflicting files with git status. Here's an example ofsuch a file:

<<<<<<< HEAD:src/clojureql/demos/derby.clj
(ns clojureql.demos.derby
=======
;; TEST ====================================================

(ns dk.bestinclass.clojureql.demos.derby
>>>>>>> Made demos code formatting more uniform.:src/dk/bestinclass/clojureql/demos/derby.clj

Fix the conflict by picking one of the offered options. Then, onceeverything is cleaned up, you can tell the merge process to accept yourchanges with git add and restart the rebase one.

$ git add .

$ git rebase --continue
Applying: Made demos code formatting more uniform.
Applying: Added build directory to ignored files.
Applying: Changed PostreSQL backend to be more like MySQL one, follow non-nulls new standard.
Applying: Updated create-table docstring to reflect recent changes.
Applying: Added a shortcut in create-table's options to make all columns non null.
Applying: Added support for multiple primary keys to create-table.
Applying: Changed EmbedConnection class to EngineConnection interface for added flexibility in Derby backend.
Applying: Removed useless prefer-method.
Applying: Added the :unique option to create-table for all backends, needs refactoring.
Applying: Refactored contraints using a list as argument for all backends.
Applying: Refactored column options for all backends, made minor changes to list-constraint and cleaned up a little.
Applying: Added new derivation for ::Generic as java.sql.Connection proxies are confused with com.mysql.jdbc.Connection ones, put prefer-method back in Postgres backend as it now cause problems.
Applying: Added primary-key, unique and non-nulls options to generic create-table.
Applying: Added the :defaults option to create-table for all backends.

We must now return to the master branch where we'll merge the rebasedpatches.

$ git merge dev
Updating b85d55b..5f771d6
Fast forward
 .gitignore                         |    1 +
 src/clojureql/backend.clj          |   49 ++++++++++++++++++++++++++++++++++-
 src/clojureql/backend/derby.clj    |   31 +++++++++++-----------
 src/clojureql/backend/mysql.clj    |   36 ++++++++++++++------------
 src/clojureql/backend/postgres.clj |   42 ++++++++++++++++++------------
 src/clojureql/demos/derby.clj      |   11 ++++---
 src/clojureql/demos/mysql.clj      |   16 ++++-------
 src/clojureql/demos/postgres.clj   |    2 +-
 src/clojureql/frontend.clj         |   18 ++++++++++---
 9 files changed, 134 insertions(+), 72 deletions(-)

That was simple, isn't it? Joking aside, this may look complex to theuninitiated, but it all make sense once you get used to it. One lastthing, after going through this, I had still to push my rebased commitsto the Gitorious clone. It didn't worked!

$ git push origin master
Enter passphrase for key '/home/budu/.ssh/id_rsa': 
To git@gitorious.org:~budu/clojureql/budu-clojureql.git
 ! [rejected]        master -> master (non-fast forward)
error: failed to push some refs to 'git@gitorious.org:~budu/clojureql/budu-clojureql.git'

That problem happen to be caused by a safety measure that prevents usersfrom pushing a "non-fast forward" branch to a remote repository. Simplyput, when you mess to much with your repository it could end up causingtroubles, here'sa messagefrom Git's mailing list archives explaining things moreproperly. You need to use the --force option to circumvent thatprotection.

$ git push --force origin master
Enter passphrase for key '/home/budu/.ssh/id_rsa': 
Counting objects: 117, done.
Compressing objects: 100% (41/41), done.
Writing objects: 100% (103/103), 17.40 KiB, done.
Total 103 (delta 70), reused 80 (delta 62)
To git@gitorious.org:~budu/clojureql/budu-clojureql.git
 + 42eb50e...5f771d6 master -> master (forced update)
=> Syncing Gitorious... [OK]

I wrote this post for those who are in a hurry and didn't had the timeto learn Git properly before contributing to some project. But rememberthat it's always better to RTFM!

Debugging ClojureQL

2009-12-14T20:57:00.004-05:00

Since last week, I embarked on a new endeavor, contributing toClojureQL. My future projectsinvolving Clojure will need to access some databases, some of whichdoesn't behave in the same way. Sure, there is alreadyclojure.contrib.sqlthat give you a wrapper around JDBC, but there are two problems withthis approach. First, JDBC is quite good at what it does, yet is not avery sophisticated tool. It gives you access to a portable subset ofSQL, that mainly support querying and updating data. This leaves a lotof advanced features out of the deal, these are still available, but ina non-portable way. The other way around, if a database system doesn'tsupport a common feature, JDBC cannot do anything about it, the drivercan though. Second, the clojure.contrib sql API has a very proceduralfeeling to it, it doesn't even let you play with an intermediary form asit executes statement directly. That's enough for basic databaseinteractions, but not for serious database agnostic development.

Lets put off advocacy and talk about something concrete. Whilecontributing to ClojureQL, I realized there was some issues withdebugging. It was working well for Postgres (which I had an instancerunning), I simply had to use the compile-sql method. A problemarise when you're testing changes that affect all backends. In thiscase, to verify the SQL generated, you need a server for each DBMS youwant to test. After looking at the code for some time, I found an easyway of compiling statements without a connection. We can create mockobjects using Clojure's proxy macro and use them instead of liveconnections. For now, it's really simple as no backend implementationsare actually using the connector. To put this idea into practice, Iwrote a macro to debug multiple databases.

(defmacro debug [db ast]
  (let [connector (.getName (db *connectors*))]
    `(compile-sql ~ast (proxy [~(symbol connector)] []))))

It uses a map containing the interfaces used by all backends withkeywords as keys.

(def *connectors* {
  :postgres org.postgresql.PGConnection
  :mysql    com.mysql.jdbc.Connection
  :derby    org.apache.derby.iapi.jdbc.EngineConnection
  :generic  Object})

With this code you can easily debug statements for every databasesClojureQL support. We can add a final touch to be able to see the SQLoutput for all backends with a single command.

(defmacro debug-and-print-all [ast]
  (let [longest (reduce max (map (comp count str) (keys *connectors*)))
        debug-and-print (fn [db] `(println (format (str "%1$-" ~longest "s : %2$s")
                                           ~(subs (str db) 1)
                                           (debug ~db ~ast))))]
    `(do ~@(map debug-and-print (keys *connectors*)))))

Finally, you can see an example output of the debug-and-show-allmacro at the REPL using my ClojureQLclone.

clojureql-test> (debug-and-print-all (create-table test [id int title text date date] :non-nulls * :primary-key id :auto-inc id :unique title))
postgres  : CREATE TABLE test (id SERIAL,title text NOT NULL,date date NOT NULL,PRIMARY KEY ("id"),UNIQUE ("title"))
mysql     : CREATE TABLE test (id int NOT NULL  AUTO_INCREMENT ,title text NOT NULL ,date date NOT NULL ,PRIMARY KEY (`id`),UNIQUE (`title`)) 
derby     : CREATE TABLE test (id int NOT NULL  GENERATED ALWAYS AS IDENTITY ,title text NOT NULL ,date date NOT NULL ,PRIMARY KEY ("id"),UNIQUE ("title"))
generic   : CREATE TABLE test (id int NOT NULL ,title text NOT NULL ,date date NOT NULL ,PRIMARY KEY ("id"),UNIQUE ("title"))

This code is not working properly on the main repository for the moment,as there's some issues with Derby and the generic backend. It's enoughfor this post, I'll go back hacking my way through ClojureQL code tofind other useful tricks and speed up version 1.0 release.

Somalia's Customary Law

2009-12-08T19:48:00.001-05:00

In recent years, there have been a growing interest for decentralization in information technologies. It started a long time ago when personal computers (and before them mini computers) replaced the gigantic computers of yore. Today many topics in software development are focused on decentralization like distributed version control, offline web applications or database sharding. This concept can be applied to many things around us, it's not only pertaining to our domain. Lately I've been thinking about what I called then a decentralized legal system. That name wasn't quite good and I learned that the correct term is polycentric law. This type of legal structure has not been studied widely, and only managed to gather interest from researchers since the beginning of the nineties. It's at that time that Tom W. Bell wrote a paper entitled "Polycentric Law" in which he described various examples of such systems like the Anglo-Saxon customary law. For him, there are six main features that embodies this concept:

Focus on individual rights
Law is enforced by reciprocal agreement between the victim(s) and the accused party
Common procedures to help maintain order
Punitive actions are centered around paying back for the wrong deeds committed
Social exclusion is used as a powerful incentive
Laws are more adaptive to traditions and customs.

Anglo-Saxon's customary law had a surety mechanism known as borh (not to be confused with a Burh) that was the foundation of their whole system of law. It was really simple, you take a dozen persons and then bind them together by making them pledge to be responsible for each others' actions. If one of them got a fine, the group must pay it, thus ill-behaving members were not very popular and could even be expelled . It's sure such a system wouldn't work well in modern time Britain, or in other prosperous countries for that matter, but at the time these were very coercive incentives. There were a lot of other similar customary law in ancient times, but we can find other kinds of decentralized systems in more civilized political structures. The Roman law for example allowed indigenous legal systems for non-Romans, some of which being polycentric, and we could also consider the European Union has a form of decentralized (or multi-layered) superstate, but it would be a stretch to call these polycentric.

This brings us to the subject of the day, Somalia's customary law, the Xeer legal system. It's what has defined Somalia since its inception and is more than thirteen centuries old. It has been and still is the principal way for Somalis to resolve their disputes despite the introduction of the Shari'a (which by the way is widely used for common civil cases like marriage, divorce, inheritance, etc.) and the Italian colonial government's attempt to get rid of traditional laws before the independence. The Xeer is considered to be completely indigenous and is perhaps the most evolved form of polycentric legal system still in use today. In spite of all the political instability, piracy and other problems plaguing this country, Somalia's economy is thriving compared to most other African nations and some people are attributing that to their customary law. Even the government of the autonomous Somaliland region have tried to make themselves more legitimate in the eyes of their citizens by appointing elders (the Xeer system's judges) into the upper house of parliament, but that apparently didn't worked so well. I must note that Xeer's popularity in that part of Somalia is more pronounced as a consequence of the more relaxed British attitude toward their colonies. Since the Somalis gained their independence in the sixties, the state of Somalia had introduced various civil law systems, but the Xeer always prevailed, even with Siad Barre efforts to forbade clanism and promote centralization.

The law is defined in terms of property rights, which make it more compensatory than punitive. This characteristic by itself position this system as a great example of the previously defined concept of polycentric law. Another aspect is shown in the way fines are being paid, not to a court or the government, but directly to the victim(s). There's also many traditions that aren't polycentric, like bigger fines for prominent members of society than for commoners and a staunch opposition to any form of taxation. It's a very secular system too, keeping religious concerns out of the way. It also normally takes precedence over the Shari'a, even though both systems generally occupy different niches. There's a saying in Somalia that say:

Diinta waa labaddali karaa, xeer se lam baddali karo

That means "One can change his religion, one cannot change the law", which shows how secularism is ingrained into Somalis collective consciousness, especially when considering the predominance of a single religion in that country for centuries. But their secularism go even further than what is known elsewhere, as the law is even considered to be separated from the government. The Xeer system is administered and enforced by civilians, not government officials.

There's an insurance mechanism not unlike the bohr surety system. Generally the family of the offender is designated to be responsible for its actions, thus rehabilitation of criminals is the business of their respective family. In extreme cases, the family can disown one of its member, who then automatically becomes an outlaw. In such case, the only option left for the criminal is to leave the country. The way everything works can bear surprising resemblance to modern systems of law. There is specialized functions that can be related to the common court and law enforcement functions:

Odayal (Judges)
Xeer Boggeyaal (Jurists)
Guurtiyaal (Detectives)
Garxajiyaal (Attorneys)
Murkhaatiyal (Witnesses)
Waranle (Police officers)

Each Somali is appointed an Odayal at birth, chosen after long deliberation by the elders of the clan. That title can be lost at some point if the community decide so, making judges more careful and taking the interest of the collectivity before their own clan. The way everything work is intimately tied to the complex Somali family clan structure. This system is even scalable, as the Xeer not only rule disputes inside and between clans, but also across regions, between alliances of clans called jilibs. I won't go into the details of court procedures as this is pretty boring stuff (comically googling "pretty boring stuff" yields `Pretty Boring Stuff': District Judges and Housing Possession Proceedings as first result!) One thing to note is the preponderance of oaths to settle complex situations, for instance when the witnesses contradict each other or if there's not enough witnesses to confirm a story. In these times, the person in question is asked to take an oath like "I swear by my virility." or "I swear by Allah.". Some oaths can even have consequences like the divorce oath that, if broken, make a marriage vow null and void. There's also a oath of innocence that the accused must take when the plaintiff fails to convince the judge of the validity of his case.

To wrap up everything, the long time appreciation of the Xeer by Somalis seems to be in sharp contrast with the usual cynicism most peoples have toward their legal system. Even though Somalia as its problems, which are less dramatic than some would have us to believe, it doesn't mean there's nothing to be learned from them.

Closure Library Tutorial: Tasks (part 1)

2009-12-03T20:39:00.003-05:00

Today we'll create something with the recently releasedClosure Library. Nothingamazing, just a simple application to manage daily tasks (I would haveliked to say a 'task manager' but that could be confusing). JavaScripthas finally matured and with this library, we nearly have all the bellsand whistles of a desktop environment graphical user interface.

The Closure Library is part of a set of tools that Google's engineershave been working on forquite some timeand they have more than proved their worth. Eventhough not directly based on it, this library has been greatlyinfluenced by the Dojo Toolkit. I've neverused that framework before, most of my experience being with jQuery, soI'm not really qualified to compare them, learning Closure will beenough for now. Lets list some of the features offered by this library:

History management
Solid and portable event handling (with a timer class and ways to delay or throttle events)
Internationalization
Basic support for spell checkers
A complete debugging and testing framework
DOM manipulation helpers
Code to help working with data sets
Cross-browser support for drawing using SVG, VML or the new HTML5 canvas element
A module system to dynamically load compiled JavaScript code
UI widgets and effects

And there's much more! All that backed up by the company that dominatethe web, how could we ask for more? So in this tutorial we'llconcentrate on two areas: UI widgets and event handling. I know, this isalready covered in theGoogle code tutorial,but I found it to be plain and boring, that library deserves more.

This tutorial will build upon the official one, just refer toit if you're lost at some point. We begin by declaring the namespaceswe'll be providing and we'll also require the goog.dom namespaceand the UI widgets we need.

goog.provide('mu.tutorial.tasks');
goog.provide('mu.tutorial.tasks.Task');

goog.require('goog.dom');
goog.require('goog.ui.CustomButton');
goog.require('goog.ui.Toolbar');
goog.require('goog.ui.ToolbarButton');
goog.require('goog.ui.Zippy');

First thing to note is that we have to provide not only the namespace,but the name of all classes in that namespace. That's because classesaren't really classes, they're still just plain JaveScriptprototypes. So class names are simply namespaces and are thus orthogonalto the inheritance mechanism. You can inherit from classes that aren'tbeing provided by any namespace. The provide and requirefunctions are only there to check for dependencies and are also used bythe debug resolver.

Now lets add something new, a toolbar. Before delving into code, we'llhave to go fetch the stylesheet and images that can be found in thegoog/demos folder. In the meantime you can also get those for thebutton widget that we'll use later on. Another file not to forget isthe common.css stylesheet that contains styles common to allwidgets. Here's all the code needed to create our toolbar.

mu.tutorial.tasks.switchPanel = function(target) {
    return function() {
        goog.dom.$('taskList').style.display = "none";
        goog.dom.$('completedTaskList').style.display = "none";
        goog.dom.$('deletedTaskList').style.display = "none";
        goog.dom.$('settings').style.display = "none";
        goog.dom.$(target).style.display = "block";
    }
}

mu.tutorial.tasks.attachToolbarButton = function(toolbar, label, tooltip, target) {
    var button = new goog.ui.ToolbarButton(label);
    button.setTooltip(tooltip);
    toolbar.addChild(button, true);
    goog.events.listen(button.getContentElement(), goog.events.EventType.CLICK,
        mu.tutorial.tasks.switchPanel(target));
}

mu.tutorial.tasks.attachToolbar = function(container) {
    var toolbar = new goog.ui.Toolbar();

    mu.tutorial.tasks.attachToolbarButton(toolbar, 'Tasks', 'List currently active tasks.', 'taskList');
    mu.tutorial.tasks.attachToolbarButton(toolbar, 'Completed', 'List completed tasks.', 'completedTaskList');
    mu.tutorial.tasks.attachToolbarButton(toolbar, 'Trash', 'List deleted tasks.', 'deletedTaskList');
    mu.tutorial.tasks.attachToolbarButton(toolbar, 'Settings', 'Settings', 'settings');

    toolbar.render(container);
}

First, we create a toolbar, then attach all buttons and finally rendereverything in the given container. There's a function to help us attachbuttons, it add a tooltip and an event listener to switch between thevarious panels. Those panels are just a bunch of divs that areshown successively by changing their display attribute, using theswitchPanel function.

For an application managing tasks, we better create a Task class. Thispart of the tutorial is very similar to Google's one. Task objects arestructurally identical to Note objects, with summary insteadtitle and description instead of content asproperties.

mu.tutorial.tasks.Task = function(data, container) {
    this.summary = data.summary;
    this.description = data.description;
    this.priority = data.priority;
    this.parent = container;
}

mu.tutorial.tasks.Task.prototype.closeEditor = function() {
  this.contentElement.innerHTML = this.description;
  this.contentElement.style.display = "block";
  this.editorContainer.style.display = "none";
};

mu.tutorial.tasks.Task.prototype.openEditor = function(e) {
  this.editorElement.value = this.description;
  this.contentElement.style.display = "none";
  this.editorContainer.style.display = "inline";
};

mu.tutorial.tasks.Task.prototype.save = function(e) {
  this.description = this.editorElement.value;
  this.closeEditor();
};

As you see, there's nothing new here. I've also included the functionsthat will be used by the editor, then again, same thing as the officialtutorial. We'll only add one new kind of action. In fact this is afunction returning a closurethat will move a task to the desired panel.

mu.tutorial.tasks.Task.prototype.clickActionButton = function(task, element, target) {
    return function(e) {
        var parent = element.parentNode;
        parent.removeChild(element);
        if (parent.childNodes.length == 0)
            mu.tutorial.tasks.noTasks(parent);

        task.parent = mu.tutorial.tasks.taskLists[target];
        if (task.parent.childNodes[0].className == 'empty')
            task.parent.removeChild(task.parent.childNodes[0]);

        task.makeDom();
        e.stopPropagation();
    };
}

Here, the noTasks function only create an h2 header tag to tellthe user there's no tasks in that panel. The taskLists namespacevariable is a simple dictionary that we'll fill later.

We're now ready to jump to the makeDom function, where we createthe DOM structure for a task, add buttons, wire up event listeners anduse the Zippy class.

mu.tutorial.tasks.Task.prototype.makeDom = function() {
    this.summaryDiv = goog.dom.createDom('div', { 'class': 'summary' }, this.summary);
    this.contentElement = goog.dom.createDom('div', { 'class': 'description' }, this.description);
    this.editorElement = goog.dom.createDom('textarea', null, '');

    this.editorContainer = goog.dom.createDom('div', {'style': 'display:none;'},
                                              this.editorElement);

    this.descriptionContainer = goog.dom.createDom('div', null,
                                                   this.contentElement,
                                                   this.editorContainer);

    var taskDiv = goog.dom.createDom('div', { 'class': 'task' },
                                     this.summaryDiv,
                                     this.descriptionContainer);

    this.parent.appendChild(taskDiv);

    this.makeButtons(this, taskDiv);

    goog.events.listen(this.contentElement, goog.events.EventType.CLICK,
                       this.openEditor, false, this);

    this.zippy = new goog.ui.Zippy(
        this.summaryDiv,
        this.descriptionContainer);
}

There is two type of button for tasks, action buttons that will use theclickActionButton function and editor buttons for the editor. Eachtype of button has a function to create them, add a class name, renderthem and add an event listener.

mu.tutorial.tasks.Task.prototype.makeActionButton = function(task, element, target, name) {
    var button = new goog.ui.CustomButton(name);

    button.addClassName('taskButton');

    button.render(element.childNodes[0]);

    goog.events.listen(
        button.getContentElement(),
        goog.events.EventType.CLICK,
        this.clickActionButton(task, element, target));
}

mu.tutorial.tasks.Task.prototype.makeEditorButton = function(element, name, callback) {
    var button = new goog.ui.CustomButton(name);

    button.addClassName('editorButton');

    button.render(element.childNodes[1].childNodes[1]);

    goog.events.listen(
        button.getContentElement(),
        goog.events.EventType.CLICK,
        callback, false, this);
}

Here's the makeButtons function.

mu.tutorial.tasks.Task.prototype.makeButtons = function(task, element) {
    if (task.parent.id == 'taskList') {
        this.makeActionButton(task, element, 'completed', 'Done');
        this.makeActionButton(task, element, 'deleted', 'Delete');
    }
    else
        this.makeActionButton(task, element, 'active', 'Undo');

    this.makeEditorButton(element, 'Save', this.save);
    this.makeEditorButton(element, 'Cancel', this.closeEditor);
}

The action buttons are attached to each tasks depending if the given taskis active or not. Active tasks can be deleted or marked as completed andwe can reactivate them after that. All tasks will remain editable fornow, so we add the editor buttons to each one of them.

Finally, the last function takes a bunch of raw task data, create Taskobjects and call their makeDom method, inserting them into thegiven container. It also fills the taskLists dictionary entry forthat list.

mu.tutorial.tasks.makeTasks = function(name, data, container) {
    if (data.length == 0)
        mu.tutorial.tasks.noTasks(container);
    else
        for (var i = 0; i < data.length; i++) {
            var task = new mu.tutorial.tasks.Task(data[i], container);
            task.makeDom();
        }

    mu.tutorial.tasks.taskLists[name] = container;
}

We still have to make use of all that code, an HTML page will containthe remaining parts. We only need one div for the toolbar, then fourothers for each sections of our application. After that we add some moreJavaScript to attach the toolbar and create some tasks to test ifeverything is working.

  <body>
    <h1>Closure Library Tutorial</h1>
    <div id="menu"></div>
    <div id="taskList"></div>
    <div id="completedTaskList"></div>
    <div id="deletedTaskList"></div>
    <div id="settings">TODO...</div>
    <script type="text/javascript">
      function main() {

          var menu = document.getElementById('menu');
          mu.tutorial.tasks.attachToolbar(menu);

          var makeTaskData = function(summary, description, priority) {
              return { 'summary': summary, 'description': description, 'priority': priority };
          };

          var taskList = document.getElementById('taskList');
          mu.tutorial.tasks.makeTasks('active', [
              makeTaskData("Make the toolbar actually do something.", "Need more studying of Toolbar demo.", 99),
              makeTaskData("Make tasks editable.", "Like in the Closure Library Notes tutorial.", 88),
              makeTaskData("Create different task lists.", "One for active tasks, another for completed ones and finally a list containing deleted tasks.", 100)
              ], taskList);

          var completedTaskList = document.getElementById('completedTaskList');
          mu.tutorial.tasks.makeTasks('completed', [], completedTaskList);

          var deletedTaskList = document.getElementById('deletedTaskList');
          mu.tutorial.tasks.makeTasks('deleted', [
              { 'summary': 'test', 'description': 'test', 'priority': 1 }
              ], deletedTaskList);
      }
      main();
    </script>
  </body>

You can grab the full HTML filehere(look at the source or use "Save Link As...") and the JavaScript usedthere. Whentesting it, you'll notice the loading time can be long for a staticpage. Using Firebug net panel, we can see that the page ask for no lessthan 54 JavaScript files, that's a lot of requests! Over the wire, thiscan quickly become a problem, especially for high latency connections. Tofix this problem, we'll use the calcdeps.py script found in thebin directory. You simply have to call it like that:

> ./closure/bin/calcdeps.py -i tasks.js -p . -o script > tasks.nodeps.js

Still, it makes our little JS file a lot heavier, it now weight in at656KB, 650 more than the original. That's a lot, but can be mitigated byserver settings like file compression and expire headers.

Here's ademoif you're not interested by setting up everything yourself. In the nexttutorial we'll learn to use sliders and color pickers to make thesettings panel do something and we'll make the active task list sortableby priority.

Any questions, corrections or insults?

Overlooking the Obvious

2009-12-02T23:33:00.005-05:00

Monday, Tim Bray posted anotherarticleon Clojure in his Concur.next series. In there is some beautiful code(as idiomatic Clojure usually is) written by John Evans, in which therewas a function I wasn't familiar with.

user> (doc merge-with)
-------------------------
clojure.core/merge-with
([f & maps])
  Returns a map that consists of the rest of the maps conj-ed onto
  the first.  If a key occurs in more than one map, the mapping(s)
  from the latter (left-to-right) will be combined with the mapping in
  the result by calling (f val-in-result val-in-latter).

Although I get the general idea I think it's preferable to performexperiments after reading some documentation, just to be sure.

user> (merge-with #(+ %1 %2) {:a 1} {:a 2 :b 3} {:a 4 :b 5 :c 6})
{:c 6, :b 8, :a 7}
user> (merge-with #(list %1 %2) {:a 1} {:a 2 :b 3} {:a 4 :b 5 :c 6})
{:c 6, :b (3 5), :a ((1 2) 4)}

It does pretty much what the docs say, it's always great to find about auseful function you didn't knew about! Let's see some real code usingthis function.

(defn- merge-map
  "Merges an inner map in 'from' into 'to'"
  [to key from]
  (merge-with merge to (select-keys from [key])))

This was extracted from Compojure source codefor response handling. It's used to merge the response's headers andsession maps when the update-response multi-method is called on anobject of the Map class. That gave me an idea: the principle of usingmerge as the with function could be generalized into afunction that recursively merges maps. Let's to do it manually.

user> (merge {:a 1} {:a 2 :b 3} {:b 4 :c 5})
{:c 5, :b 4, :a 2}
user> (merge-with merge {:a {:a 1 :b 2}} {:a {:a 3 :c 4}} {:a {:b 5 :c 6}})
{:a {:c 6, :a 3, :b 5}}
user> (merge-with (partial merge-with merge) {:a {:a {:a 1} :b {:b 2}}} {:a {:a {:a 3} :c {:c 4}}} {:a {:b {:b 5} :c {:c 6}}})
{:a {:c {:c 6}, :a {:a 3}, :b {:b 5}}}

That's surely feasible, but I can't seems to find any use for such afunction, so I'll leave it as an exercise. The complexity of this taskcertainly negate the actual benefits gained from its limited usefulnessanyway. Is there anybody who disagree?

Update: Mister Bray posted yet another article on Clojure yesterday and proggit bursted in flames with relatively interesting (but heated) discussions on Clojure versus everything under the sun. It also provoked craziness!

Wickedly Short Wikis

2009-11-18T22:20:00.024-05:00

Well well well, let's be a little more serious this time around and talkabout code. This is a technical blog (hum... interesting, a stockinstallation of emacs flyspell mode doesn't know about words like wiki,blog or even flyspell) after all, we ain't gonna talk about nothing!

So, today, I'll talk about wikis or, to be more precise, the softwarebehind them. We'll dissect a very simple implementation, just to seewhat they are all about. Basically, a wiki is a very simple concept,reduced to its core it's the combination offour principles:

Automatic link generation
Editable content
Simplified formating
Backlinks

Upon realizing this, hordes of hackers from around the world flocked towrite the shortest one for glory and fame. And thus the shortest wiki contest wasborn. As everybody expected the winner used ~~Perl~~ a mix of Perland shell script. This all happened a long time ago (in Internet time)and now I'll tears apart one of these very little beast.

I won't bother with the Perl ones as I'm not masochist. In fact, I'veactually already done it forwypy.py a couple of weeksago. It's the best Python entry and also the best for a language that isnot Perl. Even then, I've started with the 23 lines version not loosetoo much time, just enough. The actual entry to the contest was 11 lineslong and contained only 814 characters, which is still nearly four timeslarger than the winning one, that's short! In this version, the code isat least divided into four functions. It begins with a simple one,load, which returns the content of a text file if it exists, elsea string:

# ex is a shortcut for os.path.exists
def load(n): return (ex('w/'+n) and open('w/'+n).read()) or ''

This isn't really exciting, just a good usage of logical operators. Thenext one has more meat to it, let's see the code first:

def fs(s): return reduce(lambda s, r: re.sub('(?m)'+r[0], r[1], s), (('\r',''),
   ('(^|[^A-Za-z0-9?])(([A-Z][a-z]+){2,})', lambda m: (m.group(1) + '%s<a hr' \
    'ef=wypy?%s'+m.group(2)+'%s>%s</a>') % ((m.group(2),'p=','&amp;q=e','?'),
   ('','','',m.group(2)))[ex('w/'+m.group(2))]),  ('^\{\{$','\n<ul>'),
   ('^\* ','<li>'),  ('^}}$','</ul>'),  ('^---$','<hr>'),  ('\n\n','<p>'),
   ('(ht|f)tp:[^<>"\s]+','<a href="\g<0>">\g<0></a>')), cgi.escape(s))

It look awful, but upon closer inspection we see that the noise ismostly regular expressions and some compact markup in a formatstring. All in all, it's a foldtaking the escaped input string as initial value and reducing a list oftuples which are composed of a regular expression and either of a stringor a function. These happen to be the two first arguments ofre.sub which is the function used in the lambda expression fed toreduce. There's one little unexplained detail in there, why does'(?m)' is prepended to every regular expressions? After somegoogling, I've found this explanation:

Caret and dollar match after and before newlines for the remainder of the regular expression. (Older regex flavors may apply this to the entire regex.)

Well, I'll confess I don't truly understand what this is suppose tomean, but it doesn't look that important anyway, let's move on.

Now the do function, obviously the one actually doing something!It's a kind of dispatching function (using a dictionary) performingdifferent type of actions depending on the value of the first parameter:

def do(m, n): return {'get':'<h1>WyPy:<a href=wypy?p=%s&amp;q=f>%s</a></h1>(' \
   '<a href=wypy?p=%s&amp;q=e>edit me</a>)<p>%s' % (n, n, n, fs(load(n)) or n),
   'edit': '<form action=wypy?%s method=POST><h1>%s<input type=hidden name=p' \
   ' value=%s> <input type=submit></h1><textarea name=t rows=15 cols=80>%s</' \
   'textarea></form>' % (n, fs(n), n, load(n) or "Describe %s" % n), 'find': 
   ('<h1>Links: %s</h1>' % fs(n))+fs('{{\n* %s\n}}' % '\n* '.join(filter(
    lambda s: open('w/'+s).read().find(n) > -1, os.listdir('w'))))}.get(m)

It is quite simple actually, get returns a wiki page, edit is obviousand find make a page containing a list of links. The main functionthen wires up everything to be called by a CGI process and also add apage title.

def main(f): 
   n = f.get('p') or env("QUERY_STRING") or ''; n = ('HomePage',n)[n.isalpha()]
   print "Content-type: text/html; charset=utf-8\r\n\r\n<title>%s</title>" % n
   if env("REQUEST_METHOD") == "POST": open('w/'+n, 'w').write(f['t'])
   print do({'e':'edit', 'f':'find'}.get(f.get('q')) or 'get', n)

That's it, short and sweet. A last interesting point, the only code thatI had trouble understanding was this: ('HomePage',n)[n.isalpha()]}which is quite confusing on first sight. It cleverly use the implicitconversion from boolean to integer to choose between the two items of atuple. It's in fact an old school way of simulating the ternaryoperator, that feature having been added to Python 2.5 in 2006. Thattrick might be well know amongst pythonitas, but I'm not one of them asI rarelly code in Python.

While dissecting this code I've refactored (or might I say degolfed)it into a 45 lines version that you can grabhere. I've got a nearlyfinished version of this code translated to Clojure, I'll follow up onthis post once it's done, but that may take a while as I'm having funwith the recently released Closure Library.

P.S.: Ward's Wiki is down at the time of this writing so the two firstlinks of this post might not work for some time. You can try to accessthem through Google's cached links.

Hello World!

2009-11-15T19:57:00.011-05:00

Hello world! I have currently nothing interesting to say, just postingto see how posts are looking with my selected theme. I have to fill upthis post a little bit, so that I'll be able to better judge theappearance of a sufficiently large body of text. How much morecompletely useless text I still have to write for that state to beattained is not yet clear in my mind. Maybe I'll continue until I'm tootired, or not. There's a chance I'll write it in pieces, spread overmany days. Anyhow, this isn't very important, even for a post hopefullydevoid of any meaning.

Going the Lorem Ipsum way isn't an option, though, as I'm hopelesslytired of reading that text. Moreover I don't even read Latin. Whichbring me to a potentially interesting subject, that is: "How can weassess the pertinence of a given text?" It's quite hard to determine howinteresting such a subject is and I fear I could not reveal all of itsintricacies. Well, let's not discourage ourselves with these pettyconcerns and write some more. First and foremost, it's more than obviousthat pertinence is a relative concept. Not everybody would be equallyinterested in reading some collection of more of less related wordsarranged according to a somewhat coherent set of semantic rules.

And how much the actual meaning is affected by styling concerns? Can itbe actually obscured by them? Does a poorly written piece of textcontaining the same meaning as a masterpiece is necessarily of lesservalue. Maybe it can have some advantages over the better one, like itcould be of a smaller size, thus requiring less time to read. Is thisexample even possible? Theoretically and on the top of my head I'd sayyes, but that would be more an exception than a rule.

OK, now this post is starting to look like it may be big enough. Atlast, the end of the tunnel is near, I can feel it. I'll stop anytimenow...

No, that wasn't really it. There's still more words for me to write andfor you to read, however silly they are. Those two actions willevidently not be done at the same time, unless referring to myself asthe reader. And here's another potentially interesting question arising:"Can we write and read at the same time?" Well not really interesting, Iwould even say that nobody care about that question outside someneurologists somewhere found on this planet. Which bring me to anotherquestion, this one incredibly more asinine than the previous ones: "Isthere neurologists on other planets?" I'll abstain on commenting on thisone though.

In the end, I just hope I'm not making too much sense as it risks tocontradict the main objective of this post. I also wonder how manygrammatical and styling mistakes I managed to make. There's certainlysome vestigial artifacts of my mother tongue here and there. Doespunctuation is well used? Did I correctly pluralized each and everywords? Will some benevolent readers point them out for me?

P.S.: As usual I have more questions than answers!

Mu