CouchDB, ExtJS and Simile Timeline
In this post I’ll try to summarize my first experiment on using CouchDB in a solution of a real world problem.
Problem
At work we are developing a VOIP application based on Yate (an OpenSource VOIP Engine) and in this kind of application timing is a very important factor, more important than in classic Web applications.
The application is working like a charm but sometimes it encounter exceptional conditions and we feel it can be related to something we’re not handling in the right way or… at the right time ;-)
How can we trace and debug this kind of problems?
We need a complete tracelog of messages received and how our application is handling them… and we need to trace the real-deployed-application, where this exceptional conditions occurs.
Solution
The solution we’re prototyping is based on a simple and not so new architecture:
The application push useful tracing information in a central storage, and we want to search, filter, analize and visualize useful informations from it.
This kind of problem can be solved in a number of ways and technologies but our experimental solution use CouchDB as “tracing-info” storage, and a pure client-side javascript application to access the data.
Some Tech Details
Our application is developed in PHP (sigh!) so we’re using the simpler PHP binding of the CouchDB Rest API:
NOTE: PHP-on-Couch don’t support authentication so I wrote a small patch (and I will setup a github fork asap)
In our current design decision every tracing session became a new different couchdb database:
<?php
// CONNECT TO A DATABASE AND DATABASE CREATION
$client = new couchClient ('http://localhost:5984','alcadialer_tracelogs_'.$start_tracing_datetime);
if ( !$client->databaseExists() ) {
$client->createDatabase();
}
// COLLECT TRACING INFO AND STORE IT ON COUCHDB
try {
$ev->setParam("AlcaDialerCommand",get_class($command));
$response = $client->storeDoc($ev);
} catch (Exception $e) {
echo "ERROR: ".$e->getMessage()." (".$e->getCode().")<br>\n"; echo $e;
}
?>
So we’ve tracing logs databases in a CouchDB service…
now we need to navigate them in useful ways and in a timing problem “useful ways” means “visualize events on a timeline”… and the “simile timeline” is so pretty and simple to use:
Now the question is: how can we host all this stuff on CouchDB? we need a separate service?
The answer is: CouchDB is enough :-)
A more technical answer to this question can be: CouchApp is the way :-D
After CouchApp installation we can generate a CouchDB standalone application working tree (as Rails do)
$ couchapp generate myapp
$ ls -l myapp
drwxr-xr-x 4 rpl rpl 4096 2009-11-16 19:02 _attachments
-rw-r--r-- 1 rpl rpl 97 2009-11-16 07:03 couchapp.json
drwxr-xr-x 2 rpl rpl 4096 2009-11-16 07:01 lists
drwxr-xr-x 2 rpl rpl 4096 2009-11-16 07:01 shows
drwxr-xr-x 2 rpl rpl 4096 2009-11-16 07:01 updates
drwxr-xr-x 3 rpl rpl 4096 2009-11-16 07:01 vendor
drwxr-xr-x 2 rpl rpl 4096 2009-11-16 07:01 views
In our usecase we have to put all files of our javascript application in the _attachments dir:
$ ls _attachments/ -l
-rw-r--r-- 1 rpl rpl 1255 2009-11-16 19:02 index.html
drwxr-xr-x 4 rpl rpl 4096 2009-11-16 22:47 js
drwxr-xr-x 2 rpl rpl 4096 2009-11-16 19:01 style
$ ls _attachments/js -l
-rw-r--r-- 1 rpl rpl 5440 2009-11-16 22:47 application.js
drwxr-xr-x 4 rpl rpl 4096 2009-11-16 07:13 extjs
drwxr-xr-x 4 rpl rpl 4096 2009-11-16 14:52 timeline
In “_attachments/js” there is our application.js and two directories, extjs and timeline, where are our application dependencies (I think they can/could be stored under vendor directory… but I will try this in a second experiment)
From extjs and timeline directories we’ve removed unused stuff (examples, documentations etc.).
Let’s see some of the code of application.js (the interesting part of application.js code :-D).
How can we put CouchDB data in an ExtJS grid?
On CouchDB wiki there’s a CouchStore prototype for ExtJS… but it don’t work very well (it have to be reworked, I think), but in our usecase we only need to get from a CouchDB database, so we can use a classic JSONStore from the ExtJS standard lib:
var store = new Ext.data.JsonStore({
// store configs
autoDestroy: true,
url: trace'/tracelogdb/_design/messages/_view/all_by_seqid',
storeId: 'myStore',
// reader configs
root: 'rows',
idProperty: 'key',
totalProperty: 'total_rows',
restful: true,
fields: [
{name: 'id' },
{name: 'key' },
{name: 'value' },
]
});
And attach it to a GridPanel.
Our JsonStore use a design view in the tracelogdb so we need to create it (we’ve not automated the task right now):
{
"_id":"_design/messages",
"language":"javascript",
"views": {
"all_by_seqid": {
"map":"function(doc) { emit(doc.params.AlcaDialerSeqID, doc); }"
}
}
}
Now we’ve a local (to the browser) store of CouchDB documents and every row in the view have a timestamp in a sub-property of ‘value’, so we can use that value to insert and visualize events in a simile timeline component:
function TimelineInit(store) {
var eventSource = new Timeline.DefaultEventSource();
var bandInfos = [
Timeline.createBandInfo({
width: "70%",
intervalUnit: Timeline.DateTime.SECOND,
intervalPixels: 100,
eventSource: eventSource,
// THIS BAND ZOOM ON MOUSE WHEEL
zoomIndex: 0,
zoomSteps: [
{pixelsPerInterval: 280, unit: Timeline.DateTime.SECOND},
{pixelsPerInterval: 140, unit: Timeline.DateTime.SECOND},
{pixelsPerInterval: 70, unit: Timeline.DateTime.SECOND},
{pixelsPerInterval: 35, unit: Timeline.DateTime.SECOND},
{pixelsPerInterval: 400, unit: Timeline.DateTime.MINUTE},
{pixelsPerInterval: 200, unit: Timeline.DateTime.MINUTE},
{pixelsPerInterval: 100, unit: Timeline.DateTime.MINUTE}
]
}),
Timeline.createBandInfo({
overview: true,
eventSource: eventSource,
width: "10%",
intervalUnit: Timeline.DateTime.HOUR,
intervalPixels: 100
}),
];
timeline_data = {
'wiki-url':"http://fake-url/",
'wiki-section':"Fake Section",
'dateTimeFormat': 'Gregorian',
'events': []
};
var last_timestamp;
var seq = 0;
function render_event(record) {
var description = "";
// ... FORMAT EVENT DESCRIPTION FROM RECORD CONTENT
return {
durationEvent: false,
title: "(" + record.data.key + ") - " + record.data.value.name,
description: description,
start: new Date(record.data.value.origin*1000),
icon: 'js/timeline/timeline_js/images/dark-red-circle.png'
};
}
store.each(function(i) {
timeline_data.events.push(render_event(i));
});
// SYNC TIME BAND
bandInfos[1].syncWith = 0;
bandInfos[1].highlight = true;
// CREATE TIMELINE
tl = Timeline.create(document.getElementById("my-timeline"), bandInfos);
// CENTER TO THE FIRST EVENT
tl.getBand(0).setCenterVisibleDate(Timeline.DateTime.parseGregorianDateTime(timeline_data.events[0].start))
// LOAD ALL DATA
eventSource.loadJSON(timeline_data, document.location.href);
return tl;
}
We can enhance grid/timeline interactions with a listener that move timeline on grid row selection events:
listeners: {
"rowclick": function(grid,rowIndex,e) {
var selected = grid.getSelectionModel().getSelected();
window.timeline_view.getBand(0).setCenterVisibleDate(
Timeline.DateTime.parseGregorianDateTime(new Date(selected.data.value.origin*1000)));
}
},
How it looks right now?
ohhh… Yes.. it’s only a prototype but it’s soooo fun, isn’t it? :-)
Happy Hacking,
rpl