looking for a full-immersion/face-to-face HTML5, JavaScript, and Mobile Web Development training in the heart of London? Book your spot
Showing posts with label asynchronous. Show all posts
Showing posts with label asynchronous. Show all posts

Friday, June 08, 2012

Asynchronous Storage For All Browsers

I have finally implemented and successfully tested the IndexedDB fallback for Firefox so that now every browser, old or new, should be able to use this interface borrowed from localStorage API but made asynchronous.

Asynchronous Key/Value Pairs

The main purpose of this asyncStorage API is to store large amount of data as string, including base64 version of images or other files.
As it is, usually, values are the bottleneck, RAM consumption speaking, while keys are rarely such big problem.
However, while keys are retrieved asynchronously and in a non-blocking way, but kept in memory, respective values are always retrieved asynchronously in order to do not fill the available amount of RAM for our Web Application.

Database Creation/Connection

Nothing more than ...

asyncStorage.create("my_db_name", function (db, numberOfItems) {
// do stuff with the asyncStorage
});


Storing An Item

As it is for localStorage, but async

asyncStorage.create("my_db_name", function (db, numberOfItems) {
db.setItem("a", "first entry", function () {
// done, item stored
});
});

Please note that if the item was there already, it's simply replaced with the new value.

Getting An Item

As it is for localStorage, but async

asyncStorage.create("my_db_name", function (db, numberOfItems) {
db.getItem("a", function (value) {
// done, value retrieved
});
});

Please note that if the item was not previously stored, the returned value will be exactly null as it is for localStorage.

Removing An Item

As it is for localStorage, but async

asyncStorage.create("my_db_name", function (db, numberOfItems) {
db.removeItem("a", function () {
// done, no key a is present anymore
// so length is decreased already here
// and db.get("a") will send a null value
});
});


Removing All Items

As it is for localStorage, but async, and considering that only values in the specified database name will be erased, rather than all of them.

asyncStorage.create("my_db_name", function (db, numberOfItems) {
db.clear(function () {
// done, database "my_db_name" is now empty
});
});


Getting All Items Keys

If this is really what you need to do, bear in mind the API is the same used in the localStorage where indeed there's no way to retrieve all keys if not doing something like:

for (var
keys = [],
i = db.length;
i--;
) {
keys[i] = db.key(i);
}


Getting All Items

Well, the thing here is that you can store an entire object through JSON so if you need to save and get back everything, it's kinda pointless to store different keys, just use one.
However, this is how I would do this task:

for (var
object = {},
complete = function () {
alert("Done, all items in the object");
},
ongetitem = function (value, key) {
object[key] = value;
if (!--j) complete();
},
i = db.length,
j = i;
i--;
) {
db.getItem(db.key(i), ongetitem);
}


On Github, Of Course

You can find full API description in this repository. Please forgive me if the name was initially db.js, I believe the new one, asyncStorage.js, is much more appropriate (also another guy created a script called db.js so ... well, I have avoided conflicts with that library).

That's Pretty Much It

And I hope you'll start using this API to avoid blocking mobile and desktop browsers when you store a lot of data ;)

Monday, June 04, 2012

Working With Queues

Programming with queues is basically what we do in any case: it does not matter if we write code in that way, we simply think in that way.
This means that our logic flow is generally distributed in tasks where "on case X" we apply "logic/procedure Y" .. isn't it?

The Good'ol GOTO

The goto statement has been historically criticized, as well as the switch one, and in both cases is about entry and exit points in a generic workflow.
Nowadays, we can say the GOTO is not needed anymore thanks to functions, where rather than thinking "when this case occurs, goto this instruction" we call the required function in charge of that specific task providing arguments or context as we go.
We may then agree that GOTO is not really a must have while functions are, with all the power and flexibility we might need, and specially in JavaScript.

On Block-Scope

JavaScript has theoretically no block-scope concept, at least until very latest versions of ECMAScript where blocks can be written in the wild.
Blocks are cool for partially independent operations that should not affect at all the external scope/logic but if we think more about this, the usage of inline function expressions has replaced the block-scope concept for a while.
Even better, any function in JS could be considered a sort of equivalent of a block scope, with the advantage that we can re-call the same function as many times as we want, with limits in recursions, avoiding the GOTO and still using block-scopes.

All Together

What if we use as many functions as we need, in order to complete our flow, without compromising the external environment and being able to re-call segments of our flow when something goes wrong?
This can be easily done with a queue system, like the one showed below:


A Few Examples

Here a very basic example on how to use above queue system. There are two sequential things to do, waiting for some truish condition then do something.

Queue([
function (q) {
q.wait(document.body);
},
function onBodyPresent(q) {
document.body.innerHTML = "Hello queue!";
}
]);

The wait() method calls next "block" to execute, as next function, if any, only if the condition passed as argument is true, waiting otherwise "0 ms", if not specified differently, before the same "wait for" block logic is re-executed.
While this example does not show real potentials of queues based programming approach, the next one could do it.

!function () {
var
init = function (q) {
if (!result) {
number = Math.abs(prompt("factorial of:")) || 1;
result = 1;
}
q.next();
},
verify = function (q) {
if (1 < number) {
result *= number--;
q.unshift.apply(
q,
program.slice(
program.indexOf(init),
program.indexOf(verify) + 1
)
);
}
q.next();
},
showResult = function (q) {
alert(result);
q.next();
},
program = Queue([
init, verify, showResult
]).slice(),
result, number
;
}();

Above code is a factorial program: all logic blocks are known in advance and the queue is constantly re-populated until the condition in the middle is satisfied.
While performances are not the best, but generally speaking performances are never a problem when a queue logic is needed, since queues are awesome specially for asynchronous tasks that are rarely good for real-time programming, this factorial program does not use recursion, is quite easier to understand and debug, and it does not blow the RAM usage: functions are recycled as well as the queue which will never grow more than 3 indexes so just wait ... and the result at some point will appear ;-)

Asynchronous Example

As I have said already, queues are great for asynchronous tasks helping with indentation, never too many nested functions, logical workflow, each function does a little but it does it well, readability, functions are named in a semantic way but minifiers will simply shrink them once executed, and even if this does not look that OOP, I bet once we start getting use to this approach things will be easier than ever.

!function () {
// warning: this code is a proof of concept
// it won't work as it is ...
var
// query selector shortcut
$ = function (css) {
return document.querySelector(css);
},
// while user and pass fields are empty ...
login = function (q) {
q.wait(
$("#user").value.trim()
&&
$("#pass").value.trim()
);
},
// once user and pass are not empty anymore
verify = function (q) {
var xhr = new XMLHttpRequest;
xhr.open("post", "verify.php", true);
xhr.send([
"user=" + encodeURIComponent($("#user").value.trim()),
"pass=" + encodeURIComponent($("#pass").value.trim())
].join("&"));
xhr.onreadystatechange = function () {
if (xhr.readyState == 4) {
q.result = xhr.responseText;
// call next function
q.next();
}
};
},
// verify if the user exists
authorized = function (q) {
if (q.result === "ok") {
q.push(ok);
} else {
// notify the error plus re-queue logic between
// login and authorized included
q.push.apply(q, [error].concat(program.slice(
program.indexOf(login),
program.indexOf(authorized) + 1
)));
}
q.next();
},
// end of the program
ok = function (q) {
alert("Welcome!");
// could go on with the same queue
// passing through a different program
},
// warning for the user
// plus resetting fields
error = function (q) {
alert("user or pass not recognized");
$("#user").value = "";
$("#pass").value = "";
q.next();
},
// clone of the whole program
// reused later to recycle segments
program = Queue([
login, verify, authorized
]).slice()
;
}();


Where Queues Are Used

Well, almost everywhere ... Testing Frameworks are usually based on queues, specially those with asynchronous support for different tests. Same is for JavaScript or CSS loaders, based on queues when it comes to order.
Any sort of stack/Array not representing data, is usually a queue we consume during our logic ... promises are queues too, and same is for events, these are all queues.
Should I explain more? Probably yes, but I'd prefer if you have a look at these examples, use the simple Queue function I have written, and create something awesome that makes the logic of your app cleaner and better organized.
Last, but not least, did I mention that logging a queue gives us instantly a logic workflow of what's going on and accordingly with where it's going on? console.log(q.slice()) anywhere you need and you'll see all named functions you gonna deal with after the one is executing in that moment.

Friday, March 09, 2012

A Tweet Sized Queue System

The Code

This is the 138 bytes version:

function Queue(args, f) {
setTimeout(args.next = function next() {
return (f = args.shift()) ? !!f(args) || !0 : !1;
}, 0);
return args;
}

While this is the 96 one:

function Queue(a,b){setTimeout(a.next=function(){return(b=a.shift())?!!b(a)||!0:!1},0);return a}


The Why

In almost every QCon London talk I have heard the word asynchronous. Well, the aim of the tweet sized function I have just written above is to make things easier.
After I have implemented builder/JSBuilder.js for wru, so that Python is not needed anymore to build the tiny library, I needed to make builder/build.js usable in a way that asynchronous calls through node won't bother, at least visually, the normal flow of the script.
After that I have thought: "why not making a generic Queue function with Promises like approach" ?

The What

It's quite straight forward, you create a queue, you use the queue. You might eventually pollute the queue or change its order, since the queue is nothing different from a generic Array with an extra method on it: next().
Here a basic example:

Queue([
function (queue) {
// do amazing stuff asynchronously
setTimeout(function () {
queue.message = "Hello Queue";
queue.next();
}, 500);
},
function (queue) {
// do something else
alert(queue.message);
// Hello Queue
queue.next(); // not necessary at the end
// does not hurt thought
}
]);


That's Pretty Much It

So, the queue argument is the array created initially to define the ordered list of function.
This means you can recycle this variable even adding custom properties but most important is that you can change the queue via Array.prototype.splice/unshift/pop/push() methods and decide the future or the next function to execute runtime, if needed.
The only extra bit is about returning true or false in case there was a next() to execute or not.

Last, but not least, of course you can create new Queue([]) inside queues ... so, this is actually more powerful than many other queue systems I have seen so far and ... hey, 96 bytes: enjoy!

Monday, September 20, 2010

Fragment and Vertex Shaders: My Way To Load

I have finally received the fifth and amazing version of the OpenGL SuperBible book and I have already started digging into it, really well done for what I can tell.

The book is mainly focused on "real OpenGL development", something surely more suitable for tough C/C++ developers rather than Web Monkeys like me but since the book includes an OpenGL ES 2.0 related part, and since latter spec is basically what we can find in WebGL, it's always better start learning what's next knowing history and background of the used technology ... and here I am :-)

Something Already Wrong

I am not completely sure about "who started this techniques", John Resig with his micro template inside script nodes with a type text/html may be the indirect responsible for a sort of growing "monster" we can see in almost every WebGL related example ...

HTML Embedded Shaders, the Web 3.0 NO-GO

Let me pass the therm, but 3D websites are not that far away from reality. Minefield, Chrome, WebKit, and probably others, are working hard with Khronos and WebGL and I do believe official support will come later this year with next browsers releases ( ... hoping IE9 final won't "block the world" avoiding WebGL support ... )
Back in the topic, the common examples technique is a step backward in 1999 where runtime loading did not exists (aka: Ajax or scripts injection) and an HTML page was a trash bin for any kind of rubbish.

A Better Technique

We are in 2010 and we want use latest technologies, so why should we use old and deprecated one? Let me show this simple getShader function alternative:

// our shaders base path
loadShaders.base = "shader/";

// our shaders loader
function loadShaders(gl, shaders, callback) {
// (C) WebReflection - Mit Style License
function onreadystatechange() {
var
xhr = this,
i = xhr.i
;
if (xhr.readyState == 4) {
shaders[i] = gl.createShader(
shaders[i].slice(0, 2) == "fs" ?
gl.FRAGMENT_SHADER :
gl.VERTEX_SHADER
);
gl.shaderSource(shaders[i], xhr.responseText);
gl.compileShader(shaders[i]);
if (!gl.getShaderParameter(shaders[i], gl.COMPILE_STATUS))
throw gl.getShaderInfoLog(shaders[i])
;
!--length && typeof callback == "function" && callback(shaders);
}
}
for (var
shaders = [].concat(shaders),
asynchronous = !!callback,
i = shaders.length,
length = i,
xhr;
i--;
) {
(xhr = new XMLHttpRequest).i = i;
xhr.open("get", loadShaders.base + shaders[i] + ".c", asynchronous);
if (asynchronous) {
xhr.onreadystatechange = onreadystatechange;
}
xhr.send(null);
onreadystatechange.call(xhr);
}
return shaders;
}

With above minifier friendly function we can avoid shaders on the page. The natural advantage is that we can properly organize shaders in a structure like this one:

root

shader

vs // vertex shaders

ShaderName.c

fs // fragment shaders

FragmentName.c

Having a proper extension means that we can edit these files with highlighted syntax and we can edit these files a part, without looking for id or scripts inside an HTML page. Fragment and Vertex Shaders are our "application templates", so why on earth should we include them in the layout? Via loadShaders we can use cached shaders, share shaders, etc etc, and here some usage example.


vra gl = canvas.getContext("experimental-webgl");

// synchronous order, one shader
var myFragmentShader = loadShaders(gl, "fs/myFragment");

// synch, more shaders
var shaders = loadShaders(gl, [
"vs/what",
"vs/ever"
]);
// [whatShader, everShader]


// asynchronous loading
loadShaders(gl, ["fs/what", "vs/ever"], function (shaders) {
var
fragmentWhatShader = shaders[0],
vertexEverShader = shaders[1];
;
});


// asynch in "don't care mode, just cache them"
loadShaders(gl, ["fs/what", "vs/ever"], true);



Features


  • Synchronous or Asynchronous loading

  • Multiple Loads (unordered compilation for each result)

  • Ordered Results, (order respected accordingly with paths)

If I have forgotten something please point it out but even more please, whatever you like or not this solution, do not include shaders in your HTML page (OK, OK, gotcha, for some example it may make life easier but ... you know how people get examples ... right?)

Cheers :)

Monday, September 14, 2009

LiveMonitor - Asynchronous Property Monitor

Today I would like to introduce you a quite uncommon JavaScript trick, a trapped Live Object or, generally speaking, a lightweight monitor able to understand when a generic property has been changed.

About Live Objects

A live object could be described as a particular object able to change without our interaction. The most common live object example is this:

// this is the most common live object
// the HTMLCollection
var divs = document.getElementsByTagName("div");
divs.length; // let's say 4

// let's add another div inside a generic node
document.body.appendChild(
document.createElement("div")
);

divs.length; // 5!

In few words DOM searches are dynamic, which is the reason almost every selector library needs to transform the current result into a static Array.

About LiveMonitor

Specially suited for live objects, LiveMonitor is a function which aim is to notify us when the specified property change:

// LiveMonitor example
var lm = new LiveMonitor(
// the entire list of elements trapped
document.getElementsByTagName("*"),
// the property to monitor
"length"
);

// add one or more notifier
lm.onchange(function(collection){
// this, will be the lm object
this.value; // will be collection.length
collection.length; // is the new length

alert([
"All nodes collection contained ",
this.value,
" nodes but now there are ",
collection.length
].join("\n"));
});

The example is quite silly, but the concept is that as soon as the monitor will check the "length" property, in this case in the one of the trapped collection, it will fire the onchange event, notifying us that somebody did something somewhere, and this something changed our monitored property.

LiveMonitor Code


var LiveMonitor = (function(){

/** LiveMonitor :: Asynchronous Property Monitor
* @author Andrea Giammarchi
* @license Mit Style
* @blog http://WebReflection.blogspot.com/
*/

function LiveMonitor(object, property){
this.value = object[property];
this._property = property;
this._object = object;
this._event = [];
this._i = 0;
};

LiveMonitor.prototype.onchange = function onchange(callback){
this._event.push(callback);
if(this._i === 0){
var _property = this._property,
_object = this._object,
_event = this._event,
self = this
;
this._i = setTimeout(function _i(){
if(self.value !== _object[_property]){
for(var i = 0, length = _event.length; i < length; ++i){
if(_event[i].call(self, _object) === false)
break
;
};
self.value = _object[_property];
};
if(0 < self._i)
self._i = setTimeout(_i, 15)
;
}, 15);
};
};

LiveMonitor.prototype.offchange = function offchange(callback){
for(var _event = this._event, i = 0, length = _event.length; i < length; ++i){
if(_event[i] === callback)
_event.splice(i, 1)
;
};
if(_event.length === 0){
clearTimeout(this._i);
this._i = 0;
};
};

LiveMonitor.prototype.clear = function clear(){
this._event = [];
this.offchange(null);
this.value = this._object[this._property];
};

return LiveMonitor;

})();


Not Only DOM Searches

LiveMonitor could be actually used as asynchronous notifier for any kind of variable.
Let's say we have an environment able to load runtime scripts but we cannot change loaded scripts (external source inclusion).
At some point we press the "load jQuery" button and we would like to be able to be notiied when it is available ...

if(!window.jQuery){

// create a LiveMonitor instance over
// jQuery property
var jqlm = new LiveMonitor(window, "jQuery");

// add onchange event
jqlm.onchange(function(window){

// remove this event
this.offchange(arguments.callee);
// remove jqlm as well, it is not useful
// anymore for this example

// use jQuery, it's here for sure!
$("body").html("Hello LiveMonitor!");
});

// load external script
loadScript("http://external/jQuery.js");
};


Conclusion

With a cross-browser, portable, and lightweight function, we can use notifications without effort over objects, HTMLCollections, Arrays, or everything else we would like to monitor. Let's say this is a sort of asynchronous cross browser Object.prototype.watch, but this time without alchemy.