I was going to explain how to use them (which is very easy) but most importantly, why they are useful and when to use them. But instead of publishing it as a post I decided to update the docs and add all the info there so everything is centralized.

I recommend that you try to understand these methods and which kind of problems they try to solve. They are all very trivial to implement but of an extreme usefulness depending on the kind of project you are building. I also think that some of them changed the way I approach some kinds of problems. I use them so much that it’s hard to imagine coding something complex without something similar (I coded the methods clamp/loop/lerp/norm a million times).

So jump over to the docs and understand how to (ab)use these helpers. Even if you are not using amd-utils you can check the source code and implement it by yourself or extract the methods. Most of them are common knowledge but I feel that not that many people know how to use them properly or outside of the regular context, that’s why I thought some explanation would be helpful.

Docs: http://millermedeiros.github.com/amd-utils/math.html

The rule is simple, split larger functions/classes into smaller specialized ones, period. It will not only increase the readability but it will also make the code more reusable since it will be easier to override the default behavior if needed (especially if extending a class or reusing a 3rd party lib). I will try to explain how and give a basic example.

Why?

Basically because smaller functions are easier to understand and smaller files are easier to read. Try to read a very large paragraph and you will see that the information is way harder to grasp.

Another huge benefit in keeping your classes/modules small is that there is a big chance that it will have a very high cohesion (all functions are strongly-related), low cohesion is a very bad code smell. By grouping functions by context it will be very easy to identify which file should be edited and where. If the bug is on the “drop down menu” you should check the DropDown class, if you need to change the way files are written to the system than you should update the fileWriter, etc…

I’ve said it before and I’m going to say it again:

Comments are usually a sign of poor/lazy/confusing implementations. (source)

By keeping your functions small and by using descriptive names you will reduce the amount of necessary comments a lot, a few lines of well written code can say more to a developer than many pages of documentation (your mileage may vary). Don’t describe with comments what can be described with code!!

How?

Uncle Bob says that the optimal size of a functions is 1 line, and no, he is not talking about 1 line of mangled code with lots of chained methods, he is talking about 1 line of clean code that does one thing and does it well. It may seem drastic, but I must admit that it is a very good goal to keep in mind (even if you don’t follow it by heart).

Let’s use a simple example to describe it, first some CSS:

.awsum {
    background-color: #0ff;
    position: absolute;
    width: 300px;
    height: 300px;
}

Now some JS code (using jQuery for brevity):

//this whole example is an anti-pattern
function AwsumWidget(parent){
    this.root = $('<div class="awsum"/>');

    this.root
        .appendTo(parent)
        .fadeOut(0)
        .fadeIn(400)
        .click(function(evt){
            $(this).animate({
                top : Math.random() * 300,
                left : Math.random() * 300
            }, 500, 'swing');
        });
}

AwsumWidget.prototype = {

    remove : function(){
        if (! this.root) return;
        var self = this;
        this.root.fadeOut(400, function(){
            self.root.remove();
            delete self.root;
        });
    }

};

Now let’s suppose you need to create a RadWidget, it should have all the basic functionality from the AwsumWidget but it should do a different animation on click, how would you do it? Since all the logic is inside the constructor you will need to duplicate the code just to replace the click handler, not scalable/flexible at all… First step is to refactor AwsumWidget so it’s easier to extend it and overwrite just what we need, lets also abstract some hard-coded values to make it easier to update - that’s also a very good practice, don’t leave “magic” numbers in the middle of the code, specially if you use it more than once, giving a name to the value will help describe what it does:

var MAX_POS = 300,
    TRANSITION_DURATION = 400;

function AwsumWidget(parent){
    this._create(parent);
    this._animateIn();
    this._attachEvents();
}

AwsumWidget.prototype = {

    _html : '<div class="awsum"/>',

    _create : function(parent){
        this.root = $(this._html).appendTo(parent);
    },

    _attachEvents : function(){
        this.root.click(this._animateOnClick);
    },

    _animateOnClick : function(evt){
        $(this).animate({
            top : Math.random() * MAX_POS,
            left : Math.random() * MAX_POS
        }, TRANSITION_DURATION, 'swing');
    },

    _animateIn : function(){
        this.root.fadeOut(0).fadeIn(400);
    },

    remove : function(){
        if (! this.root) return;
        this._animateOut();
    },

    _animateOut : function(){
        this.root.fadeOut(TRANSITION_DURATION, $.proxy(this._destroy, this));
    },

    _destroy : function(){
        this.root.remove();
        delete this.root;
    }

};

It’s clearly more code, but by breaking the code into smaller specialized functions you can override just the functions you need and the function names already describe what the code is doing so it’s easy to follow the program logic, the RadWidget can be easily implemented as:

var MIN_SIZE = 50,
    MAX_SIZE = 300;

function RadWidget(){
    AwsumWidget.call(this);
    this.root.addClass('rad');
}

//let's just assume we are on an environment that supports Object.create
//https://developer.mozilla.org/en/JavaScript/Reference/Global_Objects/Object/create
RadWidget.prototype = Object.create(AwsumWidget.prototype);

RadWidget.prototype._animateOnClick = function(evt){
    var size = Math.max(Math.round(Math.random() * MAX_SIZE), MIN_SIZE);
    $(this).animate({
        width : size,
        height : size
    }, TRANSITION_DURATION, 'swing');
};

We reused all the code from the AwsumWidget and only changed the parts that we needed, that is the true power of breaking everything into separate functions, modular systems are more flexible by nature, since each part is treated as an individual piece you can replace them at will as long as you follow the proper interface.

Sum Up

Break your code into smaller pieces, it may seem more work but the benefits on the long run outweigh the initial overhead. And if you are thinking that it will slow down your code since you have more functions calls remember about the performance dogma, unless you have a very large loop doing something trivial the extra function calls shouldn’t be the bottleneck. It is better to have a clean/organized structure and tune it if needed than to have a huge pile of mess and not being able to identify where the real bottleneck might be or react to new requirements on a timely fashion.

By breaking your code into smaller blocks it will also help you follow the Single Responsibility Principle which I consider one of the best advices about program structure.

Try it for a while, the more you do it the more natural it will fell and it’s really impressive how much it improves the code readability and reusability. It’s very hard to foresee all the needs and later on the road it might be harder to change it, so my advise is to refactor often and keep it as modular as possible. Make code easier for you and future developers to understand and maintain, it will pay-off sooner than you expect.

That’s it!

Before coding JS-Signals I was using a very basic EventEmitter “class” that could be used to listen/dispatch arbitrary events but ever since I released JS-Signals I almost didn’t used arbitrary events anymore (because of the benefits of using a Signal), but a couple weeks ago I had to propagate changes on my model classes to the UI and the changes are coming from many different inputs, so the easiest way to keep everything in sync was to dispatch events every time my model objects updated with a new value. In that case it is way easier to use a string ID for the event type than to create a new Signal object manually for each value, since the project was already using Signals everywhere I decided to code a simple EventEmitter that would use JS-Signals internal mechanism (so I could use the advanced features if needed) but still allow arbitrary event types.

define(['signals'], function (signals) {

    /**
     * Signal Emitter
     * JS-Signals wrapper to convert it into a regular Event Emitter, which can
     * lister/dispatch arbitrary events.
     * @author Miller Medeiros
     * @version 0.1.1 (2012/01/23)
     * @license WTFPL
     */
    function SignalEmitter(){
        this._signals = {};
    }

    var _proto = SignalEmitter.prototype = {

        addListener : function(id, handler, scope, priority) {
            if (! this._signals[id]) {
                this._signals[id] = new signals.Signal();
            }
            return this._signals[id].add(handler, scope, priority);
        },

        removeListener : function(id, handler) {
            var sig = this._signals[id];
            if (! sig) return;
            sig.remove(handler);
        },

        getSignal : function(id) {
            return this._signals[id];
        },

        dispatch : function(id, args) {
            var sig = this._signals[id];
            if (! sig) return;
            if (args) {
                sig.dispatch.apply(sig, args);
            } else {
                sig.dispatch();
            }
        }

    };

    SignalEmitter.augment = function(target) {
        SignalEmitter.call(target);
        for (var key in _proto) {
            if (_proto.hasOwnProperty(key)) {
                target[key] = _proto[key];
            }
        }
    };

    return SignalEmitter;

});

You can use it as a standalone object:

var obj = new SignalEmitter();
obj.addListener('foo', console.log, console);
obj.dispatch('foo', ['bar']);

Or extend an existing object to add the SignalEmitter behavior to it:

var obj = {
    doSomething : function(args){
        this.dispatch('foo', ['lorem', 'ipsum']);
    }
};
//"inherit" from SignalEmitter object
SignalEmitter.augment(obj);

obj.addListener('foo', console.log, console);
obj.doSomething();

The method addListener returns a SignalBinding object and you can also get a reference to the underlaying Signal object by calling getSignal(id) so you will have access to all the advanced features from JS-Signals while keeping a very flexible structure.

Feel free to remove unneeded features like SignalEmitter.augment() or getSignal(), added more as a reference and because on my current project I needed both things.

Very simple code but may be useful to somebody else…

PS: please note that listening/dispatching non-existent events will fail silently (one of the main reasons why I started using Signals), use with care.

Another polemic subject and something that I’ve been delaying for a while, I’ll try to explain how I got to my conclusions and I’m sure a lot of people won’t agree until they experience the same problem or realize how often it may happen on a real project and specially if it does happen with a code you don’t “own” and/or can’t change without compromising future updates… The advice is mostly about JavaScript (since it’s what I’ve been coding more lately and where I think the issue is bigger) but it also fits other languages.

Not so long ago I had an opposite opinion about this subject - I would always set everything to private and only changed to protected/public if needed - try to understand what made me change my opinion and pay attention every time you find yourself “tied” or doing lots of hacks just because you can’t monkey-patch a simple function/variable.

Private variables and functions in JavaScript

Unlike Java/C#/AS3/PHP, JavaScript doesn’t have keywords for defining if a variable/function/object is private/public/protected/final/etc (acessor keywords)… But since it has closures and each closure has it’s own scope we can emulate some of these access controls, a private variable is simply a variable declared inside an “unreachable scope”. Example:

// global variable
var lorem = 'ipsum';

(function(){
    // the immediately invoked function expression (IIFE) will create a closure
    // all vars and functions inside this scope will be "private"
    var foo = 'bar';

    console.log(lorem); // 'ipsum'
    console.log(foo);   // 'bar'
}());

console.log(lorem); // 'ipsum'
console.log(foo);   // undefined

Because of this powerful and useful language feature a lot of people been using design patterns like the revealing module pattern to avoid polluting the global scope and to better organize their applications (including myself).

Private variables are usually a good thing since they avoid more problems than they cause, naming conflicts (two pieces of code using same name to store different info) can be very hard to track and it is very common to happen, that’s one of the main reasons why people use so many private variables, but like most things in life it also has it’s drawbacks and I’ll show a few cases where it can be an issue and how to avoid it.

Other languages

Java/C#/AS3/PHP have the concept of a protected class member, the protected member is like a private one but that can be overwritten by a child class. By using protected on all your members instead of using private you increase the flexibility of your code, if the user wants to change the default behavior of a method he can simply extend the class and overwrite it. The sad part is that private variables in JS can’t be overwritten, not even by child classes.

JavaScript is about monkey-patching

I saw this quote on Twitter a few weeks ago and I must agree:

Monkey patching in javascript is called “writing code.” – Paul Hummer

One of the greatest things about the language and what makes it so fun to code is that you can get very creative about how to solve certain problems, it gives you freedom enough to do to terrible and amazing things at the same time, you don’t feel as constrained as on a static strong-typed language…

Private variables are against monkey-patching and can make your code impossible to be reused/tweaked without a lot of manual work, one of the main reasons for that is because the variable is inside a scope that you can’t reach, not even if you extend the object, it’s like if all your variables and methods where private final (you can’t overwrite it by any means), and this process isn’t “scalable” at all.

Unit Testing

Another problem with private members is that it is harder to test them. I usually tend to think that private members doesn’t need to be tested since the public interface is already being tested, but sometimes that may not be the truth (think about a variable that holds state info or some property that needs to be swapped with a mock object). Exposing larger chunks of your codebase (as a pseudo-private member) may help you to increase your unit tests coverage and also to simplify the tests.

Solution

What I’ve been doing on some projects is to expose most of the methods and properties of objects that I know that are going to be reused in multiple projects (like open-source projects), if it isn’t meant to be called by other people or is subject to changes I simply start the name with an underscore (_), it’s a sign that you shouldn’t expect the function to be there forever (pseudo-private) but it allows the user to easily “hack” the original code to make it work as he want. I believe that code that is easy to edit is way more valuable than code that has lots of settings to try to fit all scenarios, there will be a always an edge case that isn’t covered or the application will be so complex/bloated that nobody will want to use it.

I’m not saying you should pollute the global scope with thousands of variables (ie. keep it as properties of an object, like if it was a namespace), but consider exposing things that you may want other people to be able to tweak/reuse, been doing it for a while and it saved me some headaches.

A real case where monkey-patching “saved the day”

I wanted to add JSON support to Syntastic Vim but JSONLint output contained multiple lines and making syntastic to understand the format was kinda tricky so I decided to do a pull request on JSONLint itself adding an option for toggling the error message format, if the method parseError wasn’t exposed on the lexer object the change could be way more complex. That’s the difference of taking 20min to release a new feature that affects 2 open-source projects (and that boosted my productivity a lot since now I can validate JSON files at each save) or taking 1 day or more or not even doing it…

A real example where private members made things more complex than needed

I wanted to generate HTML documentation for AMD-Utils based on a few markdown files that I had on the repository, just because it would be quicker to browse it, so I decided to code a simple generator that would read all the markdown files, process them and output to HTML. The only issue is that I got used to Github flavored markdown syntax and all my markdown files were written using this syntax, so I had to tweak the markdown parser a little bit to act like the way github renders the README files (it is not the same as the JavaScript preview) which means I had to do some pre-processing before showdown did “it’s magic” and also edit showdown source code to comment out a single line, yes, a single line!! Because Github doesn’t hard-wrap line breaks on README files while the JS preview does it.

If showdown exposed all the methods I could override the _FormParagraphs method or do some magic before and after the original _FormParagraphs() was called - basically convert all the line breaks into tokens before _FormParagraphs() and untokenize it afterwards - and I could keep the library as a normal npm dependency - so I could update it as long as they didn’t replaced the method functionality or renamed it - things could still go wrong but the “damage” would be controlled. It would also reduce the chance of updating the file without noticing that it had changes that diverged from the original lib (since the changes are on a separate file).

Another option would be to fork the project and add support for toggling the GFM settings, but that would take way more time than I wanted to spend and I would need to wait the change to be merged and pushed to npm (if the pull request was even accepted).

Conclusion

I’ve seen the same thing happening over and over again on many projects (in many different languages) and that’s why I’ve been favoring the “expose ALL the things” approach lately.

Notice that I still create many local variables and functions since it’s less verbose and not all code should/need to be heavily tweakable (I can edit the source code of my app as much as I want, no need to monkey-patch) but if you expect a lot of people to reuse your code please make it easier for them to patch it. You should warn people of the danger but assume they know what they are doing, we are all grown ups right?

Understand the reasoning behind a “best practice” so you can decide if it does apply to your case or not.

That’s it!

Further Reading

• Wikileaks To Leak 5000 Open Source Java Projects With All That Private/Final Bullshit Removed (external)
• There is no spoon

Why?

Most of my projects I just need some basic features like file concatenation, running an optimizer/compressor, some simple string replacements or running some other 3rd party command-line tool, sometimes I also need to do some things that aren’t that common (code generation, documentation, etc..) and also depending on the project my structure may not fit existing tools.

I’ve been using Ant for the past few years on almost all my projects, the good thing about Ant is that it’s very mature, can run on multiple platforms, well documented, pre-installed on Mac OS X, popular (specially for Java and AS3 developers), and almost anything you want to do you just need to search for the “term” plus “ant task” (eg. “concat files ant task”, “delete files ant task”) and the first results probably describe how to do what you need. The problem with Ant is that XML is a markup language and when you try to do things that requires a programming language (logic, loops, etc..) things get nasty really quickly so you need to extend Ant by using the script task or creating an external command line tool/app or an Ant plugin (Java), not really convenient or straightforward (too much overhead, need to learn new APIs and/or languages…)

Make and bash scripts are way more concise than Ant, but they also impose a barrier to people that doesn’t know how to edit them. Another problem with bash is that it isn’t very portable which for me is a big issue, I work with multiple devs and who knows which OS they might be using or how the build process will be triggered? It’s very important for users to be able build the project without too much trouble and also that multiple developers on the team understands how the build process work and how to edit the files, it will avoid “knowledge silos” and “code ownership”. If your team is proficient in JavaScript I suggest that the build script should be written in JavaScript, if the team is more comfortable with Python or Ruby (or anything else) than write it on these languages instead…

A few months ago I finally decided to try “vanilla” Node.js scripts and I’m very pleased with it, I see it as an excuse to learn Node.js API and it’s libraries. It’s also very fast to code since I’m very familiar with JavaScript syntax, I don’t need to spend time searching how to do things I simply code it, also since I’m not constrained by an existing tool structure I can go “nuts” and code whatever I want and the way I want it, the complexity of my scripts are only limited by my needs and deadlines.

I tend to think that tools that try to fit all scenarios end up not fitting anything properly, so I prefer code that is easy to edit/tweak than code that can be overly customized. Having the build script written on the same language as the project is a very good thing since contributors will feel more comfortable at updating it and improving it, low entry barrier is usually a good motivation for contributing.

How?

I’m not really using any specific build tool, just putting together a few npm modules as needed, kinda following the Node.js philosophy of keeping packages small and loosely coupled, some of the modules I’ve used so far and that helped a lot in the process:

• commander
  • very useful tool to generate self-documenting command-line interfaces,
    using it on most of my Node.js command-line tools.
• wrench-js
  • recursive file operations (delete, create, copy directories)
• node-glob
  • returns a list of filepaths based on a string pattern.
• handlebars
  • for templating in case you need to do some code generation and/or process
    some text files.
• UglifyJS
  • JavaScript minification.
• async
  • Asynchronous utilities and control flow.

I’m also calling external tools when needed using the child_process.exec command, so you can still call some JAR files or other command-line tools if needed…

I’m not using any build tool because I want flexibility and also because it’s very easy to put those things together, there are a plenty of Node.js modules available through npm that does all sorts of things and it’s a very good excuse to learn how to use them and to discover new libraries.

Examples

Most of the examples are very trivial, use it as a reference not as the only way of doing it.

Concat files

You just need to read the content of a file list and merge them:

// settings
var FILE_ENCODING = 'utf-8',
    EOL = '\n',
    DIST_FILE_PATH = 'dist/myAwesomeScript.js';

// setup
var _fs = require('fs');

function concat(fileList, distPath) {
    var out = fileList.map(function(filePath){
            return _fs.readFileSync(filePath, FILE_ENCODING);
        });
    _fs.writeFileSync(distPath, out.join(EOL), FILE_ENCODING);
    console.log(' '+ distPath +' built.');
}

concat([
    'foo/bar.js',
    'foo/lorem.js',
    'foo/maecennas.js'
], DIST_FILE_PATH);

It’s important to notice that I’m not concatenating files that much since I’m using r.js (RequireJS optimizer) on almost all my projects, r.js can be also used to merge CSS files (inline @imported stylesheets).

Minify JS file with UglifyJS

// settings
var FILE_ENCODING = 'utf-8';

function uglify(srcPath, distPath) {
    var
      uglyfyJS = require('uglify-js'),
      jsp = uglyfyJS.parser,
      pro = uglyfyJS.uglify,
      ast = jsp.parse( _fs.readFileSync(srcPath, FILE_ENCODING) );

    ast = pro.ast_mangle(ast);
    ast = pro.ast_squeeze(ast);

    _fs.writeFileSync(distPath, pro.gen_code(ast), FILE_ENCODING);
    console.log(' '+ distPath +' built.');
}

uglify('dist/awsum.js', 'dist/awsum.min.js');

Minify JS with google closure compiler (run external JAR)

You can run shell commands with child_process.exec. Very useful for reusing existing command-line tools and code that isn’t Node.js specific.

// --- SETTINGS ---
var COMPILER_JAR = 'build/compiler.jar';

// --- SETUP ---
var _exec = require('child_process').exec;

function compile(srcPath, distPath) {
    // exec is asynchronous
    _exec(
      'java -jar '+ COMPILER_JAR +' --js '+ srcPath +' --js_output_file '+ distPath,
      function (error, stdout, stderr) {
        if (error) {
          console.log(stderr);
        } else {
            console.log(' '+ distPath + ' built.');
        }
      }
    );
}

compile('dist/myAwesomeScript.js', 'dist/myAwesomeScript.min.js');

Use commander to parse CLI arguments

Commander.js is very useful for parsing command-line arguments, specially since it is self-documenting and extremely easy to use.

// --- SETTINGS ---
var DIST_PATH = 'dist/awsum.js';

// --- SETUP ---
var _cli = require('commander'),
    _fs = require('fs');

// global options
_cli
    .version('0.0.1')
    .option('--silent', 'suppress log messages.');

// commands
_cli
    .command('deploy')
    .description('Optimize site for deploy.')
    .action(deploy);

_cli
    .command('purge')
    .description('Delete old files from dist folder.')
    .action(purgeDeploy);

// parse commands
_cli.parse(process.argv);

function deploy() {
    purgeDeploy();
    build();
}

function purgeDeploy(){
    _fs.unlinkSync(DIST_PATH);
    if (! _cli.silent) {
        console.log(' Deleted deploy files!');
    }
}

function build(){
    // concat files here or do anything that generates the dist files
    if (! _cli.silent) {
        console.log(' Built!');
    }
}

node build -h will show the available commands and options. Running node build deploy will execute the “deploy” command, node build deploy --silent will execute the “deploy” command but suppress log messages.

Code generation / String replacements

Template engines aren’t useful only for HTML documents, they can be used for simple string replacements and also for code generation. I’ve been favoring Handlebars.js because of the “helpers” and nested paths support. I’ve been using it to add some basic info like version number, build date and even to generate some source files.

On AMD-Utils I’m using it to generate the package files and also to update the test runners (so I make sure I have unit tests to all modules).

If you are distributing the library through npm and/or have a packages.json file it is a good idea to reuse the data on the build as well so it doesn’t get out of sync. Let’s say we have a license at the top of the file like:

/**@license
 * My Awesome Lib <http://example.com>
 * Version: 0.1.0 (Tue, 03 Jan 2012 03:48:58 GMT)
 * License: MIT
 */

It could be described as:

/**@license
 * {{name}} <{{homepage}}>
 * Version: {{version}} ({{build_date}})
 * License: {{#license licenses}}
 */

And we could replace the data using handlebars:

var FILE_ENCODING = 'utf-8',
    DIST_PATH = 'dist/awsum.js';

var _handlebars = require('Handlebars'),
    _fs = require('fs');

// will generate a CSV if package.json contains multiple licenses
_handlebars.registerHelper('license', function(items){
    items = items.map(function(val){
        return val.type;
    });
    return items.join(', ');
});

var distContent = _fs.readFileSync(DIST_PATH, FILE_ENCODING);
var template = _handlebars.compile(distContent);

//reuse package.json data and add build date
var data = JSON.parse( _fs.readFileSync('package.json', FILE_ENCODING) );
data.build_date = (new Date()).toUTCString();

_fs.writeFileSync(DIST_PATH, template(data), FILE_ENCODING);

Lint JS files and prompt user to confirm before continuing added 2012/01/03

Joss Crowcroft asked for it in the comments and since it isn’t hard to implement (thanks to Commander.js .choose() method) I decided to do it.

var _cli = require('commander'),
    _jshint = require('jshint'),
    _fs = require('fs');

function lint(path, callback) {
    var buf = _fs.readFileSync(path, 'utf-8');
    // remove Byte Order Mark
    buf = buf.replace(/^\uFEFF/, '');

    _jshint.JSHINT(buf);

    var nErrors = _jshint.JSHINT.errors.length;

    if (nErrors) {
        console.log(' Found %j lint errors on %s, do you want to continue?', nErrors, path);
        _cli.choose(['no', 'yes'], function(i){
            if (i) {
                process.stdin.destroy();
                if(callback) callback();
            } else {
                process.exit(0);
            }
        });
    } else if (callback) {
        callback();
    }
}

// run
lint('dist/awsum.js', function(){
    console.log(' Built!');
});

More

There are a few Node.js general purpose build tools like node-jake and cake (coffeescript), and also some tools that assumes your project is following a specific structure and contains options for lint/concatenation/minification like smoosh, buildr.npm and grunt. I tend to prefer tools that doesn’t impose a specific structure since we never know if the constraints might become an issue or when the project requirements may change, but these tools may be a good option depending on the project.

If you need to do something that isn’t listed here search npm, there is a big chance that someone already coded a package that does what you need, if not, do it yourself and share with the open source community.

I’ve also created a gist with a RequireJS optimization example + copying/filtering files, not as easy to follow as the previous examples but might be helpful to someone trying to make something a little bit more complex.

That’s it for now.

Edit 2012/01/03: Added JSHint example with confirm prompt and link to caolan/async.
Edit 2012/05.08: Added link to gist containing a full build script.

The Workflow

1. Upload new files to a _swap directory.
2. Move current files to a _backup directory.
3. Move new files from _swap to the parent directory.
4. Test live files.
5. Delete files inside the _backup directory.
6. Win one free internet.

Why?

The main reason is because moving files on FTP is way faster than deleting them, specially if you are moving the files to a parent directory, it can reduce the downtime from a couple minutes to a few seconds depending on how many files you are updating.

Another reason is that you will have a backup of the current files on the server and you can easily revert the changes in case anything goes wrong. - You should also keep all source files on a version control system and make updates when the site has as few users as possible (usually from 9pm to 9am) to reduce the chance of ruining everything…

That’s it!

So far it’s a little rough around the edges, code can be cleaned a lot (I coded it really fast) and it isn’t as flexible/polished as I want it to be, but it’s working fine for my current needs.

The project is on Github and repository contains some example files with comments, you can check the amd-utils documentation for a live example - amd-utils documentation uses a custom template but it’s very close to mdoc default template, only extra feature is the “specs” and “source” links (since other projects probably have a different file/folder structure).

I’m not sure when I will have time to improve it but I already added a couple feature requests to the issue tracker. Watch it, fork it and contribute! If you want to report bugs and feature requests please use the Github issue tracker.

That’s it!

Doing it wrong

I already had a simple function to generate random floating point numbers inside a range, a simplified version of it looks like:

function rand(min, max) {
    return lerp(Math.random(), min, max);
}

function lerp(ratio, start, end) {
    return start + (end - start) * ratio;
}

The logic is pretty straightforward, Math.random() returns a random number from 0 to 1, the lerp() method calculates the “linear interpolation” between start and end, here is an example:

lerp(0.5, 0, 100);  // 50
lerp(0.2, 0, 10);   // 2
lerp(0.5, -10, 10); // 0

Since “lerp ratio” is also a value inside the range 0...1 it is just a matter of doing the linear interpolation to a random ratio, nothing wrong about this process… One may think that since the random float works properly it would be just fine to use Math.round() to get a random integer:

//biased! don't use this code!
function randInt(min, max) {
    return Math.round( rand(min, max) );
}

Consider this example:

randInt(-1, 1);

What is the probability of getting 0, 1 and -1 respectively? If you think it’s ~33% than you are wrong, 0 has 50% of probability since anything inside the range -0.5...0.5 is going to rounded to 0 - I already knew it before I coded the naive approach but at that time I didn’t needed any sort of precision - It is important to note that the problem only affects the min/max values no matter how large is the range.

Running this code 1000x:

// first run
-1 => 24.7%
 0 => 50.2%
 1 => 25.1%

// second run
-1 => 24.8%
 0 => 51.6%
 1 => 23.5%

Y U NO LIKE -1 and 1?

So how to do it “right”?

First thing I did was to think how I would approach the problem on the simplest way possible that wouldn’t require reading a technical paper about RNG which includes way more info than most people probably need to know…

Another option was reading Python source code (as suggested by Gabriel Laet on twitter) but my gut said the implementation was more complex than I needed so I decided to try to solve the problem myself without any external help (I like puzzles and this one didn’t looked that hard since we already have Math.random() in JavaScript).

For me the most important thing at solving this kind of problems is breaking it into smaller ones. The real issue with the previous approach is that the “range” that “covered” a single result (zero) was 50% of the full range, to solve the problem we need to break the range into 3 equal-sized steps (since the range has 3 integers) and pick a random “step”/”index” instead of simply rounding the linear interpolation.

“A problem well stated is a problem half solved” – Charles F. Kettering

Solution(s)

function randInt(min, max) {
    var stepSize = 1 / (max - min + 1),
        nSteps = Math.floor(Math.random() / stepSize);
    return min + nSteps;
}

The “step size” is calculated by dividing one by the amount of integers inside the range, so if range have 3 items it will be 1/3 === 0.3333333333333333 if range has 4 items it will be 1/4 === 0.25 and so on… (exactly what we need)

By knowing the size of the “step”, and since it is always inside the range 0 <= n <= 1, we can pick a random step by using Math.random().

If Math.random() is inside the range 0...0.3333333333333332 it should return 0 (zero steps), if it’s inside the range 0.3333333333333333...0.6666666666666665 it should return 1 (one step) and from 0.6666666666666666 to 1 it should return 2 (two steps). That is a very common math operation so I abstract it as a separate method to make clearer what the code is doing, I call this method countSteps() since most times I needed this function was to “count the number of full steps” and I haven’t seen it on any other library to follow the same convention (naming things is hard):

function countSteps(val, stepSize) {
    return Math.floor(val / stepSize);
}

So the code could be rewritten for better readability:

function randInt(min, max) {
    var stepSize = 1 / (max - min + 1);
    return min + countSteps(Math.random(), stepSize);
}

Running this code 1000x:

// first run
-1 => 31.1%
 0 => 34.7%
 1 => 34.1%

// second run
-1 => 36.1%
 0 => 33.4%
 1 => 30.4%

Not biased at all!

Solution #2 (added 2011/11/16)

After the post I got a pull request with another implementation that after a few tweaks got into this code:

function randInt(min, max) {
  var diff = max - min,
      rnd = Math.random();
  return rnd !== 1? Math.floor((diff + 1) * rnd) + min : min + diff;
}

The other solution came so fast that I didn’t even explored other options, I already had the method countSteps().. and you know,

“if you only have a hammer you tend to see every problem as a nail” – Abraham Maslow

This method does work properly because (1 -(-1) + 1) === 3 and Math.floor(Math.random() * 3) will return 0, 1 or 2 unless Math.random() === 1 which would return 3 making the method return a value greater than max.

Solution #3 (added 2011/11/16)

While I was updating the post with more info to explain the problem only happens with the max/min values I realized it could be fixed with a simpler solution while still using the regular random method but Fabio Caccamo was quicker to post it

function randInt(min, max) {
   // can't be `max + 0.5` otherwise it will round up if `rand`
   // returns `max` causing it to overflow range.
   return Math.round( rand(min - 0.5, max + 0.499999999999) );
}

It does work because the rounding issue only affect the min and max values and we are increasing the range so all the numbers have an equal chance… now I feel dumb for over-engineering a “simple” problem

Sum up

I still find the first solution to be more readable, probably because I’m very used with the countSteps method and it uses more vars/methods that ends up describing what the method does… (or maybe just because it is closer to the way my head works) But there are indeed many ways to skin a cat…

I hope it got clearer how to think about this kind of problem and possible solutions (and how to spot the problems as well). My mind is very visual so I usually draw this kind of stuff to help thinking about solutions - it took me way less time to find the solution after I drawn it on paper than I would spend searching the web or trying to decipher somebody else code - try to find a way that works for you, explaining the problem to another person also helps a lot since it makes you think about what exactly you are trying to solve.

It’s way easier to code when you know what is the real problem you are trying to solve.

This problem was simple and I wrote it quickly (solution and post) but I hope you get the idea… Maybe it’s useful on a future project. I’m still trying to find a good name for a “curried” version of randInt(-1, 1) (-1, 0, 1) I already have two methods called randBit() (0 or 1) and randSign() (-1 or 1) since these ranges are so common I think they deserves their own method, the implementation is also simpler so it has better performance – not that it should make a huge impact is most cases, for me it’s more a readability thing and not needing to pass parameters…

For more helper functions check the amd-utils on github, I’ve been improving it based on my needs and adding new stuff that may be useful when I have the opportunity or when I realize some common pattern, extract it and find a name to describe it…

PS: I hate when people use complex terms to describe simple things, that’s why I tried to explain it on the simplest way possible, by using Math terms the Mathematicians hide the knowledge from people of other domains, nowadays I can understand some of the documents but a few years ago it was like trying to read Greek…

That’s it!

Update (2011/11/2011): Added info about rounding issue only affecting min/max and added solutions #2 and #3.

SignalBinding.params (a.k.a non-semantical versioning at v0.6.3)

v0.6.3 was supposed to be v0.7.0 and v0.7.0 was supposed to be v0.8.0, my bad… I changed the API a little bit from v0.6.2 to v0.6.3 and still released it as a “patch version” (see semantic versioning for more info). So I’m going to also talk about one simple feature I added on v0.6.3 (it was an one line change and I was already using it, that’s why I released as 0.6.3).

When adding a new listener to a signal you have the option to set an Array that will be passed to the listener during execution. e.g.:

var loaded = new signals.Signal();

var loadedBinding = loaded.add(console.log, console);
//set the default parameters of binding
loadedBinding.params = ['lorem', 'ipsum'];

//will log "lorem ipsum dolor"
loaded.dispatch('dolor');

This can be very useful when you are adding listeners dynamically and want to pass some info to the listener, since js-signals already uses apply to set the execution context (this object inside handler) the implementation was very simple (updated 1 line of code) and it increased the flexibility a lot. Of course this feature shouldn’t be abused since you need to know the structure of the event handler and which parameters it does expect, but it is a nice and concise way to pass parameters and it can avoid creating a new function just for that need (on a previous project I had to wrap some functions just for that use case).

Signal.memorize / Signal.forget()

This is a feature that I’ve been pondering for a long time, it’s a way to listen to events that “happened in the past”, yep it sounds strange but that is what it does, it allows you to add a listener after the event was already dispatched and it will trigger the event handler automatically passing the previously dispatched arguments.

var started = new signals.Signal();

started.memorize = true; // default is false
started.dispatch('foo');

// add()/addOnce() will automatically fire listener if signal was dispatched before
// will log "foo" since it keep record of previously dispatched values
started.addOnce(console.log, console);

started.forget(); // forget previously dispatched values (reset signal state)
started.addOnce(console.log, console); // won't log till next dispatch (since it "forgot")
started.dispatch('bar'); // log "bar"

It can be useful for events like initialized, started, loaded, etc… It works similarly to a “promise” that was already “resolved”.

Initially I thought about implementing it as a new type of Signal, maybe called AsyncSignal or something like that, but at the end I decided to simply add the new property and method since implementation was way simpler/smaller and the usage as well, this is JavaScript not Java!!

Signal.has(listener)

Never really needed this feature on any project but since it was already being used internally and was just one line I think it is worth making it public. Maybe it can be useful to someone.

function onStart(a){
  console.log(a);
}
started.add(onStart);
started.has(onStart); // true

Universal Module Wrapper (UMD)

Until v0.6.3 js-signals was being distributed with separate files to each environment (nodejs, browser, AMD), since v0.7.0 you can use the same distribution file on all the environments and with almost no overhead, the boilerplate code is minimal and won’t affect performance.

CompoundSignal / listening to multiple Signals at once

I also spent a long time thinking about this feature, specially trying to decide if it should be merged into signals “core” or if it should be a separate project, at the end I decided to split it into it’s own repository since it isn’t that common to need this feature and it would make it easier to install it using npm. I coded an ad-hoc solution to do the same thing before on a project but once I realized I needed the exact same thing on another project I thought it was better to abstract it and create a new type of Signal that could handle it transparently and that would be more flexible as well.

The CompoundSignal works like a group of signals which will be dispatched automatically after all the signals contained by the group are dispatched. Arguments are passed to listeners as Arrays on the same order as the signals were passed to the constructor.

var endedAnimation = new signals.Signal();
var completedSomething = new signals.Signal();

// CompoundSignal is just a regular Signal that gets dispatched after the other
// signals are dispatched (and have a similar API)
var completedManyThings = new signals.CompoundSignal(endedAnimation, completedSomething);
completedManyThings.addOnce( doSomethingOnCompletedManyThings );

function doSomethingOnCompletedManyThings(paramsArr1, paramsArr2){
  //handler will receive parameters of each signal as arrays since
  //they can dispatch an arbitrary number of parameters
  console.log( paramsArr1, paramsArr2 );
}

completedSomething.dispatch('lorem', 'ipsum');

setTimeout(function(){
    endedAnimation.dispatch('ended animation');
}, 500);

//will log after 500ms: ['ended animation'], ['lorem', 'ipsum']
//note that params are on the same order as the signals were passed to
//CompoundSignal constructor not at the order they were dispatched

CompoundSignal is very powerful and flexible, although the implementation is actually very simple, check the project repository for more info and check the unit tests and source code for more details, it has some advanced features as well like overwriting previously dispatched values, dispatching same signal multiple times, resetting state, etc…

Nice thing to note is that CompoundSignal uses SignalBinding.params to simplify the logic. That’s exactly the kind of stuff that motivated me to add the SignalBinding.params, so simple and useful…

More

I hope the new additions and explanations help you to code better asynchronous applications and to decouple your objects. Signals improved the quality of my code a lot and also reduced the amount of typo errors and scope issues (the second parameter of add and addOnce can be used to set the value of the this object inside the listener), I hope it does the same for you.

Check more examples on the wiki and watch js-signals and CompoundSignal on github. For feature requests, suggestions and bug reports please use the github issue tracker. There is also an online documentation in case you need it…

Asynchronous programming is hard, use tools that make the process easier and that also provides enough flexibility.

That’s it for now.

I’ve been considering to use a CSS pre-processor like SASS, LESS, Stylus, etc, for a very long time. Every time someone asked me if I was using any of these tools/languages I would say that I’m kinda used to my current workflow and I don’t really see a reason for changing it since the problems those languages solves are not really the problems I’m having with CSS. Then yesterday I read two blog posts which made me reconsider my point of view so I decided to spend some time today studying the alternatives (once again) and porting some code to check the output and if the languages would really help to keep my code more organized/maintainable and/or if it would make the development process easier (also if they evolved on the past few years).

It takes a couple hours for an experienced developer to learn most of the features present on these languages (after you learn the first couple languages the next ones are way easier) but if you have no programming skills besides CSS/HTML and/or don’t know basic programming logic (loops, functions, scope) it will probably take a while, the command line is another barrier to CSS/HTML devs… But that isn’t the focus of this post, I’m going to talk specifically about overused/misused features. I will try to explain the most common problems I see every time someone shows a code sample or I see a project written using any of these languages/pre-processors.

Mixins

What are mixins?

Mixin is a common name used to describe that an object should copy all the properties from another object. To sum up a mixin is nothing more than an advanced copy and paste. “All” the famous pre-processors have some kind of mixin.

Dumb code duplication is dumb

Following the SCSS syntax (sass), a mixin can be described and used as:

@mixin error {
    color: #f00;
    border: 2px solid #fc0;
}

.error-default {
    @include error;
}

.error-special {
    @include error;
    background-color: #fcc;
}

Which will compile to:

.error-default {
    color: #f00;
    border: 2px solid #fc0;
}

.error-special {
    color: #f00;
    border: 2px solid #fc0;
    background-color: #fcc;
}

Note that the properties are duplicated, which is very bad, file size will increase a lot and overall performance will also be degraded if not used with care. – Imagine that on a large project with thousands of lines of code, the amount of duplicated code will be unacceptable (by my standards).

This problem isn’t specific to SASS, it is also present on LESS and Stylus and any other language/pre-processor which supports the same feature, by having a new layer of abstraction the developer won’t realize he is creating code that has lots of duplication…

Extend

LESS and Stylus doesn’t have support for anything similar to an extend, that’s why I picked SCSS (Sass) to write the code samples. A extend is like a “smarter mixin”, instead of copying and pasting the properties it will set the properties to multiple selectors at once.

.error {
    color: #f00;
    border: 2px solid #fc0;
}

.error-default {
    @extend error;
}

.error-special {
    @extend error;
    background-color: #fcc;
}

Which will compile to:

.error, .error-default, .error-special {
    color: #f00;
    border: 2px solid #fc0;
}

.error-special {
    background-color: #fcc;
}

Way closer to what a normal person would do manually… “Only” use mixins if you need to pass custom parameters. If you see yourself using the same mixin multiple times passing the same values than you should create a base “type” that is inherited by other selectors. - Compass (nice SASS framework) have a lot of mixins which I think should be base classes instead.

Extend isn’t enough

Note that extend avoids code duplication but it also causes other problems, the amount of selectors can become an issue, if you @extend the same base class multiple times you may end up with a rule that have thousands of selectors, which won’t be good for performance either and can even make the browser to crash.

Another issue is that every class you create to be used only by @extend is going to be included on the compiled file (even if not used) which can be an issue in some cases (name collisions, file size) and makes this process not viable for creating a framework like compass.

I really wish that SASS improves the way that @extend works (and that the other pre-processors also implements a similar feature) so we could create many base classes for code reuse but don’t necessarily export them. Something like:

@abstract error {
    color: #f00;
    border: 2px solid #fc0;
}

.error-default {
    @extend error;
}

.error-special {
    @extend error;
    background-color: #fcc;
}

Which would compile to:

.error-default, .error-special {
    color: #f00;
    border: 2px solid #fc0;
}

.error-special {
    background-color: #fcc;
}

PS: I know this kind of feature was already proposed before.

Extend and mixins can be bad for maintenance

Contrary to the common knowledge, extending other classes and creating mixins can degrade maintenance. Since the place where you are using the properties is far away from where the properties are being defined there is a bigger chance that you will change properties without noticing you are affecting multiple objects at once, or not realizing which elements are being affected by the changes. This is called “tight coupling”:

Tightly coupled systems tend to exhibit the following developmental characteristics, which are often seen as disadvantages:

• A change in one module usually forces a ripple effect of changes in other modules.
• Assembly of modules might require more effort and/or time due to the increased inter-module dependency.
• A particular module might be harder to reuse and/or test because dependent modules must be included.

(source: Wikipedia)

I prefer to group all my selectors by proximity, that way I make sure that when someone update a selector/property they know exactly what is going to be affected by these changes, even if that imply some code duplication.

Nesting

Another feature that a lot of people consider useful is selector nesting, so instead of repeating the selectors many times you simply nest the rules that should be applied to child elements.

#content {

    table.hl {
        margin: 2em 0;

        td.ln {
            text-align: right;
        }

    }

}

Compiles to:

#content table.hl {
    margin: 2em 0;
}

#content table.hl td.ln {
    text-align: right;
}

By abstracting the selectors it becomes very easy to be over specific and specificity is hard to handle and a bad thing for maintainability. I’ve been following the OOCSS approach and I don’t need child selectors that much so I don’t think that typing the same selector multiple times is a real problem (specially with good code completion), I know a lot of people don’t agree with that approach but for the kind of stuff I’m doing it’s been working pretty well.

Call me a weirdo but I also find nested code harder to read - since I’ve been coding non-nested CSS for more than 7 years - and also because I like to keep all my rules on the same line (easier to find proper selectors and editing properties isn’t that common)

Sum up

These tools have some cool features like the helper functions for color manipulation, variables, math helpers, logical operators, etc, but I honestly don’t think it would improve my workflow that much.

My feeling for these pre-processors is the same feeling I have for CoffeeScript, nice syntax and features but too much overhead for no “real” gain. Syntax isn’t the real problem in JavaScript for me the same way that it isn’t the real problem in CSS (and most of the languages). You still need to understand how the box-model works, specificity, cascading, selectors, floats, browser quirks, etc… you are just adding another layer of abstraction between you and the interpreted stylesheet, adding yet another barrier for future developers and increasing the chance of over-engineering. Markup may become simpler (with less classes/ids) but it comes with many drawbacks.

For me the greatest problem are developers that code CSS without the knowledge required to build a maintainable and scalable structure. A stylesheet full of mixins, if/else, loops, variables, functions, etc, will be as hard to maintain as a bloated hand-crafted stylesheet, if not harder. Developers have an inherited desire to be “clever” and that is usually a red flag.

“Everyone knows that debugging is twice as hard as writing a program in the first place. So if you’re as clever as you can be when you write it, how will you ever debug it?” – Brian Kernighan

Mixins are popular nowadays because of browser vendor prefixes, the real problem isn’t that CSS doesn’t support mixins or variables natively but that we have to write an absurd amount of vendor prefixes for no real reason since most of the implementations are similar and most of the features are only “cosmetic”. The real issue isn’t the language syntax, but the way that browsers are adding new features and people using them before they are implemented broadly (without prefixes). – This could be handled by a pre-processor that only adds the vendor prefixes (without the need of mixins or a special language) like cssprefixer. Try to find what is the real problem you are trying to solve and think about different solutions.

“It’s time to abolish all vendor prefixes. They’ve become solutions for which there is no problem, and they are actively harming web standards.” – Peter-Paul Koch

I’ve been following the OOCSS approach on most of my latest projects, and probably will keep doing it until I find a better approach. For the kind of stuff I’m coding it is more important to be able to code things fast and make updates during the development phase than to maintain/evolve the project over many months/years. I find it very unlikely to make drastic design changes without updating the markup, on the last 100 projects I coded it probably only happened 2 or 3 times. - css zen garden is a cool concept but not really that practical - Features like desaturate(@red, 10%) are cool but usually designers already provides me a color palette to be used on the whole site and I don’t duplicate the same value that much, if I do duplicate it everywhere than I can simply do a “find and replace” inside all the CSS files and call it a day, by using a function that generates a color (which you have no idea which value it will be) you can’t simply do a find and replace since you don’t know what is the value you are looking for on the source code - I prefer to simply use a color picker…

I know my experience is very different from most people so that’s why my approach is also different, your mileage may vary… If I ever need to use any of these tools it won’t be an issue (I have no strong barrier against them), I just don’t think they will save me that much time right now that would outweigh the drawbacks. Pick the tools based on the project and your workflow, it isn’t because I listed a couple issues that you should discard using a pre-processor, for many cases it would be an awesome way of generating stylesheets, just think about the drawbacks and be responsible.

“With great power comes great responsibility.” – Uncle Ben to Peter Parker

PS: I love CSS, for me it’s one of the most rewarding tasks on a website development, it’s like solving a hard puzzle…

Further Reading

• Why “variables” in CSS are harmful by Bert Bros (external)
• The Law of Leaky Abstractions by Joel Spolsky (external)
• LESS Sass. More OOCSS by Blake Haswell (external)
• There is no spoon
• How I structure CSS files
• Pragmatic CSS: using JS and inline styles

Edit 2011/11/09: added link to “The law of leaky abstractions”, small tweaks to the copy and formatting.
Edit 2012/02/04: add link to “LESS Sass. More OOCSS.” to the further reading.