Node.js, Ant, Grunt and other build tools
I started using node.js to write build scripts since last year and even wrote a post about it before. The main reason why I decided to write my build scripts in plain JS is because I want them to be flexible and easy to edit. (Use the language you and your team are familiar with).
On this post I will try to cover some issues and the main reason why I’m not using a tool like grunt, buildr.npm, smoosh and gear on my projects. I’ll focus on Grunt since it is the most popular node.js build tool out there but the issues are present on other build tools as well (written in JS or not).
A couple days ago I sent a few tweets about it:
somehow I feel that
@gruntjs is making the same mistakes as Ant. I would rather call JS methods instead of setting config options…— Miller Medeiros (@millermedeiros) August 1, 2012
it would also make more sense to just run `node grunt mytask` instead of global install, would work better on windows. /cc
@gruntjs— Miller Medeiros (@millermedeiros) August 1, 2012
now I had a “crazy” idea for a project. A node.js adapter for Ant so you can call any task as a JS method passing JSON configs.
#notdoingit— Miller Medeiros (@millermedeiros) August 1, 2012
TL;DR; Build scripts should be real scripts and not configuration files.
Which “mistakes”?
Configuration over scripting
Descriptive languages aren’t a good way to solve arbitrary needs, configuration options never fit all scenarios. It doesn’t matter if you use XML/JSON/JS, it still won’t cover them all. If every edge-case requires a new setting you will soon end up with a highly complex system that is hard to understand and maintain.
Ant and Maven failed in recognizing this, they thought people would just write new JARs that executed custom tasks and that every kind of task could be easily represented with strings, but it isn’t a trivial process, specially when the tool is so focused around configuration. Ant-Contrib tried to solve some of the problems imposed by XML by adding loops and conditionals but still feels weirder than just writing a simple JavaScript script. Functions, Array#map, Array#filter and if/else to the rescue.
It’s JavaScript running on a very fast engine! Functions calls are inexpensive and the language is very flexible. Why use a single config file when you can call multiple methods? The grunt directives looks total non-sense to me, why not a method call instead of a magic string? I understand why bundling a template language into all strings (since JS doesn’t do automatic var interpolation) but I would still favor a function call that accepts the replacements/config that way the tasks don’t need to share state and aren’t that “magical”.
Also when you have a huge amount of configuration options you need good documentation and a lot of tests to cover all cases. – Complexity lies on the exceptions – Sometimes I spend more time reading documentation than it would take me to code the same feature from scratch for my own use case. On many times code that is easy to tweak is better than code that accepts multiple options to fit all scenarios (specially if you know how to code).
A good thing on Grunt is that you can register custom tasks from inside the “gruntfile” that way you can avoid the configuration step altogether, even tho most tasks seems to favor the configuration approach and are built as plugins.
Plugins / proprietary tasks
That for me is a huge mistake. I said it before and I will say it again STOP WRITING PLUGINS!. Even if all the tasks are indeed related to a common goal (helping the dev/deploy/test automation) you are still tying your solution to a single tool/platform. I’m not saying you shouldn’t reuse other modules, NPM is great at handling conflicting dependencies and complex dependency graphs, so abuse it. There is no reason why a helper function (concat, lint, minify, etc) wouldn’t work on multiple build tools, so why lock it to a single task runner? Why bundle it with the task runner / arguments parser? Why wrap every single node.js tool into a plugin for your build system? Why put all the “helpers” inside the same namespace when you have a real module system?
You are shooting yourself in the foot, if the project stop being maintained in the future you will have a tight coupled plugin that probably won’t fit the new build environment without a lot of refactoring. Requirements change all the time and tools evolve, build small blocks that can work together instead of a single monolithic system. Write methods that receive an input and produces an output. Don’t share state across the methods, don’t share config between methods. You should treat your build scripts as you treat your own application code, build it to scale and in a way that is easy to test.
Some tasks are very useful on other contexts besides build scripts, there is no reason why they should be tied to a build tool.
I added some feedback on this grunt issue explaining how to avoid tight coupling and why it’s a bad thing.
Global runner
Installing the build tool globally is a mistake. What if changes on the tool aren’t backwards compatible and you don’t remember which version you were using before? What if different projects requires different versions of the build tool? What if a future dev doesn’t have admin rights so he can’t install it globally? What if a future dev is on Windows and don’t know that he needs to set the environment PATH variable to be able to locate global commands installed with NPM?
Running node build mytask should work everywhere without admin rights, there is no advantage in running it as build mytask (“node” is a 4 letter word, easy to type and remember).
Since the “gruntfile” is already a JS script there is no point in wrapping the whole script inside a module.exports = function(grunt){ ... }; and executing it from an external script. You could simply require grunt as a common node.js library and call methods directly on it, the script would be more portable (could be called from other tools).
It was also a mistake to name the “gruntfile” with the same name as the grunt command since on windows the system will try to open the js file instead of running the command (unless you configure it to not do it).
Reinventing the “same” wheel
OK, I really like to reinvent the wheel, but in some cases that isn’t the smartest decision one can make. Specially when your wheel doesn’t work as good as an existing one and it consumes more time than you are willing to spend.
Yesterday I did a research and almost all the tools listed on NPM with the “build” tag have different implementations of the same tasks.
It will take an absurd amount of time before all the relevant tasks present on battle tested tools like Ant, Maven, Gradle are re-implemented on node.js specially if everyone starts coding the exact same tasks for every single node-driven build tool and just committing the same mistakes instead of experimenting and arguing about what could be improved.
Does it really make sense to every single build tool to re-implement arguments parsing since libs like nopt, commander.js and node-optimist already exists? Does it really make sense to have a rigid structure for you build tasks since you can just reuse one of these option parsers, write a few functions and glue together some other independent modules? If a task “depends” on another one you can simply call the other method before executing it or use something like Async.js to run the tasks in parallel or in sequence. (each “task” can be a separate function that receives a callback)
Learn from other peoples mistakes, don’t commit them all over again using a different technology.
Addendum
My intention with this post was to make people question their ideas/workflows so they can improve the tools and/or come up with “better” solutions. Depending on your needs the tools discussed above might be more than enough.
Try to keep your build files as clean as possible so other devs can understand how it works and can make changes to the file, you certainly don’t want to create a knowledge silo around the build script, problems can rise on different environments and requirements usually change.
I got really curious if it was possible to distribute Ant as a standalone package and execute it without needing to install so I ended up coding an adapter for node.js (even tho I said I wouldn’t do it). That way I can avoid rewriting some of the tasks all over again and can use it as an “utility belt” for now. More info about it on the project repository. The idea was inspired by Gradle since it leverages Ant tasks internally.
Automate all the things!
