Introducing BuilDj: Software Project Definition Format that GNOME deserves

In the GNOME beer event, I had a nice chat with ebassi about the problems around our build configuration system and how things like CMake, SCons and Waf do not get the full picture and do not cover some of the really strange corncercases that autotools supports and therefore, coming up with a solution would be a 1 year work with a team of really experienced engineers.

As stubborn as I am, I decided to prove him wrong so I give you BuilDj:

The Problem

My main problem with our current Autotools situation are these:

• It is not portable as it encourages strong use of Bash and command line tools which ends up making it really hard to use on a non POSIX system, you know, like that used by 92% of the desktop users, even if the source code itself is perfectly portable.
• It gets on your way, a lot. Sometimes I refrained from writing some testcases just to avoid to add the Autotools boilerplate.
• Really bad documented, still.
• You need to learn M4/Autoconf, Automake, Make and Bash to use it properly, as if learning C, C++ or Vala and worrying about the problem the programmer is trying to solve is not hard enough.
• None extends it because it's hard to extend.
• Its hard to read and understand even if you eventually wrote it yourself, makes code refactoring a big pain.
• You simply can't parse autotools and have an IDE or a continous integration tool to understand what's there.
• The most important one, it scares peoples away, damaging our mindshare, making potential contributors go away. It drains the fun out of our platform.

Requirements

We need a human+machine friendly project description format, that it's pleasant to the eye when you read it, that it's intuitive enough to let you understand what's going on even if you never saw it before.
A format that gets out of your way!

It should support all of the common tasks a GNOME maintainer does (in-line .pc and .desktop file definition, mkenums, gobject introspection support, cross-compilation support, pkg-config oriented, xdg mimetype registration/definition, integration with intltool), but it should suppor them in a meaningful and unobtrusive way.

It should not be a programming language, but support embedable programming extensions with a well documented API, so that IDEs can integrate it.

My Proposal

 A JSON description format called BuilDj that is build-system agnostic (although its reference implementation is done with Waf)

{
"project":
{
"name": "BuilDj Test",
"version": "0.0.1",
"url": "http://www.codethink.co.uk"
},
"requires":
{
"gtk+-2.0":
{
"type": "package",
"version": "2.14",
"mandatory": "True"
}
},
"targets":
{
"my_shared_lib":
{
"type": "sharedlib",
"tool": "cc",
"input": ["lib.c"],
"version": "1.2.3"
},
"my_gtk_program":
{
"type": "program",
"tool": "cc",
"input": ["gtk_program.c"],
"uses": ["my_shared_lib"],
"packages": ["gtk+-2.0"]
},
"my_vala_program":
{
"type": "program",
"tool": "vala",
"input": ["vala_program.vala"],
"packages": ["gtk+-2.0"]
}
}

I don't even need to explain what that means right? By the way, this stuff already works, check the git repository.

Currently it is implemented as a waf wscript that parses the project.js json file, I'm not really interested in entering the build system wars but focuing on having a reference implementation of the format.
I choosed waf because it was the only one that offered most of the features I want as an approachable API and it only adds python as a dependency.

Implementations of the format in other systems are more than welcome, but current development will stick to waf in the foreseeable future.

I'm planning to propose and mentor this work as a Summer of Code project so that we can implement the missing basic features and support for a few GNOME apps.

There's a lot of work to do, support for C++, library and function checking, system type sizes, full cross compilation support. We already have some mockups and plans for those, and the waf maintainer has shown himself quite happy to accept patches upstream for the general purpose tools and that make things easier.

Despite the missing features, it surprisingly itches some of my own scratches already.

P.S. Anjuta and MonoDevelop guys, if you're listening, I'd love to get your feedback!