Year: 2018

Next week I’m going to participate in a session about package managers for different languages – we’ll present pip (python), composer (PHP), and npm (JavaScript). In sharing ideas with my colleagues I realized how many of the struggles of modern JavaScript tooling and workflow is based upon the fact that the language didn’t have a built-in module system until recently, so I’ve decided to write down my thoughts as preparation for the talk.

A brief history of modularity

For a more in-depth explanation, please, check this evolution of modularity in JavaScript – it has many details hard to find in other places.

JavaScript was created in 1995 without modules, and every <script>  shared the same global scope. We learned how to work with functions and namespacing to gain a minimum ability to divide our code into pieces, but as the importance of JavaScript grew, the pressure to have better alternatives was more intense as well.

In the server, Node.js became the de facto standard and its module choice, CommonJS, the format used in the incipient node package manager,npm . That was 2010, 15 years after the JavaScript release. Browser-side nobody could afford or was interested in a unified module system, so alternatives bloomed and the market became more fragmented. With time, the use of npm skyrocketed and so the importance of CommonJS, even for browsers.

How to make CommonJS available in the browser

It may be worth to pause and remember that at that point we still didn’t have a module system for browsers. We had something that was simple enough to work with and hack. The core of the CommonJS API is quite simple, and has two pieces:

• require: a function to import some code that’s somewhere else.
• module.exports: a variable to hold the code to be exported.

Let’s say that I have an input.js file in CommonJS format:

var constants = require( './constants' );
console.log( constants.HELLO + ' ' + constants.WORLD );

And the corresponding constants.js contains:

module.exports = {
    HELLO: 'HELLO',
    WORLD: 'WORLD',
};

I can’t add those files to the browser through the <script>  tag and expect them to work. That’s invalid JavaScript as browsers understand it.

How do we make it valid JavaScript? Well, something we can do is to copy the modules to the same file, wrap them in a function (so their internal variables don’t collide) and expose the necessary keywords through the function arguments:

// Scope the first module
function( require, module ) {
    var constants = require( './constants' );
    console.log( constants.HELLO + ' ' + constants.WORLD );
}

// Scope the second module
function( require, module ) {
    module.exports = {
        HELLO: 'HELLO',
        WORLD: 'WORLD',
    };
}

This can be included in the browsers! It won’t fail, but it will also do nothing.

OK, next step, let’s implement require : it is a function that takes a module identifier and returns its module.exports object. We can do that:

// Implement require
var modules = {};
var require = function( moduleId ){
    var tmpModule = {};
    modules[ moduleId ]( require, tmpModule );
    return tmpModule.exports;
}

// Scope and register the first module
var input = function( require, module ) {
    var constants = require( './constants' );
    console.log( constants.HELLO + ' ' + constants.WORLD );
}
modules[ './input.js' ] = input;

// Scope and register the second module
var constants = function( require, module ) {
    module.exports = {
        HELLO: 'HELLO',
        WORLD: 'WORLD',
    };
}
modules[ './constants' ] = constants;

It looks a bit better, but still does nothing and we ended up adding a lot of variables to the global scope.

Let’s fix this by scoping the code within an IIFE (so it doesn’t pollute the global scope) and execute the main module, the entry point of our program (./input.js in our example):

(function() {
  // Implement require
  var modules = {};
  var require = function( moduleId ) {
    var tmpModule = {};
    modules[ moduleId ]( require, tmpModule );
    return tmpModule.exports;
  };

  // Scope and register the first module
  var input = function( require, module ) {
    var constants = require( './constants' );
    console.log( constants.HELLO + ' ' + constants.WORLD );
  };
  modules[ './input' ] = input;

  // Scope and register the second module
  var constants = function( require, module ) {
    module.exports = {
      HELLO: 'HELLO',
      WORLD: 'WORLD',
    };
  };
  modules[ './constants' ] = constants;

  // Execute the main module
  var module = {};
  modules[ './input' ]( require, module );
})();

This is it! We’ve transformed our initial CommonJS modules into something that is executable in today browsers.

This exercise would need quite a bit of work to be production-ready, but the fundamental steps are there and it’s not difficult to see that’s easily automated by tools. This is mostly what Webpack does when it transpiles CommonJS to IIFE. Rollup seems to be a bit more elegant but its strategy is similar. If you’re curious, check the runnable code they generate.

The transition to ESModules

The success of npm and CommonJS  taught the browser ecosystem a valuable lesson: a better workflow was possible. Some attempts were made to replicate the registry plus format formula but, eventually, npm / CommonJS won .

Flash forward to 2015 and the ES6 standard introduces modules in the JavaScript language. Three years after that, browser adoption and node support are still not universal or complete, but everybody agrees that it will. The whole ecosystem seems to be on board and execution environments, tools, libraries, and authors are preparing for what that means.

If we believe that, npm will continue to be the central registry to distribute JavaScript for the foreseeable future, but CommonJS will no longer be the default module format. In this moment of transition, not everything is clear or necessarily better. The ashes of the module wars are still warm, but they won’t be forever.