Cross platform JavaScript with Browserify – Sharing Code Between Node.js and the Browser

Full stack JavaScript! Easier said than done. Especially if you want to use some of your JavaScript code in your Node.js backend as well as in the browser. There are quite a few ways how to achieve that, none of which is entirely straightforward. This post will explain how to use browserify together with Grunt to set up a JavaScript project that can be used in the browser and in Node.js.

The biggest concern here is the module system. In Node.js you typically write CommonJS modules . On the client the options are more varied. A lot of projects use no module system at all. Besides that, the most popular module system is probably AMD with implementations like RequireJS . Since CommonJS modules are loaded synchronously and AMD is all about asynchronous module loading, there are some fundamental differences.

If you want to publish a public library that can be used in Node.js and in the browser, things become even more complicated. People include third party libraries in a number of ways, for example with an AMD loader or by just placing a script tag in their html and expecting the library to export a global variable. You might want to enable your users to go with whatever style they prefer, but this means extra work for you as the library author. This issue is usually solved by using a UMD (universal module definition), but in the context of Node.js/Browser compatible code the UMD might conflict with the CommonJS module system.

Besides the module system, both platforms have different capabilities, like the DOM API that is present in the browser but not in Node and the Node core modules with things like filesystem access, operating system access and the streams API, all of which are not available in the browser.

So the questions that need to be answered are these:

• How to write modular code?
• How to make the code compatible with different module loaders (CommonJS, AMD, …)?
• How to make the code compatible with the different capabilities of both platforms (Node.js and browser)?
• How to expose your own modules to third party code?
• How to write your tests only once but run them in Node.js and in the browser?

Possible Solutions

As of now, these issues are not trivially solved. However, there are already quite a few approaches to tackle these problems. One option is to add some boilerplate code to each module to make it compatible with CommonJS as well as AMD. Dr. Axel Rauschmayer wrote a good summary of the different flavours of this method. amdefine works in a similar fashion.

In the future, this problem might be solved by the module system that comes with EcmaScript 6. Since ES6 is not there yet and major browser support will not be available for quite some time, this is not relevant today (or maybe it is, with the help of Traceur or this transpiler or this module loader polyfill ).

Another, quite different approach is browserify . With browserify, you can write CommonJS modules — that is, you code your modules like in any plain vanilla Node.js project with exports/module.exports and require. Then you process your code with browserify which will turn your modules and all their transitive dependencies into a single blob of code that works in the browser.

The fact that there are various solutions to the problem doesn’t make it easier. When starting out with your first cross platform module, the options at hand are confusing and getting everything set up is still more complicated than it should be.

In this blog post we will examine the CommonJS + browserify approach and also discuss its advantages and drawbacks.

CommonJS Modules Primer

If you have written Node.js code before, you know how this works and can skip this section. If not, here’s a short primer on CommonJS modules.

Each module goes to its own file. Everything in that file is private to that module, unless you explicitly export it with the exports keyword. So given the following module file named example_module.js:

1var var1 = 42;
2var var2 = 'foo';
3 
4function someFunction() {
5  return var2;
6}
7 
8exports.exportedVar = 'bar';
9 
10exports.exportedFunction = function() {
11  console.log(var1 + someFunction());
12}

Only exportedVar and exportedFunction can be accessed from other modules. var1, var2 and someFunction are private to this module. To use this module from another module you would use the following code:

1var exampleModule = require('./example_module');
2console.log(exampleModule.exportedVar);         // bar
3console.log(exampleModule.exportedFunction());  // 42foo
4console.log(exampleModule.var1);                // undefined
5console.log(exampleModule.var2);                // undefined
6console.log(exampleModule.someFunction());      // TypeError: Object #<Object> has 
7                                                // no method 'someFunction'

If the string given to require is a file system path (usually a relative path starting with a dot) the module is imported from the given file system location. To use modules from a package that has been installed via npm or Node core modules you would use the module name without any prefix, that is require('modulename').

Browserify

From the possible solutions for cross platform JavaScript that have been mentioned above, browserify stands out with a number of advantages:

• No boilerplate code, this is obviously a plus on the DRY side.
• CommonJS module syntax is simple and straightforward (I like it better than AMD syntax, but this is probably a matter of taste).
• Browserify completely separates the concern of how many HTTP requests are needed until all JavaScript code is loaded from the granularity of your source code modules. With RequireJS, each module is loaded asynchronously and thus triggers an HTTP request. Of course, there are tools to optimize RequireJS code into a single file to work around this problem.
• You can also browserify your tests and then run them in a browser of your choice.
• Replacing (shimming) dependencies for the browser build is simple and straightforward.
• You can use ci.testling.com for cross-browser compatibility tests. This alone would be reason enough to use browserify in any project because ci.testling.com just rocks. (More on that below.)
• Browserify solves the problem of different capabilities of Node.js and browsers in a very clever way: There are shims for many Node core modules, so calls to the Node core API do something reasonable in the browser.

Of course, browserify also has some drawbacks:

• You always need a build step to test your code in the browser, even in development. This is not the case with RequireJS. However, browserify is reasonable fast and if you use grunt watch or watchify your turnaround time will be fine.
• When your code base grows, you might need to decide carefully if all modules should be browserified into a single file or if you want to split up and create multiple browserified files which depend on each other. Of course this might also be an important consideration when using other module systems.
• If you use frameworks that have a module concept of their own (like, say, AngularJS ), their might be some confusion when trying to map framework modules to CommonJS modules.

Grunt

Browserify requires some sort of build step to turn your Node.js code into browser code. If you consider this a no-go, then browserify might not be for you. (The boilerplate code solutions mentioned above do not require any build step.) And since we need some build step anyway, we can as well use some more tooling goodness for our JavaScript code.

The build process should include the following steps:

• Browserify the code,
• Run the tests in Node.js and in the browser – preferably the same test code.
• Minify the browser build
• Run other standard tasks, like linting the the code

A canonical choice for JavaScript builds is Grunt . It is easily the most popular JavaScript based build tool with a plethora of available plug-ins. Gulp is a new interesting alternative to Grunt, but I did not yet have a chance to give it a try (although I have heard a lot of praise for Gulp from trustworthy sources). And – because we are talking about the JavaScript community with its strong NIH symptom – there is now an even newer contender called Fez.

By the way, the creator of browserify (James Halliday aka substack) seems to despise build tools like Grunt and rather relies completely on npm and shell scripting for his build needs. In the end, this is largely a matter of taste.

All this said, we go with Grunt here. To use grunt, you should install the module grunt-cli from npm globally, so do npm install -g grunt-cli. Depending on your OS you might need a sudo in front of that.

Our basic (empty) Gruntfile.js, which will be our starting point, looks like this:

1'use strict';
2 
3module.exports = function(grunt) {
4 
5  // configure grunt
6  grunt.initConfig({
7 
8    pkg: grunt.file.readJSON('package.json'),
9 
10  });
11 
12  // Load plug-ins
13  // grunt.loadNpmTasks('grunt-contrib-whatever');
14 
15  // define tasks
16  grunt.registerTask('default', [
17    // No tasks, yet
18  ]);
19};

and here’s the basic package.json for the project:

1{
2  "name": "mymodule",
3  "version": "0.0.1",
4  "author": "Your Name",
5  "description": "description of your project...",
6  "scripts": {
7    "test": "grunt"
8  },
9  "main": "./mymodule"
10}

This assumes that the main entry point to your module is named mymodule.js and is in the same folder as the package.json and the Gruntfile.js.

In the following sections we will add more and more stuff to the gruntfile to do all the things stated earlier.

Note for the impatient: If you would rather skip all explanations and just see the code, the final result is here: https://github.com/basti1302/browserify-grunt-mocha-template . Feel free to copy/paste/fork/whatever this project or use parts of it for your own project.

Configuring The Grunt Build

Let’s work through the requirements for the build one by one.

Adding Grunt Plug-ins And Load Tasks

We will need to add some existing Grunt plug-ins to the mix. The process is always the same, so here’s a quick explanation on how that works, in case you are not familiar with Grunt:

• Find the plug-in on https://npmjs.org/ . This will give you the name of the package.
• In your project folder, do npm install package-name--save-dev. This will download the latest version of the plug-in to your project’s node_modules folder and add it to the devDependencies section of your package.json.
• Add a line grunt.loadNpmTasks('package-name'); to the Gruntfile where it says // Load plug-ins.
• Each plug-in defines a Grunt task. The name of the task is usually different from the package name of the plug-in. To find out the task name, refer to the documentation for the plug-in on GitHub or the npm web site.
• Add the necessary configuration to the object literal that is passed to grunt.initConfig. Use the task name (see above) as the key.
• Add the tasks as a subtask to the default task.

If that sounds a bit confusing at first, don’t worry. The following examples will demonstrate it step by step.

Linting

This one is easy and standard Grunt business. We install grunt-contrib-jshint by doing npm install grunt-contrib-jshint --save-dev, load the task in the Gruntfile and add some config, like this:

1module.exports = function(grunt) {
2 
3  // configure grunt
4  grunt.initConfig({
5    pkg: grunt.file.readJSON('package.json'),
6 
7    jshint: {
8      files: [
9        '**/*.js',
10        '!node_modules/**/*',
11      ],
12      options: {
13        jshintrc: '.jshintrc'
14      }
15    },
16  });
17 
18  // Load plug-ins
19  grunt.loadNpmTasks('grunt-contrib-jshint');
20 
21  // define tasks
22  grunt.registerTask('default', [
23    'jshint',
24  ]);
25};

This tells JSHint to process all of our own JavaScript files but not the stuff from node_modules (third party stuff) and to use the file .jshintrc for configuration options. You can find an example for a JSHint configuration here. We als added the grunt.loadNpmTask line to have Grunt load the plug-in code and registered the task as a sub-task of our default task.

If you run grunt now, your sources will be linted by JSHint.

Running Tests In Node.js

This section assumes that you use Mocha for your tests and that the tests reside in a subfolder named test. If you use a different test framework, you’ll need a different Grunt plug-in.

Install grunt-mocha-test by doing npm install grunt-mocha-test --save-dev, load the plug-in in the Gruntfile as before with grunt.loadNpmTasks('grunt-mocha-test'); and add mochaTest as the second subtask to your default task. The relevant config snippet looks like this:

1grunt.initConfig({
2 
3    pkg: grunt.file.readJSON('package.json'),
4 
5    jshint: {
6      // ... see above
7    },
8 
9    // run the mocha tests via Node.js
10    mochaTest: {
11      test: {
12        options: {
13          reporter: 'spec'
14        },
15        src: ['test/**/*.js']
16      }
17    },
18 
19  });
20 
21  ...

If you run grunt now, your tests will be run in Node.js after linting.

Building The Browser Lib

Now for the interesting part: We will use browserify to combine all our CommonJS modules together into one file that can be used in the browser. As usual, install the plug-in first: npm install grunt-browserify --save-dev, add grunt.loadNpmTasks('grunt-browserify') and add 'browserify' as the last subtask to the default task.

Here’s our initial grunt-browserify config:

1browserify: {
2      standalone: {
3        src: [ '<%= pkg.name %>.js' ],
4        dest: './browser/dist/<%= pkg.name %>.standalone.js',
5        options: {
6          standalone: '<%= pkg.name %>'
7        }
8      },
9    },

Update: The grunt-browserify API has changed since writing this post. The code above works for grunt-browserify 1.x. For grunt-browserify 2.x this would need to be:

1browserify: {
2      standalone: {
3        src: [ '<%= pkg.name %>.js' ],
4        dest: './browser/dist/<%= pkg.name %>.standalone.js',
5        options: {
6          bundleOptions: {
7            standalone: '<%= pkg.name %>'
8          }
9        }
10      },
11    },

See https://github.com/jmreidy/grunt-browserify/issues/165 for details.

This will tell browserify to use '<%= pkg.name="" %="">.js' as the entry point. <%= pkg.name="" %=""/> is a placeholder that Grunt replaces with the value of the name attribute from package.json file, so src resolves to mymodule.js. The result will be written to browser/dist/mymodule.standalone.js.

This browserify build will be used by users of our module. By using the standalone option (in the options object) we tell browserify to include a UMD (universal module definition), so it can be used via an AMD module loader like RequireJS or by simply placing a script tag in the page, which registers mymodule as a global var.

Run grunt to check if the browserified build is created correctly.

You could create a simple html page and include the browserified lib with a script tag. There is an example in the GitHub repo and there is also an example using RequireJS. Both examples are actually using the minified version, which we’ll cover next.

One more thing: The code created by browserify will probably not pass JSHint, so we should augment our JSHint config a bit to exclude the browserified files:

1jshint: {
2      files: [
3        '**/*.js',
4        '!node_modules/**/*',
5        '!browser/dist/**/*',
6        '!browser/test/**/*', 
7      ],
8      options: {
9        jshintrc: '.jshintrc'
10      }
11    },

(Excluding browser/test doesn’t make sense at this point but it will soon, so you can as well add that now.)

Minifying The Browser Lib

We use grunt-contrib-uglify to minify the browser build. Install it via npm, load the module in Grunt and add the uglify subtask to the default task (after the browserify subtask). Configuration as follows:

1// Uglify browser libs
2    uglify: {
3      dist: {
4        files: {
5          'browser/dist/<%= pkg.name %>.standalone.min.js':
6              ['<%= browserify.standalone.dest %>'],
7        }
8      }
9    },

Browserifying The Tests

Next on our list is the ability to run our Node.js-based tests (be it Mocha, Jasmine or whatever) in a browser. For this, we will also run our test suite through browserify. But we also need to make shure that the our browserified tests can access our production code. To make this easier, we produce a slightly different build of the production code via browserify. Add the following to the grunt-browserify config section:

1// browserify everything
2    browserify: {
3      standalone: {
4        // ... see above
5      },
6 
7      require: {
8        src: [ '<%= pkg.name %>.js' ],
9        dest: './browser/dist/<%= pkg.name %>.require.js',
10        options: {
11          alias: [ './<%= pkg.name %>.js:' ]
12        }
13      },
14 
15      tests: {
16        src: [ 'browser/test/suite.js' ],
17        dest: './browser/dist/browserified_tests.js',
18        options: {
19          external: [ './<%= pkg.name %>.js' ],
20          // Embed source map for tests
21          debug: true
22        }
23      }
24    },

This configuration will cause browserify to build three files. We already discussed the standalone build. The require build will also include the production code but it will not contain a UMD. Instead we will be able to require('mymodule) from other, separate browserified modules that have been created with an --external parameter.

The tests build will include the browserified tests. We need to browserify the tests to be able to run the Mocha tests in the browser while writing the tests as clean, simple CommonJS Mocha tests. This build must not include the module under test, so we use the external parameter to exclude mymodule from this file.

With this in place, you can already manually execute the tests in your browser. All you need is a html page that acts as a test runner. See here for an example.

Serving Static Files With Grunt and Connect

Opening the test runner html file via a file:// URL is okay, but not the real deal. So we use a handy little plug-in named grunt-contrib-connect to start a simple http server for serving static files inside the Grunt build and tear it down when the build has finished.

Here is the configuration:

1connect: {
2      server: {},
3    },

Don’t forget grunt.loadNpmTasks('grunt-contrib-connect'); and adding the task to the
default task list.

Running Tests In The Browser Automatically In Each Build

So far we have already run the tests successfully in the browser as a manual action. Of course we want the in-browser tests as a part of our automated build. For this, we use grunt-mocha-phantomjs, which runs the tests in PhantomJS , a headless browser.

After the usual chore of doing npm install grunt-mocha-phantomjs --save-dev (which will also install PhantomJS if it isn’t installed yet) and adding grunt.loadNpmTasks('grunt-mocha-phantomjs'); as well as 'mocha_phantomjs' to the list of tasks, we configure it like this:

1// run the mocha tests in the browser via PhantomJS
2    'mocha_phantomjs': {
3      all: {
4        options: {
5          urls: [
6            'http://127.0.0.1:8000/browser/test/index.html'
7          ]
8        }
9      }
10    },

This will start PhantomJS and points it to the given URL (which is the grunt-contrib-connect server we configured in the previous step). It also takes care of failing the build when Mocha tests fail in the test run inside PhantomJS.

Reducing Turnaround Time With Watchify

If you run the complete Grunt build on each change, your turnaround time will suffer. For larger code bases this will become unbearable over time. You can use grunt watch to automatically re-run a part of your build whenever a file is changed but you need to make a decision which Grunt tasks will be part of the run triggered by grunt watch and which steps will only be run if you explicitly start them from the command line.

Another option is watchify , which watches your browserify input files and rebuilds the browserify bundle when one of them changes. This is similar to using grunt watch together with grunt-browserify, but it’s considerably faster because watchify caches stuff from one browserify run to the other which grunt-browserify does not do.

You can start watchify like this:

1watchify --entry mymodule.js --outfile browser/dist/mymodule.standalone.js

Putting It All Together

We have added a considerable amount of stuff to our Gruntfile by now. To recap, here is the complete Gruntfile.js . Actually this repository is a working project template with some dummy modules, a readily configured Grunt build, package.json and Mocha tests.

The Art Of Shimming

Browserify comes with one big caveat: If you just “naively” browserify any given module, you might end up with a very large resulting JavaScript file, especially if the module has a lot of transitive dependencies. For code that targets the browser, this is obviously undesirable. However, there are ways to take care of that. First, when writing your cross platform module you need to be more cautious about which dependencies to include. The size of the dependency and the number of the dependency’s dependencies is now even more important than for a module that only targets Node.js.

The other important countermeasure to get the file size down is shimming, that is, substituting certain modules with a replacement, a shim, during browserification. This section explains how to do that.

When using browserify directly from the beginning, it is usually a bit easier. Whenever you add a dependency, watch out how this changes the size of the browserified file. Also, pay attention to which transitive dependencies the new dependency pulls in. If you think that you do not need all transitive dependencies, try to exclude some (see below) and see if your tests still work.

When you introduce browserify to an already existing project, it might be a bit more fiddly to find out which modules should be shimmed, or, which modules contribute the most to the resulting file size. Inspect the browserified JavaScript file to see what is in there, this might give you a start. The rest is a bit of try and error to find out which modules can be excluded/shimmed.

Shimming Modules

The browser field in the package.json is used to define what will be shimmed (substituted) during browserification . Here is an example:

1{
2  ...
3  "dependencies": {
4    "module-from-npm": "~0.2.12",
5  },
6  ...
7  "browser": {
8    "./lib/local-module.js": "./browser/lib/shim/local-module.js",
9    "module-from-npm": "./browser/lib/shim/module-from-npm.js",
10    "util": "./browser/lib/shim/node-util.js",
11    "./lib/optional": false
12  }
13  ...
14}

The browser object contains four key-value pairs. The key of each pair is always the module that is to be shimmed. The value is the actual shim that will be used as a substitute in the browserify build. In this examples, all browser shims come from ./browser/lib/shim but this is totally a matter of taste.

The first key identifies a local module of this package, that is, a module file that sits in the same directory as the package.json or in a subdirectory of that directory. These are always given as a relative path (the same path that you would use to require it from a file in the main directory), and they always need to have the file suffix.

The second key identifies a module from a package that has been installed via npm and which lives somewhere under node_modules. These are identified just by the module name that would be used in a require statement.

The third key is similar, but this time a Node core module is shimmed. Syntactically, there is no difference to the second example.

Actually, most Node core modules are shimmed anyway by browserify and the shims are probably fine. However, since each shim needs to cover all functionality of the shimmed node core module, they are quite large. If you only use a tiny fraction of the functionality of some core module, you might be better of (in terms of file size) to provide your own, reduced shim.

The fourth key-value pair is special since its value is simply false instead of a shim. This means that the module will be completely omitted from the browserify build. If the module is normally included in the build because it is required somewhere, but you are completely sure that no functionality of the module is ever used at runtime, this will make browserify leave it out. Also, the module is not scanned for require calls, so transitive dependencies of this module will also be ignored.

Using ci.testling.com

Another strong advantage of using browserify is that it also enables you to use ci.testling.com. Testling takes all of your code, browserifies it, and runs your tests – in every possible browser that you might want to support, including all major browser like Firefox, Chrome, IE, Safari and also mobile browsers (Android stock browser, iPhone, iPad, …). And it not only runs one version of, for example, IE, but all of them! From IE6 to IE10 (IE11 will probably be added soon). The complete list of browsers and versions is available as a json file on testling.com. Tests can be written as TAPE tests or with Mocha, but the service also accepts any JavaScript test framework out there, as long as there is custom TAP reporter for it (Jasmine is supported in this way, for example).

To enable your project for ci.testling.com, you need to add a testling field to your package.json. Here is an example that configures which browsers you want your tests to run in and sets up testling to use Mocha tests:

1"testling": {
2    "browsers": [
3      "ie8", "ie9", "ie10", "ie10",
4      "ff/15", "ff/20", "ff/25", "ff/nightly",
5      "chrome/29", "chrome/canary",
6      "opera/15", "opera/next",
7      "safari/6.0"
8    ],
9    "harness" : "mocha",
10    "files": "test/*.js"
11  },

The service from ci.testling.com is free for open source projects and to set it up you simply add a web hook to your GitHub project. Paid private instances are also available for closed source projects.

When to go Cross Platform — And When Not

Of course not every JavaScript project is suitable for making it a cross platform module. Code that uses the file system, a database connection, or responds to http requests probably makes no sense in the browser – this is probably classical backend-only code. I say “probably”, because with things like IndexedDB , Web Storage and File API browser are currently becoming more and more powerful by the day, they already can do amazing things that we’re considered backend-only only a few years back.

On the other hand, if code works heavily with the browser’s DOM API, it might not be very useful in the backend. Again, this is not a hard “No go” because there are DOM API modules for Node.js (jsdom , for instance).

Note that if your project is frontend only it can still be a very good idea to write it as CommonJS modules and build it with browserify.

In the end, it all depends on you specific use case and constraints.

Conclusion

In my very humble opinion, browserify rocks and everybody should use it, no matter if you want to reuse your code on client and server or if you are writing a frontend only thing. Granted, that is a very bold statement but I truly believe that browserify is a very powerful tool and it enable you to use one of the best package managers out there and leverage the huge and insanely fast growing ecosystem of npm.

If you want to share frontend and backend code, using browserify even becomes more awesome. Setting up a project might not be completely trivial, but the examples here can be a good starting point.

I would love to hear about your experience with Browserify and the other tools mentioned in this blog post. Leave a comment below!