HTML in Node.js

 by Tomasz Jakut

Tomasz Jakut

@comandeer

Senior Software Engineer / Web Standards Expert

JavaScript developer, web conservative. Web standards and accessibility advocate.

. Posted on 2020-04-20

Sometimes we can be using a technology for years and not realize that it hides some small, yet super useful, feature deep inside. This is also true for Node.js and its require(). Did you know that it allows you to import HTML files into your JavaScript application?

# HTML in Node.js?!

Imagine that you are creating a complex web application. You have just noticed that you spend too much time handling HTML templates. You must load such an HTML file, convert it into a template and finally replace data placeholders with real data.

What if you could do something like this instead?

const template = require( './templates/hello.html' );

console.log( template.render( {
  user: 'Comandeer'
} ) );

Instead of creating separate templates for every HTML file, you just import an HTML file that is automagically transformed into a proper template (e.g. a Hogan one). Simple and beautiful, in the spirit of webpack — but implemented in the production environment, not the development one.

It would be nice if this worked, wouldn’t it?

# Extending require()

As a matter of fact, you can actually achieve this using a barely known feature of require(): the extensions property. This property contains file extensions (e.g. .js, .cjs etc.) supported by require() as well as callbacks handling these extensions.

Thanks to this, adding support for an HTML file will simply mean adding yet another callback to require.extensions:

require.extensions[ '.html' ] = () => {};

The callback takes two parameters: an object representing the module being loaded and its path.

Every loaded module must be compiled into JavaScript code before being available in Node.js applications. The compilation step is done via the _compile function of the loaded module.

To see how it works, you can check the Node.js source code. Your hook will do it in a similar way:

const { readFileSync } = require( 'fs' );

require.extensions[ '.html' ] = ( module, path ) => {
  const html = readFileSync( path, 'utf8' ); // 1
  const code = `const hogan = require( 'hogan.js' );
                const template = hogan.compile( \`${ html }\` );

                module.exports = template;`; // 2

  module._compile( code, path ); // 3
};

In the beginning, you fetch the content of the HTML file (1). Then you insert it into the code of a very simple JavaScript module that wraps HTML into a Hogan template (2). The code prepared in this way is then compiled using module._compile (3).

And this is all — your JavaScript application is now able to import HTML files!

# Arrgh!

Unfortunately, in real world hooks are often more complex, like @babel/register transpiling JavaScript code right before importing it. The pirates library that makes it easier to add hooks was created for such cases:

const { readFileSync } = require( 'fs' );
const { addHook } = require( 'pirates' );

addHook(
  ( code, path ) => {
    const html = readFileSync( path, 'utf8' );

    return `const hogan = require( 'hogan.js' );
            const template = hogan.compile( \`${ html }\` );

            module.exports = template;`;
  }, // 1
  { exts: [ '.html' ] } // 2
);

The hook is added using the addHook() function. It takes the module transforming function as the first parameter (1) and the options object as the second one (2).

The only option you will use in this case is the exts one that contains an array of file extensions handled by the hook.

There is also a matcher option that takes a function. It checks if the file with the provided path should be transformed by the hook. In this case, you want to transform all HTML files with the hook, so you can skip the matcher option.

# Wait a minute…

Is it even a good idea to extend require() in this way?

Well, yes, but also no.

No, because importing modules will last longer the more steps you add to this process (like adding some code, transpiling, handling image files, etc.). Additionally, you may have noted that the official documentation claims that require.extensions is deprecated since version 0.10.0…

Yes, because… there is no other way to do it. Even if it is not described directly in the official documentation and it is not recommended to use, the vast part of the Node.js ecosystem is based on it and therefore require.extensions simply cannot be removed. Especially when there is no alternative to it.

# What about ES modules?

The newest versions of Node.js (12+) introduced — still experimental — support for ES modules. To be honest, their syntax is much more pleasant to work with than the old CommonJS one. Additionally, ESM in Node.js has their own mechanism for extending the module loader. However, it is still in flux and changes very often. In my humble opinion, it is very risky to use it now, and as a result, it may be better to stick to require.extensions for a while.

Having said that, I must admit that the syntax of ESM hooks is much friendlier than require() hooks — mainly because the new ones do not depend on some voodoo magic not covered in any documentation.

# Demo

Demo versions of all three described methods (a “manual” require() hook, a require() hook created using pirates and a probably outdated hook for ESM) are available in the sample GitHub repository.

Have fun with importing HTML!