Replace asset-packagist with custom package repositories

Replace asset-packagist with custom package repositories

Drupal Commerce contributor Ryan Hovland of ProCycle reached out today in the Drupal Slack #commerce channel to let me know the asset-packagist Composer repository is defunct. This repository allowed projects like Commerce Kickstart to download JavaScript libraries managed by npm to a Drupal codebase via Composer.

We still don't know why or how the domain lapsed, but it may have been related to business disruption caused by Russia's invasion of Ukraine, as the repository is maintained by a Kyiv based team called HiQDev. Whether that's the cause or not, I'd like to pause and remember our friends in the Drupal community who are suffering as a result of the war and express my hope for their safety and the end of hostilities. For me that includes several team members we've retained in partnership with Lemberg Solutions. Thinking of you all. 😔🙏🏼

In the meantime, if you're maintaining a project that uses asset-packagist, whether a public repository or your own website based on a template that included it, you may have noticed the following message on a build or deploy:

  [Composer\Downloader\TransportException]
  curl error 28 while downloading https://asset-packagist.org/packages.json: Connection timeout after 10003 ms

Remove asset-packagist

You have a few options to recover and secure your builds. You'll notice in your project's composer.json file the following entry in your repositories array (with or without the asset-packagist key / trailing comma):

        "asset-packagist": {
            "type": "composer",
            "url": "https://asset-packagist.org"
        },

This instructs Composer to look for packages in the repository served at that URL. It included wrapped versions of JavaScript libraries managed via npm and Bower so PHP projects could build them into their codebases without having to include these other package managers in their build processes. It was nice while it worked, but now that it's defunct, you risk failing builds or, at worst, the incorporation of malicious code into your codebase should a bad actor acquire the domain.

The first thing to do is to remove this repository from your composer.json. If your project doesn't actually have any dependencies in the npm-asset or bower-asset namespaces, it should continue to build just fine. You're good to go! 🥳

However, if composer update now produces errors about missing packages like the following, you have more work to do:

  Problem 1
    - Root composer.json requires npm-asset/select2, it could not be found in any version, there may be a typo in the package name.

Review your build process

You only have a few next steps. The simplest would be to review your dependencies to see if you're actually using the packages you were fetching from the repository. If you're no longer using them, remove them from your codebase and carry on.

As long-time Drupal contributor Chris Burgess points out, you might also consider improving your build process to make use of the package managers your dependencies prefer. This will ensure you can continue to make use of version constraints to fetch patch or minor version updates to these projects.

Define custom packages

Finally, you can also do what I ended up doing based on guidance from Paul Mitchum in the Drupal Slack #contribute channel. (Sidebar: apparently he's currently a Drupal developer for Wizards of the Coast - very cool! 🧙🏼‍♂️) You can continue to use Composer to fetch JavaScript libraries (or anything, really) by defining a custom package in your repositories array.

When you do this, you can choose to fetch the project from its Git repository by defining a vcs repository. I opted not to do this, because I wanted to fetch a specific artifact rather than check out a branch with little observability of the commits that may have been pushed since my last build.

Instead, I chose to fetch a specific artifact by defining a custom package. By doing this, I could point Composer to a .zip archive served by GitHub based on a specific tag or commit I knew to be compatible with my project. For the Select2 library, which we use to improve the UX of large select lists in the Drupal back-end, that meant defining the following package in my repositories array:

        {
            "type": "package",
            "package": {
                "name": "select2/select2",
                "version": "4.1.0-rc.0",
                "type": "drupal-library",
                "dist": {
                    "type": "zip",
                    "url": "https://github.com/select2/select2/archive/refs/tags/4.1.0-rc.0.zip"
                }
            }
        }

Now, in my dependencies array (either require or require-dev as needed), I can instruct Composer to fetch the package using the name I set above:

        "select2/select2": "4.1.0-rc.0",

Note that you don't have the same options here regarding version constraints. Once you make this switch, it's now up to you to track new releases and update your composer.json file accordingly. 👀

Put the libraries in their place

Composer is responsible not just for downloading code packages but also for putting them in the right place in your codebase. For most PHP packages, this means installing them into your vendor directory. However, Drupal projects make use of composer/installers plugin to put things like contributed modules and themes into the right subdirectories of the project's document root.

Drupal projects using asset-packagist likely used the oomphinc/composer-installers-extender plugin to define two new installer types and to instruct Composer to install the packages into the site's libraries directory:

        "installer-types": [
            "bower-asset",
            "npm-asset"
        ],
        "installer-paths": {
            "web/libraries/{$name}": [
                "type:drupal-library",
                "type:bower-asset",
                "type:npm-asset"
            ]
        },

You're no longer using the packages defined by asset-packagist, so you can remove its npm and Bower related installer types and their related installer paths. Since you defined your custom package as a drupal-library, the specified .zip archives will now be installed into the expected web/libraries subdirectory (or equivalent document root based on your project's configuration).

Just copy the distribution files

That said, one of my colleagues suggested one final improvement. These libraries almost always include unnecessary files you may not want in your document root, even if it's just a normal version of a .js or .css file and a minified version. In the case of Select2, it's actually quite a lot of cruft related to developing with the library that you simply don't need to make use of the Select2 Drupal module.

Instead of installing the entire archive into your site's libraries folder, you can copy over just the files you need. For Drupal modules that define JavaScript libraries, this means looking at the module's .libraries.yml file to see what it actually looks for to determine whether or not a library has been installed successfully. In the case of Select2, it looks like this:

select2.min:
  remote: https://github.com/select2/select2
  license:
      name: MIT
      url: https://github.com/select2/select2/blob/master/LICENSE.md
      gpl-compatible: true
  js:
    /libraries/select2/dist/js/select2.min.js: { minified: true }
  css:
    component:
      /libraries/select2/dist/css/select2.min.css: {}
  dependencies:
    - core/jquery.once
    - core/drupal

You can see that it expects the Drupal site to include a minified .js file and a minified .css file in a dist subfolder of the library. To only place those files in your site's document root, you need to change your installer path for these packages and make use of the Drupal Composer scaffold to copy just the files you want.

For Commerce Kickstart, I chose to install custom packages outside the document root in a libraries directory like so:

            "libraries/{$name}": [
                "furf/jquery-ui-touch-punch",
                "select2/select2"
            ],

I then used the scaffold to copy the distribution files expected by the related Drupal modules into the right place:

        "drupal-scaffold": {
            "locations": {
                "web-root": "./web"
            },
            "overwrite": true,
            "file-mapping": {
                "[web-root]/libraries/jquery-ui-touch-punch/jquery.ui.touch-punch.min.js": "libraries/jquery-ui-touch-punch/jquery.ui.touch-punch.min.js",
                "[web-root]/libraries/select2/dist/js/select2.min.js": "libraries/select2/dist/js/select2.min.js",
                "[web-root]/libraries/select2/dist/css/select2.min.css": "libraries/select2/dist/css/select2.min.css"
            }
        },

As always, there are certainly more than one ways to skin this cat. 🙀

You can see what I did to replace asset-packagist with custom packages in this commit to Commerce Kickstart, but adapt as needed to suit your project and feel free to share any other thoughts or suggestions in the comments!