Opinion / Post

The world of Node.js and its package manager, NPM, has revolutionised how we think about and manage dependencies in software development. However, this ecosystem is not without its challenges. One such challenge is the proliferation of micropackages, which, while seemingly innocuous, can cumulatively create a significant burden for developers and organisations alike. In this blog post, we will explore the nature of this issue, its underlying causes, potential solutions, and examine a real-world example to illustrate the sheer insanity of the problem.

The Plague

The term “micropackage” refers to small, often single-function packages in the NPM ecosystem. These packages typically accomplish very specific, sometimes trivial, tasks. On the surface, this approach seems to promote modularity and reusability, hallmarks of efficient software development. However, in practice, it leads to a phenomenon I refer to as the “micropackage plague”.

A Worked Example

To truly grasp the absurdity of the micropackage problem, let’s consider the case of developer Jon Schlinkert¹, known for creating an extensive array of such packages. Schlinkert’s repositories include a series of packages for ANSI colors, each dedicated to a single color: ansi-red, ansi-green, ansi-blue, and so on. This approach not only fragments what could be a single, manageable package into several smaller ones but also bloats the dependency tree unnecessarily.

Furthermore, Schlinkert’s portfolio includes packages like is-odd and is-even, which offer simple functionality to check if a number is odd or even. While these functions are elementary and could be easily integrated into larger projects, they exist as standalone packages. The is-even package takes ludicrousness to another level by depending on is-odd to determine if a number is even.

And it doesn’t stop there. Here is everything npm install is-even actually pulls in:

Through all these micropackages, he seems to have achieved something. If you’re a web developer you probably have some of his code on your computer. If you go to your node_modules and do a little grep -r jonschlinkert it’s pretty likely he’ll show up. Through creating hundreds of packages that depend on each other, you end up with ansi-red being downloaded 6.5M times a week, when nobody is going and including ansi-red themselves. We have entered a weird fractal of CV-boosting where NPM metrics are gamified, to make you look like a more impactful developer than you maybe are.

These examples highlight the core issues of the micropackage plague:

1. Dependency Bloat and Redundancy: Instead of a single package to handle ANSI colors, there are multiple packages, one for each color. This leads to a bloated node_modules directory and an inflated dependency tree.
2. Unnecessary Complexity: For something as straightforward as checking if a number is even, relying on another package (is-odd) adds unnecessary complexity and dependency.
3. Maintenance Overhead: Each of these packages, no matter how small, requires ongoing maintenance for updates, security, and compatibility. This only gets more difficult as the number of packages in active use increases.

This level of fragmentation and redundancy in package management illustrates just how far the micropackage issue can go, impacting the efficiency and manageability of JS development in general.

The Cause

The root of the micropackage problem lies in a combination of cultural and technical factors within the Node.js and NPM community. The culture of open-source contribution in the Node.js ecosystem encourages the sharing of any and all code, no matter how small. This philosophy, while commendable for promoting collaboration, sometimes leads to an excessive focus on quantity over quality.

While the technical simplicity of creating and publishing packages to NPM commendably lowers the barrier to entry, it also presents a double-edged sword. This accessibility encourages many developers to contribute small, often superfluous packages for tasks that could be efficiently integrated into larger applications or libraries.

Furthermore, there’s a tendency to prioritise individual developer metrics, such as the number of packages created or download counts, over the practical utility of the package. This can lead to developers creating multiple small packages to inflate these metrics, rather than focusing on contributing to more substantial, robust projects. As mentioned earlier, NPM metrics are also quite gamified, which means micropackages are commonly used to abuse the metrics’ flaws to make the author’s profile look better.

The Solution

I don’t have a silver bullet, but a few things would nudge it the right way.

Stop treating download counts as a measure of impact. Transitive downloads in particular tell you nothing about whether anyone actually chose your package. If ansi-red gets 6.5M downloads a week and nobody asked for it, that number is noise. As long as it looks good on a CV, people will keep farming it.

Tooling could help too. I’d love a dependency tree viewer that makes it painfully obvious when half your node_modules is one-liners, and audit tools that flag a project for dragging in a silly number of tiny or unmaintained packages.

Prefer bigger, well-maintained libraries over one package per function. Nobody needs a package per colour. If you care about bundle size, make the library modular, don’t split it into fifty packages.

And honestly, if it’s one line, just write the line. You don’t need a dependency to check if a number is even.

How much this blog post will do to combat it? Unfortunately, likely nothing at all.

What next?

I had a discussion about this with my friend Daniel, and he suggested I try to map the space to find the most prolific micropackage authors. I like the idea of taking this a step further and trying to apply the principles of epidemiology to this micropackage plague. Expect another post sometime soon with follow up results to this.