Opinion / Post

---

The world of Node.js and its package manager, NPM, has revolutionised how we think about and manage dependencies in software development. However, this ecosystem is not without its challenges. One such challenge is the proliferation of micropackages, which, while seemingly innocuous, can cumulatively create a significant burden for developers and organisations alike. In this blog post, we will explore the nature of this issue, its underlying causes, potential solutions, and examine a real-world example to illustrate the sheer insanity of the problem.

The Plague

The term “micropackage” refers to small, often single-function packages in the NPM ecosystem. These packages typically accomplish very specific, sometimes trivial, tasks. On the surface, this approach seems to promote modularity and reusability, hallmarks of efficient software development. However, in practice, it leads to a phenomenon I refer to as the “micropackage plague”.

A Worked Example

To truly grasp the absurdity of the micropackage problem, let’s consider the case of developer Jon Schlinkert¹, known for creating an extensive array of such packages. Schlinkert’s repositories include a series of packages for ANSI colors, each dedicated to a single color: ansi-red, ansi-green, ansi-blue, and so on. This approach not only fragments what could be a single, manageable package into several smaller ones but also bloats the dependency tree unnecessarily.

Furthermore, Schlinkert’s portfolio includes packages like is-odd and is-even, which offer simple functionality to check if a number is odd or even. While these functions are elementary and could be easily integrated into larger projects, they exist as standalone packages. The is-even package takes ludicrousness to another level by depending on is-odd to determine if a number is even.

Through all these micropackages, he seems to have achieved something. If you’re a web developer you probably have some of his code on your computer. If you go to your node_modules and do a little grep -r jonschlinkert it’s pretty likely he’ll show up. Through creating hundreds of packages that depend on each other, you end up with ansi-red being downloaded 6.5M times a week, when nobody is going and including ansi-red themselves. We have entered a weird fractal of CV-boosting where NPM metrics are gamified, to make you look like a more impactful developer than you maybe are.

These examples highlight the core issues of the micropackage plague:

1. Dependency Bloat and Redundancy: Instead of a single package to handle ANSI colors, there are multiple packages, one for each color. This leads to a bloated node_modules directory and an inflated dependency tree.
2. Unnecessary Complexity: For something as straightforward as checking if a number is even, relying on another package (is-odd) adds unnecessary complexity and dependency.
3. Maintenance Overhead: Each of these packages, no matter how small, requires ongoing maintenance for updates, security, and compatibility. This only gets more difficult as the number of packages in active use increases.

This level of fragmentation and redundancy in package management illustrates just how far the micropackage issue can go, impacting the efficiency and manageability of JS development in general.

The Cause

The root of the micropackage problem lies in a combination of cultural and technical factors within the Node.js and NPM community. The culture of open-source contribution in the Node.js ecosystem encourages the sharing of any and all code, no matter how small. This philosophy, while commendable for promoting collaboration, sometimes leads to an excessive focus on quantity over quality.

While the technical simplicity of creating and publishing packages to NPM commendably lowers the barrier to entry, it also presents a double-edged sword. This accessibility encourages many developers to contribute small, often superfluous packages for tasks that could be efficiently integrated into larger applications or libraries.

Furthermore, there’s a tendency to prioritise individual developer metrics, such as the number of packages created or download counts, over the practical utility of the package. This can lead to developers creating multiple small packages to inflate these metrics, rather than focusing on contributing to more substantial, robust projects. As mentioned earlier, NPM metrics are also quite gamified, which means micropackages are commonly used to abuse the metrics’ flaws to make the author’s profile look better.

The Solution

Addressing the micropackage plague requires a multi-prong approach:

1. Promote Best Practices: Encourage the development community to favor more comprehensive libraries over micropackages for trivial tasks. This can be achieved through education, community guidelines, and highlighting best practices in popular forums and documentation.
2. Improve Dependency Management Tools: Tools can be developed or improved to help manage and visualise dependency trees, making it easier to identify and eliminate unnecessary micropackages.
3. Security Auditing: Enhance security auditing tools to automatically flag projects with an excessive number of dependencies, particularly those that are known to be vulnerable or poorly maintained.
4. Community-led Initiatives: Initiatives like code reviews or community-driven standards can help in vetting packages for quality and utility before they become widely adopted.
5. Redefine Metrics of Success: Shift the focus from the quantity of contributions to the quality and impact of contributions within the Node.js ecosystem.
6. Encourage Modular but Comprehensive Packages: Foster a development culture that values packages that are modular but still comprehensive enough to reduce the need for numerous tiny dependencies.

In conclusion, while the micropackage issue is a significant challenge in the NPM ecosystem, it is not insurmountable. Through community effort, education, tooling improvements, and a shift in cultural priorities, we can mitigate the negative impacts of this plague and steer towards a more efficient and secure future in software development.

How much this blog post will do to combat it? Unfortunately, likely nothing at all.

What next?

I had a discussion about this with my friend Daniel, and he suggested I try to map the space to find the most prolific micropackage authors. I like the idea of taking this a step further and trying to apply the principles of epidemiology to this micropackage plague. Expect another post sometime soon with follow up results to this.