-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 - --- title: "Tiny Packages, Big Challenges in NPM" date: 2024-01-08T05:41:13Z draft: false category: essay excerpt: "Opinion piece on the micropackage plague in npm, using Jon Schlinkert's ANSI utilities and even is-even as examples of dependency bloat. I outline the perverse incentives, the operational pain, and a few cultural and tooling fixes I'd love to see." - --- # Opinion / Post - --- The world of Node.js and its package manager, NPM, has revolutionised how we think about and manage dependencies in software development. However, this ecosystem is not without its challenges. One such challenge is the proliferation of micropackages, which, while seemingly innocuous, can cumulatively create a significant burden for developers and organisations alike. In this blog post, we will explore the nature of this issue, its underlying causes, potential solutions, and examine a real-world example to illustrate the ~wave:sheer insanity~ of the problem. :whatthefuck: ## The Plague The term "micropackage" refers to small, often single-function packages in the NPM ecosystem. These packages typically accomplish very specific, sometimes trivial, tasks. On the surface, this approach seems to promote modularity and reusability, hallmarks of efficient software development. However, in practice, it leads to a phenomenon I refer to as the "micropackage plague". ### A Worked Example To truly grasp the absurdity of the micropackage problem, let's consider the case of developer Jon Schlinkert[^1], known for creating an extensive array of such packages. Schlinkert's repositories include a series of packages for ANSI colors, each dedicated to a single color: `ansi-red`, `ansi-green`, `ansi-blue`, and so on. This approach not only fragments what could be a single, manageable package into several smaller ones but also bloats the dependency tree unnecessarily. Furthermore, Schlinkert's portfolio includes packages like `is-odd` and `is-even`, which offer simple functionality to check if a number is odd or even. While these functions are elementary and could be easily integrated into larger projects, they exist as standalone packages. The `is-even` package takes ludicrousness to another level by depending on `is-odd` to determine if a number is even. And it doesn't stop there. Here is everything `npm install is-even` actually pulls in: ![is-even 1.0.0 depends on is-odd 0.1.2, which depends on is-number 3.0.0, then kind-of 3.2.2, then is-buffer 1.1.6: five packages and 4,213 bytes of code to compute n % 2 === 0](is-even-chain.svg) Through all these micropackages, he seems to have achieved something. If you're a web developer you probably have some of his code on your computer. If you go to your `node_modules` and do a little `grep -r jonschlinkert` it's pretty likely he'll show up. Through creating hundreds of packages that depend on each other, you end up with `ansi-red` being downloaded 6.5M times a week, when nobody is going and including `ansi-red` themselves. We have entered a weird fractal of CV-boosting where NPM metrics are gamified, to make you look like a more impactful developer than you maybe are. These examples highlight the core issues of the micropackage plague: 1. **Dependency Bloat and Redundancy**: Instead of a single package to handle ANSI colors, there are multiple packages, one for each color. This leads to a bloated node_modules directory and an inflated dependency tree. 2. **Unnecessary Complexity**: For something as straightforward as checking if a number is even, relying on another package (`is-odd`) adds unnecessary complexity and dependency. 3. **Maintenance Overhead**: Each of these packages, no matter how small, requires ongoing maintenance for updates, security, and compatibility. This only gets more difficult as the number of packages in active use increases. This level of fragmentation and redundancy in package management illustrates just how far the micropackage issue can go, impacting the efficiency and manageability of JS development in general. ## The Cause The root of the micropackage problem lies in a combination of cultural and technical factors within the Node.js and NPM community. The culture of open-source contribution in the Node.js ecosystem encourages the sharing of any and all code, no matter how small. This philosophy, while commendable for promoting collaboration, sometimes leads to an excessive focus on quantity over quality. While the technical simplicity of creating and publishing packages to NPM commendably lowers the barrier to entry, it also presents a double-edged sword. This accessibility encourages many developers to contribute small, often superfluous packages for tasks that could be efficiently integrated into larger applications or libraries. Furthermore, there's a tendency to prioritise individual developer metrics, such as the number of packages created or download counts, over the practical utility of the package. This can lead to developers creating multiple small packages to inflate these metrics, rather than focusing on contributing to more substantial, robust projects. As mentioned earlier, NPM metrics are also quite gamified, which means micropackages are commonly used to abuse the metrics' flaws to make the author's profile look better. ## The Solution I don't have a silver bullet, but a few things would nudge it the right way. Stop treating download counts as a measure of impact. Transitive downloads in particular tell you nothing about whether anyone actually chose your package. If `ansi-red` gets 6.5M downloads a week and nobody asked for it, that number is noise. As long as it looks good on a CV, people will keep farming it. Tooling could help too. I'd love a dependency tree viewer that makes it painfully obvious when half your `node_modules` is one-liners, and audit tools that flag a project for dragging in a silly number of tiny or unmaintained packages. Prefer bigger, well-maintained libraries over one package per function. Nobody needs a package per colour. If you care about bundle size, make the library modular, don't split it into fifty packages. And honestly, if it's one line, just write the line. You don't need a dependency to check if a number is even. How much this blog post will do to combat it? Unfortunately, likely nothing at all. ## What next? I had a discussion about this with my friend [Daniel](https://danielmroz.co.uk), and he suggested I try to map the space to find the most prolific micropackage authors. I like the idea of taking this a step further and trying to apply the principles of epidemiology to this micropackage plague. Expect another post sometime soon with follow up results to this. [^1]: In the case you're reading this Jon, sorry, it's nothing personal. -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEJtHIQX7gIiqRWdSs/RuAbpOsR8kFAmrxaZcACgkQ/RuAbpOs R8kavA//VNxrwArCwqc0GP2OJKaAN6GtXXE1f8Cj2XuRN+Wzbkttlq8cJW9Govch seUv4WgI388Chd31/aO+Q68ZrCwQD3QFSdJa8RASoZIlWsNa7A5d8hNW26AgiAlG zCdqL3DJpaZUDwgBA5XfF2T8gkDKb3Jnge7tsTy3F2xrqAMbP8XKhArC+NVraHc8 cQ9dSItAiC2RLAgHm56HlUc2yIAPy4DA+K49cUQZV+U4A2gj8FCg9Cdnq6KxZct/ aXEbbyoTDPcEdnubHVJnaFPKFDsiTrPNSWNPBeNenBda4P8c7y/xHmeiVv+VQZo3 AWLWoXEzdbO9R2T689PzaOdcS9UoAxZqYvoOeIudbj9zfgFJ55sD/2ut4bASzpZZ xQeGzP8RIQtJocOzlJqaOKnZEaieAA8JXrrZ4XiY59Yn5tcv5943iKFzUW/vV3Zd fPjLSg+nYdMzC0U9aBPP/NkEQVEvsLUHD4DSxdZCZskCZUiHvASSjizIXT0B/D41 kXHk2DlU97XNmWRK5kZ5B0cPKF3XF5ib5Weo0DXBMpZ+oEDCGvb5WBxIhAyq+QQz 9o+XlW0yBTUR8XIGCVZc+nKJdc/o0WOk5P/eDlo7WgAe77906lMvrqyHXK7jLAnn W2DyBdIj6gaHNMrvTlvgGX1wKSq+ZnwacynSSCQAmHDVsdmsbbM= =HGLw -----END PGP SIGNATURE-----