Over the years our legacy APIs have not had rate-limiting built into them, other than the implicit, informal rate limiting caused by performance bottlenecks. Most of the time, for most users of our public APIs, this has been sufficient. As the registry grows, however, we’ve seen heavier use of our APIs, some of which has been heavy enough to prompt us to take action. If we can identify the user and suspect it’s a bug, we reach out. Sometimes we simply block IPs when the usage is at levels that affect other people’s ability to use the APIs.
This isn’t ideal, and we’d rather give clear signals to API users what the allowed rates are and when they are being rate-limited. We’re therefore rolling out more explicit rate-limiting for all registry apis. Your tools should be ready to handle responses with http status code 429, which means that you have exceeded the allowed limit.
We will be allowing logged-in users to make requests at a higher rate than anonymous users.
For some services we have already begun limiting the amount of data that can be requested at a time. For example, we limit package search requests to queries that are at least three characters long. We have also taken steps to prevent API users from exploiting bugs to work around that limit.
We’ve also re-instituted limits in our downloads API. Our previous implementation of this API limited data ranges to well under a year for performance reasons. Our current implementation performs much better but turned out to have its breaking point as well,. You may now request at most 18 months of data at a time for single-package queries. Bulk package data queries are limited to at most 128 packages and 365 days of data.
We reserve the right to further limit API usage without warning when we see a pattern of requests causing the API to be unusable for most callers. We’ll follow up with documentation in these cases. Our primary goal is to prevent API use from either deliberately or accidentally making the service unresponsive for other users.
All of our existing registry API documentation is available in the registry repo on GitHub, and you can find the most up-to-date statements about our rate-limiting policies there.

npm’s newest wombat is… an actual wombat. Teacup is a female wombat joey being nursed and raised at the Sleepy Burrows Wombat Sanctuary in Gundaroo, Australia. When npm adopted her shortly after she arrived at Sleepy Burrows in July, Teacup weighed just under 200 grams (7 oz.), but her caretakers have done their best to simulate life in her mother’s pouch, providing milk and rubbing her with hemp oil (be warned: cuteness ahead!) to help her regulate body temperature.
A month and a half later, Teacup has grown to over 800 grams (28 oz.), is growing a healthy layer of hair and is learning how to walk. We will be paying very close attention as she matures, because now “watching baby wombat videos” counts as legitimate workday activity. Stay tuned for updates!
If you’d like to support the Sleepy Burrows Wombat Sanctuary, you can learn more, and lose the next week of your life watching all the videos, on their website.
npm, Inc. and I will continue to throw our weight behind our values, including diversity and inclusivity in the Node.js project. I am encouraged to see that the Node.js Foundation board also recognizes the importance of these values, and is taking steps to correct the failures in project governance that risked calling their commitment into doubt.
There is tremendous risk if the Node.js Foundation doesn’t decisively expand its community of open source contributors. The Node.js ecosystem is larger than ever. Its continued growth depends on technical innovation, and innovation requires a healthy culture. Any project will suffer without contributions from a broad selection of its members, and any project will lose relevance if its leaders don’t actively promote inclusive conduct.
Node.js developers are an extremely diverse community who care deeply about inclusivity, and are not shy about expressing themselves through direct action. The Node.js project is stronger when they speak up.
I am confident that the leaders of Node.js Foundation will take the right actions to put this challenge behind us. It isn’t the first time that the community has spoken up about its needs, and I hope it isn’t the last. I am extremely proud of the community we’ve all built together, and excited to see it continue to grow and mature.

This piece is a part of our Customer Convos series. We’re sharing stories of how people use npm at work. Want to share your thoughts? Drop us a line.
Q: Hi! Can you state your name and what you do?
A: Hi! I’m Jan and I’m an iOS developer at Clue.
How’s your day going?
Chilling with my cat, so purrrrrretttty good.
Tell me the story of npm at your company.
Our products are two mobile apps for iOS and Android. We write some logic in JavaScript so that we don’t have to do it twice on both platforms and can share it. Using a real package manager to handle that instead of someone going from time to time “hey, we should maybe update the JS in the apps, huh?” is pretty nice.
Can you tell us a story about a specific package you wanted to make that private packages really enabled you to do?
Some of the core logic of our app would be really error-prone and tricky to re-write for each of our platforms — but it has a bunch of proprietary logic, so its being private was a must-have.
Does your company do open source? How do you negotiate what you keep private and public?
We do have a GitHub org and few repos up there with little helper things that we’ve built over the years, but it’s not an important part of our work. We’ve recently been talking internally about carving out bits and pieces that would be useful in broader contexts and open-sourcing those, but nothing concrete yet.
To people who are unsure what they could use private packages for, how would you explain the use case?
By making analogy to GitHub private orgs/repos. You know how your source code is in a private repo? Well, the build artifacts of your JS library can be, too!
How’s the day-to-day experience of using private packages?
Pretty seamless! I have few nitpicks about the web interface (getting to the private package takes way too many clicks, and I’d love to see a version history), but otherwise I can’t say I’ve noticed any problems.
Oh! there was an issue earlier this year when the person who set up the org left the company. I remember people complaining about the process of transferring the ownership being a PITA, but I wasn’t super involved with that, so I don’t really remember the specifics…
Editor’s note: We’re always happy to help! If you have any issues, please reach out to support at [email protected].
Would you recommend that another org or company use private packages or orgs? Why?
Yes. “Please stop copy-pasting files between repos.”
Any questions I didn’t ask that you wish I did?
Nyup, I think you got it covered.
Any cool npm stuff your company has done publicly that you’d like to promote?
Sadly not…
Here’s another small big release, with a handful bunch of fixes and a couple of small new features! This release has been incubating rather longer than usual and it’s grown quite a bit in that time. I’m also excited to say that it has contributions from 27 different folks, which is a new record for us. Our previous record was 5.1.0 at 21. Before that the record had been held by 1.3.16 since December of 2013.

If you can’t get enough of the bleeding edge, I encourage you to check out our canary release of npm. Get it with npm install -g npmc. It’s going to be seeing some exciting stuff in the next couple of weeks, starting with a rewriten npm dedupe, but moving on to… well, you’ll just have to wait and find out.
d080379f6 [email protected] Updates extract to use tar@4, which is much faster than the older tar@2. It reduces install times by as much as 10%. (@zkat)4cd6a1774 0195c0a8c #16804 [email protected] Update publish to use tar@4. tar@4 brings many advantages over tar@2: It’s faster, better tested and easier to work with. It also produces exactly the same byte-for-byte output when producing tarballs from the same set of files. This will have some nice carry on effects for things like caching builds from git. And finally, last but certainly not least, upgrading to it also let’s us finally eliminate fstream—if you know what that is you’ll know why we’re so relieved. (@isaacs)1ac470dd2 #10382 If you make a typo when writing a command now, npm will print a brief “did you mean…” message with some possible alternatives to what you meant. (@watilde)20c46228d #12356 When running lifecycle scripts, INIT_CWD will now contain the original working directory that npm was executed from. Remember that you can use npm run-script even if you’re not inside your package root directory! (@MichaelQQ)be91e1726 4e7c41f4a [email protected]: Fixes a number of issues on Windows and adds support for several more languages: Korean, Norwegian (bokmål and nynorsk), Ukrainian, Serbian, Bahasa Indonesia, Polish, Dutch and Arabic. (@zkat)2dec601c6 #17142 Add the new commit-hooks option to npm version so that you can disable commit hooks when committing the version bump. (@faazshift)bde151902 #14461 Make output from npm ping clear as to its success or failure. (@legodude17)b6d5549d2 #17844 Make package-lock.json sorting locale-agnostic. Previously, sorting would vary by locale, due to using localeCompare for key sorting. This’ll give you a little package-lock.json churn as it reshuffles things, sorry! (@LotharSee)44b98b9dd #17919 Fix a crash where npm prune --production would fail while removing .bin. (@fasterthanlime)c3d1d3ba8 #17816 Fail more smoothly when attempting to install an invalid package name. (@SamuelMarks)55ac2fca8 #12784 Guard against stack overflows when marking packages as failed. (@vtravieso)597cc0e4b #15087 Stop outputting progressbars or using color on dumb terminals. (@iarna)7a7710ba7 #15088 Don’t exclude modules that are both dev & prod when using npm ls --production. (@iarna)867df2b02 #18164 Only do multiple procs on OSX for now. We’ve seen a handful of issues relating to this in Docker and in on Windows with antivirus. (@zkat)23540af7b #18117 Some package managers would write spaces to the _from field in package.json’s in the form of name @spec. This was causing npm to fail to interpret them. We now handle that correctly and doubly make sure we don’t do that ourselves. (@IgorNadj)0ef320cb4 #16634 Convert any bin script with a shbang a the start to Unix line-endings. (These sorts of scripts are not compatible with Windows line-endings even on Windows.) (@ScottFreeCode)71191ca22 #16476 [email protected] Running an install with --ignore-scripts was resulting in the the package object being mutated to have the lifecycle scripts removed from it and that in turn was being written out to disk, causing further problems. This fixes that: No more mutation, no more unexpected changes. (@addaleax)459fa9d51 npm/read-package-json#74 #17802 [email protected] Use unix-style slashes for generated bin entries, which lets them be cross platform even when produced on Windows. (@iarna)5ec72ab5b #18229 Make install.sh find nodejs on debian. (@cebe)b019680db #10846 Remind users that they have to install missing peerDependencies manually. (@ryanflorence)3aee5986a #17898 Minor punctuation fixes to the README. (@AndersDJohnson)e0d0a7e1d #17832 Fix grammar, format, and spelling in documentation for run-script. (@simonua)3fd6a5f2f #17897 Add more info about using files with npm pack/npm publish. (@davidjgoss)f00cdc6eb #17785 Add a note about filenames for certificates on Windows, which use a different extension and file type. (@lgp1985)0cea6f974 #18022 Clarify usage for the files field in package.json. (@xcambar)a0fdd1571 #15234 Clarify the behavior of the files array in the package-json docs. (@jbcpollak)cecd6aa5d #18137 Clarify interaction between npmignore and files in package.json. (@supertong)6b8972039 #18044 Corrected the typo in package-locks docs. (@vikramnr)6e012924f #17667 Fix description of package.json in npm-scripts docs. (@tripu)48d84171a f60b05d63 [email protected] Perf improvements. (@zkat)f4650b5d4 [email protected]: Serialize writes to the same file so that results are deterministic. Cleanup tempfiles when process is interrupted or killed. (@ferm10n) (@iarna)96d78df98 80e2f4960 4f49f687b 07d2296b1 a267ab430 #18176 #18025 Move the lifecycle code out of npm into a separate library, npm-lifecycle. Shh, I didn’t tell you this, but this portends to some pretty cool stuff to come very soon now. (@mikesherov)0933c7eaf #18025 Force Travis to use Precise instead of Trusty. We have issues with our couchdb setup and Trusty. =/ (@mikesherov)afb086230 #18138 Fix typos in files-and-ignores test. (@supertong)3e6d11cde #18175 Update dependencies to eliminate transitive dependencies with the WTFPL license, which some more serious corporate lawyery types aren’t super comfortable with. (@zkat)ee4c9bd8a #16474 The tests in test/tap/lifecycle-signal.js, as well as the features they are testing, are partially broken. This moves them from being skipped in CI to being disabled only for certain platforms. In particular, because npm spawns its lifecycle scripts in a shell, signals are not necessarily forwarded by the shell and won’t cause scripts to exit; also, shells may report the signal they receive using their exit status, rather than terminating themselves with a signal. (@addaleax)9462e5d9c #16547 Remove unused file: bin/read-package-json.js (@metux)0756d687d #16550 The build tools for the documentation need to be built/installed before the documents, even with parallel builds. Make has a simple mechanism which was made exactly for that: target dependencies. (@metux)
Recently, there’s been some buzz around the next great architectural shift in systems. There is a rising interest in the evolution of decentralized edge computing as a core part of that shift.
For over two years, npm has been using edge computing concepts to ensure that the developer experience for users of npm Enterprise, our private registry product, matches the experience of using the centralized, cloud-hosted version of the npm Registry.
Here’s why we’re doing that, and how:
Many enterprises have strict requirements that prevent them from using cloud-hosted products for critical parts of their infrastructure. This approach makes sense from a regulatory compliance perspective, but it makes life inconvenient for developers within those companies who wish to take advantage of open-source code from the npm Registry, or who wish to use npm to share and reuse their own code with their colleagues.

npm Enterprise allows developers at big companies to run a version of the npm Registry behind their firewall. Of course, it wouldn’t be enough for enterprise customers to simply deploy a fresh install of npm Enterprise with an empty registry. Much of npm’s value comes from the 500,000 packages available on the public registry, and being able to combine these packages with private code. Without access to these packages, developers would waste time reinventing a lot of wheels.
npm Enterprise lets companies mix public packages from the public registry with private code stored within their private registry without risks or complexity.
We designed npmE so that each npmE server is a private edge node to the public registry. Each npmE instance can replicate select parts or all of the npm Registry to offer this functionality to end users. It also provides additional local services that are only accessible to these users, based on a company’s unique requirements.

Our customers are able to configure their private registry as a full mirror of the public Registry to decrease latency, cut bandwidth costs, and offer npm Registry to end users who are restricted from accessing the public internet. Alternately, they may selectively mirror npm’s Registry using specific whitelists managed by the admin console or the npm command line.
When combined with npmE add-ons which enforce code quality, evaluate how packages and their dependencies are licensed, and scan for security vulnerabilities, this architecture gives companies total control of the public packages their developers may use.
At the same time, these developers may find, share, and re-use proprietary code by publishing to their private local registry. Private code stays private by never leaving the company’s own infrastructure.
End users don’t have to think about where each package is located; they can just pull from the npm Enterprise server. Behind the scenes, the server automatically determines the scope and proxy of the package pull.

There are two primary challenges of an edge node architecture:
For many years, deploying and maintaining a private instance of this kind of architecture would have been prohibitively difficult for everyone but the most advanced IT organizations. These sorts of enterprise software installations took several months to implement and involved manual processes of configuring servers, runtimes, and components. Every enterprise IT org would have taken responsibility for the ops role of their enterprise instance.
Fortunately, it’s now much easier to deploy and maintain private edge nodes thanks to technologies like containerization, orchestration and scheduling platforms.
Deployment and management are now baked into the design and development effort of modern applications. Generally, this creates reproducible and consistent cloud-native deployments, but it also is becoming the foundation of modern enterprise software deployments. This automation and inherent portability allows our customers to deploy into their own environments without deep knowledge of our architecture.
Of course, it isn’t quite as easy and magical as it all sounds. We initially built out our own containerized installation methods by packing all of our services into a single container. This approach still required npm Enterprise customers to be quite technical to complete the deployment. The system also lacked the tooling for managing versions, customers, backups, and updates.
After a few months of banging our heads against the wall, we decided that we were dedicating too many resources to deploying and managing of Enterprise instances. We switched over to a platform called Replicated to be our enterprise version management platform. Replicated’s platform provides workflows for integrating our existing CI/CD release pipeline with our enterprise release channels.
Similar to the way that npm packages are versioned for automatic updates, we use discrete versioned images for each of the services that make up npm Enterprise. We organize these into specific channels provided — “stable,” “beta”, and “unstable” — and when we promote a set of images to “stable,” Replicated automatically notifies our customers that an update to npm Enterprise is available and makes the update as simple as a single click. Our customers don’t have to manually update services, and we don’t have to manually push containers around in order to keep the edge nodes of the npm Registry on recent versions.

Beyond deployment and management, there also was the problem of developing enterprise-specific features such as change management processes, LDAP integration, and an admin dashboard, which enterprise customers require but which fall outside our core product expertise. Many of these features are included (or at least made easier) in the Replicated platform and provide a consistent experience that enterprise IT admins are now familiar with.
These sorts of enterprise ready features are important to our Enterprise customers, but since they aren’t a core part of our value proposition, it has made a ton of sense for us to leverage a partner to power these as much as possible.
The state of art in edge node architecture is still evolving, but it is gaining more traction in a variety of use cases. An increasing quantity of JS developers rely on npm, and as a result, an increasing number of enterprises will need npm Enterprise. For developers to be effective it’s imperative that they benefit from the global Registry of npm packages.
By partnering with Replicated to pioneer an architecture that delivers on that promise while reducing management overhead and satisfying security requirements, we can see an emerging future that embraces the distributed nature of the internet. To learn more about Replicated’s technology, visit their site.
npm’s core and enterprise offerings are constantly improving.To try out a fully private, enterprise edge-node instance of npm, just reach out or download a free trial today.
Editor’s note: This is a guest post from Jenn Schiffer, who originally posted it on her blog extremely online and incredibly logged on (you can see the original post here). We feel 100% in agreement about the importance of an inclusive, collaborative, and supportive community and code ecosystem.
I was having lunch the other day with a very cool local dev evangelist and among the many interesting topics we covered in the dev rel world, the one that stuck out was how developers tend to over-worry about whether their contributions to a code ecosystem are actually valuable. This comes up a lot when people ask me if it’s “okay” to post a static site on Glitch or use it to save code snippets or prototype throwaway things (yes!).
When I say “code ecosystem” I mean places like the npm registry, GitHub, and even Glitch. and when I say that developers worry about their contributions there, i mean that we have an actual problem where we are:
I’ve had many debates with past coworkers, pals, and collaborators about whether a node module that returns the number of seconds in a minute belongs in npm (sure!) or if it’s okay to have a GitHub repo that only contains a readme file listing one’s favorite pizzerias (definitely!) and I find it hard to believe that it’s the ecosystem that detractors of this kind of use are worried about. Honestly, I think it has to do with their idea of the value of repo and module count and contributions.
The open source community has a big problem with projecting our insecurities about our own metrics onto other developers and it’s a bad cycle that makes people of all levels of experience worry about whether their contributions are some fake idea of “worthy” or not. It’s been used as a tool of harm against women in the community (“she hardly has any green on her GitHub”) which in part is harm against the entire community and our code ecosystems.
As developers we need to stop giving weight to these metrics and chill tf out when it comes to judging our peers. As ecosystem owners, we need to better moderate and have more discussions with the community about what we expect of our users with regards to both social behavior and code contributions. I don’t want people judging Glitch users on how many Glitch projects they have or if they are mostly “just remixes,” much like I witness with GitHub.
We call Glitch “the friendly community where you’ll build the app of your dreams” — and that dream app can be a static site, or a markdown file listing your favorite pizzerias - hey, it may inspire you to learn how to evolve it into a map of those places using the google maps api, or not! All of our dreams are different and so all of our contributions to Glitch will be different, and I think that is a big part of what will keep Glitch a friendly community.
On August 1, a user notified us via Twitter that a package with a name very similar to the popular cross-env package was sending environment variables from its installation context out to npm.hacktask.net. We investigated this report immediately and took action to remove the package. Further investigation led us to remove about 40 packages in total.
On July 19 a user named hacktask published a number of packages with names very similar to some popular npm packages. We refer to this practice as “typo-squatting”. In the past, it’s been mostly accidental. In a few cases we’ve seen deliberate typo-squatting by authors of libraries that compete with existing packages. This time, the package naming was both deliberate and malicious—the intent was to collect useful data from tricked users.
All of hacktask’s packages have been removed from the npm registry.
Adam Baldwin of Lift Security also looked into this incident to see if there were any other packages, not owned by hacktask, with the same package setup code. He has every file in the public registry indexed by content hash to make scans like this possible. He did not find any other instances of that specific file with those contents.
Following is a list of hacktask’s packages, with a count of total downloads from 7/19 to 7/31.
Download counts for these packages are larger in the last two days because of public interest in the problem. The numbers from before exposure are more revealing of the effect of the malware. Note that 30-40 downloads is typical for any public package published to the registry, from registry mirrors automatically downloading copies. From this you can see that the real danger came from the crossenv package, which had nearly 700 downloads, with some secondary exposure from the jquery typosquats. But even in that case, most of the downloads come from mirrors requesting copies of the 16 versions of crossenv published. Our estimate is that there were at most 50 real installations of crossenv, probably fewer.
babelcli: 42 cross-env.js: 43 crossenv: 679 d3.js: 72 fabric-js: 46 ffmepg: 44 gruntcli: 67 http-proxy.js: 41 jquery.js: 136 mariadb: 92 mongose: 196 mssql-node: 46 mssql.js: 48 mysqljs: 77 node-fabric: 87 node-opencv: 94 node-opensl: 40 node-openssl: 29 node-sqlite: 61 node-tkinter: 39 nodecaffe: 40 nodefabric: 44 nodeffmpeg: 39 nodemailer-js: 40 nodemailer.js: 39 nodemssql: 44 noderequest: 40 nodesass: 66 nodesqlite: 45 opencv.js: 40 openssl.js: 43 proxy.js: 43 shadowsock: 40 smb: 40 sqlite.js: 48 sqliter: 45 sqlserver: 50 tkinter: 45
If you downloaded and installed any of these packages, you should immediately revoke and replace any credentials you might have had in your shell environment.
hacktask’s email address is banned from using npm. In this era of throwaway email addresses, that is not sufficient to prevent the human being behind it from trying again, but we felt it was a necessary gesture.
We are supporting ^Lift Security and the Node Security Project in their ongoing work to do static analysis of public registry packages, but this will not find every problem. Determining if a package contains malicious content when published is, of course, equivalent to the halting problem and therefore not something we can do.
We’re discussing various approaches to detecting and preventing publication—either accidental or malicious—of packages with names very close to existing packages. There are programmatic ways to detect this, and we might use them to block publication. We’re using the Smyte service to detect spam as it is published to the registry, and will be experimenting with using it to detect other kinds of violations of our terms of service.
Please do reach out to us immediately if you find malware on the registry. The best way to do so is by sending email to [email protected]. We will act to clean up the problem and find related problems if we can.

You probably know ^Lift Security for its work as the Node Security Project, which reviews the most popular of the half-million packages in the npm Registry to find security vulnerabilities. However, you might not know that ^Lift also reviews the npm registry itself.
Since npm was founded, we have worked with Adam Baldwin and his team to conduct periodic security reviews of the code that we use to power the world’s largest software registry. Their methods include penetration tests and code audits of the contents of all private packages and of all of npm’s operations.
We work with ^Lift’s engineers to get a much-needed second opinion about our work. They clearly explain tradeoffs and priorities in their code reviews and give us actionable suggestions for mitigating risks.
Earlier this week, ^Lift completed another penetration test of the registry and I am currently reviewing their report of what they found. As always, they’ve shown us that we have things to do to tighten up our operations, including using HSTS and changing the ways some of our APIs operate.
In npm, Inc.’s three and a half years of operations, there has never been an incident in which a stranger exploited a vulnerability to steal user credentials — but our work to improve security is never done. Every change to a system can have security implications, and we’re constantly working on a lot of things! Every npm user and package on the npm registry is safer because of ^Lift’s reviews.
“We’re dedicated to creating good processes so that a solid foundation exists for new features to help users protect their accounts and the software they publish such as two factor authentication or package signing,” Adam said.
“Security is always at odds when it comes to other business objectives. From ^Lift’s perspective, npm does a great job prioritizing security when necessary and also taking it seriously. We don’t have to go to extreme lengths to convince them when something is an issue. npm takes the time to understand and make it a part of its overall business plans.”
npm will always favor transparency — we’re happy to describe, in specific detail, our security processes and policies, including how you can help. Take a look, share your feedback, and watch this space. We’ll be sharing the security processes and features we add in order to keep the npm community in the loop.
The #npm channel on irc.freenode.net is being devoiced. That means: if you’re not a moderator in the channel, you won’t be able to post there. Instead, you’ll be redirected to this message.
As an official communication channel, IRC is difficult for us to adequately moderate and maintain. A welcoming and compassionate npm community is important to us, and as a small team, moderating IRC effectively has proven very difficult. Since #npm is probably always going to be seen as an “official” npm venue, we cannot in good conscience continue to let it fall into disrepair.
Here are some much better places to seek help, guidance, and support, which are actively moderated and maintained, either by us or by others.
#Node.js channel on irc.freenode.net is a good place to go for general purpose Node.js or JavaScript chat. And if you create an unofficial #npm channel, we won’t stop you, but we also can’t commit to helping you moderate it.