the npm blog

tl;dr - Update to npm v6.13.4 as soon as possible on all your systems to fix a vulnerability allowing arbitrary path access.

The Vulnerabilities

In versions of npm prior to 6.13.3 (and versions of yarn prior to 1.21.1), a properly constructed entry in the package.json bin field would allow a package publisher to modify and/or gain access to arbitrary files on a user’s system when the package is installed.

In versions of npm prior to 6.13.4 (and all versions of yarn as of this announcement), it was possible for a globally-installed package with a binary entry to overwrite an existing binary in the target install location.  (That is, not any arbitrary file on the system, but any file in /usr/local/bin.)

A mitigating factor for both vulnerabilities is that a malicious actor would have to get their victim to install the package with the specially crafted bin entry.  However, as we have seen in the past, this is not an insurmountable barrier.

Current Risk

The npm, Inc. security team has been scanning the registry for examples of this attack, and have not found any published packages in the registry with this exploit.  That does not guarantee that it hasn’t been used, but it does mean that it isn’t currently being used in published packages on the registry.

We will continue monitoring, and will take action to prevent any bad actors from exploiting this vulnerability in the future.  However, we cannot scan all possible sources of npm packages (private registries, mirrors, git repositories, etc.), so it is important to update as soon as possible.

The Fix

The package.json parsing libraries in use in npm v6.13.3 were updated such that they would sanitize and validate all entries in the bin field to remove leading slashes, . and .. path entries, and other means of path escape, using the well tested and highly reliable path utility built into Node.js.

The fix was reviewed by npm, Inc.’s security team, and showed that it prevented the arbitrary path manipulation reported.

The bin script linking libraries in use in npm v6.13.4 were updated such that, when installing binary entries of top-level globally installed packages, they will only overwrite existing binary files if they are currently installed on behalf of the same package being installed.  For example, npm install –global foo could overwrite /usr/local/bin/foo if and only if /usr/local/bin/foo is currently a link to a previously installed version of foo.

This second fix was also reviewed by npm, Inc.’s security team, and showed that it prevented the bin file overwriting exploit.

To patch both vulnerabilities, please run npm install -g [email protected] as soon as possible.

Thank you!

The vulnerabilities were responsibly reported by Daniel Ruf, and we greatly appreciate Daniel’s help in keeping our community safe.

If you think you’ve found a security issue in the npm CLI, we encourage you to report it to [email protected] and we’ll help you through the triage and disclosure process.

Second of two important bugfix releases this week. Please enjoy it!

npm install -g npm@latest

6.13.4 (2019-12-11)

BUGFIXES

• 320ac9aee npm/bin-links#12 npm/gentle-fs#7 Do not remove global bin/man links inappropriately (@isaacs)

DEPENDENCIES

• 52fd21061 [email protected] (@isaacs)
• d06f5c0b0 [email protected] (@isaacs)

We’re pleased to announce today the launch of npm Pro, an affordable new tool designed for independent JavaScript developers, and well-suited for consultant work, personal projects, and side hustles. We’re also announcing npm Teams, the new name for the product formerly known as npm Orgs (the feature set remains the same). And finally, we’re seizing this opportunity to talk a bit about our product landscape, and where that’s headed next. Read on to learn more.

Introducing npm Pro and npm Teams

Today’s launch of npm Pro is intended to bring independent developers – whether full-time consultants, part-time developers, or anyone working on a side project – a product clearly made for them. Key features of npm Pro include unlimited public packages, unlimited private packages, and the ability to share ownership of packages with other Pro developers. In addition, Pro inherits some key npm Free functionality and features with: you’ll get basic support and npm Audit (which alerts you to unsafe code), and your dedicated namespace is the same as  your user name. At $7 a month, it’s an affordable option for anyone who needs private packages, and works mostly solo or with a small number of collaborators, but for whom team-based packages and permission sets would be overkill. 

Along with rolling out this new product, we’re renaming an existing one: What was npm Orgs is now called npm Teams. The focus of this product today, and of our plans for its future, is on team development and collaboration, and this new name better reflects that. It shares two core features with npm Pro – unlimited public and private packages – and it also inherits the benefits of npm Free, but it features team-based management of packages and permissions. In addition, it allows you to create a namespace distinct from user name, so the team project is not permanently tied to any one user. 

Our future plans for both Pro and Teams include additional features and functionality specifically designed for each, and you can expect to see us rolling those out over the next year and beyond. (Something on your wishlist? Let us know.)

The npm Product Landscape

Philosophically, we’re fans of the Unix principle that favors small, sharp tools designed for specific purposes, and today’s announcements are about being a bit truer to that philosophy. Admittedly, calling them small might be a stretch, but with Pro and Teams now distinct products, we have the opportunity for further sharpening – two different paths for feature development, each aimed at a different user and set of use cases. And together with npm Free, our most basic product, and npm Enterprise, our tool designed for very large teams at very large companies, we’re closer to our vision of a product line capable of serving the full breadth of the community. Stay tuned though; we’re busily at work on all of the above, as well as some new products in the general area of security and compliance, and we will have more to announce soon.

Finally – and of course – these products sit alongside, and help pay for, the two products you probably know best: the npm Registry and the npm CLI. While npm, Inc. is a regular company like any other, we pay all of the costs (staff, maintenance, and operation) of operating the Registry and building and delivering the CLI, but make it available to the community for free. That’s a labor of love for every one of us here, so we hope you continue to find them valuable as we continue to improve them. But we hope you will consider trying our paid products npm Pro and npm Teams as well.

Just a quick little release to update some deps and fix a few very annoying bugs.

Come and get it!

npm i -g npm@latest

Full release notes:

6.13.3 (2019-12-09)

DEPENDENCIES

• 19ce061a2 [email protected] Properly normalize, sanitize, and verify bin entries in package.json.
• 59c836aae [email protected]
• fb4ecd7d2 [email protected]
  • 5f33040 #476 npm/pacote#22 npm/pacote#14 fix: Do not drop perms in git when not root (isaacs, @darcyclarke)
  • 6f229f7 sanitize and normalize package bin field (isaacs)
• 1743cb339 [email protected]

A new npm version has been released!

Get it in the usual ways:

npm i -g npm@latest

6.13.2 (2019-12-03)

BUG FIXES

• 4429645b3 #546 fix docs target typo (@richardlau)
• 867642942 #142 fix(packageRelativePath): fix ‘where’ for file deps (@larsgw)
• d480f2c17 #527 Revert “windows: Add preliminary WSL support for npm and npx” (@craigloewen-msft)
• e4b97962e #504 remove unnecessary package.json read when reading shrinkwrap (@Lighting-Jack)
• 1c65d26ac #501 fix(fund): open url for string shorthand (@ruyadorno)
• ae7afe565 #263 Don’t log error message if git tagging is disabled (@woppa684)
• 4c1b16f6a #182 Warn the user that it is uninstalling npm-install (@Hoidberg)

From November 21-25, the npm registry experienced periodic service degradation. Alerted to increasing error rates from our monitoring systems and reports from the npm community, our incident response team began investigations on Thursday and has since identified the root cause and implemented mitigations that have stabilized registry service.

Root Cause 

Starting Thursday, an npm user launched a bot to aggressively crawl the registry––against our terms of service––querying a mix of existing and non-existing packages. One of the most important security aspects of the registry is to ensure that we do not acknowledge the existence or non-existence of private packages to unauthorized users. In order to do that while also making use of caching by our content delivery network, we have engineered systems that are capable of returning the required authorization information quickly and accurately.

The behavior by this user put undue strain on the systems that are used to authenticate whether a user may identify if a package exists, retrieve, and cache it for future retrievals.

Timeline

Nov 21, 18:20 UTC –– aggressive bot queries begin. The npm incident response team begins investigation and blocks bot activity while exploring long-term options. System load dissipates.

Nov 22, 19:50 UTC –– npm registry fully stable.

Nov 25, 01:20 UTC –– load spike on npm services returns.

Nov. 25, 12:18 UTC –– Further CDN updates deployed.

Nov. 25, 12:38 UTC –– npm registry fully stable.

Retrospective

As we have discussed here before, the npm public registry, like the JavaScript ecosystem as a whole, is experiencing exponential growth. Open systems like the registry require a basic level of mutual trust between the users and the operators of that system. Our terms of service are crafted to bolster that trust and to ensure quality of service to the entire JavaScript community. 

The npm operations team works continually to maintain quality of service and to enforce those terms of service.

This is the third in a series of blog posts we’re running to preview and gather input on the new security insights API we’re developing. 

Previous posts

• Part 1: Package publication Insights
• Part 2: Malware

Today’s topic: Behavioral Analysis

A lot of stuff happens when you install an npm package. npm downloads and extracts dependencies, but it also runs install hooks. These install hooks can have all kinds of side effects from compiling binaries, downloading other dependencies or maybe something a bit more malicious in nature. Post-install scripts are the most popular malware infection method right now. 

In an effort to understand this further and to make side effects (malicious or not) transparent, the npm security team has been hard at work building infrastructure to do behavioral analysis of npm packages at scale.

What is behavioral analysis?

For every package published to the registry, we run the package installation process inside a controlled and instrumented environment. Anything that goes in or out, gets added or deleted is captured and stored. This type of analysis is especially useful in situations with obfuscated source code or binary artifacts where reverse engineering or static analysis is particularly difficult. It does however product a mountain of data–data that you’ll be able to sift through effectively with the security insights API

What will the security insights API have available?

We’re going to initially launch with three pieces of behavioral metadata exposed. Through this analysis, we will have a lot more data than you are ever going to want to sift through. To make it useful, we’re going to pare back what we’re going to make initially available and, as our capabilities grow, we’ll continue to expose more data.

The initial release of the API will likely contain the following info from our behavioral analysis on a package:

• DNS requests (hosts contacted / looked up)
• processes executed (including command lines and arguments)
• network traffic summary

Real world example: cryptocurrency malware

Let’s take a look at a simple cryptocurrency malware sample. This sample runs coin mining software on install similar to what we have seen in the past.

Figure 1: Request

Figure 2: Response

In this example, we’re going to query for the answers returned by the dns requests made. It very obviously shows a site that is out of place for a typical npm install.

Figure 3: Request

Figure 4: Response

Investigating further by requesting the command line and its arguments for this package shows a suspicious binary being executed, extremely useful information when trying to understand what might be going on with a package without installing or running it yourself.

How would you use it?

How cool would it be to know if a package is making insecure network requests before you install it or be able to diff two versions to see what might have changed?

We’ve already seen a lot of interesting ideas from the previous insights we’ve shared. Given this new information we’d love your input on what you would like to see in the npm security insights API and what you would build with it. Sign up for the private beta and let us know ›

A new npm version has been released! This fixes some bugs and includes changes on the docs by the community!

Get it in the usual ways:

npm i -g npm@latest

6.13.1 (2019-11-18)

BUG FIXES

• 938d6124d #472 fix(fund): support funding string shorthand (@ruyadorno)
• b49c5535b #471 should not publish tap-snapshot folder (@ruyadorno)
• 3471d5200 #253 Add preliminary WSL support for npm and npx (@infinnie)
• 3ef295f23 #486 print quick audit report for human output (@isaacs)

TESTING

• dbbf977ac #278 added workflow to trigger and run benchmarks (@mikemimik)
• b4f5e3825 #457 feat(docs): adding tests and updating docs to reflect changes in registry teams API. (@nomadtechie)
• 454c7dd60 #456 fix git configs for git 2.23 and above (@isaacs)

DOCUMENTATION

• b8c1576a4 30b013ae8 26c1b2ef6 9f943a765 c0346b158 8e09d5ad6 4a2f551ee 87d67258c 5c3b32722 b150eaeff 7555a743c b89423e2f #463 #285 #268 #232 #485 #453 docs cleanup: typos, styling and content (@claudiahdz) (@XhmikosR) (@mugli) (@brettz9) (@mkotsollaris)

DEPENDENCIES

• 661d86cd2 [email protected] (@claudiahdz)

npm developer Ruy Adorno writes about his experience cutting his first npm release and two new features available in the CLI ›

https://dev.to/ruyadorno/npm-6-13-0-7f3

Happy Tuesday! Here on the Community & Open Source Team we’ve been working hard, in front of and behind the scenes, to provide real value and unlock developer potential. With that in mind, I’m happy to announce a number of updates/releases we’ve landed this morning & would like to detail.

Who We Are?

First off, you may have seen some new and old faces in the wild recently, so I thought I’d take a moment to reacquaint you with who our Community & Open Source Team is:

• Isaac Schlueter - Chief Open Source Technology Officer @isaacs
• Claudia Hernández - Sr. Software Engineer @claudiahdz
• Michael Perrotte - Software Engineer - @mikemimik
• Ruy Adorno - Software Engineer @ruyadorno
• Darcy Clarke - Engineering Manager @darcyclarke

Open Source Product Development

As the stewards of the Open Source practice at npm we’ve begun a process to revamp how we plan, prioritize & communicate the development of tools & services supporting our community.

🧹 Clean Up

We began cleaning up our existing repositories back in August but the efforts will continue into Q4 to address licensing, docs, test coverage, archival of legacy work & more. With 285 Public, “active”, repositories, we’ve got our work ahead of us but are confident we can make contributing to any of our projects a seamless experience going forward.

⚙️ Standardization

Part of cleaning up our projects has been to standardize much our project config. We’ve taken the first steps to documenting our default labels and other config within a open-source-boilerplate-project and .github projects. The implementation of these settings is currently handled by Probot’s Settings app & GitHub’s native capabilities, respectively (both of which we’re learning to love).

🗣 Open RFC Meetings

In early September we began running a bi-weekly Open RFC call and have recently started to stream these calls on our YouTube Channel. We’ve opened a Poll to determine the best meeting days/times and hope you’ll give us your feedback on how we can continue to improve these discussions.

📅 Events Calendar

If you want to ensure you don’t miss one of the Open RFC calls, or are interested in the various meetups, conferences and events our team is attending, you can subscribe to our new Public Events Calendar.

✅ Status & Project Boards

Inspired by the work of Wes Todd on Express’ statusboard we’ve created a high-level project statusboard to keep track of the health of our various projects. This repository has doubled as our temporary, internal, issue tracker but will soon evolve into a critical tool for keeping an eye on the state of open source at npm.

As well, as of this morning, we’ve made public our GitHub Project board where you can see the status of the various initiatives we have in-progress day to day and week to week.

ℹ️ Archiving npm.community

Last year we announced we were closing issues on npm/cli & pointing users to the npm community forums; While we’ve appreciated all of the support we saw with this change, the team strongly believes that we can better address feedback and interact with our users by being as close as possible to where our source code resides.

What this means?

• The Discourse forums have been set to read-only mode as of this morning
• We’ll be migrating references to outstanding #bugs topics into npm/cli/issues later today
• Going forward, reports for bugs or issues can be filed against npm/cli/issues or any one of our existing projects
• Feature requests, discussions and/or RFCs will continue to reside in npm/rfcs
• Requests for help, diagnostics and other support questions can be triaged via our new landing page that will live on npm.community, launching later today

Over the last year we’ve been humbled by the community support and want to give a huge shout out to some of our most active community members:

• John Gee @shadowspawn with over 1,000+ posts 🎉 🙌🏻
• Lars Willighagen @larsgw with almost 500 posts 🎉 🙌🏻
• Markus Tacker @coderbyheart
• Godmar @godmar
• Joshua @legodude17
• Daniel Stockman @evocateur
• Brian Olore @olore
• Steven @styfle
• Matt Travi @travi
• Daniel Ruf @DanielRuf
• Artem Varaksa @arch
• Igor Pellegrini @Pictor13
• Chris Brody @brodybits
• Jacob Page @DullReferenceExcptn
• Jordan Harband @ljharb
• Beni Von Cheni @BeniCheni
• Simon Kurtz @simonua

🎉 Supporting Open Source Maintainers

As of npm v6.13 we’ve added support for a brand new funding field in package.json along with a corresponding npm fund sub-command.

Post install you will now see output that describes the number of packages that have defined funding information. You can opt-out of this prompt by using the --no-fund flag if you so choose.

At the end of August, we made a promise to the community to invest time & effort to better support package maintainers. This work is just the first, small step toward creating a means/mechanism for a more sustainable open source development ecosystem.

A big shout-out goes to all the collaborators/contributors to this on-going work:

• Node Foundation: Package Maintenance Working Group
• Feross Aboukhadijeh @feross (ie. thanks & funding projects)
• Kyle Mitchell @kemitchell (ie. sustainability & support contributions)

As noted back in August, we’d love to partner/collaborate with any interested parties that want to make a difference in this space. We’ll be looking to improve this funding experience and, overall, what we can offer the community in the months to come.

🎉 CLI Docs

As of npm v6.13 we’ve bundled a new look to the CLI docs. Running npm help <command> --viewer browser will give you an offline sneak peak at what is to come for our docs.npmjs.com site in the weeks ahead.

ℹ️ npm v7 Status Update

Quite a lot of work has gone into npm v7 over the last several months, but much of it has been under the hood, refactoring components which will allow us to deliver some long-awaited features.

⏭ Faster, Better, Streamier

pacote, cacache, make-fetch-happen, ssri, and npm-registry-fetch, have all been ported to use the latest minipass streams instead of the hodge podge of core and community streaming libraries they were using before. Using a deterministic streaming library has allowed us to track down and eliminate entire classes of race conditions that arose from using a mix of Promises and async streams.

As part of this, we’ve also released minipass-fetch, which is a port of node-fetch to minipass streams.

pacote, npm’s package fetcher, has been completely rewritten from the ground up. In the process, we were able to unlock the ability to fetch the state of a package from a GitHub pull request (ie. your code reviews just got a whole lot easier 🎉).

Examples:

npm install github:username/reponame#pull/123
# or...
npx username/reponame#pull/123

cacache, npm’s content-addressable cache layer, has also gotten a major overhaul, tracking down some long-standing bugs and race conditions in the process.

ssri, cacache, npm-registry-fetch, and pacote all now have 100% test coverage, and we’re making progress on make-fetch-happen at this very moment. This work will make it easier to fix bugs and add new features with confidence in the future while the improved streaming performance will make installs that much faster.

🗂 Better package.json#files handling

Once upon a time, in the dark days before node-tar, npm’s implementation of .npmignore files was to use tar’s --exclude option. That confused people, who expected it to work like .gitignore, and so it was made that way, first with fstream-ignore.

Some time later, we decided that an include list would be better than an exclude list, and so the package.json "files" list was born. The initial implementation was to put a ! in front of all the entries in that list, and treat it like a sort of “reverse ignore” file.

That has caused a lot of problems in the intervening time. And, it’s unnecessarily slow. npm-packlist version 2 (which npm v7 will use) will treat the files list as a set of files to walk, rather than a set of reverse exclusion patterns. Unless you’re doing something very tricky with .npmignore files and package.json "files", you won’t notice a difference, and it’ll just remove a pile of weird edge cases that resulted in .DS_Store files showing up in packages.

⏱ Dropping Support for Node v6 & v8

Starting with npm v7, only Node.js versions 10 and up will be supported. Node.js v8 falls off the LTS maintenance window as of January 1st, 2020. Since npm v7 won’t be out of beta until after that window has closed, it only makes sense to move past these unmaintained releases.

This change also allows us to migrate fully to native Promises, and clear away some workarounds and hacks that were cluttering up the codebase.

🌲 The Tree Doctor

All the code in the npm CLI that deals with package trees is being refactored (and in many cases, entirely reimplemented) in a new module called arborist.

This will make it possible to easily add workspaces, properly handle peer dependencies, and behave as users expect with dependency trees created by both yarn and pnpm.

📦 Installing Peer Deps By Default

In one of our more controversial RFCs, @isaacs suggested that we start installing peer deps again. This would get rid of some error messages that persistently annoy users, especially newcomers to React and other ecosystems where peer dependencies are used heavily.

As Maël Nison and a few others have pointed out, this could be fairly disruptive, as many users have adapted to the peerDependencies “not installed by default” behavior.

npm’s current behavior, where it ignores peerDependencies entirely at tree building time, is a problem. Even if we back off from installing peer deps by default, we’ll still have to construct trees mindful of the conflicts that peer dependencies can create, and enable users to install peerDependencies with an opt-in command, so the implementation will need to be there regardless.

We expect to explore this more fully during the beta period, and either ratify npm/rfcs#43 or modify it based on what we learn.

📅 Beta Release Schedule

We hope to have arborist functionally complete in the next few weeks, with a preview of the full CLI release by the end of the year. After that, we’ll be bugging you all to try it out, with the expectation that we can promote npm v7 to latest after a few months of baking.

You’ll be the first to know when the beta launches and we hope you’ll help us vet all this hard work by trying it out and giving us your feedback as we continue to improve 🙂

Keep your eyes peeled for more updates and releases from our team in the weeks to come.