Package management is an integral part of programming with Node.js. More than just a program, the Node package manager (npm) is an entire ecosystem. In this tutorial, get familiar with npm and all of its components:

The npm registry of software packages.
The npm command line interface (CLI) program used to manage those packages.
The npm website, which you can use to get help and to get involved.

Get the code

The code you need to follow along with the examples in this tutorial is in my GitHub repo.

Get the code

The npm registry

The npm registry contains hundreds of thousands of open source Node packages, which are backed by a large and engaged developer community.

You probably inferred from its frequent appearance in "Explore Node.js basic concepts" and "Introduction to the event loop in Node.js" that the npm registry is the place to go for modules to use in your Node applications.

It's one thing to locate and install a package when you know its name, but let's say you're looking for a kind of software and not a particular package. How would you find what you need in the registry?

As an example, suppose you're looking for a JavaScript linter. To find one of the many linters in the npm registry, go to the npm website. Type "linter" in the search field at the top of the page, and you'll see the following result:

Figure 1. Searching npmjs.com for 'linter' Searching npmjs.com for `linter`.

That's a lot of choices! With more than 3,000 packages matching the keyword "linter," how will you decide which one to use?

For a start, you can narrow the search a bit. JavaScript is an ECMAScript implementation, so let's see what happens after adding "ecmascript" to the search.

Figure 2. Searching npmjs.com for 'ecmascript linter' Searching npmjs.com for `ecmascript linter` to narrow the search.

Okay, that's better. It's easier to sift through 50 packages than 3,000. But wouldn't it be helpful to narrow the search even further?

Package scores

The search capability for npmjs.com (provided by npms.io) offers 3 key metrics to help you:

Popularity
Quality
Maintenance

Every package in the npm registry is subject to the algorithm that computes these scores. The metrics mean pretty much what you think they should, but if you want a more detailed explanation, check out the About page at npms.io.

Scores are presented in a bar graph format at the top right corner of each entry using p for popularity, q for quality, and m for maintenance. Hover your mouse over each link and the associated numerical score should pop up. (I'm using Chrome; your mileage may vary.)

You can also search packages directly at npms.io, where the same search for ecmascript linter produced 53 results:

Figure 3. Searching npms.io for ecmascript linter Searching npms.io for ecmascript linters

In this example, let's say you choose the package with the highest average of quality, popularity, and maintenance. You end up with eslint, which scores 97, 85, and 100 (respectively) for an average of 94.

The npm CLI

Once you choose a package, you need to install it, which brings us to the npm command line interface (CLI).

The CLI is a program that acts as the package manager for your Node programs. It ships with Node.

The npm you know

You've already used npm several times in this learning path. In "Explore Node.js basic concepts," you used npm to install packages like nodemon:

npm install -g nodemon

You've also used npm to run server.js (via npm start), run functional tests (via npm test), and even run custom scripts from package.json (via npm run load-test and npm run start-dev).

Now we'll look more closely at npm's functionality and what you can do with it.

Install a local package with npm

In "Create your first Node.js application," you used npm to install the sqlite3 module, which was specified as a dependency in the package.json file for the shopping list application:

Listing 1. /Node.js/Course/Unit-6/package.json

{
  "name": "shopping-list",
  "version": "1.0.0",
  "description": "Shopping List",
 .
 .
 .
  "homepage": "https://github.com/jstevenperry/IBM-Code#readme",
  "dependencies": {
    "sqlite3": "^4.0.1"
  }
}

When npm is supplied with no arguments, it installs all modules listed in the dependencies object in package.json. (You'll learn more about package.json in "Node.js dependency management".)

Install a new set of packages

We'll use npm install with a different package.json than the one we used in the previous tutorial. This will install a new set of packages and dependencies.

To start, in my GitHub repo you will see the following package.json. Note that this file includes a pretest script to run the eslint JavaScript linter.

Listing 2. package.json with the eslint dependency

{
  "name": "Unit-7",
  "version": "1.0.0",
  "description": "Sample code for Unit 7",
  "main": "logger.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "pretest": "eslint --ignore-path ../../.gitignore ."
  },
  "keywords": [],
  "author": "J Steven Perry",
  "license": "Apache-2.0",
  "repository": {
    "type": "git",
    "url": "git+https://github.com/jstevenperry/IBM-Code"
  },
  "devDependencies": {
    "babel-eslint": "^8.2.5",
    "eslint": "^5.1.0",
    "eslint-config-strongloop": "^2.1.0"
  }
}

If you run npm install, you will get output like this:

$ npm install
added 164 packages from 190 contributors and audited 333 packages in 3.368s
found 0 vulnerabilities

$

Get info about installed software

To see which version of eslint was installed, run the npm list eslint command:

$ npm list eslint
Unit-7@1.0.0 /Users/sperry/home/development/projects/IBM-Code/Node.js/Course/Unit-7
└── eslint@5.1.0

$

To see the full dependency tree in node_modules, run npm list with no module specified. The output looks like this (abbreviated because it's quite lengthy):

$ npm list
Unit-7@1.0.0 /Users/sperry/home/development/projects/IBM-Code/Node.js/Course/Unit-7
├─┬ babel-eslint@8.2.5
│ ├─┬ @babel/code-frame@7.0.0-beta.44
│ │ └─┬ @babel/highlight@7.0.0-beta.44
│ │   ├── chalk@2.4.1 deduped
│ │   ├── esutils@2.0.2 deduped
│ │   └── js-tokens@3.0.2 deduped
.
.
│ │ └── estraverse@4.2.0
│ └── eslint-visitor-keys@1.0.0
├─┬ eslint@5.1.0
│ ├─┬ ajv@6.5.2
│ │ ├── fast-deep-equal@2.0.1
│ │ ├── fast-json-stable-stringify@2.0.0
│ │ ├── json-schema-traverse@0.4.1
│ │ └─┬ uri-js@4.2.2
│ │   └── punycode@2.1.1
│ ├─┬ babel-code-frame@6.26.0
.
.
│ │ ├── lodash@4.17.10 deduped
│ │ ├─┬ slice-ansi@1.0.0
│ │ │ └── is-fullwidth-code-point@2.0.0 deduped
│ │ └── string-width@2.1.1 deduped
│ └── text-table@0.2.0
└── eslint-config-strongloop@2.1.0
$

Once eslint is installed, you can find out what kind of error messages it produces.

To run eslint, run the pretest script from Listing 2, this time using the npm run pretest command:

$ npm run pretest

> Unit-7@1.0.0 pretest /Users/sperry/home/development/projects/IBM-Code/Node.js/Course/Unit-7
> eslint --ignore-path ../../.gitignore .


/Users/sperry/home/development/projects/IBM-Code/Node.js/Course/Unit-7/logger.js
  91:10  error  'setMessageOnlyOutput' is defined but never used  no-unused-vars
  92:3   error  'messageOnlyOutput' is not defined                no-undef

✖ 2 problems (2 errors, 0 warnings)

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! Unit-7@1.0.0 pretest: `eslint --ignore-path ../../.gitignore .`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the Unit-7@1.0.0 pretest script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/sperry/.npm/_logs/2018-07-11T21_13_15_041Z-debug.log
$

It isn't uncommon for a linter to discover multiple issues the first time you run it. I actually cleaned up logger.js to capture the above output.

Additionally, you can configure both the strongloop config and eslint rules. You can see this in the .eslintrc.json file in the source code, where I've overridden the max-len rule (80 characters seems conservative to me).

Install a global package using npm

Just like local packages, you've already had some practice with global packages. In "Explore Node.js basic concepts," you installed nodemon, which is a global package. As I mentioned there, the global install adds the package to your PATH so you can run it from anywhere on your computer, which is especially useful for CLI packages like nodemon.

The command to install a global package is npm install -g PACKAGE_NAME, where PACKAGE_NAME is the name of the package, such as nodemon.

Local versus global installation

The developers at npm have a rule of thumb for when to install packages globally and for when not to:

If the package is a command-line tool that you need to run from anywhere on your computer, install it globally.
If the package is an application dependency, install it locally.

npm utilities

npm can do more than just simple installations. Two of my favorite utilities for the npm CLI are npm prune and npm doctor.

npm prune

If you're like me, you experiment while developing, trying out multiple options before landing on your final solution. Unfortunately, this can lead to extraneous dependencies, which are installed packages that you don't use.

As an example, let's say you want to try fluid-eslint, so you install it:

$ npm install fluid-eslint
+ fluid-eslint@2.10.2
added 72 packages from 91 contributors and audited 795 packages in 4.877s
found 0 vulnerabilities
$ npm list fluid-eslint
Unit-7@1.0.0 /Users/sperry/home/development/projects/IBM-Code/Node.js/Course/Unit-7
└── fluid-eslint@2.10.2
$

At some point, you decide not to use fluid-eslint anymore. You remove it from your package.json, but removing a package from package.json doesn't alter the contents of node_modules. fluid-eslint and its dependencies are still in node_modules.

An installed top-level package that isn't listed as a dependency in package.json -- and isn't a dependency of another package -- is considered extraneous. What you need is a way to remove extraneous dependencies, which is where npm prune comes in handy.

Removing extraneous dependencies

To identify extraneous packages, run npm list. All extraneous packages will be listed at the bottom of the dependency tree, along with an error message:

$ npm list
Unit-7@1.0.0 /Users/sperry/home/development/projects/IBM-Code/Node.js/Course/Unit-7
├─┬ babel-eslint@8.2.5
│ ├─┬ @babel/code-frame@7.0.0-beta.44
│ │ └─┬ @babel/highlight@7.0.0-beta.44
│ │   ├── chalk@2.4.1 deduped
│ │   ├── esutils@2.0.2 deduped
│ │   └── js-tokens@3.0.2 deduped
│ ├─┬ @babel/traverse@7.0.0-beta.44
.
.
│ │ ├─┬ slice-ansi@1.0.0
│ │ │ └── is-fullwidth-code-point@2.0.0 deduped
│ │ └── string-width@2.1.1 deduped
│ └── text-table@0.2.0
├── eslint-config-strongloop@2.1.0
└── fluid-eslint@2.10.2 extraneous

npm ERR! extraneous: fluid-eslint@2.10.2 /Users/sperry/home/development/projects/IBM-Code/Node.js/Course/Unit-7/node_modules/fluid-eslint
$

Notice that fluid-eslint is listed as extraneous.

Now you can use the npm prune CLI utility to remove fluid-eslint and its dependencies from node_modules:

$ npm prune
removed 72 packages and audited 333 packages in 1.59s
found 0 vulnerabilities

$

If you run npm install with npm list fluid-eslint, you'll see all traces are gone:

$ npm install
audited 333 packages in 1.434s
found 0 vulnerabilities

Ix:~/home/development/projects/IBM-Code/Node.js/Course/Unit-7 sperry$ npm list fluid-eslint
Unit-7@1.0.0 /Users/sperry/home/development/projects/IBM-Code/Node.js/Course/Unit-7
└── (empty)
$

npm doctor

npm doctor is a handy utility to make sure your npm environment is running correctly. Its output looks like this:

$ npm doctor
Check                               Value                        Recommendation
npm ping                            OK
npm -v                              v6.1.0
node -v                             v10.6.0
npm config get registry             https://registry.npmjs.org/
which git                           /usr/local/bin/git
Perms check on cached files         ok
Perms check on global node_modules  ok
Perms check on local node_modules   ok
Verify cache contents               verified 5660 tarballs
$

It runs a number of checks, including an npm ping to make sure your computer can connect to the npm registry (at registry.npmjs.org), a version check of node and npm, and the path to your git program, among others.

For more information about npm doctor, check out its docs page.

More about npm

A complete guide to npm is beyond the scope of this course, especially since that information already exists in the npm docs. If you want to know more, be sure to check the npm documentation. You'll use npm throughout the rest of this learning path, so I encourage you to become familiar with it.

Alternatives to npm

While npm is the oldest Node.js package manager (going back to version 0.0.1 released in January 2010), it isn't the only one. In this section, I show you a few you should know about, starting with what is probably the most popular npm replacement: Yarn.

Yarn

Yarn is a package manager for Node that works with an npm registry just like the npm CLI does. It was developed by Facebook and released in October 2016.

Yarn offers a number of improvements over npm, including:

Yarn uses an npm registry?

I say Yarn uses an npm registry because at the time of this writing registry.yarnpkg.com is actually a reverse proxy to registry.npmjs.org, mostly for performance reasons. While there is a yarnpkg GitHub issue proposing to deprecate registry.yarnpkg.com, the registry in use should be invisible to most Node developers.

Offline access: Yarn lets you access packages that have already been downloaded, whereas the npm CLI requires Internet access to run npm install (unless you use npm pack or an add-on like offline-npm).
License checking : Yarn offers built-in license checks for dependencies, whereas npm requires an add-on like license-checker.

Install Yarn

You install Yarn from the npm registry, as a global package using this command: npm install -g yarn. Yarn works with package.json just like npm. To install a package, you just use yarn install instead of npm install.

cd IBM-Code/Node.js/Course/Unit-7
rm -Rf node_modules
yarn install

You should see output like this:

Ix:~/src/projects sperry$ cd IBM-Code/Node.js/Course/Unit-7/
Ix:~/src/projects/IBM-Code/Node.js/Course/Unit-7 sperry$ rm -Rf node_modules/
Ix:~/src/projects/IBM-Code/Node.js/Course/Unit-7 sperry$ yarn install
yarn install v1.7.0
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Building fresh packages...
✨  Done in 2.32s.
Ix:~/src/projects/IBM-Code/Node.js/Course/Unit-7 sperry$

Then you can run yarn pretest just like you did npm run pretest (Yarn figures out that pretest is a script and lets you skip the run sub-command):

Ix:~/src/projects/IBM-Code/Node.js/Course/Unit-7 sperry$ yarn pretest
yarn run v1.7.0
$ eslint --ignore-path ../../.gitignore .

/Users/sperry/home/development/projects/IBM-Code/Node.js/Course/Unit-7/logger.js
  91:10  error  'setMessageOnlyOutput' is defined but never used  no-unused-vars
  92:3   error  'messageOnlyOutput' is not defined                no-undef

✖ 2 problems (2 errors, 0 warnings)

error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Notice that you get the same output from eslint as you did from npm.

Yarn solves the mystery

Just like npm, Yarn features a number of very handy utilities. why is one of my favorites.

When perusing a package.json or node_modules directory, you might wonder why a given package is used. In Yarn, you don't have to wonder. As an example, let's say you're in node_modules and see a package called acorn:

Ix:~/src/projects/IBM-Code/Node.js/Course/Unit-7/node_modules sperry$ yarn why acorn
yarn why v1.7.0
[1/4] 🤔  Why do we have the module "acorn"...?
[2/4] 🚚  Initialising dependency graph...
[3/4] 🔍  Finding dependency...
[4/4] 🚡  Calculating file sizes...
=> Found "acorn@5.7.1"
info Reasons this module exists
   - "eslint#espree" depends on it
   - Hoisted from "eslint#espree#acorn"
   - Hoisted from "eslint#espree#acorn-jsx#acorn"
info Disk size without dependencies: "68MB"
info Disk size with unique dependencies: "68MB"
info Disk size with transitive dependencies: "68MB"
info Number of shared dependencies: 0
✨  Done in 0.19s.

Looks like acorn-jsx needs the acorn package, which is used by espree, which is used by eslint.

The full dependency path is eslint > espree > acorn-jsx > acorn from this line:

   - Hoisted from "eslint#espree#acorn-jsx#acorn"

It even tells you how much disk space it's taking up, which is pretty cool.

More package managers for Node

The Node ecosystem includes other package managers, but most are not drop-in replacements for npm; instead, they offer certain features that run in addition to npm.

Webpack: A module bundler, mostly for the browser.
Bower: A quasi-drop-in replacement for npm, mostly for the browser. (But see: How to migrate away from Bower).
RequireJS: Mostly a require() replacement for Node modules.
pnpm: A complete drop-in replacement for npm (like Yarn).

The npm website

Use the npm website when you need help or other information about Node. You can use it to access Node documentation, and it's also where you can get involved as a developer in Node.js projects.

Get help

The npm website offers:

Extensive Node.js documentation.
A npm support page with links to various channels on Twitter and GitHub.
Status pages for the npm registry, website, and CLI.

Get involved

Let's say you've created an awesome package and want to share it with the world via the npm registry. Now what?

To publish an npm package at registry.npmjs.org you'll need to set up an npm account.

Once you've verified your email address, you'll be able to share your new package in the npm registry. You'll need to follow certain standards and procedures.

Once you're signed in, the front page has some interesting metrics:

Figure 4. The npm front page The npm front page, which you can view once you're signed into your account.

As of this writing, the front page lists 800,000+ packages for Node!

Conclusion

In this tutorial, you learned that npm is three things:

A registry of software packages for Node.
A command line interface (CLI) program (npm) used to manage Node packages in your software.
A website where you can get help and get involved.

Check the references to learn more about the npm ecosystem, the npm CLI, and the npm website.

In "Node.js dependency management," you'll dive deeper into managing dependencies in Node.

Test your understanding

Answer true or false

npm is the oldest CLI for interacting with the npm registry.
A drop-in replacement for npm means that you can only use it for require() and certain npm functions.
It is not possible to install a local package unless it is specified in the dependencies section of package.json.

Check your answers

Choose the best answer

Which of the following are drop-in replacement(s) for npm?

A. Zeke

B. tower

C. flake

D. Yarn

E. All of the above
Which of the following combined commands will print out PACKAGE_NAME's currently installed version (assuming MODULE_NAME is specified in package.json)?

A. npm install && npm --curver MODULE_NAME

B. npm install && npm list MODULE_NAME

C. yarn install && yarn list MODULE_NAME

D. npm install && npm MODULE_NAME --version

E. A and D

F. B and C

G. None of the above

Check your answers

Fill in the blank

The npm ecosystem consists of three components: the npm __, the npm __, and the npm __.
Provide the command to install Yarn globally using npm: npm __.
To remove extraneous packages from your Node application's node_modules directory, you use the npm __ command.

Check your answers

Check your answers

Answers for true or false questions

Answer: True: While there are other CLI clients, npm was released in January of 2010.
Answer: False: A drop-in replacement for npm is a program that functions like the npm CLI. For example, instead of npm install -g nodemon you could use yarn global add nodemon to install nodemon globally.
Answer: False: Although it's not recommended, you can install a local dependency without a package.json file. You would use the command: npm install PACKAGE_NAME, where PACKAGE_NAME is the name of the package you want to install.

Answers for multiple choice questions

Answer: Yarn is the only npm drop-in replacement of the names listed.
Answer: F (B and C): Once packages have been installed by npm or Yarn, the list subcommand works for either CLI. Note that you should not mix-and-match; in other words, do not install with one tool and list with the other.

Answers for fill in the blank questions

Answer: registry, CLI, website
Answer: install -g yarn (or i -g yarn), full command: npm install -g yarn
Answer: prune

`npm` alternatives

Yarn: A drop-in replacement for npm.
pnpm : Another drop-in replacement for npm.
Webpack: A module bundler, mostly for the browser.
Bower: A quasi-drop-in replacement for npm.
RequireJS: Mostly a require() replacement for Node modules.

Introduction to Node package manager

Introduction to Node package manager