Ben Nadel
On User Experience (UX) Design, JavaScript, ColdFusion, Node.js, Life, and Love.
I am the chief technical officer at InVision App, Inc - a prototyping and collaboration platform for designers, built by designers. I also rock out in JavaScript and ColdFusion 24x7.
Meanwhile on Twitter
Loading latest tweet...
Ben Nadel at InVision Office 2012 (New York City) with:

Where Does Node.js And Require() Look For Modules?

Posted by Ben Nadel

In the last week or so, I've started to experiment with building server-side Javascript code for Node.js. When you build a Node.js application, you can use the global require() method to load and cache Javascript modules. These modules provide localized memory spaces that can house singletons and class definitions. When you invoke the require() method, Node.js searches a number of system paths looking for the module in question. This morning, I was brought to the edge of sanity trying to figure out what that searching algorithm was. After failing to find anything useful in the Node.js docs regarding the require() method, Jeremy Martin came to my rescue, pointing me to the documentation on Modules.

If you want to load a local, relative Javascript module into a Node.js application, you can simply use the require() method in conjunction with relative (or absolute) file paths:

  • var moduleA = require( "./module-a.js" );
  • var moduleB = require( "../../module-b.js" );
  • var moduleC = require( "/my-library/module-c.js" );

NOTE: The (.js) file extension is optional. Node.js will add it for you when attempting to load the file. It will also attempt to load the given module as a directory with an index file (index.js) or a package description (package.json).

Ok, so that stuff makes sense - relative file paths are something that I can wrap my head around. But what about when no file path is used? Where does Node.js look if you attempt to include a module by name alone:

  • var moduleA = require( "module-a" );
  • var moduleB = require( "module-b.js" );

This is where I really started to lose my sanity this morning! When you refer to a Node.js module by name, Node.js uses a complex searching algorithm to locate the requested source code.

First, Node.js looks to see if the given module is a core module - Node.js comes with many modules compiled directly into the executable binary (ex. http, fs, sys, events, path, etc.). These core modules will always take precedence in the loading algorithm.

If the given module is not a core module, Node.js will then begin to search for a directory named, "node_modules". It will start in the current directory (relative to the currently-executing Javascript file in Node) and then work its way up the folder hierarchy, checking each level for a node_modules folder.

Once Node.js finds the node_modules folder, it will then attempt to load the given module either as a (.js) Javascript file or as a named sub-directory. And, if it finds the named sub-directory, it will then attempt to load the file in various ways. So, for example, if you make a request to load the module, "utils":

  • var utils = require( "utils" );

... Node.js will perform a hierarchical directory search for "node_modules" and "utils" in the following ways:

  • ./node_modules/utils.js
  • ./node_modules/utils/index.js
  • ./node_modules/utils/package.json

If it still can't find the file in this directory-spidering manner, Node.js will then proceed to look at the directory paths outlined in the "require.paths" array. The paths values in this array default to the paths defined, in part, by the environmental variable, NODE_PATH; but, they can be updated programmatically within a Node.js application.

NOTE: From what I have read, programmatically updating the "require.paths" value is considered bad practice.

If I log my default require.paths value to the console, here is what I get:

[
'/usr/local/lib/node',
'/Users/ben/.node_modules',
'/Users/ben/.node_libraries',
'/usr/local/Cellar/node/0.4.5/lib/node'
]

I write all of this in an attempt to hammer this Node.js module search overview into my head. If you want to know more about Node.js's module search algorithm, check out the documentation on Modules. There, you can find a much more in-depth explanation of the search algorithm including module path caching and package descriptions.

Ok - now I can start to breathe a bit easier. I'm serious - I almost lost it a few times trying to get to the bottom of this.




Reader Comments

hey ben,

while you're exploring node.js, you should consider using Node Package Manager (NPM) (http://npmjs.org/) ... it's a great utility for managing additional modules, and can help remove some of the guesswork.

nice writeup!

Reply to this Comment

@Keegan,

I actually started off this morning by installing NPM - I wanted to play with NowJS. The one-line installer for NPM (curl command) didn't work; had to download and build the make file.

I finally did get it installed and then, subsequently, installed NowJS. But, I couldn't for the life of me figure out the directory structure that NowJS was using; now, could I figure out how it was loading dependencies. That's how I got lost down the module-location algorithm rabbit hole :)

Now, I'm feeling a *little* bit more in control. Thanks!

Reply to this Comment

Thanks for posting this. I spent a good chunk of last weekend trying to figure out where to put my modules and ultimately never found the answer.

NodeJS is still so new that it's not as easy to find answers as it for most other server-side languages.

Reply to this Comment

This is helpful but is there any chance you can point me towards a discussion of the same methodology applied to client side includes? It would seem like the same methodology would be followed but it does not seem to be!?!?!

Reply to this Comment

thnkz for the post! i came to kno bout NODE_PATH env variable - set that variable and cld install modules globally now!

Reply to this Comment

Ah thanks Ben - I'm a noob, and after mastering the "Hello World" bit I thought, ok I'm going to write a bit of code, where should I put it? 48 hours later I find this - saved my sanity! - Terry.

Reply to this Comment

This also saved me from losing my mind, so thanks! total noob to nodejs and have spent all day trying to figure out how the hell it wasn't finding any modules. a new folder named node_modules in my projects directory and voila! thanks again!

Reply to this Comment

"If the given module is not a core module, Node.js will then begin to search for a directory named, "node_modules". It will start in the current directory (relative to the currently-executing Javascript file in Node)"

Thanks!

That was the clue for my case. I had to
cp -r node_modules ~/Demo...
This was while in ~/local/node. My index.js was in ~/Demo.

Reply to this Comment

just do a strace or dtruss when running your code you will surely see how the path to your script is resolved.

e.g.

$> strace node myapp.js

Reply to this Comment

Post A Comment

?
You — Get Out Of My Dreams, Get Into My Comments
Live in the Now
Oops!
Comment Etiquette: Please do not post spam. Please keep the comments on-topic. Please do not post unrelated questions or large chunks of code. And, above all, please be nice to each other - we're trying to have a good conversation here.