Where Does Node.js And Require() Look For Modules?

Posted April 13, 2011 at 11:24 AM by Ben Nadel

Tags: Javascript / DHTML

In the last week or so, I've started to experiment with building server-side Javascript code for Node.js. When you build a Node.js application, you can use the global require() method to load and cache Javascript modules. These modules provide localized memory spaces that can house singletons and class definitions. When you invoke the require() method, Node.js searches a number of system paths looking for the module in question. This morning, I was brought to the edge of sanity trying to figure out what that searching algorithm was. After failing to find anything useful in the Node.js docs regarding the require() method, Jeremy Martin came to my rescue, pointing me to the documentation on Modules.

If you want to load a local, relative Javascript module into a Node.js application, you can simply use the require() method in conjunction with relative (or absolute) file paths:

  • var moduleA = require( "./module-a.js" );
  • var moduleB = require( "../../module-b.js" );
  • var moduleC = require( "/my-library/module-c.js" );

NOTE: The (.js) file extension is optional. Node.js will add it for you when attempting to load the file. It will also attempt to load the given module as a directory with an index file (index.js) or a package description (package.json).

Ok, so that stuff makes sense - relative file paths are something that I can wrap my head around. But what about when no file path is used? Where does Node.js look if you attempt to include a module by name alone:

  • var moduleA = require( "module-a" );
  • var moduleB = require( "module-b.js" );

This is where I really started to lose my sanity this morning! When you refer to a Node.js module by name, Node.js uses a complex searching algorithm to locate the requested source code.

First, Node.js looks to see if the given module is a core module - Node.js comes with many modules compiled directly into the executable binary (ex. http, fs, sys, events, path, etc.). These core modules will always take precedence in the loading algorithm.

If the given module is not a core module, Node.js will then begin to search for a directory named, "node_modules". It will start in the current directory (relative to the currently-executing Javascript file in Node) and then work its way up the folder hierarchy, checking each level for a node_modules folder.

Once Node.js finds the node_modules folder, it will then attempt to load the given module either as a (.js) Javascript file or as a named sub-directory. And, if it finds the named sub-directory, it will then attempt to load the file in various ways. So, for example, if you make a request to load the module, "utils":

  • var utils = require( "utils" );

... Node.js will perform a hierarchical directory search for "node_modules" and "utils" in the following ways:

  • ./node_modules/utils.js
  • ./node_modules/utils/index.js
  • ./node_modules/utils/package.json

If it still can't find the file in this directory-spidering manner, Node.js will then proceed to look at the directory paths outlined in the "require.paths" array. The paths values in this array default to the paths defined, in part, by the environmental variable, NODE_PATH; but, they can be updated programmatically within a Node.js application.

NOTE: From what I have read, programmatically updating the "require.paths" value is considered bad practice.

If I log my default require.paths value to the console, here is what I get:

[
'/usr/local/lib/node',
'/Users/ben/.node_modules',
'/Users/ben/.node_libraries',
'/usr/local/Cellar/node/0.4.5/lib/node'
]

I write all of this in an attempt to hammer this Node.js module search overview into my head. If you want to know more about Node.js's module search algorithm, check out the documentation on Modules. There, you can find a much more in-depth explanation of the search algorithm including module path caching and package descriptions.

Ok - now I can start to breathe a bit easier. I'm serious - I almost lost it a few times trying to get to the bottom of this.




Reader Comments

Apr 13, 2011 at 12:05 PM // reply »
2 Comments

hey ben,

while you're exploring node.js, you should consider using Node Package Manager (NPM) (http://npmjs.org/) ... it's a great utility for managing additional modules, and can help remove some of the guesswork.

nice writeup!


Apr 13, 2011 at 12:09 PM // reply »
11,314 Comments

@Keegan,

I actually started off this morning by installing NPM - I wanted to play with NowJS. The one-line installer for NPM (curl command) didn't work; had to download and build the make file.

I finally did get it installed and then, subsequently, installed NowJS. But, I couldn't for the life of me figure out the directory structure that NowJS was using; now, could I figure out how it was loading dependencies. That's how I got lost down the module-location algorithm rabbit hole :)

Now, I'm feeling a *little* bit more in control. Thanks!


Apr 15, 2011 at 10:11 PM // reply »
1 Comments

Thanks for posting this. I spent a good chunk of last weekend trying to figure out where to put my modules and ultimately never found the answer.

NodeJS is still so new that it's not as easy to find answers as it for most other server-side languages.


Feb 23, 2012 at 7:29 AM // reply »
1 Comments

export NODE_DEBUG=module is your friend...


Mar 4, 2012 at 6:15 AM // reply »
1 Comments

@martin cleaver THANK YOU :)


Mar 28, 2013 at 6:10 AM // reply »
1 Comments

there's a node module called "rekiure"

it allows you to "require" without using of relative paths

its a big time saver when it comes to testing/refactoring

https://npmjs.org/package/rekuire

super easy to use


Jun 15, 2013 at 11:33 AM // reply »
1 Comments

This is helpful but is there any chance you can point me towards a discussion of the same methodology applied to client side includes? It would seem like the same methodology would be followed but it does not seem to be!?!?!


Post A Comment

Comment Etiquette: Please do not post spam. Please keep the comments on-topic. Please do not post unrelated questions or large chunks of code. And, above all, please be nice to each other - we're trying to have a good conversation here.

Please review the following issues:

Author Name:


Author Email:

Author Website:

Comment:

Supported HTML tags for formatting: <strong>bold</strong>   <em>italic</em>   <code>code</code>







  • Help Wanted - Find Your Next ColdFusion Job
Ben Nadel's Company - Epicenter Consulting Recent Blog Comments
Jun 19, 2013 at 8:17 AM
My Experience With AngularJS - The Super-heroic JavaScript MVW Framework
@Prateek, to match a word or text you should use .toContain('word') that's a jasmine reference. website is : http://pivotal.github.io/jasmine/ ... read »
Jun 19, 2013 at 8:10 AM
My Experience With AngularJS - The Super-heroic JavaScript MVW Framework
Hi Guys, Actually i am doing e2e test of angular js of my project but i am not getting one thing that is how to press enter key through the test when my form is filled as i am not using a button but ... read »
Jun 18, 2013 at 9:20 PM
Mapping AngularJS Routes Onto URL Parameters And Client-Side Events
I couldn't find examples of passing multiple arguments using the when() routing statement so figured out through trial and error that you can pass multiple arguments using the following format: .whe ... read »
Jun 18, 2013 at 3:39 PM
Experimenting With The Amazon Simple Storage Service (S3) API Using ColdFusion
Hi Ben, THANKS! While not bleeding edge, it is new to me & I like learning new things every day! ... read »
Jun 18, 2013 at 12:30 PM
Disabling Auto-Correct And Auto-Capitalize Features On iPhone Inputs
Also spellcheck="false" should be mentioned as part of html5 specs ... read »
Jun 18, 2013 at 8:40 AM
Using Named Functions Within Self-Executing Function Blocks In Javascript
Hi Ben, you forgot to mention the most important thing for named self-executing functions - they can be referenced by name ONLY inside their execution context (which is parens in this case), it mean ... read »
dee
Jun 18, 2013 at 7:01 AM
My Safari Browser SQLite Database Hello World Example
hai ben, this program is really good i could understand the concept but i dint know how to save it and how to open it as you have done in the video can u give that details pls ... read »
Jun 18, 2013 at 6:04 AM
Clearing Inline CSS Properties With jQuery
Thanks a lot for for post! It helped me a lot... after being stuck since 24 hrs.. found solution from your post. Thanks again! ... read »
InVision App - Prototyping Made Beautiful With Prototyping Tools