Where Does Node.js And Require() Look For Modules?
var moduleA = require( "./module-a.js" ); var moduleB = require( "../../module-b.js" ); var moduleC = require( "/my-library/module-c.js" );
NOTE: The (.js) file extension is optional. Node.js will add it for you when attempting to load the file. It will also attempt to load the given module as a directory with an index file (index.js) or a package description (package.json).
Ok, so that stuff makes sense - relative file paths are something that I can wrap my head around. But what about when no file path is used? Where does Node.js look if you attempt to include a module by name alone:
var moduleA = require( "module-a" ); var moduleB = require( "module-b.js" );
This is where I really started to lose my sanity this morning! When you refer to a Node.js module by name, Node.js uses a complex searching algorithm to locate the requested source code.
First, Node.js looks to see if the given module is a core module - Node.js comes with many modules compiled directly into the executable binary (ex. http, fs, sys, events, path, etc.). These core modules will always take precedence in the loading algorithm.
var utils = require( "utils" );
... Node.js will perform a hierarchical directory search for "node_modules" and "utils" in the following ways:
If it still can't find the file in this directory-spidering manner, Node.js will then proceed to look at the directory paths outlined in the "require.paths" array. The paths values in this array default to the paths defined, in part, by the environmental variable, NODE_PATH; but, they can be updated programmatically within a Node.js application.
NOTE: From what I have read, programmatically updating the "require.paths" value is considered bad practice.
If I log my default require.paths value to the console, here is what I get:
I write all of this in an attempt to hammer this Node.js module search overview into my head. If you want to know more about Node.js's module search algorithm, check out the documentation on Modules. There, you can find a much more in-depth explanation of the search algorithm including module path caching and package descriptions.
Ok - now I can start to breathe a bit easier. I'm serious - I almost lost it a few times trying to get to the bottom of this.
Want to use code from this post? Check out the license.
while you're exploring node.js, you should consider using Node Package Manager (NPM) (http://npmjs.org/) ... it's a great utility for managing additional modules, and can help remove some of the guesswork.
I actually started off this morning by installing NPM - I wanted to play with NowJS. The one-line installer for NPM (curl command) didn't work; had to download and build the make file.
I finally did get it installed and then, subsequently, installed NowJS. But, I couldn't for the life of me figure out the directory structure that NowJS was using; now, could I figure out how it was loading dependencies. That's how I got lost down the module-location algorithm rabbit hole :)
Now, I'm feeling a *little* bit more in control. Thanks!
Thanks for posting this. I spent a good chunk of last weekend trying to figure out where to put my modules and ultimately never found the answer.
NodeJS is still so new that it's not as easy to find answers as it for most other server-side languages.
export NODE_DEBUG=module is your friend...
@martin cleaver THANK YOU :)
there's a node module called "rekiure"
it allows you to "require" without using of relative paths
its a big time saver when it comes to testing/refactoring
super easy to use
This is helpful but is there any chance you can point me towards a discussion of the same methodology applied to client side includes? It would seem like the same methodology would be followed but it does not seem to be!?!?!
thnkz for the post! i came to kno bout NODE_PATH env variable - set that variable and cld install modules globally now!
Ah thanks Ben - I'm a noob, and after mastering the "Hello World" bit I thought, ok I'm going to write a bit of code, where should I put it? 48 hours later I find this - saved my sanity! - Terry.
This also saved me from losing my mind, so thanks! total noob to nodejs and have spent all day trying to figure out how the hell it wasn't finding any modules. a new folder named node_modules in my projects directory and voila! thanks again!
That was the clue for my case. I had to
cp -r node_modules ~/Demo...
This was while in ~/local/node. My index.js was in ~/Demo.
just do a strace or dtruss when running your code you will surely see how the path to your script is resolved.
$> strace node myapp.js
Grateful for the post, I was deep in the woods last night over this question
But, but....what is then the point of doing a global install ?
I installed the express module globally and yet I get this error message. A global installation should allow accessing modules from any where.
sudo npm install -g express
(sudo because -g fiddles with /usr )
Nagesh, the node.js philosophy (which I don't necessarily agree with) is that global install is only for modules that add commands to your system, not for modules that act as libraries for your own projects. For libraries, you are encouraged to install the libraries locally, within each project you develop.
Take the express module. When you install express globally it creates an 'express' command that can be accessed from any shell. (Actually these days you might have to also install the 'express-generator' module as well but that's beside the point of this discussion.) If somebody has created a complete node.js application that you want to add to your system, 'npm install -g' is probably the right way to install it.
I can see where the node.js philosophy comes from. All these open source modules are developing rapidly and may have very specific version dependencies on other modules. Maintaining this on a global basis on your system could result in the kind of problem that's called 'DLL hell' on Windows systems. node.js modules are relatively small and it's easy to keep local copies within each project.
However, coming from a C++ background, where the 3rd party libraries you use can be quite enormous (I'm thinking off Qt as a prime example), the idea of every program I write having its own copy of those libraries is dreadful. When you install something like Qt for development it usually puts its libraries (and DLLs on Windows) in one specific place and you reference -that copy- in your projects.
The people working on node.js -have- provided a 'preferred' way to work with globally installed modules if that's your choice. For example, if you have installed express globally, you can go into the main directory of your project and enter this command:
npm link express
This will create a node_modules directory in your project, but then under there it will create a soft link to the globally installed express module (wherever that happens to be on your system). At this point this even works on Windows, using 'junctions' which is the Windows equivalent of a soft link. You can then write 'require('express')' and it will find the global module through the soft link.