How to store Node.js deployment settings/configuration files?_问答_开发者

I have been working on a few Node apps, and I've been looking for a good pattern of storing deployment-related settings. In the Django world (where I come from), the common practise would be to have a settings.py file containing the standard settings (timezone, etc), and then a local_settings.py for deployment specific settings, ie. what database to talk to, what memcache socket, e-mail address for the admins and so on.

I have been looking for similar patterns for Node. Just a config file would be nice, so it does not have to be jammed in with everything else in app.js, but I find it important to have a way to have server-specific configuration in a file that is not in source control. The same app could well be deployed across different servers with wildly 开发者_JAVA百科different settings, and having to deal with merge conflicts and all that is not my idea of fun.

So is there some kind of framework/tool for this, or does everyone just hack something together themselves?

I use a package.json for my packages and a config.js for my configuration, which looks like:

var config = {};

config.twitter = {};
config.redis = {};
config.web = {};

config.default_stuff =  ['red','green','blue','apple','yellow','orange','politics'];
config.twitter.user_name = process.env.TWITTER_USER || 'username';
config.twitter.password=  process.env.TWITTER_PASSWORD || 'password';
config.redis.uri = process.env.DUOSTACK_DB_REDIS;
config.redis.host = 'hostname';
config.redis.port = 6379;
config.web.port = process.env.WEB_PORT || 9980;

module.exports = config;

I load the config from my project:

var config = require('./config');

and then I can access my things from config.db_host, config.db_port, etc... This lets me either use hardcoded parameters, or parameters stored in environmental variables if I don't want to store passwords in source control.

I also generate a package.json and insert a dependencies section:

"dependencies": {
  "cradle": "0.5.5",
  "jade": "0.10.4",
  "redis": "0.5.11",
  "socket.io": "0.6.16",
  "twitter-node": "0.0.2",
  "express": "2.2.0"
}

When I clone the project to my local machine, I run npm install to install the packages. More info on that here.

The project is stored in GitHub, with remotes added for my production server.

You can require JSON files as of Node v0.5.x (referencing this answer)

config.json:

{
    "username" : "root",
    "password" : "foot"
}

app.js:

var config = require('./config.json');
log_in(config.username, config.password);

Much later, I found a pretty good Node.js module for managing configuration: nconf.

A simple example:

var nconf = require('nconf');

// First consider commandline arguments and environment variables, respectively.
nconf.argv().env();

// Then load configuration from a designated file.
nconf.file({ file: 'config.json' });

// Provide default values for settings not provided above.
nconf.defaults({
    'http': {
        'port': 1337
    }
});

// Once this is in place, you can just use nconf.get to get your settings.
// So this would configure `myApp` to listen on port 1337 if the port
// has not been overridden by any of the three configuration inputs
// mentioned above.
myApp.listen(nconf.get('http:port'));

It also supports storing settings in Redis, writing configuration files, and has a fairly solid API, and is also backed by one of the more well-respected Node.js shops, Nodejitsu, as part of the Flatiron framework initiative, so it should be fairly future-proof.

Check out nconf at Github.

My solution is fairly simple:

Load the environment config in ./config/index.js

var env = process.env.NODE_ENV || 'development'
  , cfg = require('./config.'+env);

module.exports = cfg;

Define some defaults in ./config/config.global.js

var config = module.exports = {};

config.env = 'development';
config.hostname = 'dev.example.com';

//mongo database
config.mongo = {};
config.mongo.uri = process.env.MONGO_URI || 'localhost';
config.mongo.db = 'example_dev';

Override the defaults in ./config/config.test.js

var config = require('./config.global');

config.env = 'test';
config.hostname = 'test.example';
config.mongo.db = 'example_test';

module.exports = config;

Using it in ./models/user.js:

var mongoose = require('mongoose')
, cfg = require('../config')
, db = mongoose.createConnection(cfg.mongo.uri, cfg.mongo.db);

Running your app in test environment:

NODE_ENV=test node ./app.js

You might also look to dotenv which follows the tenets of a twelve-factor app.

I used to use node-config, but created dotenv for that reason. It was completely inspired by ruby's dotenv library.

Usage is quite easy:

var dotenv = require('dotenv');
dotenv.load();

Then you just create a .env file and put your settings in there like so:

S3_BUCKET=YOURS3BUCKET
SECRET_KEY=YOURSECRETKEYGOESHERE
OTHER_SECRET_STUFF=my_cats_middle_name

That's dotenv for nodejs.

Are you guys using npm to start your scripts (env etc) ?

If you use .env files you can include them in your package.json and use npm to source/start them.

Example:

{
  "name": "server",
  "version": "0.0.1",
  "private": true,
  "scripts": {
    "start": "node test.js",
    "start-dev": "source dev.env; node test.js",
    "start-prod": "source prod.env; node test.js"
  },
  "dependencies": {
    "mysql": "*"
  }
}

then run the npm scripts:

$ npm start-dev

Its described here https://gist.github.com/ericelliott/4152984 All credit to Eric Elliot

You might also look to node-config which loads configuration file depending on $HOST and $NODE_ENV variable (a little bit like RoR) : documentation.

This can be quite useful for different deployment settings (development, test or production).

Just do a simple settings.js with exports:

exports.my_password = 'value'

Then, in your script, do a require:

var settings = require('./settings.js');

All your settings now will be availabe via settings variable:

settings.my_password // 'value'

Convict is another option that adds a schema for validation. Like nconf, it supports loading settings from any combination of environment variables, arguments, files, and json objects.

Example from the README:

var convict = require('convict');
var conf = convict({
  env: {
    doc: "The applicaton environment.",
    format: ["production", "development", "test"],
    default: "development",
    env: "NODE_ENV"
  },
  ip: {
    doc: "The IP address to bind.",
    format: "ipaddress",
    default: "127.0.0.1",
    env: "IP_ADDRESS",
  },
  port: {
    doc: "The port to bind.",
    format: "port",
    default: 0,
    env: "PORT"
  }
});

Getting started article: Taming Configurations with node-convict

I'm going to throw my hat into the ring here because none of these answers address all the critical components that pretty much any system needs. Considerations:

Public configuration (that can be seen by the frontend) vs private configuration (guy mograbi got this one right). And ensuring these are kept separate.
Secrets like keys
Defaults vs environment-specific overrides
Frontend bundles

Here's how I do my configuration:

config.default.private.js - In version control, these are default configuration options that can only be seen by your backend.
config.default.public.js - In version control, these are default configuration options that can be seen by backend and frontend
config.dev.private.js - If you need different private defaults for dev.
config.dev.public.js - If you need different public defaults for dev.
config.private.js - Not in version control, these are environment specific options that override config.default.private.js
config.public.js - Not in version control, these are environment specific options that override config.default.public.js
keys/ - A folder where each file stores a different secret of some kind. This is also not under version control (keys should never be under version control).

I use plain-old javascript files for configuration so I have the full power of the javascript langauge (including comments and the ability to do things like load the default config file in the environment-specific file so they can then be overridden). If you want to use environment variables, you can load them inside those config files (tho I recommend against using env vars for the same reason I don't recommend using json files - you don't have the power of a programming language to construct your config).

The reason each key is in a separate file is for installer use. This allows you to have an installer that creates keys on-machine and stores them in the keys folder. Without this, your installer might fail when you load your configuration file that can't access your keys. This way you can traverse the directory and load any key files that are in that folder without having to worry about what exists and what doesn't in any given version of your code.

Since you probably have keys loaded in your private configuration, you definitely don't want to load your private config in any frontend code. While its probably strictly more ideal to completely separate your frontend codebase from your backend, a lot of times that PITA is a big enough barrier to prevent people from doing it, thus private vs public config. But there's two things I do to prevent private config being loaded in the frontend:

I have a unit test that ensures my frontend bundles don't contain one of the secret keys I have in the private config.
I have my frontend code in a different folder than my backend code, and I have two different files named "config.js" - one for each end. For backend, config.js loads the private config, for frontend, it loads the public config. Then you always just require('config') and don't worry about where it comes from.

One last thing: your configuration should be loaded into the browser via a completely separate file than any of your other frontend code. If you bundle your frontend code, the public configuration should be built as a completely separate bundle. Otherwise, your config isn't really config anymore - its just part of your code. Config needs to be able to be different on different machines.

You can use Konfig for environment specific config files. It loads json or yaml config files automatically, it has default value and dynamic configuration features.

An example from Konfig repo:

File: config/app.json
----------------------------
{
    "default": {
        "port": 3000,
        "cache_assets": true,
        "secret_key": "7EHDWHD9W9UW9FBFB949394BWYFG8WE78F"
    },

    "development": {
        "cache_assets": false
    },

    "test": {
        "port": 3001
    },

    "staging": {
        "port": #{process.env.PORT},
        "secret_key": "3F8RRJR30UHERGUH8UERHGIUERHG3987GH8"
    },

    "production": {
        "port": #{process.env.PORT},
        "secret_key": "3F8RRJR30UHERGUH8UERHGIUERHG3987GH8"
    }
}

In development:

> config.app.port
3000

In production, assume we start application with $ NODE_ENV=production PORT=4567 node app.js

> config.app.port
4567

More details : https://github.com/vngrs/konfig

I will create a folder as config a file naming as config.js and later I will use this file wherever required as below

Example of config.js

module.exports = {
    proxyURL: 'http://url:port',
    TWITTER: {
        consumerkey: 'yourconsumerkey',
        consumerSecrete: 'yourconsumersecrete'
    },
    GOOGLE: {
        consumerkey: 'yourconsumerkey',
        consumerSecrete: 'yourconsumersecrete'
    },
    FACEBOOK: {
        consumerkey: 'yourconsumerkey',
        consumerSecrete: 'yourconsumersecrete'
    }
}

Then if i want to use this config file somewhere

I will first import as below

var config = require('./config');

and I can access the values as below

const oauth = OAuth({
    consumer: {
        key: config.TWITTER.consumerkey,
        secret: config.TWITTER.consumerSecrete
    },
    signature_method: 'HMAC-SHA1',
    hash_function(base_string, key) {
        return crypto.createHmac('sha1', key).update(base_string).digest('base64');
    }
});

Just use npm module config (more than 300000 downloads)

https://www.npmjs.com/package/config

Node-config organizes hierarchical configurations for your app deployments.

It lets you define a set of default parameters, and extend them for different deployment environments (development, qa, staging, production, etc.).

$ npm install config
$ mkdir config
$ vi config/default.json


{
      // Customer module configs
      "Customer": {
        "dbConfig": {
          "host": "localhost",
          "port": 5984,
          "dbName": "customers"
        },
        "credit": {
          "initialLimit": 100,
          // Set low for development
          "initialDays": 1
        }
      }
}



$ vi config/production.json

{
  "Customer": {
    "dbConfig": {
      "host": "prod-db-server"
    },
    "credit": {
      "initialDays": 30
    }
  }
}



$ vi index.js

var config = require('config');
//...
var dbConfig = config.get('Customer.dbConfig');
db.connect(dbConfig, ...);

if (config.has('optionalFeature.detail')) {
  var detail = config.get('optionalFeature.detail');
  //...
}


$ export NODE_ENV=production
$ node index.js

A bit late (just 10 year) but I use a config.js structured like this:

const env = process.env.NODE_ENV || 'development';

var config_temp = {
    default:{
        port: 3000,
        mysql_host: "localhost",
        logging_level: 5,
        secret_api_key: process.env.SECRET_API_KEY
    },
    development: {
        logging_level: 10
    },
    production: {
        port: 3001,
        mysql_host: "not-localhost"
    }
};
var config = {
    ...config_temp.default, 
    ...config_temp[env]
}
module.exports = config;

and I load the config with:

var config = require('./config');
var port = config.port;

In this way:

The reading of the env variable is included in the config.js file so I can avoid this ugliness: require('./config')[process.env.NODE_ENV || 'development'].
The file config.js can be uploaded in the code 's repo because sensitive variables continue to be handled with process.env.
If the same element is contained in both default:{ and custom_env:{ only the second is kept.
There are no dedicated folders and multiple files (like in config)

I know this is a really old post. But I want to share my module for configuring environment variables, I think it is very flexible solution. Here is the module json-configurator

var configJson = {
  'baseUrl': 'http://test.com',
  '$prod_baseUrl': 'https://prod.com',
  'endpoints': {
    'users': '<%= baseUrl %>/users',
    'accounts': '<%= baseUrl %>/accounts'
    },
  foo: 'bar',
  foobar: 'foobar',
  $prod_foo: 'foo in prod',
  $test_foo: 'foo in test',
  deep:{
    veryDeep: {
      publicKey: 'abc',
      secret: 'secret',
      $prod_secret: 'super secret'
    }
  }
};

var config = require('json-configurator')(configJson, 'prod');

console.log(config.deep.veryDeep.secret) 
// super secret 

console.log(config.endpoints.users)
// https://prod.com/users

Then you can use process.env.NODE_ENV to get all the variables for your environment.

It's better to separate 'development' and 'production' configs.

I use following way: Here is my config/index.js file:

const config = {
    dev : {
        ip_address : '0.0.0.0',
        port : 8080,
        mongo :{
            url : "mongodb://localhost:27017/story_box_dev",
            options : ""
        }
    },
    prod : {
        ip_address : '0.0.0.0',
        port : 3000,
        mongo :{
            url : "mongodb://localhost:27017/story_box_prod",
            options : ""
        }
    }
}

For require the config use following:

const config = require('../config')[process.env.NODE_ENV];

Than you can use your config object:

const ip_address = config.ip_address;
const port = config.port;

I am a bit late in the game, but I couldn't find what I needed here- or anywhere else - so I wrote something myself.

My requirements for a configuration mechanism are the following:

Support front-end. What is the point if the front-end cannot use the configuration?
Support settings-overrides.js - which looks the same but allows overrides of configuration at settings.js. The idea here is to modify configuration easily without changing the code. I find it useful for saas.

Even though I care less about supporting environments - the will explain how to add it easily to my solution

var publicConfiguration = {
    "title" : "Hello World"
    "demoAuthToken" : undefined, 
    "demoUserId" : undefined, 
    "errorEmail" : null // if null we will not send emails on errors. 

};

var privateConfiguration = {
    "port":9040,
    "adminAuthToken":undefined,
    "adminUserId":undefined
}

var meConf = null;
try{
    meConf = require("../conf/dev/meConf");
}catch( e ) { console.log("meConf does not exist. ignoring.. ")}




var publicConfigurationInitialized = false;
var privateConfigurationInitialized = false;

function getPublicConfiguration(){
    if (!publicConfigurationInitialized) {
        publicConfigurationInitialized = true;
        if (meConf != null) {
            for (var i in publicConfiguration) {
                if (meConf.hasOwnProperty(i)) {
                    publicConfiguration[i] = meConf[i];
                }
            }
        }
    }
    return publicConfiguration;
}


function getPrivateConfiguration(){
    if ( !privateConfigurationInitialized ) {
        privateConfigurationInitialized = true;

        var pubConf = getPublicConfiguration();

        if ( pubConf != null ){
            for ( var j in pubConf ){
                privateConfiguration[j] = pubConf[j];
            }
        }
        if ( meConf != null ){
              for ( var i in meConf ){
                  privateConfiguration[i] = meConf[i];
              }
        }
    }
    return privateConfiguration;

}


exports.sendPublicConfiguration = function( req, res ){
    var name = req.param("name") || "conf";

    res.send( "window." + name + " = " + JSON.stringify(getPublicConfiguration()) + ";");
};


var prConf = getPrivateConfiguration();
if ( prConf != null ){
    for ( var i in prConf ){
        if ( prConf[i] === undefined ){

            throw new Error("undefined configuration [" + i + "]");
        }
        exports[i] = prConf[i];
    }
}


return exports;

Explanation

undefined means this property is required
null means it is optional
meConf - currently the code is target to a file under app. meConf is the overrides files which is targeted to conf/dev - which is ignored by my vcs.
publicConfiguration - will be visible from front-end and back-end.
privateConfiguration - will be visible from back-end only.
sendPublicConfiguration - a route that will expose the public configuration and assign it to a global variable. For example the code below will expose the public configuration as global variable myConf in the front-end. By default it will use the global variable name conf.

app.get("/backend/conf", require("conf").sendPublicConfiguration);

Logic of overrides

privateConfiguration is merged with publicConfiguration and then meConf.
publicConfiguration checks each key if it has an override, and uses that override. This way we are not exposing anything private.

Adding environment support

Even though I don't find an "environment support" useful, maybe someone will.

To add environment support you need to change the meConf require statement to something like this (pseudocode)

if ( environment == "production" ) { meConf = require("../conf/dev/meConf").production; }

if ( environment == "development" ) { meConf = require("../conf/dev/meConf").development; }

Similarly you can have a file per environment

 meConf.development.js
 meConf.production.js

and import the right one. The rest of the logic stays the same.

an alt example I just used because I wanted more flexibility than a typical .json file but didn't want it abstracted away into a library which would require a dependency is something like this. Basically, exporting a function invoked immediately which returned an object with values I wanted set. Gives a lot of flexibility.

     module.exports = function(){
       switch(node_env){
         case 'dev':
           return
           { var1 = 'development'};
         }
    }();

There is a much better explanation with full example here. Using Config Files in Node.js

npm i config

In config/default.json
{
    "app": {
        "port": 3000
    },
    "db": {
        "port": 27017,
        "name": "dev_db_name"
    }
}

In config/production.json
{
    "app": {
        "port": 4000
    },
    "db": {
        "port": 27000,
        "name": "prod_db_name"
    }
}

In index.js

const config = require('config');

let appPort = config.get('app.port');
console.log(`Application port: ${appPort}`);

let dbPort = config.get('db.port');
console.log(`Database port: ${dbPort}`);

let dbName = config.get('db.name');
console.log(`Database name: ${dbName}`);

console.log('NODE_ENV: ' + config.util.getEnv('NODE_ENV'));

$ node index.js
Application port: 3000
Database port: 27017
Database name: dev_db_name
NODE_ENV: development

For production
$ set NODE_ENV=production
$ node index.js
Application port: 4000
Database port: 27000
Database name: prod_db_name
NODE_ENV: production

In addition to the nconf module mentioned in this answer, and node-config mentioned in this answer, there are also node-iniparser and IniReader, which appear to be simpler .ini configuration file parsers.

Here is a neat approach inspired by this article. It does not require any additional packages except the ubiquitous lodash package. Moreover, it lets you manage nested defaults with environment-specific overwrites.

First, create a config folder in the package root path that looks like this

package
  |_config
      |_ index.js
      |_ defaults.json
      |_ development.json
      |_ test.json
      |_ production.json

here is the index.js file

const _ = require("lodash");
const defaults = require("./defaults.json");
const envConf = require("./" + (process.env.NODE_ENV || "development") + ".json" );
module.exports = _.defaultsDeep(envConf, defaults);

Now let's assume we have a defaults.json like so

{
  "confKey1": "value1",
  "confKey2": {
    "confKey3": "value3",
    "confKey4": "value4"
  }
}

and development.json like so

{
  "confKey2": {
    "confKey3": "value10",
  }
}

if you do config = require('./config') here is what you will get

{
  "confKey1": "value1",
  "confKey2": {
    "confKey3": "value10",
    "confKey4": "value4"
  }
}

Notice that you get all the default values except for those defined in environment-specific files. So you can manage a config hierarchy. Using defaultsDeep makes sure that you can even have nested defaults.

I just recently released a small module to load any type of configuration files. It's pretty straight-forward, you can check it at https://github.com/flesler/config-node

You can use pconf: https://www.npmjs.com/package/pconf

Example:

var Config = require("pconf");
var testConfig = new Config("testConfig");
testConfig.onload = function(){

  testConfig.setValue("test", 1);
  testConfig.getValue("test");
  //testConfig.saveConfig(); Not needed

}

I used Dotenv-Flow for configuration management.

This is working as expected. It's very often that you have multiple environments like local, dev, staging, and production. Just flow these steps to create your own environments.

1. npm i dotenv-flow.

2. Create files like .env | .env.dev | .env.prod.

For testing purposes copy this content

.env

DATABASE_HOST=global
DATABASE_PORT=global
DATABASE_USER=global
DATABASE_PASS=global
DATABASE_NAME=global

.env.dev

DATABASE_NAME=dev
DATABASE_PASS=dev

.env.prod

DATABASE_NAME=prod
DATABASE_PASS=prod

Now create a test file use these environment variables.

test.js

console.log('database host:', process.env.DATABASE_HOST);
console.log('database port:', process.env.DATABASE_PORT);
console.log('database user:', process.env.DATABASE_USER);
console.log('database pass:', process.env.DATABASE_PASS);
console.log('database name:', process.env.DATABASE_NAME);

Now use these commands to run your script.

node -r dotenv-flow/config test.js
node -r dotenv-flow/config test.js --node-env=dev
node -r dotenv-flow/config test.js --node-env=prod

If you create these environment variables files in a specific folder like in my case I have created these files in envs folder then use the below command.

node -r dotenv-flow/config test.js --dotenv-flow-path=./envs
node -r dotenv-flow/config test.js --dotenv-flow-path=./envs --node-env=dev
node -r dotenv-flow/config test.js --dotenv-flow-path=./envs --node-env=prod

For those who are visiting this old thread here is a package I find to be good.

https://www.npmjs.org/package/config

I tryied some of suggested solutions here, but was not sattisfied with them, so I created my own module. It is called mikro-config and the main difference is that it honors convention over configuration, so you can just require the module and start using it.

You store your configuration in either plain js, or json files from /config folder. First it loads default.js file, then all other files from /config directory, then it loads environment specific configuration based on $NODE_ENV variable.

It also allows to override this configuration for local development with local.js or environment specific /config/env/$NODE_ENV.local.js.

You can take at look at it here:

https://www.npmjs.com/package/mikro-config

https://github.com/B4nan/mikro-config

For long, I used to use the approach mentioned in the solution here. There is a concern however, about security of the secrets in clear text. You can use another package on top of config so that the security bits are taken care of.

Check this out: https://www.attosol.com/secure-application-secrets-using-masterkey-in-azure-key-vault/

How we do it with TypeScript.

export const loadConfig = () => {
    const configLoadeded = configLoader.util.toObject() as any
    Config = configLoadeded
}

export interface ConfigI {
    productName: string;
    productId: string;
    googleCloudApiKey: string;
}

These days, when working with databases it is the easiest not to deal with configuration files at all, because deployment environments are easier to set up with just a single environment variable, call it DB_CONNECTION, for example, and pass it any additional configuration data as required.

configuration data example:

const config = {
    userIds: [1, 2, 3],
    serviceLimit: 100,
    // etc., configuration data of any complexity    
};
// or you can read it from a config file

Create a connection string, with extra parameters that the database driver doesn't care about:

import {ConnectionString} from 'connection-string';

const cs = new ConnectionString('postgres://localhost@dbname', {
    user: 'user-name',
    password: 'my-password',
    params: {
        config
    }  
});

Then we can generate the resulting string for storing it in the environment:

cs.toString();
//=>postgres://localhost:my-password@dbname?config=%7B%22userIds%22%3A%5B1%2C2%2C3%5D%2C%22serviceLimit%22%3A100%7D

So you store this in your environment, let's say, DB_CONNECTION, and within the client process you can just read it via process.env.DB_CONNECTION:

const cs = new ConnectionString(process.env.DB_CONNECTION);

const config = JSON.parse(cs.params?.config); // parse extra configuration
//=> { userIds: [ 1, 2, 3 ], serviceLimit: 100 }

This way you will have both the connection, and all the extra configuration needed, all within a single environment variable, no need messing with configuration files.

Try with properties-gen https://www.npmjs.com/package/properties-gen

Pretty similar to node-config which loads a configuration base file and extends it with an ext file depending on the environment you set, but this is a cli that runs on demand and is fully configurable.

npx properties-gen init

To create the cli configuration, then install it in your project

npm install properties-gen --save-dev

Define your configuration files (base and extensions) and run the generate command before build process or before starting dev server in your project.

{
  "name": "myApp",
  "scripts": {
    "config": "properties-gen generate",
    "dev": "npm run config && next dev",
    "build": "npm run config && next build",
    "start": "next start"
  }
}

One cool thing is you can define multiple configuration groups, in case you need to generate multiple outputs for example a client and server specific files.