Running LevelDB in Windows Azure Websites

Datetime:2016-08-23 00:49:09          Topic: Windows Azure  Leveldb  DataBase           Share

Running LevelDB in Windows Azure Websites

LevelDB is becoming an interesting space in the Node.js community. Some healthy innovation is showing how a modular approach to building your own database has some compelling benefits.

Don’t know what LevelDB is? Find out more here .

Windows Azure Websites offers a fast and convenient way to get Node.js applications running quickly. It’s backed by a persistent disk which is replicated for you, so why not make use of that free space by putting your database on there?

Here are some steps to getting  LevelDB  running in a Node.js application on Windows Azure Websites.

Prerequisites

Before you start, you must be using Windows (the 8.1 preview is currently free to download), and have the following installed:

Why 32 bit? Because node-gyp seems to target the processor architecture of the version of Node you have installed, and the shared (free) version of Windows Azure Websites is 32 bit only.

Adding LevelDB

Use NPM to add LevelDB to your app. The level modules includes levelup and leveldown:

$ npm install level

This will also compile the native leveldown module.

If you have any problems at this stage, I recommend cloning the leveldown repository , running `node-gyp configure`, and opening the .sln file located in the build directory. You’ll need to configure the solution to build Release mode.

Once you’ve installed level, you can start using the database in your code like this:

// create a database 
var db = require('level')('db');

// put a value
db.put('key1', 'value1');

// get a value
db.get('key1', function(err, data){
  console.log(data);
});

See the LevelUp documentation for more examples.

Configuring Git

(I’m assuming you’re using git to deploy this to Azure one way or another!)

Adding the modules to git

Because Leveldown is a native module, you can’t compile it on Azure. Therefore you must add the compiled binary to your git repo. That’s straight forward enough, but you’ll find that you have 60MB of dependencies, which is  quite a bit to upload to the Azure Website repo. It’s probably worth removing some of the unnecessary files before you add anything to git.

Trim down the unnecessary files

You can remove everything from this folder, with the exception of the ‘leveldown.node’ file (which is the DLL we just compiled).

node_modules\level\node_modules\leveldown\build\Release

You can also do away with this folder:

node_modules\level\node_modules\leveldown\deps

That should bring you down to about 1.2MB.

I found that large git pushes can take a very long time to Azure Websites. I think some throttling takes place after 4MB or so.

Ignore the database files

You should also add the ‘db’ folder (or whatever you choose to name your database) to your .gitignore file, as you don’t want updates to your application to overwrite your database.

Running in Azure

You should now be able to push your application to Windows Azure Websites, and your data will be persisted in LevelDB. However, there are a couple of rules you must stick to:

  1. You must keep the number of Node processes to exactly 1 . You can scale up to a ‘shared’ or ‘standard’ model, but you can’t scale out. LevelDB cannot be hosted within multiple processes.
  2. You are limited to 1GB of data on the disk

Conclusions

LevelDB offers a fast and convenient way to host a database in an Azure Web Site.

Websites run in a 32bit mode, I even found that the 63bit switch didn’t make any difference, but as long as we have the 32bit version of node installed we can compile the native module for this platform.

It’s worth keeping your git repo small by pruning out some of the build artifacts. I did this by deleting the files, but you could do the same with a .gitignore file.

You should keep your database files out of the repository using .gitignore.

You should be careful about the number of Node processes you have running, given the constraints of LevelDB.

It would be interesting to benchmark this against table storage to see if there any performance gains, perhaps another bog post?





About List