Node.js and Oracle NoSQL Database

Datetime:2016-08-23 02:24:20          Topic: Oracle  Node.js  Nosql           Share

Oracle NoSQL Database is an interesting option to consider when you want a schemaless, fast, scale-able database which can provide relaxed (eventual) consistency. Oracle provides a Node.js driver for this database. In this blog I'll describe how to install Oracle NoSQL database and how to connect to it from a Node.js application.

The Node.js driver provided by Oracle is currently in preview version 3.3.7. It uses NoSQL client version 12.1.3.3.4 which does not work with 4.x versions of NoSQL database, so I downloaded Oracle NoSQL Database, Enterprise Edition 12cR1 (12.1.3.3.5) from here (the version number was closest to the version number of the client software).

NoSQL installation

To get NoSQL Database up and running, I followed the steps described in the installation manual  here . This was quite educational as it showed me the parts a NoSQL database consists of. If you want to do a quick installation, you can do the steps described  here . Do mind that the {} variable references do not work in every shell. On Oracle Linux 7.2 (inside Virtualbox) I did the following. First download kv-ee-3.3.5.zip here (previous versions on the Oracle NoSQL download page).

I made sure a Java was installed. I used a Oracle Java 1.8 64 bit.

I created the user oracle

mkdir -p /home/oracle/nosqldb/data

mkdir -p /home/oracle/nosqldb/root

cd /home/oracle/nosqldb

unzip kv-ee-3.3.5.zip

I updated /etc/environment

KVHOME="/home/oracle/nosqldb/kv-3.3.5"

KVDATA="/home/oracle/nosqldb/data"

KVROOT="/home/oracle/nosqldb/root"

Logout and login to make the environment active

Next I created an initial boot config

java -jar $KVHOME/lib/kvstore.jar makebootconfig -root $KVROOT -store-security none -capacity 1 -harange 5010,5030 -admin 5001 -port 5000 -memory_mb 1024 -host localhost -storagedir $KVDATA

I started the Oracle NoSQL Database Storage Agent(SNA).

nohup java -Xmx256m -Xms256m -jar $KVHOME/lib/kvstore.jar start -root $KVROOT &

And I configured it:

java -Xmx256m -Xms256m -jar $KVHOME/lib/kvstore.jar runadmin -port 5000 -host localhost <<EOF

configure -name mystore

plan deploy-zone -name "LocalZone" -rf 1 -wait

plan deploy-sn -zn zn1 -host localhost -port 5000 -wait

plan deploy-admin -sn sn1 -port 5001 -wait

pool create -name LocalPool

show topology

pool join -name LocalPool -sn sn1

topology create -name topo -pool LocalPool -partitions 10

topology preview -name topo

plan deploy-topology -name topo -wait

show plan

EOF

You can test if it is running by executing:

java -Xmx256m -Xms256m -jar $KVHOME/lib/kvstore.jar ping -port 5000 -host localhost

Oracle NoSQL Database basics

NoSQL Database has built-in load balancing, sharding and several other features related to high availability quite clearly integrated as an essential part of the software and not as some kind of add-on. Read more about it  here .

Also different consistency models are available. You can sacrifice immediate consistency to gain more performance. Read more about that  here .

It provides an Admin console to look at the topology and execution of plans. The console does not allow you to do actual changes to the configuration. For that you can use  a CLI .

It also allows you to browse log files

And look at performance details for specific nodes.

For development, kvlite available. It is a simple single node store which can be used locally. Read more here . When using kvlite, you do not need to do the configuration as described in the installation.

Node.js application

Installation of the Node.js module is easy. Just do a npm install nosqldb-oraclejs. The NoSQL Node.js driver page gives a piece of code which you can use to test your application. The default installation of NoSQL however as described above causes a port conflict with the proxy server which is started. This port conflict is not immediately clear as it gives you the below exception.

{ NoSQLDB-ConnectionError: Error with NoSQL DB Connection: Error verifying the proxy connection

NoSQLDB-ConnectionError: Error with NoSQL DB Connection: Connection timeout

at /home/oracle/nodejsnosql/node_modules/nosqldb-oraclejs/lib/store.js:277:25

at Timeout._onTimeout (/home/oracle/nodejsnosql/node_modules/nosqldb-oraclejs/lib/store.js:181:7)

at tryOnTimeout (timers.js:228:11)

at Timer.listOnTimeout (timers.js:202:5)

I changed the port and some log settings and used this test application . When I run it with node app.js I get the following output:

[2016-08-16 19:05:02.717] [INFO]  [at Object.startProxy (/home/oracle/nodejsnosql/node_modules/nosqldb-oraclejs/lib/proxy.js:353:10)] [PROXY] Start proxy

Connected to store

Table is created.

Inserting data...

Reading data...

Writing row #0

Writing row #1

Reading row #0

{ id: 0, name: 'name #0' }

Reading row #1

{ id: 1, name: 'name #1' }

Closing connection...

[2016-08-16 19:05:09.630] [INFO]  [at Store.close (/home/oracle/nodejsnosql/node_modules/nosqldb-oraclejs/lib/store.js:299:12)] Store close

Store connection closed.

Shutting down proxy.

Proxy closed.

How does this Node.js module work?

I read that the proxy translates network activity between the Node.js module and the Oracle NoSQL Database store. The proxy can be spawned as a child process (JVM) from the Node.js module. A JavaScript Thrift client (see  Thrift protocol ) has been generated with the Apache Thrift compiler which is used by the module to communicate with the proxy. The proxy then uses kvclient to connect to the database. This would be something like the image below.

I wondered what the performance cost would be of having a Java proxy and two translations between the Node.js module and the NoSQL database. It would be interesting to compare the bare RMI Java client performance with the Node.js module performance and to compare performance of a query executed from within the database with a query executed from outside by the RMI kvclient to determine the performance cost of the different hops/translations. I can understand the usage of Thrift though since it provides a relatively easy way to create clients in different languages.





About List