How to set up Apache Storm on mac using brew

Datetime:2016-08-23 00:32:35          Topic: ZooKeeper           Share

I am learning about Apache Storm and the guide had long instructions on how to install Apache Storm . However the steps were too lengthy. I installed the single node using brew and it piece was breeze. I did not had to spend time figuring out out if I was installing latest version.

Pre-requisites:

Make sure that you have ‘ HomeBrew ‘ installed. One reason that I prefer ‘brew’ is that I do not have to remember where things are installed and brew manages and fixes any updates that have been introduced. It also makes installation and uninstall super easy. Everything this is saved under /url/local/Cellar.

Step 1: Install Zookeeper

‘ brew install zookeeper’

After the installation is complete, you will see this message.

To have launchd start zookeeper at login:

ln -sfv /usr/local/opt/zookeeper/*.plist ~/Library/LaunchAgents

Then to load zookeeper now:

launchctl load ~/Library/LaunchAgents/homebrew.mxcl.zookeeper.plist

Or, if you don’t want/need launchctl, you can just run:

zkServer start

I prefer not to launch zookeeper at login as it will sow down my mac login. So I prefer to start and stop Zookeeper using the command-

zkServer start/stop’

Step 2: Install ZeroMQ

‘brew install zeromq’

Step 3: Install Apache storm

‘brew install storm’

This above command will install everything in Cellar folder and will have a symlink in ‘/usr/local/opt/storm’ folder.

Step 4: Update the storm config file storm.yaml.

This file will be in /usr/local/opt/storm/libexec/conf folder.

After opening this file you will see that everything is commented out. So we will add the following.

storm.zookeeper.servers:

– “localhost”

# – “server2”

#

nimbus.host: “localhost”

nimbus.thrift.port: 6627

ui.port: 8772

storm.local.dir: “/Users/dinesharora/storm/data”

java.library.path: /usr/lib/jvm”

supervisor.slots.ports:

– 6700

– 6701

– 6702

– 6703

If you will notice that I have added “storm.local.dir”. Make sure that whatever folder you create, they have the right permissions because I initially added “ /home/user/storm/data ” and did not realize that you cannot create any directory under “ /home ” folder. This was causing issues when I started nimbus and supervisor.

Step 5: start zookeeper, nimbus, supervisor and ui

Below are the location on shell scripts. Start them in the given sequence

‘zkServer start’

‘/usr/local/opt/storm/libexec/bin/storm nimbus’

‘/usr/local/opt/storm/libexec/bin/storm supervisor’

‘/usr/local/opt/storm/libexec/bin/storm ui’

Step 6: Check that everything is running smoothly.

To do that run the below command.

‘jps’

The above command should give the following result.

Dineshs-MBP:bin dinesharora$ jps

5282 supervisor

5267 nimbus

5460 core

5735 Jps

4235 QuorumPeerMain

Step 7: Check the server index page. Access the url – ‘http://localhost:8772/index.html’.

If you are able to access the page then your installation is successful.

P.S. If you upgraded to new OSX – El Captain then you might have issues with brew itself. Do yourself a favor and reinstall brew again.

~ Stir a storm





About List