Chapter 2
Installing HyperDex

HyperDex provides multiple installation methods that provide easy access to HyperDex in a variety of environments. Those looking to play around with HyperDex and go through the tutorials may find it easiest to start with the quickstart Docker container. For Linux users, the easiest and most convenient way to install HyperDex is to use pre-compiled binary packages. Users of OS X have the option to install using Homebrew packages. Finally, HyperDex is also distributed as a set of tarballs so that power users may install HyperDex in a custom manner on their platform of choice.

2.1 Quick Installation with Docker

HyperDex supports running within Docker containers for easy deployment. With Docker, an application is isolated in its own container, unable to change the workstation outside the container. This makes it easy to rapidly deploy new applications without worrying about having to cleanup or revert any changes they may make to the OS. This sample environment is fully feature-complete, and applications developed against such an environment will work with distributed and sharded HyperDex clusters with zero application level changes.

For HyperDex, we can leverage containers to setup short-lived clusters that may be used for one-off tasks, rapid prototyping, or for going through the HyperDex tutorial—all without any commitment or modification to our base OS. The hyperdex/quickstart container includes everything necessary to run a single-node HyperDex cluster, including the coordinator, daemon, and Python bindings.

Running the HyperDex quickstart container is a simple three-step process:

1.
We’ll need Docker installed to use this container, which may be done by following the Docker installation instructions.
2.
We then fetch the HyperDex with the command docker pull hyperdex/quickstart.
3.
To start the cluster, run docker run --net=host -t -i hyperdex/quickstart. When all goes well, you’ll see the following message indicating your cluster is ready for use:
...  
daemon: I1230 19:59:50.547586    21 daemon.cc:492] reconfiguration complete...  
The transient HyperDex cluster is now online.  
 
This is a transient HyperDex cluster.  
You can connect to this cluster at address=172.17.3.27, port=1982.

With this quickstart cluster, both the coordinator and daemon logs will be displayed on stdout. You can connect to the cluster at athe address and port listed, and can stop the cluster at any time using the command displayed on your console. When you stop the cluster, the HyperDex cluster will be destroyed, and taken offline automatically. To fully clean up any disk space in use by the cluster, run docker rm b6ae7c60c275, where b6ae7c60c275 is the ID of your quickstart container.

2.2 Installing on CentOS/RedHat Using Binary Packages

The following code snippet will install HyperDex on any 64-bit CentOS 6 or 7 system. The script configures the HyperDex repository, imports the HyperDex signing key, and installs the hyperdex package. This script is available in the HyperDex tarball or repository at doc/install/centos-packages.sh.

  cat >> /etc/yum.conf << EOF
  [hyperdex]
  name=hyperdex

  baseurl=http://centos.hyperdex.org/base/\$basearch/\$releasever
  enabled=1
  gpgcheck=1
  EOF

  rpm --import http://centos.hyperdex.org/hyperdex.gpg.key
  yum install -y epel-release
  yum install -y hyperdex

Note: Users installing on Red Hat Enterprise Linux may have to follow the instructions to install Extra Packages for Enterprise Linux 1 prior to running the above code.

2.3 Installing on Debian Using Binary Packages

The following code snippet will install HyperDex on any 64-bit Debian 7 (Wheezy) system. The script configures the HyperDex repository, imports the HyperDex signing key, and installs the HyperDex package. This script is available in the HyperDex tarball or repository at doc/install/debian7-packages.sh.

  wget -O - http://debian.hyperdex.org/hyperdex.gpg.key | apt-key add -
  cat >> /etc/apt/sources.list.d/hyperdex.list << EOF

  deb [arch=amd64] http://debian.hyperdex.org wheezy main
  EOF
  apt-get update
  apt-get install -y hyperdex

2.4 Installing on Ubuntu Using Binary Packages

The following code snippets will install HyperDex on 64-bit Ubuntu systems. Pick the script that corresponds to the distribution you are running. Each script configures the HyperDex repository, imports the HyperDex signing key, and installs the HyperDex package. To figure out which version of Ubuntu you are using, cat /etc/os-release and look for the corresponding version and codename.

2.4.1 Ubuntu 12.04 Precise Pangolin

This script is available in the HyperDex tarball or repository at doc/install/ubuntu12.04-packages.sh.

  wget -O - http://ubuntu.hyperdex.org/hyperdex.gpg.key | apt-key add -
  cat >> /etc/apt/sources.list.d/hyperdex.list << EOF

  deb [arch=amd64] http://ubuntu.hyperdex.org precise main
  EOF
  apt-get update
  apt-get install -y hyperdex

2.4.2 Ubuntu 14.04 Trusty Tahr

This script is available in the HyperDex tarball or repository at doc/install/ubuntu14.04-packages.sh.

  wget -O - http://ubuntu.hyperdex.org/hyperdex.gpg.key | apt-key add -
  cat >> /etc/apt/sources.list.d/hyperdex.list << EOF

  deb [arch=amd64] http://ubuntu.hyperdex.org trusty main
  EOF
  apt-get update
  apt-get install -y hyperdex

2.5 Installing on Fedora Using Binary Packages

The following code snippet will install HyperDex on any 64-bit Fedora 20 system. The script configures the HyperDex repository, imports the HyperDex signing key, and installs the hyperdex package. This script is available in the HyperDex tarball or repository at doc/install/fedora-packages.sh.

  cat >> /etc/yum.conf << EOF
  [hyperdex]
  name=hyperdex
  baseurl=http://fedora.hyperdex.org/base/\$basearch/\$releasever

  enabled=1
  gpgcheck=1
  EOF
  rpm --import http://fedora.hyperdex.org/hyperdex.gpg.key
  yum install -y hyperdex

2.6 Installing on Linux Using Universal Binaries

HyperDex is also available as a self-contained universal binary that works on most glibc-based Linux distributions. To use this installation, first download the requisite tarball, and then you can install HyperDex like this:

  tar xzvf hyperdex-1.5.dev-linux-amd64.tar.gz -C /usr/local
  export PATH=/usr/local/hyperdex/bin:${PATH}

  hyperdex --version

2.7 Installing on OS X Using Homebrew

HyperDex is available on Mac OS X using the Homebrew package manager. The following script, available in the HyperDex tarball or repository at doc/install/homebrew.sh may be used to install HyperDex on OS X:

  brew tap HyperDex/hyperdex && brew install hyperdex

2.8 Installing From Source

Installing from source tarballs is a straightforward process that works on most any recent Linux distribution. Using the source tarballs provides power users with control over the installation process and the opportunity to customize the HyperDex installation.

To install from source, we must first prepare the system by installing a compiler, and dependencies that are not managed by the HyperDex team. This process will be highly system specific, but should be familiar to most users who install from source. The HyperDex team and community are always happy to provide assistance to users installing from source.

On systems like Ubuntu, we can prepare the system for the from-source install with these commands:

  sudo apt-get install -y build-essential libgoogle-glog-dev \
      libpopt-dev make pkg-config wget

  
  wget https://github.com/jedisct1/libsodium/releases/download/1.0.1/libsodium-1.0.1.tar.gz

  tar xzvf libsodium-1.0.1.tar.gz
  cd libsodium-1.0.1
  ./configure
  make
  sudo make install

With the prereqs installed, installing HyperDex from tarballs may be accomplished with this script:

  TARBALL_PO6=libpo6-0.5.2
  TARBALL_E=libe-0.8.2
  TARBALL_BUSYBEE=busybee-0.5.2
  TARBALL_LEVELDB=hyperleveldb-1.2.2

  TARBALL_REPLICANT=replicant-0.6.4
  TARBALL_MACAROONS=libmacaroons-0.2.0
  TARBALL_TREADSTONE=libtreadstone-0.1.0

  TARBALL_HYPERDEX=hyperdex-1.5.0
  
  for tarball in \
      ${TARBALL_PO6} ${TARBALL_E} ${TARBALL_BUSYBEE} \

      ${TARBALL_LEVELDB} ${TARBALL_REPLICANT} ${TARBALL_MACAROONS} \
      ${TARBALL_TREADSTONE} ${TARBALL_HYPERDEX}

  do
      wget http://hyperdex.org/src/${tarball}.tar.gz
      tar xzvf ${tarball}.tar.gz

      pushd ${tarball}
      ./configure
      make
      sudo make install
      popd
  done
2.8.0.1 Changing the Installation Directory

By default HyperDex installs files in /usr/local. If you’d like to install HyperDex elsewhere, you can specify the installation prefix at configure time. For example, to install HyperDex in /opt/hyperdex:

  export PKG_CONFIG_PATH=/opt/hyperdex/lib/pkgconfig
  ./configure --prefix=/opt/hyperdex

Check the --help option to configure for more ways to tune where HyperDex places files.

2.8.1 Troubleshooting

This section walks through some common installation problems that might be encountered while installing HyperDex from source.

2.8.1.1 Missing libtool

If your system is missing libtool, you’ll see an error message like the following:

  configure.ac:6: installing ‘./install-sh’
  configure.ac:6: installing ‘./missing’
  Makefile.am:48: Libtool library used but ‘LIBTOOL’ is undefined
  Makefile.am:48:   The usual way to define ‘LIBTOOL’ is to add ‘LT_INIT’
  Makefile.am:48:   to ‘configure.ac’ and run ‘aclocal’ and ‘autoconf’ again.
  Makefile.am:48:   If ‘LT_INIT’ is in ‘configure.ac’, make sure
  Makefile.am:48:   its definition is in aclocal’s search path.
  Makefile.am: installing ‘./depcomp’
  autoreconf: automake failed with exit status: 1

To correct this issue, install the libtool package using your distribution’s package manager.

2.8.1.2 Missing pkgconfig

If your system is missing pkgconfig, you’ll see an error message like the following:

  ./configure: line 18348: syntax error near unexpected token ‘PO6,’
  ./configure: line 18348: ‘PKG_CHECK_MODULES(PO6, libpo6 >= 0.3.1)’

To correct this issue, install the pkg-config package (sometimes called pkgconfig) using your distribution’s package manager.

2.9 Verifying Installation

Once you have HyperDex installed, you should be able to view the built-in help. If the following commands provide meaningful output, then it is very likely that HyperDex is installed correctly and ready for use.

  hyperdex help
  hyperdex daemon --help

2.10 Upgrading to 1.6

The 1.6 release introduces new features that are backwards incompatible with previous releases, most notably a binary document format. Before upgrading to 1.6, backup your data. Once 1.6 is installed, copy your data into a newly-deployed 1.6 cluster.

2.11 Upgrading to 1.5

HyperDex 1.5 is backwards compatible with the 1.4 on-disk format. To upgrade from 1.4 to 1.5, take a backup and restart the cluster from the backup.

2.12 Upgrading to 1.4

Upgrading to 1.4 is a two step process. First, take a full backup of your cluster using the provided backup tools. Start the 1.4 cluster from the backup. This will upgrade the coordinator to 1.4, and the daemons will work with the old data directory.

Alternatively, take a backup, and restart the coordinator from the backed-up coordinator state on the same address. Then do a rolling restart of the daemons.

The 1.4 release’s on-disk format is backwards incompatible with the 1.3 and prior releases. HyperDex daemons will automatically upgrade to the 1.4 format when first launched.

2.13 Upgrading to 1.3

Upgrading to 1.3 is a two step process. First, take a full backup of your cluster using the provided backup tools. Start the 1.3 cluster from the backup. This will upgrade the coordinator to 1.3, and the daemons will work with the old data directory.

Alternatively, take a backup, and restart the coordinator from the backed-up coordinator state on the same address. Then do a rolling restart of the daemons.

2.14 Upgrading to 1.2

The 1.2 release introduces new features that are backwards incompatible with previous releases. Before upgrading to 1.2, backup your data. Once 1.2 is installed, copy your data into a newly-deployed 1.2 cluster.

1https://fedoraproject.org/wiki/EPEL