Chapter 2
Installing HyperDex

HyperDex provides two means of installation for users. The easiest and most convenient way to install HyperDex is to use precompiled binary packages. Packages are available for a several platforms and more platforms will be supported as resources permit. For users who need more control over their installation, HyperDex provides the option to compile from source tarballs.

2.1 Installing Binary Packages

The HyperDex team maintains repositories for Debian, Ubuntu, and Fedora so that the latest version is always conveniently available.

2.1.1 Debian

To access the Debian repository, add the following to /etc/apt/sources.list:

  deb http://debian.hyperdex.org wheezy main

Subsequent invocations of the package manager may complain about the absence of the relevant package signing key. You can download the Debian public key and add it with:

  apt-key add hyperdex.gpg.key

The following code may be copied and pasted into a root terminal to quickly setup the Debian repository and install HyperDex:

  wget -O - http://debian.hyperdex.org/hyperdex.gpg.key \
  | apt-key add -
  wget -O /etc/apt/sources.list.d/hyperdex.list \
      http://debian.hyperdex.org/hyperdex.list
  aptitude update
  aptitude install hyperdex

2.1.2 Ubuntu

To access the Ubuntu repository, add the following to /etc/apt/sources.list:

  deb [arch=amd64] http://ubuntu.hyperdex.org precise main

Subsequent invocations of the package manager may complain about the absence of the relevant package signing key. You can download the Ubuntu public key and add it with:

  apt-key add hyperdex.gpg.key

The following code may be copied and pasted to quickly setup the Ubuntu repository and install HyperDex:

  sudo wget -O /etc/apt/sources.list.d/hyperdex.list \
      http://ubuntu.hyperdex.org/precise.list
  sudo wget -O - http://ubuntu.hyperdex.org/hyperdex.gpg.key \
  | sudo apt-key add -
  sudo apt-get update
  sudo apt-get install hyperdex

Change precise.list to reflect the version of Ubuntu that you are using.

2.1.3 Fedora

To access the Fedora repository, add the following to /etc/yum.conf:

  [hyperdex]
  name=hyperdex
  baseurl=http://fedora.hyperdex.org/
  enabled=1
  gpgcheck=0

To install HyperDex, issue the following command as root:

  yum install hyperdex

2.1.4 Package Names

Packages are named consistently across platforms. On all architectures, the hyperdex meta-package pulls in all HyperDex packages. For most users, installing the hyperdex package and any language packages will be sufficient.

For completeness, here is a list of all packages:

hyperdex-daemon
This package contains the HyperDex daemon that runs on each storage node. It required on every storage node.
libhyperdex-client
This package contains the client library for C/C++ bindings. It is required on all platforms which will access HyperDex.

On Debian and Ubuntu systems, this will have a small number appended to the package name indicating the version of the package contained within.

python-hyperdex-client
This provides the python module hyperdex.client. This package is only required for systems that need to interact with HyperDex from Python.
libhyperdex-coordinator
This package provides the coordinator for a HyperDex cluster. This package is required only on systems which will serve as the coordinator for the cluster.
replicant
This package provides part of the HyperDex coordinator and is only necessary on systems which will serve as the coordinator for the cluster.

Most packages are complemented by development and debug packages. In the development package, there are header files and static libraries. The debug packages provide symbols which will aid in providing tracebacks to the HyperDex developers. Please consult your package manager to find these packages.

2.2 Installing Source Tarballs

An alternative to installing a prepackaged binary is to install from source tarballs. This is a straightforward process that should work on most any recent Linux distribution for which there isn’t a prepackaged binary. We’ll first list the prerequisites to installing HyperDex from a source tarball. Then, we’ll describe how to configure HyperDex. Finally, we’ll describe the installation step.

2.2.1 Prerequisites

HyperDex has a minimal number of prerequisites for installation from source. Although we list all prerequisites in this section for completeness, the HyperDex configuration step will automatically warn about any missing dependencies.

Required Dependencies:

Dependencies for Python Bindings:

Dependencies for Java Bindings:

Dependencies for Yahoo! Cloud Serving Benchmark (YCSB):

2.2.2 Configuring

HyperDex uses the Autotools to make configuration and installation as straightforward as possible. After extracting the HyperDex tarball, you’ll need to configure HyperDex. The simplest configuration installs HyperDex in its default location (/usr/local) using the C++ compiler found on the system. The configuration is performed in the directory extracted from the tarball and looks like:

  ./configure

This basic configuration will configure the HyperDex daemon and native client library components to be built; however it omits several useful options for configuring HyperDex. The rest of this section will highlight common ways to configure HyperDex. Unless otherwise noted, all options should work well together.

2.2.2.1 Java Bindings

HyperDex does not build Java bindings by default. To enable the Java bindings, you must pass --enable-java-bindings to ./configure like so:

  ./configure --enable-java-bindings

If any of the prerequisites are missing ./configure should fail.

Java bindings are currently unavailable in 1.3.0.

2.2.2.2 Python Bindings

HyperDex does not build Python bindings by default. To enable the Python bindings, you must pass --enable-python-bindings to ./configure like so:

  ./configure --enable-python-bindings

If Python or its headers cannot be found, ./configure will fail.

2.2.2.3 Yahoo! Cloud Serving Benchmark

HyperDex provides all the source code necessary to build a HyperDex driver for the YCSB benchmark. If Java bindings are enabled, then YCSB can be built with --enable-ycsb.

  ./configure --enable-ycsb

Note that YCSB must be in your Java CLASSPATH. Configure cannot detect YCSB by itself.

2.2.2.4 Changing the Installation Directory

By default HyperDex installs files in /usr/local. If you’d like to install HyperDex elsewhere, you can specify the installation prefix at configure time. For example, to install HyperDex in /opt/hyperdex:

  ./configure --prefix=/opt/hyperdex

Check the --help option to configure for more ways to tune where HyperDex places files.

2.2.3 Installing

Once configured, HyperDex is simple to build and install. Keep in mind that the following commands may fail if the installation directory is not writable by the current user.

  make
  make install
  ldconfig

2.3 Installing from Git

Developers wishing to contribute to the development of HyperDex may build HyperDex directly from Git. Building from Git is as straightforward as building from source tarballs, but requires a few extra dependencies and some setup before the ./configure step.

In order to build the HyperDex repository, you’ll need to have the following utilities installed. Most of these utilities are prepacked for Linux distributions. Note that since these dependencies are only required for building from Git, they will not be detected at ./configure time and instead make will fail with an error message.

Required Dependencies:

You’ll need to build po6, e, BusyBee, HyperLevelDB Replicant, and HyperDex from Git, as the development version often introduces across repositories.

For each of these repositories, you may build and install the code with:

  autoreconf -i
  ./configure
  make
  make install
  ldconfig

2.3.1 Troubleshooting

2.3.1.1 Missing libtool

If your system is missing libtool, you’ll see an error message like the following:

  configure.ac:6: installing ‘./install-sh’
  configure.ac:6: installing ‘./missing’
  Makefile.am:48: Libtool library used but ‘LIBTOOL’ is undefined
  Makefile.am:48:   The usual way to define ‘LIBTOOL’ is to add ‘LT_INIT’
  Makefile.am:48:   to ‘configure.ac’ and run ‘aclocal’ and ‘autoconf’ again.
  Makefile.am:48:   If ‘LT_INIT’ is in ‘configure.ac’, make sure
  Makefile.am:48:   its definition is in aclocal’s search path.
  Makefile.am: installing ‘./depcomp’
  autoreconf: automake failed with exit status: 1
2.3.1.2 Missing pkgconfig

If your system is missing pkgconfig, you’ll see an error message like the following:

  ./configure: line 18348: syntax error near unexpected token ‘PO6,’
  ./configure: line 18348: ‘PKG_CHECK_MODULES(PO6, libpo6 >= 0.3.1)’

2.4 Verifying Installation

Once you have HyperDex installed, you should be able to view the built-in help. If the following commands provide meaningful output, then it is very likely that HyperDex is installed correctly and ready for use.

  hyperdex help
  hyperdex daemon --help

2.5 Upgrading to 1.3

Upgrading to 1.3 is a two step process. First, take a full backup of your cluster using the provided backup tools. Start the 1.3 cluster from the backup. This will upgrade the coordinator to 1.3, and the daemons will work with the old data directory.

Alternatively, take a backup, and restart the coordinator from the backed-up coordinator state on the same address. Then do a rolling restart of the daemons.

2.6 Upgrading to 1.2

The 1.2 release introduces new features that are backwards incompatible with previous releases. Before upgrading to 1.2, backup your data. Once 1.2 is installed, copy your data into a newly-deployed 1.2 cluster.