Warp Performance Benchmarks

This page provides a comparison between Warp, and the open-source HyperDex system upon which it is based. If you land on this page from another website be sure to check out the HyperDex performance benchmarks to get an intuition for how the HyperDex system performs.

TPC-C

We evaluated the performance of Warp using an implementation of the TPC-C benchamrk. The TPC-C benchmark simulates an e-commerce application by specifying a mixed transaction workload. The workload specified by TPC-C is inherently difficult to process because it includes both read-heavy and update-heavy transaction profiles and the update-heavy transactions often include updates to a small number of keys in the system. For instance, the new-order transaction generates the order's identifier using a sequentially-increasing counter associated with one of one-hundred districts. The payment transaction increments the year-to-date totals for one of one-hundred districts and one of ten warehouses. Consequently, the workload necessarily requires that transactions operate on a small amount of shared data.

http://hyperdex.org/images/performance/data/tpcc-kv/throughput.svg

We compared Warp against HyperDex to quantify the overhead of introducing transactions on top of the TPC-C workload. The graph to the right shows that Warp's throughput is 96% the throughput of HyperDex. For completeness, we ran the same benchmark against an implementation of two-phase commit built on top of HyperDex. Warp achives more than three times higher throughput than this experimental implementation.

The average Warp transaction takes just 1.8% longer than the equivalent non-transactional operations in HyperDex. The key to ensuring this low overhead Warp's is the structure of linear transactions. Linear transactions contact the same number of servers, and perform the same amount of I/O per server, as the non-transactional equivalents. Consequently, the primary cost of a group of operations does not change when the group is performed within the context of a transaction.

Summary & Discussion

Using a standard transactional benchmark we evaluated HyperDex and HyperDex Warp. Overall, the overhead introduced by wrapping operations within a transaction is almost insignificant -- in the single digits for the workload we tested. Warp achieves performance on-par with HyperDex itself, while offering significantly the stronger guarantee of serializable transactions.

There's more to Warp than just it's high performance. We give a high-level overview of what makes Warp on our What's Warp page. Those who want to dig a little deeper can read our paper, "Warp: Lightweight Multi-Key Transactions for Key-Value Stores" available on our papers page.