Home // DBKDA 2012, The Fourth International Conference on Advances in Databases, Knowledge, and Data Applications // View article

Sparse Dictionaries for In-Memory Column Stores

Authors:
Johannes Wust
Jens Krueger
Martin Grund
Uwe Hartmann
Hasso Plattner

Keywords: Column-store; In-memory databases; Dictionary compression; Write-optimized Store; Read-optimized Store

Abstract:
A common approach to achieve high read and write performance for column-oriented in-memory databases is to separate the data store into a read-optimized main partition and a write-optimized differential buffer. The differential buffer has to be merged into the main partition periodically to preserve read performance and increase memory efficiency. If data is dictionary-compressed, this merge process becomes time-consuming since it involves rewriting the whole main tables if dictionary mappings are changed. To reduce the merge time, we introduce the concept of sparse dictionaries, which can avoid rewriting the whole table in many cases. The basic idea is to place gaps in the dictionaries of the main partition which allows us to merge the differential buffer into the main partition without expensive recompression of the main tables. We leverage known data characteristics to optimize our algorithms for enterprise applications.

Pages: 25 to 33

Copyright: Copyright (c) IARIA, 2012

Publication date: February 29, 2012

Published in: conference

ISSN: 2308-4332

ISBN: 978-1-61208-185-4

Location: Saint Gilles, Reunion

Dates: from February 29, 2012 to March 5, 2012