Clever Geek Handbook

Key / Value database for storing binary data

I am looking for a lightweight, reliable and fast key / value database for storing binary data. Simple without a server. Most of the popular underlying databases like CDB and BerkeleyDB don't actually store BLOB

. What could be the best choice that I have missed?

My current choice is SQLite, but it is too advanced for my simple use.

+3


source to share


6 answers


As stated earlier, BerkeleyDB supports opaque values ​​and keys, but I would suggest a better alternative: LevelDB.

LevelDB:

Google is your friend :), so much so that they even provide you with a built-in database: A fast and lightweight key / value database library from Google.



Features:

  • The keys and values ​​are arbitrary byte arrays.
  • The data is stored sorted by key.
  • Subscribers can provide a custom compare function to override the sort order.
  • The main operations are Put (key, value), Get (key), Delete (key).
  • Several changes can be made in one atomic batch.
  • Users can create a temporary snapshot to get a consistent view of the data.
  • Forward and backward iteration is supported by data.
  • Data is automatically compressed using the Snappy compression library.
  • External activity (file system operations, etc.) is relayed through the virtual interface so that users can customize the interaction of the operating system.
  • Detailed documentation on how to use the library is included in the source code.
+7


source


What makes you think that BerkDB cannot store binary data? From their docs:

The keys and content arguments are objects described in the typedef. Byte indicates the string of dsize bytes pointed to by dptr. Arbitrary binary data as well as plain text strings are allowed.



Also see their examples :

money = 122.45;
key.data = &money;
key.size = sizeof(float);
...
ret = my_database->put(my_database, NULL, &key, &data, DB_NOOVERWRITE);
+3


source


If you don't need "multiple writes" (only a few readers work), want something small and want something that is available for almost every Linux, you can take a look at gdbm, which is similar to berkeley db, but much simpler. Also maybe not so fast.

In almost the same area, there are things like tokyocabinet, qdbm and the already mentioned leveldb.

Berkeley db and sqlite are ahead of them because they support multiple authors. Sometimes berkeley db is the desaster version.

The main thing about gdbm: it is already on every Linux, without problems with the version, small.

+1


source


What OS are you using? For Windows, you can check out ESE , which is a very robust storage engine that comes as part of the OS. It includes Active Directory, Exchange, DNS, and several other Microsoft server products and is used by many third party products ( RavenDB comes to mind as a good example).

0


source


You can take a look at http://www.codeproject.com/Articles/316816/RaptorDB-The-Key-Value-Store-V2 my friend Databases.

0


source


Using sqlite is now straight forward with the new readfile (X) and write (X, Y) functions available from 2014-06. For example, to save blob in database from file and retrieve it again

     CREATE TABLE images(name TEXT, type TEXT, img BLOB);
     INSERT INTO images(name,type,img) VALUES('icon','jpeg',readfile('icon.jpg'));
     SELECT writefile('icon-copy2.jpg',img) FROM images WHERE name='icon';

see https://www.sqlite.org/cli.html "File I / O Functions"

0


source






More articles:


All Articles