Iterators, and Snapshots for LevelDB UWP

Datetime:2016-08-23 00:45:35          Topic: Leveldb           Share

Introduction

Inlast article I introduced the basics of LevelDB. I introduced the basics on how to install the library and use basic Get, Put, Delete, and Batch operations. In this article I will go a little deeper and introduce iterators, and snapshots feature of LevelDB.

Creating Snapshots

Snapshots are extremely powerful constructs that let's you freeze a particular view for your read operations. Snapshots provide consistent read-only views for all the read operations. When reading values from database it's quite possible other instances of DB objects are making changes to database. In certain scenarios you might want to freeze a snapshot of database at particular point in time. This is where Snapshot object will help you. Now since Snapshot has to keep some additional data and resources while you are doing reads it must be released once you are doing with your operation. Creating and releasing snapshot is super simple:

using (var snapshot = db.GetSnapshot())
{
    ...
}

If you are not writing using blocks in your code, be sure to call snapshot.Dispose() .

Using Snapshots

Each ReadOption object can take a Snapshot property, and each read operation requires you to pass ReadOption object. Once you have snapshot you can use it as following code illustrates:

using (var snapshot = db.GetSnapshot())
{
    var val = db.Get(new ReadOptions { Snapshot = snapshot }, Slice.FromString("foo"));
}

Iterators

Iterators ofcourse are required when you want to iterate over the data stored in your key-value store. Now I won't go into details of comparators but by default (LevelDB UWP) orders data by key bytes lexicographically. This behaviour can be customized with comparators, but it's out of scope for this article. So the best way to summarize the default key order is imagining my key value store has following key value paris inserted:

db.Put(Slice.FromString("b"), Slice.FromString("Two"))
db.Put(Slice.FromString("d"), Slice.FromString("Three"))
db.Put(Slice.FromString("a"), Slice.FromString("One"))

It would be stored in store in following order: a => One, b => Two, d => Three . This means iterating from begining to end you will encounter keys in order of a, b, c since a < b < c . Keeping this in mind let's create an iterator that iterates over all key value pairs in database:

using (var itr = db.NewIterator(new ReadOptions()))
{
    itr.SeekFirst();
    while (itr.Valid())
    {
        byte[] key = itr.Key().ToByteArray();
        byte[] val = itr.Value().ToByteArray();
        itr.Next();
    }
}

As you can see just like anything else an iterator must be disposed when you don't need it anymore. Iterator has methods like Seek , SeekToFirst , SeekToLast , Valid , Next , and Prev to navigate between your records ( Checkout documentation ). Additionally it provides method Key, and Value to read the slices themselves.

Using Seek you can do powerful things like iterating over records with key X..Y, for example:

using (var itr = db.NewIterator(new ReadOptions()))
{
    itr.Seek(Slice.FromString("record001"));
    while (itr.Valid())
    {
        var keySlice = itr.Key();
        if (keySlice.ToString() == "record007") break;
        // use keySlice
        itr.Next();
    }
}

Same can be done in reverse order:

using (var itr = db.NewIterator(new ReadOptions()))
{
    itr.Seek(Slice.FromString("record007"));
    while (itr.Valid())
    {
        var keySlice = itr.Key();
        if (keySlice.ToString() == "record001") break;
        // use keySlice
        itr.Prev();
    }
}

Just remeber doing iteration in reverse order is an expensive operation and thus a little slower. Try using forward iterations whenever possible.

Combining Snapshots and Iterators

We can fuse iterators with snapshots to create really powerful features like getting set of key-value pairs at a particular instance in time while mutations are happening (something close to MVCC). It can't be any simpler:

using (var snapshot = db.NewSnapshot())
using (var itr = db.NewIterator(new ReadOptions({ Snapshot = snapshot })))
{
  // use itr
}

Combining WriteBatches and Iterators with proper synchronization can yield powerful results. Imagination is your limit.

Conclusion

This completes the basic usage of LevelDB. There is more advanced stuff that I will visit in future articles. You can look into detailed documentation by going to LevelDB UWP Wiki .





About List