SisoDb – First code released

Recently I started playing with Lucene.Net and Json.Net for the reason of creating my own solution for persisting hierarchial-structures of data. The goal is to make it schemaless and really simple. The result was: SisoDb. As of right now it got support for:

  • Commitable Unit of work
  • Insert
  • Update
  • DeleteByKey
  • GetByKey

Roadmap

My focus in the next couple of spent hours will lie on:

  • Querying support both for Indexes as well as free text querying using Lucene
  • Version-control for concurrency detection
  • Some support for relations
  • Distrubuted proxy

Some basics

SisoDb persists Structures. The only demand that is puts on the structures being persisted is that they must have a property that contains a Guid which acts as the unique-identifier for the Structure. By conventions, this member should be named Id but you can configure this via schemas. It can be System.Nullable or System.Guid. You don’t have to provide a value, since SisoDb will generate this for you upon insert. Contained objects in the Structure should not have an Id-property. A Structure should be seen as an aggregate where contained objects and properties are leafs of data, either represented of simple value types (strings, ints, decimals, datetimes) or complex types (your own types). The Structure will be represented in the database as Json, which is produced using Newtonsofts Json.Net library. Which means, you can use the attributes of this library to affect how the structure is serialized and deserialized. As of right now there is one underlying provider, which stores the Structures (the Json, Key and Indexes) using Lucene.Net. I’m thinking of building a provider that uses MsSql as the storage media, but as of right now, it’s Lucene that is used.

An example

Create a Structure

The example model is really simple. A Customer-entity containing some simple properties for naming as well as Billing-address and Delivery-address, which is represented by a custom complex type, Address.

[Serializable]
public class Customer
{
    public Guid Id { get; set; }
    public string Firstname { get; set; }
    public string Lastname { get; set; }
    public ShoppingIndexes ShoppingIndex { get; set; }
    public DateTime CustomerSince { get; set; }
    public Address BillingAddress { get; set; }
    public Address DeliveryAddress { get; set; }

    public Customer()
    {
        ShoppingIndex = ShoppingIndexes.Level0;
        BillingAddress = new Address();
        DeliveryAddress = new Address();
    }
}

[Serializable]
public class Address
{
    public string Street { get; set; }
    public string Zip { get; set; }
    public string City { get; set; }
    public string Country { get; set; }
}

[Serializable]
public enum ShoppingIndexes
{
    Level0 = 0,
    Level1 = 10,
    Level2 = 20,
    Level3 = 30
}

Create a Database

An instance of a SisoDatabase implementation is supposed to be long lived. It holds meta-data/schema-information about the structures being persisted. The SisoDatabase takes an IConnectionInfo which lets you define which underlying database-implementation to use as well as a connectionstring. When using the LuceneIO-provider, the connectionstring should contain the path to the directory where the data (indexes to use the correct Lucene terminology) should be stored.

var connectionInfo = new SisoConnectionInfo(StorageProviders.LuceneIo, DbPath);
var database = new SisoDatabase(connectionInfo);

This is something you would put in your IoC-container or at least in a factory.

Create a Unit-of-work & Insert a customer

The Unit-of-work is designed to be short lived. It is used for performing CRUD-operations against the database. It implements IDisposable and it’s recomended to be used in conjunction with the using-statement. If you don’t call Commit, ongoing changes aren’t flushed against the database. If you don’t use the using-statement make sure to call Dispose since it ensures that write-locks are being released.

var customer = new Customer
{
    Firstname = "Daniel",
    Lastname = "Wertheim",
    ShoppingIndex = ShoppingIndexes.Level1,
    CustomerSince = DateTime.Now,
    BillingAddress =
        {
            Street = "The street",
            Zip = "12345", City = "The City",
            Country = "Sweden"
        }
};

using (var unitOfWork = database.CreateUnitOfWork())
{
    unitOfWork.Insert(customer);
    unitOfWork.Commit();
}

Refetch and Update Customer

As of right now the only query-support is GetByKey which lets you retrieve a single item by a key-value (Guid).

using (var unitOfWork = database.CreateUnitOfWork())
{
    var customer = unitOfWork.GetByKey<Customer>(id);
    customer.ShoppingIndex = ShoppingIndexes.Level2;
    customer.Firstname = "Hans";

    unitOfWork.Update(customer);
    unitOfWork.Commit();
}

Delete Customer

As of right now you can only delete via the key-value.

using (var unitOfWork = database.CreateUnitOfWork())
{
    unitOfWork.DeleteByKey<Customer>(customer.Id);
    unitOfWork.Commit();
}

Custom key

As told, per default, you don’t have to provide any schema-information as long as the root-entity (structure) contains an Id:Guid or Id:Nullable -member. If you want to use another key-member, this is how you do it.

database.Schemas.Register<Customer>(
    builder => builder
        .SetKey(c => c.UId)
        .AddIndex(c => c.Lastname));

That’s it for now. The current code can be found here.

//Daniel

SisoDb – Some code preview

I thought I would share some code of how my SisoDb (Simple-Structure-Oriented-Db) looks like.

As of right now you have to provide info about which member contains the key. This will change and will instead be convention based.

var schemaBuilder = new ManualSchemaBuilder();
schemaBuilder.Register<Customer>(builder => builder.SetKey(c => c.Id));

You can also use SchemaBuilder.AddIndex to provide specific indexes that you wan’t to query more efficiently.

var schemaBuilder = new ManualSchemaBuilder();
schemaBuilder.Register<Customer>(
    builder => builder
        .SetKey(c => c.Id)
        .AddIndex(c => c.Lastname));

The next step is to create a database. Supported options are I/O-based or a Virtual (in-memory) database. The virtual only lasts as long as the UnitOfWork lives. The database is designed to be long-lived and is something you would keep alive, preferable using your IoC-container.

var connectionInfo = new LuceneConnectionInfo(LuceneDbTypes.Io, @"C:\#Temp\SisoDb\ConsoleSample");
var database = new LuceneDatabase(connectionInfo);

var schemas = schemaBuilder.GetSchemas();
database.RegisterSchemas(schemas);

Now we are ready to add some data, so lets create a Customer and add it using UnitOfWork

var customer = new Customer
                    {
                        Firstname = "Daniel",
                        Lastname = "Wertheim",
                        ShoppingIndex = ShoppingIndexes.Level1,
                        CustomerSince = DateTime.Now
                    };

using (var unitOfWork = database.CreateUnitOfWork())
{
    unitOfWork.Insert(customer);
    unitOfWork.Commit();
}

That’s it. If Commit() isn’t called everything is rolled back.

I will soon publish the source code on Google Code.

//Daniel

Writing my own NoSql DB?

Yesterday I got a thought:

why not write something very simple that can store object-graphs without mappings and other fuss.

Yes I know there’s MongoDb, RavenDb and several others, but it’s always a great deal of fun to write something of your own. So, inspired by Ayende’s technology choices, I spent a few hours last night just fiddling around with Lucene.Net and Json.Net. The result:

A simple model

public class Address
{
    public string Street { get; set; }
    public string Zip { get; set; }
    public string City { get; set; }
    public string Country { get; set; }
}

public class Customer
{
    [Key]
    public Guid? Id { get; set; }

    [Index]
    public string Firstname { get; set; }

    [Index]
    public string Lastname { get; set; }

    [Index]
    public int ShoppingIndex { get; set; }

    public Address BillingAddress { get; private set; }
    public Address DeliveryAddress { get; private set; }

    public Customer()
    {
        BillingAddress = new Address();
        DeliveryAddress = new Address();
    }
}

Consuming a Storage-provider

var customer = new Customer
                    {
                        Id = Guid.NewGuid(),
                        Firstname = "Daniel",
                        Lastname = "Wertheim",
                        ShoppingIndex = 99
                    };
customer.DeliveryAddress.Country = "Sweden";

var store = new LuceneStructureStore();
store.Insert(customer);

var refetched = store.GetByKey<Customer>(customer.Id.ToString());
...
...

Maybe it will grow to something useful. In the meantime I will continue my work with my MongoDB-provider, Simple-MongoDB.

//Daniel