Unveiling Gavran: RavenDB re-written in C

Unveiling Gavran: RavenDB re-written in C

RavenDB is written in C# and .NET, unlike most of the database engines out there. The other databases are mostly written in C, C++ and sometimes Java.

I credit the fact that I wrote RavenDB in C# as a major part of the reason I was able to drive it forward to the point it is today. That wasn’t easy and there are a number of issues that we had to struggle with as a result of that decision. And, of course, all the other databases at the playground look at RavenDB strangely for being written in C#.

In RavenDB 4.0, we have made a lot of architectural changes. One of them was to replace some of the core functionality of RavenDB with a C library to handle core operations. Here is what this looks like:

EXPORT int32_t rvn_allocate_more_space(int64_t new_length_after_adjustment, void *handle, void **new_address, int32_t *detailed_error_code);

EXPORT int32_t rvn_open_journal_for_writes(const char *file_name, int32_t transaction_mode, int64_t initial_file_size, int32_t durability_support, void **handle, int64_t *actual_size, int32_t *detailed_error_code);

EXPORT int32_t rvn_close_journal(void* handle, int32_t* detailed_error_code);

EXPORT int32_t rvn_write_journal(void* handle, void* buffer, int64_t size, int64_t offset, int32_t* detailed_error_code);

EXPORT int32_t rvn_open_journal_for_reads(const char *file_name, void **handle, int32_t *detailed_error_code);

However, there is still a lot to be done in this regard and that is just a small core.

Due to the COVID restrictions, I found myself with some time on my hands and decided that I can spare a few weekends to re-write RavenDB from scratch in C. I considered using Rust, but that seemed like to be over the top.

The results of that can be seen here. I kept meticulous records of the process of building this, which I may end up publishing at some point. Here is an example of how the code looks like:

static result_t btree_defrag(txn_t* tx, page_t* p) { void* buffer; ensure(txn_alloc_temp(tx, PAGE_SIZE, &buffer)); memcpy(buffer, p->address, PAGE_SIZE); memset(p->address + p->metadata->tree.floor, 0, PAGE_SIZE - p->metadata->tree.floor); p->metadata->tree.ceiling = PAGE_SIZE; uint16_t* positions = p->address; size_t max_pos = p->metadata->tree.floor / sizeof(uint16_t); for (size_t i = 0; i < max_pos; i++) { uint64_t size; uint16_t cur_pos = positions[i]; void* end = varint_decode( varint_decode(buffer + cur_pos, &size) + size, &size); if (p->metadata->tree.page_flags == page_flags_tree_leaf) { end++; // flags } uint16_t entry_size = (uint16_t)(end - (buffer + cur_pos)); p->metadata->tree.ceiling -= entry_size; positions[i] = p->metadata->tree.ceiling; memcpy(p->address + p->metadata->tree.ceiling, buffer + cur_pos, entry_size); } return success(); }

The end result is that I was able to take all the knowledge of building and running RavenDB for so long and create a compatible system in not that much code. When reading the code, you’ll note methods like defer() and ensure(). I’m using compiler extensions and some macro magic to get a much nicer language support for RAII. That is pretty awesome to do in C, even if I say so myself and has dramatically reduced the cognitive load of writing with manual memory management.

An, of course, following my naming convention, Gavran is Raven in Croatian.

I’ll probably take some time to finish the actual integration, but I have very high hopes for the future of Gavran and its capabilities. I’m currently running benchmarks, you can expect them by May 35th.