Docstore database engine for SWI-Prolog
Today I released the version 1.0.0 of my database engine for SWI-Prolog. It is an in-process in-memory
transactional engine that journals changes into a log file. It started as a project to get more flexibility out of
persistency.pl. The existing library was not thread-safe which required
wrapping every call with a mutex. I also had structured my other code
around property lists (or option lists, which are just terms like [key(value)]). That required conversion from one representation to
another.
Here is some example usage:
?- ds_insert(vehicle{year: 1926, make: chrysler, model: imperial}).
?- ds_insert(vehicle{year: 1953, make: chevrolet, model: corvette}).
?- ds_insert(vehicle{year: 1954, make: cadillac, model: fleetwood}).
?- ds_all(vehicle, List).
List = [
vehicle{'$id':'f3012622-cacb-4d7f-a22a-c36305274a80',
make:chrysler, model:imperial, year:1926},
vehicle{'$id':'23418d47-5835-41ff-a6b8-8748f3b2163e',
make:chevrolet, model:corvette, year:1953},
vehicle{'$id':'8c79f80f-d43e-4fad-a1bb-5fca23a195e0',
make:cadillac, model:fleetwood, year:1954}].
This is not the first database system I have built. I have done that in Turbo Pascal, too, in high school. It had excellent support for record files (typed files). But I think it had no transactions or anything else to support consistency. When the power goes out in the middle of write, some data might get not written. I found that transactions are easy to implement in Prolog. I currently use in-memory log during the transaction and write it into the real log file at the end of the transaction. Each transaction must end with a marker term in the log file. When the end term is missing the transaction is considered failed and the logged actions from it are discarded. This makes consistency easy to get. In my engine I only allow to run single transaction concurrently. This makes it less powerful than real databases but it is simpler to implement.
SWI-Prolog 7.x got a new data structure called dicts (they are also used in the
example above). At the first I disliked them but now I see that they are good alternative for property lists. They are supposed to be more
efficient, too. I had not seen them when I started the database project but decided to convert all of its predicates to accept/retrieve dicts. The
internal storage uses eav(id, key, value) representation anyway, not dicts nor property lists. This makes journaling individual key
value updates cheaper. The journal, by the way, is just a text file consisting of assert/retract calls and transaction begin/end markers.
The database supports hooks before_save and before_remove. Hooks will participate in the currently running transaction.
Failing hook or one that throws an exception will discard the transaction and changes made by it.
The old version of the database is powering this blog. I have not yet migrated all other libraries onto SWI 7.x and dicts but I hope to do it soon.
It's also my first Prolog pack I have published. I accidentally published version 0.0.1 when I tried out installation on my machine. SWI asks to "verify package status" when you install a pack. I had no idea it actually publishes it when you answer Y. Well, at least it's released now.