Lode — as in geological deposit or a vein of minerals — is a PStore of object data that can be mined for valuable information.

As noted in the PStore documentation all objects are marshaled which is not without caveats and dangers but, if you only need a simple object store, PStore is a solution. Lode takes this a step further by allowing you to have a pipeline workflow along with a Domain Specific Language (DSL) for creating, updating, finding, and deleting records.

Features

  • Built atop PStore.

  • Uses the Railway Pattern via Dry Monads for fault tolerant pipelines.

  • Emphasizes use of Hash, Data, Struct, or whole value objects in general.

  • Great for XDG caches, lightweight file-based databases, or simple file stores in general.

Requirements

  1. Ruby.

Setup

To install with security, run:

# 💡 Skip this line if you already have the public certificate installed.
gem cert --add <(curl --compressed --location https://alchemists.io/gems.pem)
gem install lode --trust-policy HighSecurity

To install without security, run:

gem install lode

You can also add the gem directly to your project:

bundle add lode

Once the gem is installed, you only need to require it:

require "lode"

Usage

To use, create a Lode instance and then use database-like messages to interact with your table of records as desired. For example, the following creates a table of links and stores them within the demo.store file and then interacts with those records:

lode = Lode.new "demo.store"

lode.write :links do
  create({id: 1, url: "https://one.com"})
  create({id: 2, url: "https://2.com"})
  create({id: 3, url: "https://three.com"})
end

# Success({:id=>3, :url=>"https://three.com"})
# (only the last record created is answered back)

lode.write(:links) { create({id: 4, url: "https://four.io"}) }
# Success({:id=>4, :url=>"https://four.io"})

lode.write(:links) { create({id: 4, url: "https://four.io"}) }
# Failure("Record exists for id: 4.")

lode.write(:links) { update({id: 1, url: "https://one.demo"}) }
# Success({:id=>1, :url=>"https://one.demo"})

lode.write(:links) { update({id: 5, url: "https://five.bogus"}) }
# Failure("Unable to find id: 5.")

lode.write(:links) { upsert({id: 2, url: "https://two.com"}) }
# Success({:id=>2, :url=>"https://two.com"})

lode.write(:links) { upsert({id: 5, url: "https://five.demo"}) }
# Success({:id=>5, :url=>"https://five.demo"})

lode.read(:links) { find 1 }
# Success({:id=>1, :url=>"https://one.demo"})

lode.read(:links) { find 13 }
# Failure("Unable to find id: 13.")

lode.write(:links) { delete 2 }
# Success({:id=>2, :url=>"https://two.com"})

lode.write(:links) { delete 13 }
# Failure("Unable to find id: 13.")

lode.read(:links, &:all)
# Success(
#   [
#     {:id=>1, :url=>"https://one.demo"},
#     {:id=>3, :url=>"https://three.com"},
#     {:id=>4, :url=>"https://four.io"},
#     {:id=>5, :url=>"https://five.demo"}
#   ]
# )

The default configuration is set up to use a primitive Hash which is the default behavior when using PStore. Everything answered back is a result monad as provided by the Dry Monads gem so you can leverage the Railway Pattern to build robust, fault tolerant, pipelines.

Configuration

Lode can be configured using a block or a keyword argument. The following are identical:

# With block.
lode = Lode.new "demo.store" do |config|
  config.mode = :max
  config.table = Lode::Tables::Value
  config.primary_key = :name
end

# With keyword argument.
configuration = Lode::Configuration[mode: :max, table: Lode::Tables::Value, primary_key: :name]
lode = Lode.new "demo.store", configuration:

The default configuration consists of the following attributes:

Lode::Configuration[
  store: PStore,
  mode: :default,
  table: Lode::Tables::Dictionary,
  primary_key: :id,
  registry: {}
]

Each key can be configured as follows:

  • store: Any object that adhere’s to the PStore Object API. You’ll most likely never need to change this but is available if desired. Default: PStore.

  • mode: The mode determines PStore behavior and can be one of the following:

    • :default: The default mode and is identical to PStore.new path.

    • :thread: Ensures a thread safe PStore instance is created. This is identical to PStore.new path, true.

    • :file: Ensures a file safe PStore instance is created. This is identical to setting store.ultra_safe = true on a PStore instance.

    • :max: Ensures a thread and file safe PStore instance is created for situations where you need maximum safety.

  • table: Defines the type of table used to interact with your records. The following values are supported:

    • Lode::Tables::Dictionary: The default value which allows you to interact with a Hash of records but would also work with any object that can respond to #[] and #[]=.

    • Lode::Tables::Value: Allows you to interact with whole value objects like Data, Struct, or whole value objects in general which have attribute readers and writers.

  • primary_key: Defines the primary key used when interacting with your table of records (useful when finding or upserting records). Default: :id.

  • registry: Used for registering default settings for your tables. This is not meant to be used directly but is documented for transparency.

Paths

Upon initialization, and when given a file, the file is only created once you start saving records. Although, when given a nested path, the full parent path will be created in anticipation of the file eventually being created. Example:

# The file, "demo.store", is not created until data is saved.
Lode.new "demo.store"

# The path, "a/nested/path", will be created so `demo.store` can eventually be saved.
Lode.new "a/nested/path/demo.store"

Registry

The registry is part of the configuration and directly accessible via a Lode instance. The registry allows you to customize individual table behavior as desired. For instance, you could have a Hash table or value table (i.e. Data, Struct, etc). Additionally, each table can have different primary keys too. The registry accepts three arguments in this format:

key, model:, primary_key:

The default model is a Hash but could be Data, Struct, or any value object. The default primary key is :id but could be any attribute that uniquely identifies a record. This means the following is identical when registering default table settings:

# Initialization with registration.
lode = Lode.new("demo.store") { |config| config.register :links, primary_key: :slug }

# Direct registration.
lode = Lode.new "demo.store"
lode.register :links, primary_key: :slug

Given the above, you could now create and find link records by slug like so:

lode.write(:links) { upsert({id: 1, slug: :demo, url: "https://demo.com"}) }
lode.read(:links) { find :demo }

# Success({:id=>1, :slug=>:demo, :url=>"https://demo.com"})

Keep in mind that the registry only defines default behavior. You can override default behavior by specifying a key. Example:

lode.read(:links) { find 1, key: :id }
# Success({:id=>1, :slug=>:demo, :url=>"https://demo.com"})

Even though the default primary key was registered to be :slug, we were able to use :id instead. The optional :key keyword argument is also available for all table methods.

Tables

As mentioned when configuring a Lode instance, two types of tables are available to you. The default (i.e. Lode::Tables::Dictionary) allows you to interact with Hash records which is compatible with default PStore functionality. Example:

lode = Lode.new "demo.store"
lode.write(:links) { upsert({id: 1, url: "https://one.com"}) }
# Success({:id=>1, :url=>"https://one.com"})

The second, and more powerful table type, is a value object table (i.e. Lode::Tables::Value). Here’s an example using a Data model:

Model = Data.define :id, :url

lode = Lode.new("demo.store") do |config|
  config.table = Lode::Tables::Value
  config.register :links, model: Model
end

lode.write :links do
  upsert({id: 1, url: "https://one.com"})
  upsert Model[id: 2, url: "https://two.com"]
end

lode.read(:links, &:all)
# Success([#<data Model id=1, url="https://one.com">, #<data Model id=2, url="https://two.com">])

The above would work with a Struct or any value object. One of many conveniences when using value objects — as shown above — is you can upsert records using a Hash or an instance of your value object.

Each table supports the following methods:

  • #primary_key: Answers the primary key as defined when the table was registered or the default key (i.e. :id).

  • #all: Answers all records for a table.

  • #find: Finds an existing record by primary key or answers a failure if not found.

  • #create: Creates a new record by primary key or answers a failure if record already exists.

  • #update: Updates an existing record by primary key or answers a failure if the record can’t be found.

  • #upsert: Creates or updates a new or existing record by primary key.

  • #delete: Deletes an existing record by primary key.

All of the above (except #primary_key) support an optional :key keyword argument which allows you to use a different key that is not the primary key if desired.

Read/Write

You’ve already seen a few examples of how to read and write to your object store but, to be explicit, the following are supported:

  • #read: Allows you to only read data from your object store in a single transaction. Any write operation will result in an exception.

  • #write: Allows you to write (and read) records in a single transaction.

Both of the above methods require you to supply the table name and a block with operations. Since a table name must always be supplied this means you can interact with multiple tables within the same file store or you can write different tables to different files. Up to you. Here’s an example of a basic write and read operation:

lode = Lode.new "demo.store"

# Read Only
lode.read(:links) { find 1 }

# Write/Read
lode.write(:links) { upsert({id: 1, url: "https://demo.com"}) }

Attempting to write within a read transaction will result in an error. For example, notice delete is being used within the read transaction which causes an exception:

lode.read(:links) { delete 1 }
# in read-only transaction (PStore::Error)

For those familiar with PStore behavior, a write and read operation is the equivalent of the following using PStore directly:

require "pstore"

store = PStore.new "demo.store"

# Write/Read
store.transaction do |store|
  store[:links] = store.fetch(:links, []).append({id: 1, url: "https://demo.com"})
end

# [{:id=>1, :url=>"https://demo.com"}]

# Read Only
store.transaction(true) { |store| store.fetch(:links, []).find { |record| record[:id] == 1 } }
# {:id=>1, :url=>"https://demo.com"}

Store

If at any time you need access to the original PStore instance, you can ask for it. Example:

lode = Lode.new "demo.store"
load.store

# #<PStore:0x000000010c592178 @abort=false, @filename="demo.store", @lock=#<Thread::Mutex:0x000000010c5fbfd8>, @thread_safe=false, @ultra_safe=false>

Development

To contribute, run:

git clone https://github.com/bkuhlmann/lode
cd lode
bin/setup

You can also use the IRB console for direct access to all objects:

bin/console

Tests

To test, run:

bin/rake

License

Security

Code of Conduct

Contributions

Developer Certificate of Origin

Versions

Community

Credits