a datomic interface

Author: Chris Zheng  (
Library: v0.3.1-SNAPSHOT
Date: 19 December 2014
Generated By: MidjeDoc

1   Introduction

2   Tutorial

  2.1   Step One
    2.1.1   Schema
    2.1.2   Connection
    2.1.3   Writing
    2.1.4   Reading
    2.1.5   Restrictions
    2.1.6   Time Travel
  2.2   Step Two
    2.2.1   Enums
    2.2.2   Updating
    2.2.3   Selection
    2.2.4   Java
  2.3   Step Three
    2.3.1   Refs
    2.3.2   Walking
    2.3.3   Data Access
    2.3.4   Pointers
    2.3.5   Playground

3   Example - Bookstore

  3.1   Definition and Setup
  3.2   Updating Data
  3.3   Different Semantics, Same Result
  3.4   Expanding the Business
  3.5   Moving Stock

4   Example - Schoolyard

  4.1   Data Setup
  4.2   Datomic
  4.3   Querying
  4.4   Datalog Generation
  4.5   Expressivity


a datomic interface

Author: Chris Zheng  (
Library: v0.3.1-SNAPSHOT
Date: 19 December 2014
Generated By: MidjeDoc

1    Introduction


Add to project.clj dependencies

[im.chit/adi "0.3.1-SNAPSHOT"].

All the functionality can be loaded using:

(require '[adi.core :as adi])


adi makes creating complex applications easy. Built on top of datomic, adi was especially designed to model complexity, relationships and partial data. Having said that, adi is for the faint of heart. It has been built especially for the creative, crazy and super ambitious. Small apps with simple data models are very easy to define and explore. But why stop there? The bigger, more intertwined the data model becomes, the more adi will have its chance to shine.

So what exactly is adi?

The key to understanding adi lies in understanding the power of a schema. The schema dictates what you can do with the data. Instead of limiting the programmer, the schema should exhance him/her, much like what a type system does for programmers but without being as restrictive. Once we have a schema, there should be no difference in the shape of the query and the shape of the data that we get back from the query. We shouldn't have to play around turning objects into records, objects into queries, etc. Furthermore, data can be queried and inserted in ANY shape or form, as long as it follows the conventions specified within that schema.

2    Tutorial

So open up a repl in a project with adi installed and require load up the library:

(require '[adi.core :as adi])

We are off and away!

2.1    Step One

2.1.1    Schema

A simple adi schema is constructed as follows:

(def schema-1
  {:account/user     [{:type :string      ;; (1)
                       :cardinality :one
                       :unique :value     ;; (2)
                       :required true}]   ;; (3)
   :account/password [{:required true     ;; (1) (3)
                       :restrict ["password needs an integer to be in the string"
                                  #(re-find #"\d" %)]}] ;; (4)
   :account/credits  [{:type :long
                       :default 0}]})

There are a couple of things to note about this particular schema:

  1. We specified the :type for :account/user to be :string and :cardinality to be :one. However, because these are default options, we can optionally leave them out for :account/password.
  2. We want the value of :account/user to be unique.
  3. We require that both :account/user and :account/password to be present on insertion.
  4. We are checking that :account/password contains at least one number
  5. We set the default amount of :account/credits to be 0

2.1.2    Connection

So having a basic schema, an adi datastore can be constructed. Note that we are connecting to a standard datomic in memory store url. The philosophy of adi is that it should work seamlessly with datomic. From experience, adi should provide enough expressiveness to do about 95% of all the CRUD in an application. The user can perform more complicated or optimised queries by dropping back into the datomic api if needed:

(def ds (adi/connect! "datomic:mem://adi-examples-step-1" schema-1 true true))

The last two arguments are flags for reset? and install-schema?. To blow away the old datastare and construct a brand new one, set both flags as true. If connecting to an already existing datastore set both flags as false (by default) otherwise all data will be lost.

2.1.3    Writing

Once a datastore has been established, lets add some records. Note that multiple records can be added by using a vector of multiple records:

(adi/insert! ds {:account {:user "angela"   :password "hello1"}})

(adi/insert! ds [{:account {:user "billy"    :password "pass1"}}
                 {:account {:user "carl"     :password "pass1"}}])

2.1.4    Reading

Now that there is data, we can then do a search for the record using select.

(adi/select ds {:account {:password "pass1"}})

=> #{{:account {:user "billy", :password "pass1", :credits 0}}
     {:account {:user "carl", :password "pass1", :credits 0}}}

We can use the :first option to pull the first entry of the set. This is useful when you know that there is only one result from the query. We can also use the :ids option to add the entity :db/id field onto the data:

(adi/select ds {:account {:user "angela"}} :first true :ids true)

=> {:db {:id 17592186045418}, :account {:user "angela", :password "hello1", :credits 0}}

For option keys such as :first and :ids, we can just use the keyword itself, so this invocation works as well

(adi/select ds {:account {:user "angela"}} :first :ids)

=> {:db {:id 17592186045418}, :account {:user "angela", :password "hello1", :credits 0}}

2.1.5    Restrictions

We now learn what we can't do and how the schema helps in keeping our data regular. Lets try to add in some incomplete data:

(adi/insert! ds {:account {:credits 10}})

=> (throws Exception "The following keys are required: #{:account/password :account/user}")
(adi/insert! ds {:account {:user "adi"}})

=> (throws Exception "The following keys are required: #{:account/password}")

As can be seen by the examples above, because we have set the :required attribute to true, both :account/password and :account/user are needed before the data is considered valid input. There is also an additional :restrict attribute for the password field and this takes care of additional validation on inputs. We see below that the insert fails because the password has to contain at least one integer:

(adi/insert! ds {:account {:user "adi" :password "hello"}})

=> (raises-issue {:failed-restriction true})

Any data that lies outside of the schema will also cause the insert to fail. This can be disabled through using access models but for now, we will let the insert fail on insertion of an :account/type field

(adi/insert! ds {:account {:user "adi" :password "hello1" :type :vip}})

=> (raises-issue {:nsv [:account :type] :no-schema true})

Field uniqueness is something that datomic supports natively. We can trigger a failed transaction by attempting to insert another user named billy into the system:

(adi/insert! {:account {:user "billy" :password "pass2"}})

=> throws

2.1.6    Time Travel

A native feature of datomic allows users to access the state of the database at any point in time. This is also supported by adi. We can use transactions to list all the transactions involving a particular attribute:

(adi/transactions ds :account/user)

=> [1001 1003]

So we can see that there are two transactions involving the :account/user attribute. Lets narrow in and find the transaction number involving the user angela. Note that we inserted angela before billy carl:

(adi/transactions ds :account/user "angela")

=> [1001]

We can also see when the transaction has occured. This of course will be different for everyone:

(adi/transaction-time ds 1001)  ;; -> #inst "2014-10-29T00:17:09.961-00:00"

=> #(instance? java.util.Date %)

We can use the :at option to input either a transaction number or the time. It can be seen that at 1001, only a single record is in the datastore, whilst at 1003, all three records have be added.

(adi/select ds :account :at 1001)

=> #{{:account {:user "angela", :password "hello1", :credits 0}}}
(adi/select ds :account :at 1003)

=> #{{:account {:user "angela", :password "hello1", :credits 0}}
     {:account {:user "billy", :password "pass1", :credits 0}}
     {:account {:user "carl", :password "pass1", :credits 0}}}

There is nicer format for adi such that the schema as well as the connection object is prettied up to display an abriged schema as well as the time and id of the last transaction.

(println ds)

;; #adi{:connection #connection{1003 #inst "2014-10-29T00:31:28.206-00:00"},
;;      :schema #schema{:account {:credits :long, :password :string, :user :string}}}

2.2    Step Two

2.2.1    Enums

There is an intrinsic concept of enums in datomic, seen in the website's schema documentation. adi just takes this explicitly and incorporates it into the schema. We see that there is a new entry for the schema - the account/type attribute which is of type :enum:

(def schema-2
  {:account {:user     [{:required true
                         :unique :value}]
             :password [{:required true
                         :restrict ["password needs an integer to be in the string"
                                    #(re-find #"\d" %)]}]
             :credits  [{:type :long
                         :default 0}]
             :type     [{:type :enum        ;; (2)
                         :default :free
                         :enum {:ns :account.type
                                :values #{:admin :free :paid}}}]}})

Some comments regarding this particular definition compared to the one in Step One:

  1. Note that instead of using :account/<attr> to specify attributes in a flat hashmap, we can just nest theattributes under the :account namespace. This allows for much better readability and saves a couple of characters.
  2. The :enum type is a special :ref type that is defined here. Thelibrary provides it easy to install and manage them. We put them under a common namespace (:account.type) and givethem allowed values #{:admin :free :paid}

2.2.2    Updating

Lets connect to a brand new database and insert some data. Note the different ways of nesting data. There is a correspondence between a nested hashmap and a flat hashmap having keys representing data-paths. adi takes advantage of this correspondence to give allow users more semantic freedom of how to represent their data:

(def ds (adi/connect! "datomic:mem://adi-examples-step-2" schema-2 true true))

(adi/insert! ds {:account {:user "adi1"
                           :password "hello1"}
                 :account/type :paid})

(adi/insert! ds [{:account {:password "hello2"
                            :type :account.type/admin}
                  :account/user "adi2"}
                 {:account {:user "adi3"
                            :credits 1000}
                  :account/password "hello3"}])

Lets take a look at all the :admin accounts:

(adi/select ds {:account/type :admin})

=> #{{:account {:user "adi2", :password "hello2", :credits 0, :type :admin}}}

We can make adi1 into an :admin and then do another listing:

(adi/update! ds {:account/user "adi1"} {:account/type :admin})

(adi/select ds {:account/type :admin})

=> #{{:account {:user "adi1", :password "hello1", :credits 0, :type :admin}}
     {:account {:user "adi2", :password "hello2", :credits 0, :type :admin}}}

If we attempt to add an value of :account.type/<value> that is not listed, an exception will be thrown:

(adi/insert! ds {:account {:user "adi4"
                           :password "hello4"
                           :type :vip}})

=> throws

2.2.3    Selection

There are many ways of selecting data. We have already seen the basics:

(adi/select ds {:account/credits 1000} :first)

=> {:account {:user "adi3", :password "hello3", :credits 1000, :type :free}}

Adding a :pull model will filter out selection options:

(adi/select ds {:account/type :free} :first
            :pull {:account {:credits :unchecked
                               :type :unchecked}})

=> {:account {:user "adi3", :password "hello3"}}

Another feature is that a list can be used to input an expression. In the examples below, the '(> ? 10) predicate acts as the adi equivalent of using an actual predicate #(> % 10). If there is no ?, it is assumed that the first argument is ?. Note that all the three queries below give the same results:

(adi/select ds {:account/credits '(> 10)} :first)

=> {:account {:user "adi3", :password "hello3", :credits 1000, :type :free}}
(adi/select ds {:account/credits '(> ? 10)} :first)

=> {:account {:user "adi3", :password "hello3", :credits 1000, :type :free}}
(adi/select ds {:account/credits '(< 10 ?)} :first)

=> {:account {:user "adi3", :password "hello3", :credits 1000, :type :free}}

2.2.4    Java

Java expressions can also be used because these functions are executed at the transactor end:

(adi/select ds {:account/user '(.contains "2")} :first)

=> {:account {:user "adi2", :password "hello2", :credits 0, :type :admin}}
(adi/select ds {:account/user '(.contains ? "2")} :first)

=> {:account {:user "adi2", :password "hello2", :credits 0, :type :admin}}
(adi/select ds {:account/user '(.contains "adi222" ?)} :first)

=> {:account {:user "adi2", :password "hello2", :credits 0, :type :admin}}

2.3    Step Three

2.3.1    Refs

The most significant feature that adi has implemented on top of datomic is the way that it deals with :ref types. We add more attributes to our schema, this time, we include a :book namespace:

(def schema-3
 {:account {:user     [{:required true
                      :unique :value}]
          :password [{:required true
                      :restrict ["password needs an integer to be in the string"
                                 #(re-find #"\d" %)]}]
          :type     [{:type :enum
                      :default :free
                      :enum {:ns :account.type
                             :values #{:admin :free :paid}}}]
          :credits  [{:type :long
                      :default 0}]
          :books    [{:type :ref
                      :cardinality :many
                      :ref  {:ns :book}}]}
:book   {:name    [{:required true
                    :fulltext true}]
         :author  [{:fulltext true}]}})

Lets connect and print out the datastore:

(def ds (adi/connect! "datomic:mem://adi-example-step-3" schema-3 true true))
(println ds)
;; #adi{:connection #connection{1000 #inst "2014-10-29T03:28:33.478-00:00"},
;;        :schema #schema{:account {:books :&book<*>,
;;                                  :credits :long,
;;                                  :type :enum,
;;                                  :password :string,
;;                                  :user :string},
;;                        :book {:accounts :&account<*>,
;;                               :author :string,
;;                               :name :string}}}

Note that the schema shows a couple of strange symbols - :&book<*> and `:&account<*>. These are symbols that say that :account/book is an attribute of type :ref that is pointing to the :book namespace. The <*> means that there is multiple cardinality associated with the attribute. Having done that, lets add some data to our datastore:

(adi/insert! ds {:account {:user "adi1" :password "hello1"}})

(adi/insert! ds {:account {:user "adi2" :password "hello2"
                           :books #{{:name "The Count of Monte Cristo"
                                     :author "Alexander Dumas"}
                                    {:name "Tom Sawyer"
                                     :author "Mark Twain"}
                                    {:name "Les Miserables"
                                     :author "Victor Hugo"}}}})

2.3.2    Walking

We start off by listing all the accounts:

(adi/select ds :account)

=> #{{:account {:user "adi1" :password "hello1" :credits 0 :type :free}}
     {:account {:user "adi2" :password "hello2" :credits 0 :type :free}}}

We can use :pull to specify a model to pull.

(adi/select ds :account
            :pull {:account {:books :checked}})

=> #{{:account {:credits 0 :type :free :password "hello1" :user "adi1"}}
     {:account {:books #{{:author "Mark Twain" :name "Tom Sawyer"}
                         {:author "Victor Hugo" :name "Les Miserables"}
                         {:author "Alexander Dumas" :name "The Count of Monte Cristo"}}
                :credits 0 :type :free :password "hello2" :user "adi2"}}}

Using :id instead of :checked will pull book ids. This is very useful for copying references

(adi/select ds :account
            :pull {:account {:books :id}})

=> #{{:account {:credits 0, :type :free, :password "hello1", :user "adi1"}}
     {:account {:credits 0, :type :free, :password "hello2", :user "adi2"
                :books #{17592186045425 17592186045426 17592186045424}}}}

2.3.3    Data Access

We will now use the :access option instead of :pull. Essentially, they do the same thing except tha :access limits the data model on the way in and on the way out, whilst :pull limits the data model on the way out only. We can look at a case where both are the same. In the case below, using :access or :pull does not matter and will yield the same result:

(adi/select ds :account
            :access {:account {:books {:author :unchecked}
                               :credits :unchecked
                               :type :unchecked}})

=> #{{:account {:user "adi1" :password "hello1"}}
     {:account {:user "adi2" :password "hello2"
                :books #{{:name "Tom Sawyer"}
                         {:name "The Count of Monte Cristo"}
                         {:name "Les Miserables"}}}}}


In the case where they differ, this is the result of using the :pull option:

(adi/select ds {:account {:books/author "Victor Hugo"}}
            :pull {:account {:books {:author :unchecked}
                               :credits :unchecked
                               :type :unchecked}})

=> {:account {:books #{{:name "Tom Sawyer"}
                       {:name "The Count of Monte Cristo"}
                       {:name "Les Miserables"}}
              :password "hello2" :user "adi2"}}


And this is the result of using the :access option. The difference is that because the :account/book/author path is :unchecked, the operation raises an exception to say that a query like this is not allowed:

(adi/select ds {:account {:books/author "Victor Hugo"}}
            :access {:account {:books {:author :unchecked}
                               :credits :unchecked
                               :type :unchecked}})

=> (raises-issue {:key-path [:account :books :author]
                  :nsv [:book :author]
                  :data "Victor Hugo"
                  :not-allowed true})


To fix this problem limiting searches to the :account/books path we can use both :access and :pull models for fine-tuning control over how our data is accessed:

(adi/select ds {:account {:books/author "Victor Hugo"}}
            :access {:account {:books :checked}}
            :pull {:account {:books {:author :unchecked}
                               :credits :unchecked
                               :type :unchecked}})

=> {:account {:books #{{:name "Tom Sawyer"}
                       {:name "The Count of Monte Cristo"}
                       {:name "Les Miserables"}}
              :password "hello2" :user "adi2"}}

2.3.4    Pointers

Entity :db/id keys are essentially pointers to data. We can add references to other enities just by using copying these :db/id keys around. Instead of pulling data, we can use the :pull-ids option to pull a set of entity ids associated with the search:

(adi/select ds :account :return :ids)

=> #{17592186045421 17592186045423}

Having this is super nice because we can just use these like pointers. We can add The Book and the Sword to our datastore and link them to both our user accounts straight away:

(let [account-ids (adi/select ds :account :return :ids)]
  (adi/insert! ds [{:book {:name "The Book and the Sword"
                           :author "Louis Cha"
                           :accounts account-ids}}]))

Now a search on all the books that adi has yields one result:

(adi/select ds {:book {:accounts/user "adi1"}})

=> #{{:book {:name "The Book and the Sword" :author "Louis Cha"}}}

2.3.5    Playground

Lets do a couple more inserts and selects just to show off some different features

Path Reversal

Inserts can use both forward and backward references. In this case, we are adding a book with a bunch of accounts:

(adi/insert! ds {:book {:name "Charlie and the Chocolate Factory"
                        :author "Roald Dahl"
                        :accounts #{{:user "adi3" :password "hello3" :credits 100}
                                    {:user "adi4" :password "hello4" :credits 500}
                                    {:user "adi5" :password "hello5" :credits 500}}}})

(adi/select ds {:account {:books/author "Roald Dahl"}}
            :pull {:account {:password :unchecked
                               :credits :unchecked
                               :type :unchecked}})

=> #{{:account {:user "adi3"}}
     {:account {:user "adi4"}}
     {:account {:user "adi5"}} }

Fulltext searches

Fulltext searches are avaliable on schema attributes defined with :fulltext true:

(adi/select ds {:book/author '(?fulltext "Louis")}
            :pull {:book {:accounts :checked}} :first)

=> {:book {:author "Louis Cha" :name "The Book and the Sword"
           :accounts #{{:credits 0 :type :free :password "hello2" :user "adi2"}
                       {:credits 0 :type :free :password "hello1" :user "adi1"}}}}

Deletion controlled by models:

(adi/delete-all! ds {:book/author "Roald Dahl"}
                 :access {:book {:accounts :checked}})

(adi/select ds :account)

=> #{{:account {:user "adi2", :password "hello2", :credits 0, :type :free}}
     {:account {:user "adi1", :password "hello1", :credits 0, :type :free}}}

3    Example - Bookstore

Lets go ahead and model a network of bookstores around the world. We apply the knowledge of :ref and :enum types from previous tutorial to good use:

3.1    Definition and Setup

The schema for our bookstore model can be seen in Figure 1. It is a rather simplistic model. The two main concepts being a Book and a Store. They are connected together through an Inventory, which keeps how many books there are in a store:

fig.1  -  Schema Diagram

The actual schema code is defined as follows:

(def schema-4
 {:book   {:name    [{:required true
                    :fulltext true}]
         :author  [{:fulltext true}]}

:inventory {:count [{:type :long}]
            :cover [{:type :enum
                     :enum {:ns :cover.type
                            :values #{:hard :soft}}}]
            :book    [{:type :ref
                       :ref  {:ns :book}}]
            :store   [{:type :ref
                       :ref  {:ns :store}}]}
:store  {:name    [{:required true
                    :fulltext true}]
         :address [{:type :ref
                    :ref  {:ns :address}}]}

:address {:country [{:required true}]}})

Once again, the adi datastore is created and the seed stores created

(def ds (adi/connect! "datomic:mem://adi-example-step-4" schema-4 true true))

(adi/insert! ds [{:address {:country "USA"
                            :stores #{{:name "Canyon Books"}
                                      {:name "Capital Books"}}}}
                 {:db/id [[:AUS]]
                  :address {:country "Australia"}}
                 {:store {:address [[:AUS]]
         :name "Koala Books"}}])

We can see our stores from Australia:

(adi/select ds {:store {:address/country "Australia"}})

=> #{{:store {:name "Koala Books"}}}

As well as the USA:

(adi/select ds {:store {:address/country "USA"}})

=> #{{:store {:name "Canyon Books"}} {:store {:name "Capital Books"}}}

3.2    Updating Data

We can now start adding some books to our Australian store:

(adi/update! ds {:store {:address/country "Australia"}}
             {:store/inventories #{{:count 10
                                    :cover :hard
                                    :book {:name "The Count of Monte Cristo"
                                           :author "Alexander Dumas"}}
                                   {:count 5
                                    :cover :hard
                                    :book {:name "Tom Sawyer"
                                           :author "Mark Twain"}}
                                   {:count 3
                                    :cover :soft
                                    :book {:name "Les Miserables"
                                           :author "Victor Hugo"}}}})

Query for the inventory containing a book in our network starting with "Les". Notice the use of the :first keyword, this will return a single result as opposed to a set of results:

(adi/select ds {:inventory {:book/name '(?fulltext "Les")}} :first)

=> {:inventory {:count 3, :cover :soft}}

update-in! is a very useful function as it can be used to set values across references. In this case, we set the inventory count of "Tom Sawyer" in "Koala Books" to 4:

(adi/update-in! ds {:store/name "Koala Books"}
                [:store/inventories {:book/name "Tom Sawyer"}]
                {:count 4})

We can look at the inventory of a specific book and store by specifying more search keys:

(adi/select ds {:inventory {:book/name "Tom Sawyer"
                            :store/name "Koala Books"}} :first)

=> {:inventory {:count 4, :cover :hard}}

We can also return all the books in the network, along with their inventories and stores. This is done by providing an :access <MODEL> pair of args. In this case, we specify that linked refs in :inventories and :stores are also returned:

(adi/select ds :book :access {:book {:inventories {:store :checked}}})

=> #{{:book {:author "Mark Twain" :name "Tom Sawyer"
             :inventories #{{:store {:name "Koala Books"}
                             :count 4 :cover :hard}}}}
     {:book {:author "Victor Hugo" :name "Les Miserables"
             :inventories #{{:store {:name "Koala Books"}
                             :count 3  :cover :soft}}}}
     {:book {:author "Alexander Dumas" :name "The Count of Monte Cristo"
             :inventories #{{:store {:name "Koala Books"}
                             :count 10 :cover :hard}}}}}

3.3    Different Semantics, Same Result

Lets test out some other functions, starting with retract!. Retract takes a set of keys to remove from an entity:

(adi/retract! ds {:inventory {:book/name "Tom Sawyer"
                              :store/name "Koala Books"}}

The result of the retract is that the cover information is missing when we do a search:

(adi/select ds {:inventory {:book/name "Tom Sawyer"
                            :store/name "Koala Books"}} :first)

=> {:inventory {:count 4}}

We can update the cover information using update-in!:

(adi/update-in! ds {:book/name "Tom Sawyer"}
                [:book/inventories {:store/name "Koala Books"}]
                {:cover :soft})

(adi/select ds {:inventory {:book/name "Tom Sawyer"}} :first)

=> {:inventory {:count 4, :cover :soft}}

Oops. we made a mistake. The cover can also be changed using update! using slightly different semantics:

(adi/update! ds {:inventory {:book/name "Tom Sawyer"
                             :store/name "Koala Books"}}
             {:inventory/cover :hard})

(adi/select ds {:inventory {:book/name "Tom Sawyer"}} :first)

=> {:inventory {:count 4, :cover :hard}}

The choice of using update! and update-in! is purely in its communication:

Another way to reach the inventory entity is to start by finding the store, then following the :store/stock link. This time, we are changing the :inventory/count value to 3:

(adi/update-in! ds {:store/name "Koala Books"}
                [:store/inventories {:book/name "Tom Sawyer"}]
                {:count 3})

(adi/select ds {:inventory {:book/name "Tom Sawyer"}} :first)

=> {:inventory {:count 3, :cover :hard}}

3.4    Expanding the Business

Lets replicate the stock at Koala Books to Capital Books. So basically, we are creating inventory records that have a link to the already existing books. It is very simple to do using adi:

(let [koala-inventories (adi/select ds {:inventory {:store/name "Koala Books"}}
                                    :pull {:inventory {:book :id}})]
  (adi/update! ds {:store/name "Capital Books"}
               {:store/inventories (set (map :inventory koala-inventories))}))

Now, lets look at where we can find Tom Sawyer in our stores:

(adi/select ds {:book/name "Tom Sawyer"} :pull {:book {:inventories {:store :checked}}} :first)

=> {:book {:author "Mark Twain", :name "Tom Sawyer",
           :inventories #{{:cover :hard, :count 3, :store {:name "Koala Books"}}
                          {:cover :hard, :count 3, :store {:name "Capital Books"}}}}}

A fire burnt all the copies of the Count of Monte Cristo from Koala Books. Instead of setting the count to 0, we can use delete! to get rid of the inventory entity entirely

(adi/delete! ds {:inventory {:store/name "Koala Books"
                             :book/name '(?fulltext "Count")}})

We see that now there is only one inventory record:

(adi/select ds {:book/name '(?fulltext "Count")}
            :pull {:book {:inventories {:store :checked}}} :first)

=> {:book {:author "Alexander Dumas", :name "The Count of Monte Cristo",
           :inventories #{{:cover :hard, :count 10, :store {:name "Capital Books"}}}}}

The copies of Tom Sawyer exceeded expectations and sold out. We will delete the inventory record but this time using delete-in!:

(adi/delete-in! ds {:store/name "Capital Books"}
                [:store/inventories {:book/name "Tom Sawyer"}])

(adi/select ds {:book/name "Tom Sawyer"}
            :pull {:book {:inventories {:store :checked}}} :first)

=> {:book {:author "Mark Twain", :name "Tom Sawyer",
           :inventories #{{:cover :hard, :count 3, :store {:name "Koala Books"}}}}}

3.5    Moving Stock

Capital Books ran into financial problems and had to close down. All the stock was transfered to Canyon Books. In order to perform the whole lot in one transaction, the macro sync-> is used:

(let [inventory-ids (adi/select ds {:inventory {:store/name "Capital Books"}}
                                :return :ids)]
  (adi/sync-> ds
              (adi/update! {:store/name "Canyon Books"}
                           {:store/inventories inventory-ids})
              (adi/delete! {:store/name "Capital Books"})))

Now we can see that Capital Books is no more:

(adi/select ds :store :pull {:store {:inventories {:cover :unchecked
                                                   :book {:author :unchecked}}}})

=> #{{:store {:name "Koala Books"
              :inventories #{{:book {:name "Les Miserables"} :count 3}
                             {:book {:name "Tom Sawyer"} :count 3}}}}
{:store {:name "Canyon Books"
         :inventories #{{:book {:name "The Count of Monte Cristo"} :count 10}
                        {:book {:name "Les Miserables"} :count 3}}}}}

4    Example - Schoolyard

4.1    Data Setup

We want to model a simple school, and we have the standard information like classes, teachers students. The schema for our bookstore model can be seen in Figure 1. It is a rather simplistic model. This is actually much like the Bookstore example with a couple more fields.

fig.2  -  Schema Diagram

(def schema-5
{:class   {:type    [{:type :keyword}]
           :name    [{:type :string}]
           :accelerated [{:type :boolean}]
           :teacher [{:type :ref                  ;; <- Note that refs allow a reverse
                      :ref  {:ns   :teacher       ;; look-up to be defined to allow for more
                             :rval :teaches}}]}   ;; natural expression. In this case,
 :teacher {:name     [{:type :string              ;; we say that every `class` has a `teacher`
                       :fulltext true}]
           :canTeach [{:type :keyword             ;; so the reverse will be defined as a
                       :cardinality :many}]       ;; a `teacher` `teaches` a class
           :pets     [{:type :keyword
                       :cardinality :many}]}
 :student {:name     [{:type :string}]
           :siblings [{:type :long}]
           :classes    [{:type :ref
                       :ref   {:ns   :class
                               :rval :students}   ;; Same with students
                         :cardinality :many}]}})

Once again, the adi datastore is created:

(def ds (adi/connect! "datomic:mem://adi-example-step-5" schema-5 true true))
(println ds)
;; #adi{:connection #connection{1001 #inst "2014-10-29T19:05:09.697-00:00"}
;;      :schema #schema{:teacher {:pets :keyword<*>
;;                                :canTeach :keyword<*>
;;                                :teaches :&class<*>
;;                                :name :string}
;;                      :class   {:type :keyword
;;                                :name :string
;;                                :teacher :&teacher
;;                                :students :&student<*>
;;                                :accelerated :boolean}
;;                      :student {:classes :&class<*>
;;                                :name :string
;;                                :siblings :long}}}

Now here is the fun part. Lets fill in the data. This is one way of filling out the data. There are many other ways. Note that it is object-like in nature, with links defined through ids. If it doesn't contain an id, the record is automatically created. The example is slightly contrived mainly to show-off some different features of adi:

(def class-data                      ;;; Lets See....
  [{:db/id [[:MATHS]]
    :class {:type :maths             ;;; There's Math. The most important subject
            :name "Maths"            ;;; We will be giving all the classes ids
            :accelerated true}}      ;;; for easier reference

   {:db/id [[:SCIENCE]]              ;;; Lets add Science and Art
    :class {:type :science
            :name "Science"
            :accelerated false}}

   {:db/id [[:ART]]
    :class {:type :art
            :name "Art"
            :accelerated false}}

   {:teacher {:name "Mr. Blair"                       ;; Here's Mr Blair
              :teaches #{[[:ART]]
                         [[:SCIENCE]]}                ;; He also teaches Science
              :canTeach #{:maths :science}
              :pets    #{:fish :bird}}}

   {:teacher {:name "Mr. Carpenter"                   ;; This is Mr Carpenter
              :canTeach #{:sports :maths}
              :pets    #{:dog :fish :bird}
              :teaches #{{:+ {:db/id [[:SPORTS]]}      ;; He teaches sports
                          :type :sports
                          :name "Sports"
                          :accelerated false
                          :students #{{:name "Jack"   ;; There's Jack
                                       :siblings 4    ;; Who is also in EnglishB and Maths
                                       :classes #{{:+ {:db/id [[:ENGLISHB]]}
                                                   :type :english
                                                   :name "English B"
                                                   :accelerated false
                                                   :students #{{:name  "Anna"  ;; There's also Anna in the class
                                                                :siblings 1
                                                                :classes #{[[:ART]]
                                                               {:name    "Charlie"
                                                                :siblings 3
                                                                :classes  #{[[:ART]]}}
                                                               {:name    "Francis"
                                                                :siblings 0
                                                                :classes #{[[:MATHS]]}}
                                                               {:name    "Harry"
                                                                :siblings 2
                                                                :classes #{[[:SCIENCE]]}}}}}}}}}}}
   {:db/id [[:ENGLISHA]]       ;; What about English A ?
    :class {:type :english
            :name "English A"
            :accelerated true
            :teacher {:name "Mr. Anderson"   ;; Mr Anderson is the teacher
                      :teaches  #{[[:MATHS]]
                                  [[:ENGLISHB]]}   ;; He also takes Maths
                      :canTeach :maths
                      :pets     :dog}
            :students #{{:name "Bobby"   ;; And the students are listed
                         :siblings 2
                         :classes  #{[[:MATHS]]}}
                        {:name "David"
                         :siblings 5
                         :classes #{[[:SCIENCE]]
                        {:name "Erin"
                         :siblings 1
                         :classes #{[[:ART]]}}
                        {:name "Kelly"
                         :siblings 0
                         :classes #{[[:SCIENCE]]

Okay... our data is defined... and...

(adi/insert! ds class-data)

BAM!! We are now ready to do some Analysis

4.2    Datomic

By now, you should be familiar with this query:

(adi/select ds {:student/name "Harry"})

=> #{{:student {:name "Harry", :siblings 2}}}

What we want to explore is how this query relates to the Datomic API. So lets go under the hood a little bit and check out the :db/id of the entity that we are gettin back:

(adi/select ds {:student/name "Harry"} :first :return :ids)
=> 17592186045426

Lets now do a raw datomic query.

(datomic/q '[:find ?x :where
             [?x :student/name "Harry"]]
           (datomic/db (:connection ds)))

=> #{[17592186045426]}

As can be seen, the select function is just a more succinct version of q with many added features.

4.3    Querying

There is a query method that is halfway between select and q in terms and is convenient for dropping back into datalog queries. We see more examples of the query for Harry's id:

(adi/query ds '[:find ?x :where
                [?x :student/name "Harry"]]
           []  :first :return :ids)
=> 17592186045426

And again, the same query with Harry passed in as a parameter:

(adi/query ds '[:find ?x
                :in $ ?name
                [?x :student/name ?name]]
           ["Harry"] :first :return :ids)
=> 17592186045426

Lets do a query for all the students in maths:

(->> (adi/query ds
                '[:find ?x :in $ ?class :where
                  [?x :student/classes ?c]
                  [?c :class/type ?class]]
     (mapv #(-> % :student :name)))

=> ["Bobby" "Francis" "David" "Kelly"]

Here is the equivalent result using select

(->> (adi/select ds {:student {:classes/type :maths}})
     (mapv #(-> % :student :name)))

=> ["Bobby" "Francis" "David" "Kelly"]

4.4    Datalog Generation

Now the cool thing is that select actually generates a datalog query first and then runs it against datomic. We can access the datalog query via the :raw option:

(adi/select  ds {:student {:classes/type :maths}} :raw)

=> #{[:find ?self :where
      [?self :student/classes ?e25242]
      [?e25242 :class/type :maths]]}

So very complex queries can be built up

(adi/select ds {:student {:classes/teacher {:name '(?fulltext "Blair")}
                          :siblings '(> 2)}})

=> #{{:student {:name "Charlie", :siblings 3}}
     {:student {:name "David", :siblings 5}}}

Lets check out what the generated datalog query is:

(adi/select ds {:student {:classes/teacher {:name '(?fulltext "Blair")}
                          :siblings '(> 1)}} :raw)
=> #{[:find ?self :where
      [?self :student/siblings ?e_28962]
      [(> ?e_28962 2)]
      [?self :student/classes ?e28960]
      [?e28960 :class/teacher ?e28961]
      [(fulltext $ :teacher/name "Blair")
       [[?e28961 ?e_28963]]]]}

As can be seen by this example, the adi query is much much more succinct. We can now take the output and stick it into query to get the same result as before:

(adi/query ds '[:find ?self :where
                [?self :student/siblings ?e_28962]
                [(> ?e_28962 2)]
                [?self :student/classes ?e28960]
                [?e28960 :class/teacher ?e28961]
                [(fulltext $ :teacher/name "Blair")
                 [[?e28961 ?e_28963]]]]

=> #{{:student {:name "Charlie", :siblings 3}}
     {:student {:name "David", :siblings 5}}}

So which one will you prefer to be using?

4.5    Expressivity

Find all classes that are taught by Mr Anderson:

(adi/select ds {:class/teacher {:name "Mr. Anderson"}})

=> #{{:class {:type :english, :name "English A", :accelerated true}}
     {:class {:type :english, :name "English B", :accelerated false}}
     {:class {:type :maths, :name "Maths", :accelerated true}}}

Find the teacher that teaches a student called Harry with a pet bird

(->> (adi/select ds {:teacher {:teaches {:students/name "Harry"}
                               :pets :bird}})
     (mapv #(-> % :teacher :name)))

=> ["Mr. Blair"]

Find all students taught by Mr. Blair:

 (adi/select ds {:student/classes {:teacher/name "Mr. Blair"}})
 (mapv #(-> % :student :name)))

=> ["Charlie" "Anna" "David" "Harry" "Erin" "Kelly"]

Find all the students that have class with teachers with a pet fish

 (adi/select ds {:student/classes {:teacher/pets :fish}})
 (mapv #(-> % :student :name)))

=> ["Charlie" "Jack" "Anna" "David" "Harry" "Erin" "Kelly"]

Not that you'd ever want to write a query like this but you can. Find the class with the teacher that teaches a student that takes the class taken by Mr. Carpenter.

 (adi/select ds {:class/teacher
                  {:classes/teacher {:name "Mr. Carpenter"}}}})
 (mapv #(-> % :class :name)))

=> ["Science" "English B" "English A" "Art" "Sports" "Maths"]

Note that the search path :class/teacher/teaches/students/classes/teacher/name will also work.