hara.test easy to use test framework

Author: Chris Zheng  (z@caudate.me)
Date: 11 November 2016
Repository: https://github.com/zcaudate/hara
Version: 2.4.8

1    Introduction

hara.test is a test framework based off of the midje syntax, containing macros and helpers for easy testing and verification of functions

1.1    Installation

Add to project.clj dependencies:

[im.chit/hara.test "2.4.8"]

1.2    Motivation

hara.test serves to provide a light-weight test framework based upon a subset of operations defined by midje. The framework allows for rapid in-repl testing of the whole project as well as very easy shell integration. It abides by the following principles:

  • test data should be explict for both input and output
  • test cases can be read like documentation
  • test suites contains information about the functions that are being tested

1.3    Running

hara.test can be run in the repl by:

(use 'hara.test)

(run)

or in the shell:

> lein run -m hara.test :exit

2    Index

3    API

3.1    Basics

For those that are familiar with midje, it's pretty much the same thing. We define tests in a fact expression:

(fact "lets test to see which if numbers are odd"

  1 => odd?

  2 => odd?)

;; Failure  [hara_test.clj]
;;    Info  "lets test to see which if numbers are odd"
;;    Form  2
;;   Check  odd?
;;  Actual  2

Or for the more grammatically correct, a facts expression:

(facts "lets test to see which if numbers are even?"

  1 => even?

  2 => even?)
  
;; Failure  [hara_test.clj]
;;    Info  "lets test to see which if numbers are even?"
;;    Form  1
;;   Check  even?
;;  Actual  1

The arrow => forms an input/output syntax and can only be in the top level fact form. This means that it cannot be nested arbitrarily in let forms as in midje. This has the effect simplifying the codebase as well as forcing each individual test to be more self-sufficient. The arrow is flexible and is designed so that the input/output syntax can be kept as succinct as possible. Other checks that can be performed are given as follows:

(facts "Random checks that show-off the range of the `=>` checker"

  ;; check for equality
  1 => 1         

  ;; check for being in a set
  1 => #{1 2 3}  

  ;; check for class
  1 => Long
  
  ;; check for function
  1 => number?  

  ;; check for pattern
  "one" => #"ne")

3.2    Metadata

Metadata can be placed on the fact/facts form in order to provide more information as to what exactly the fact expression is checking:

^{:refer hara.test/fact :added "2.4" :tags #{:api}}
(fact "adding metadata gives more information"

  (+ 1 2 3) => (+ 3 3))

Metadata allows the test framework to quickly filter through what test are necessary, as well as to enable generation of documentation and docstrings through external tools.

3.3    Checkers



all ^

checker that allows `and` composition of checkers

v 2.4
(defn all
  [& cks]
  (let [cks (mapv base/->checker cks)]
    (common/checker
     {:tag :all
      :doc "Checks if the result matches all of the checkers"
      :fn (fn [res]
            (->> cks
                 (map #(base/verify % res))
                 (every? base/succeeded?)))
      :expect cks})))
link
(mapv (all even? #(< 3 %)) [1 2 3 4 5]) => [false false false true false]

any ^

checker that allows `or` composition of checkers

v 2.4
(defn any
  [& cks]
  (let [cks (mapv base/->checker cks)]
    (common/checker
     {:tag :any
      :doc "Checks if the result matches any of the checkers"
      :fn (fn [res]
            (or (->> cks
                     (map #(base/verify % res))
                     (some base/succeeded?))
                false))
      :expect cks})))
link
(mapv (any even? 1) [1 2 3 4 5]) => [true true false true false]

anything ^

a checker that returns true for any value

v 2.4
link
(anything nil) => true (type anything) => hara.test.common.Checker

contains ^

checker for maps and vectors

v 2.4
(defn contains
  [x & modifiers]
  (cond (map? x)
        (contains-map x)

        (sequential? x)
        (contains-vector x (set modifiers))

        :else
        (throw (Exception. "Cannot create contains checker"))))
link
((contains {:a odd? :b even?}) {:a 1 :b 4}) => true ((contains {:a 1 :b even?}) {:a 2 :b 4}) => false ((contains [1 2 3]) [1 2 3 4]) => true ((contains [1 3]) [1 2 3 4]) => false

contains-in ^

shorthand for checking nested maps and vectors

v 2.4
(defmacro contains-in
  [x]
  "A macro for nested checking of data in the `contains` form"
  (cond (map? x)
        `(contains ~(reduce-kv (fn [out k v]
                                 (assoc out k `(contains-in ~v)))
                               {}
                               x))
        (vector? x)
        `(contains ~(reduce (fn [out v]
                              (conj out `(contains-in ~v)))
                            []
                            x))
        :else x))
link
((contains-in {:a {:b {:c odd?}}}) {:a {:b {:c 1 :d 2}}}) => true ((contains-in [odd? {:a {:b even?}}]) [3 {:a {:b 4 :c 5}}]) => true

exactly ^

checker that allows exact verifications

v 2.4
(defn exactly
  [v]
  (common/checker
   {:tag :exactly
    :doc "Checks if the result exactly satisfies the condition"
    :fn (fn [res] (= (common/->data res) v))
    :expect v}))
link
((exactly 1) 1) => true ((exactly Long) 1) => false ((exactly number?) 1) => false

is-not ^

checker that allows negative composition of checkers

v 2.4
(defn is-not
  [ck]
  (let [ck (base/->checker ck)]
    (common/checker
     {:tag :is-not
      :doc "Checks if the result is not an outcome"
      :fn (fn [res]
            (not (ck res)))
      :expect ck})))
link
(mapv (is-not even?) [1 2 3 4 5]) => [true false true false true]

just ^

exact checker for maps and vectors

v 2.4
(defn just
  [x & modifiers]
  (cond (map? x)
        (just-map x)

        (vector? x)
        (just-vector x (set modifiers))

        :else
        (throw (Exception. "Cannot create just checker"))))
link
((just {:a odd? :b even?}) {:a 1 :b 4}) => true ((just {:a 1 :b even?}) {:a 1 :b 2 :c 3}) => false ((just [1 2 3 4]) [1 2 3 4]) => true ((just [1 2 3]) [1 2 3 4]) => false ((just [3 2 4 1] :in-any-order) [1 2 3 4]) => true

just-in ^

shorthand for exactly checking nested maps and vectors

v 2.4
(defmacro just-in
  [x]
  "A macro for nested checking of data in the `just` form"
  (cond (map? x)
        `(just ~(reduce-kv (fn [out k v]
                             (assoc out k `(just-in ~v)))
                           {}
                           x))
        (vector? x)
        `(just ~(reduce (fn [out v]
                          (conj out `(just-in ~v)))
                        []
                        x))
        :else x))
link
((just-in {:a {:b {:c odd?}}}) {:a {:b {:c 1 :d 2}}}) => false ((just-in [odd? {:a {:b even?}}]) [3 {:a {:b 4}}]) => true

satisfies ^

checker that allows loose verifications

v 2.4
(defn satisfies
  [v]
  (common/checker
   {:tag :satisfies
    :doc "Checks if the result can satisfy the condition:"
    :fn (fn [res]
          (let [data (common/->data res)]
            (cond (= data v) true
                  
                  (class? v) (instance? v data)

                  (map? v) (= (into {} data) v)
                  
                  (vector? v) (= data v)
                  
                  (ifn? v) (boolean (v data))

                  (checks/regex? v)
                  (cond (checks/regex? data)
                        (= (.pattern ^Pattern v)
                           (.pattern ^Pattern data))
                        
                        (string? data)
                        (boolean (re-find v data))

                        :else false)
                  
                  :else false)))
    :expect v}))
link
((satisfies 1) 1) => true ((satisfies Long) 1) => true ((satisfies number?) 1) => true ((satisfies #{1 2 3}) 1) => true ((satisfies [1 2 3]) 1) => false ((satisfies number?) "e") => false ((satisfies #"hello") #"hello") => true

throws ^

checker that determines if an exception has been thrown

v 2.4
(defn throws
  ([]  (throws Throwable))
  ([e] (throws e nil))
  ([e msg]
   (common/checker
    {:tag :throws
     :doc "Checks if an exception has been thrown"
     :fn (fn [{:keys [^Throwable data type]}]
           (and (= :exception type)
                (instance? e data)
                (if msg
                  (= msg (.getMessage data))
                  true)))
     :expect {:exception e :message msg}})))
link
((throws Exception "Hello There") (common/map->Result {:type :exception :data (Exception. "Hello There")})) => true

throws-info ^

checker that determines if an `ex-info` has been thrown

v 2.4
(defn throws-info
  ([]  (throws-info {}))
  ([m]
   (common/checker
    {:tag :raises
     :doc "Checks if an issue has been raised"
     :fn (fn [{:keys [^Throwable data type]}]
           (and (= :exception type)
                (instance? clojure.lang.ExceptionInfo data)
                ((contains m) (ex-data data))))
     :expect {:exception clojure.lang.ExceptionInfo :data m}})))
link
((throws-info {:a "hello" :b "there"}) (common/evaluate '(throw (ex-info "hello" {:a "hello" :b "there"})))) => true

3.4    Runner



print-options ^

output options for test results

v 2.4
(defn print-options
  ([] (print-options :help))
  ([opts]
   (cond (set? opts)
         (alter-var-root #'common/*print*
                         (constantly opts))

         (= :help opts)
         #{:help :current :default :list :disable :reset :all}

         (= :current opts) common/*print*
         
         (= :default opts)
         #{:print-thrown :print-failure :print-bulk}
         
         (= :list opts)
         #{:print-thrown :print-success :print-facts :print-facts-success :print-failure :print-bulk}

         (= :disable opts)
         (alter-var-root #'common/*print* (constantly #{}))
         
         (= :reset opts)
         (alter-var-root #'common/*print* (constantly #(print-options :default)))
         
         (= :all opts)
         (alter-var-root #'common/*print* (constantly (print-options :list))))))
link
(print-options) => #{:disable :reset :default :all :list :current :help} (print-options :default) => #{:print-bulk :print-failure :print-thrown} (print-options :all) => #{:print-bulk :print-facts-success :print-failure :print-thrown :print-facts :print-success}

run-namespace ^

run tests for namespace

v 2.4
(defn run-namespace
  ([] (run-namespace (.getName *ns*)))
  ([ns]
   (run-namespace ns (project/project)))
  ([ns project]
   (run-namespace ns common/*settings* project))
  ([ns settings project]
   (binding [*warn-on-reflection* false
             common/*settings* (merge common/*settings* settings)
             common/*print* (or (:print settings) common/*print*)]
     (println "n")
     (println (-> (format "---- Namespace (%s) ----" (str ns))
                  (ansii/style  #{:blue :bold})))
     (let [all-files (project/all-files (:test-paths common/*settings*)
                                        {}
                                        project)
           facts (accumulate (fn [id sink]
                               (when-let [path (get all-files ns)]
                                 (binding [common/*path* path]
                                   (prn path)
                                   (load-file path)))))
           results (interim facts)]
       (event/signal {:test :bulk :results results})
       (reduce-kv (fn [out k v]
                    (assoc out k (count v)))
                  {}
                  results)))))
link
(run-namespace 'hara.class.checks-test) => {:files 1, :thrown 0, :facts 5, :checks 9, :passed 9, :failed 0} ;; ---- Namespace (hara.class.checks-test) ---- ;; ;; Summary (1) ;; Files 1 ;; Facts 5 ;; Checks 9 ;; Passed 9 ;; Thrown 0 ;; ;; Success (9)

run ^

run tests for entire project

v 2.4
(defn run
  ([] (run (project/project)))
  ([project]
   (run common/*settings* project))
  ([settings project]
   (binding [*warn-on-reflection* false
             common/*settings* (merge common/*settings* settings)
             common/*print* (or (:print settings) common/*print*)]
     (let [all-files (project/all-files (:test-paths common/*settings*)
                                        {}
                                        project)
           proj (:name project)]
       (println "n")
       (println (-> (format "---- Project (%s%s) ----" (if proj (str proj ":") "") (count all-files))
                    (ansii/style  #{:blue :bold})))
       (println "")
       (let [facts (accumulate (fn [id sink]
                                 (doseq [[ns path] (-> all-files seq sort)]
                                   (println (ansii/style ns  #{:blue}))
                                   (binding [common/*path* path]
                                     (load-file path)))))
             results (interim facts)]
         (event/signal {:test :bulk :results results})
         (reduce-kv (fn [out k v]
                      (assoc out k (count v)))
                    {}
                    results))))))
link
(run) ;; ---- Project (im.chit/hara:124) ---- ;; documentation.hara-api ;; documentation.hara-class ;; documentation.hara-common ;; documentation.hara-component ;; .... ;; .... ;; hara.time.data.vector-test ;; ;; Summary (99) ;; Files 99 ;; Facts 669 ;; Checks 1151 ;; Passed 1150 ;; Thrown 0 ;; ;; Failed (1) (run {:test-paths ["test/hara"] :include ["^time"] :print #{:print-facts}}) => {:files 8, :thrown 0, :facts 54, :checks 127, :passed 127, :failed 0} ;; Fact [time_test.clj:9] - hara.time/representation? ;; Info "checks if an object implements the representation protocol" ;; Passed 2 of 2 ;; Fact [time_test.clj:16] - hara.time/duration? ;; Info "checks if an object implements the duration protocol" ;; Passed 2 of 2 ;; ...

3.5    Options

Options for run and run-namespace include:

  • specifing the :test-paths option (by default it is "test")
  • specifing :include and :exclude entries for file selection
  • specifing :check options:
    • :include and :exclude entries:
      • :tags so that only the :tags that are there on the meta data will run.
      • :refers can be a specific function or a namespace
      • :namespaces refers to specific test namespaces
  • specifing :print options for checks

Some examples can be seen below:

(run {:checks {:include [{:tags #{:web}}]} ;; only test for :web tags
      :test-paths ["test/hara"]}) ;; check out "test/hara" as the main path
=> {:files 0, :thrown 0, :facts 0, :checks 0, :passed 0, :failed 0}

Only test the hara.time-test namespace

(run {:checks {:include [{:namespaces #{'hara.time-test}}]}})
=> {:files 1, :thrown 0, :facts 32, :checks 53, :passed 53, :failed 0}

;; Summary (1)
;;   Files  1
;;   Facts  32
;;  Checks  53
;;  Passed  53
;;  Thrown  0
;;
;; Success (53)

Only test facts that refer to methods with hara.time namespace:

(run {:test-paths ["test/hara"]
      :checks {:include [{:refers '#{hara.time}}]}})
=> {:files 1, :thrown 0, :facts 32, :checks 53, :passed 53, :failed 0}
;; Summary (1)
;;   Files  1
;;   Facts  32
;;  Checks  53
;;  Passed  53
;;  Thrown  0
;;
;; Success (53)

Only pick one file to test, and suppress the final summary:

(run {:test-paths ["test/hara"]
      :include    ["^time"]
      :print      #{:print-facts}})
=> {:files 8, :thrown 0, :facts 54, :checks 127, :passed 127, :failed 0}
;;   Fact  [time_test.clj:9] - hara.time/representation?
;;   Info  "checks if an object implements the representation protocol"
;; Passed  2 of 2

;;   Fact  [time_test.clj:16] - hara.time/duration?
;;   Info  "checks if an object implements the duration protocol"
;; Passed  2 of 2

;; ...