Data Macros

Expanding abbreviated data literals

Posted on Monday, 4 January, 2016

Here's an excerpt from a new article I'm working on for the JUXT blog. It's about yet another gem that falls out of Prismatic's amazing Schema library, the library that just keeps on giving. I use this technique in yada for isolating the abbreviated forms of resource-models from the rest of the logic. It works surprisingly well.

Schema coercions can be used to transform abbreviated data into canonical data. This allows you to create 'short-hands' for your data structures. This is analogous to Clojure's macros where the reader expands short-hand forms into 'canonical' macro-less Clojure which can be more readily evaluated.

This needs an example.

Imagine we have a data structure that contains an action and an access requirement needed to perform the action.

For example, to ensure that :do-something is only available to Bob, we could write:

{:action :do-something
 :allow {:user "bob"}}

We can define a schema for this :-

 '[schema.core :as s]
  [schema.coerce :as sc])

(s/defschema AccessRequirement
  {(s/optional-key :user) String
   (s/optional-key :role) String}
  (s/constrained not-empty)))

(s/defschema Actions
 [{:action s/Keyword
   :allow AccessRequirement}])

If we are specifying users a lot, it might become desirable to permit the following short-hand form :-

{:action :do-something
 :allow "bob"}

to expand automatically to :-

{:action :do-something
 :allow {:user "bob"}}

So if instead of providing the access requirement as a map, we just use a string, we mean that to be a map with the string as the value of the :user entry.

We can define this short-hand as follows :-

(def AccessReqMapping
  (fn [x]
    (string? x) {:user x}
    :otherwise x))})

(def access-right-coercer
  (sc/coercer Actions

So what is access-right-coercer? It's a function that not only checks the input conforms to the schema, but will also give it a nudge if it comes across a short-hand, transforming this :-

{:action :do-something
 :allow "bob"}

into this :-

{:action :do-something
 :allow {:user "bob"}}

Note, that this function will only coerce if it detects a short-hand, so it's safe to use with an already expanded value. If the value doesn't conform to the schema, and can't be made to, an exception is thrown.

If we expand out all the short-hand forms using declarative coercers, we can avoid coding for them later in our data transformation logic. This reduces work, complexity and ultimately bugs.

Just like macros in the Clojure language, abbreviated data expansion is a technique that scales, recursively! In fact, it's used exhaustively in yada (in the yada.schema namespace) to give numerous abbreviation possibilities to the designer of its resource-models.

For example, yada's syntax for declaring the set of representations a resource is capable of producing is quite complex. But the short-hand is easy.

So this :-

 [{:media-type "text/html"}]}

can be abbreviated to this :-

{:produces "text/html"}

Likewise, this :-

(require '[])

   (fn [ctx]
    (io/file "me.jpg")}}}

(for resources where the GET response is constant and independent of the request) can be usefully shortened to this :-

(require '[])

 {:get (io/file "me.jpg")}}

Like code macros, data macros are recursive. For example, yada supports multiple authentication realms. In fact, the code assumes it. However, for most cases a single authentication realm suffices. How can we support the complex case without making the common case overly verbose? Data Macros to the rescue!

 {:realm "default"
  :scheme "Basic"
  :authenticator X}}

expands into :-

    [{:scheme "Basic"
      :authenticator X}]}}}

Unlike code macros, we can compose data macros by merging the coercion mappings. Since functions are the values of these coercion mappings, the only limiting factor is how much of a performance penalty we are willing to spend.

If data literals in Clojure weren't powerful enough already, imagine what you can with a powerful macro facility.