[Main website]

dok/spec/metaprogramming

Metaprogramming is the management of code as normal data: i.e. code can be analyzed, transformed and/or generated. Metaprogramming is useful for:

  • extending Dok with Domain Specific Languages (DSL)
  • analyzing and optimizing code
  • supporting different run-time environments
  • creating libraries that can interact with developers for improving their usage
  • adding macro for reducing boiler-plate code
  • Aspect Oriented Programming (AOP) where various aspects are fused toghether also using compiler plugins
  • extending the compiler with ad-hoc optimization pass taking in consideration the domain of the DSL

In Dok, metaprogramming is meant to be useful and effective only if used inside Dokmelody IDE.

Quick syntax

  • {DSL| ... |} code to execute using a DSL
  • [DSL| ... |] a definition in a DSL, o a quasi-quotation of a value
  • ${DSL| ...} expansion of code in a quasi quotation
  • $DSL-ref reference to an external DSL variable
  • :tag ... meta-info associated to current lexical scope
  • :^tag ... meta-info associated to the previously defined element

Example of code using DSL for type and data definitions

This is an example of code in CL for PEG parser, with:

  • the grammar
  • the parser creation
  • the parser result
sentence <-  article? subject verb ( preposition? article? (object / object preposition object ))
spaces <- [ ]*
article <- spaces ("the" / "the two" / "a")
preposition <- spaces ("on" / "at" / "to" / "with")
subject <- spaces ("man" / "men" / "dog" / "dogs" / "cat")
verb <- spaces ("sat" / "saw" / "shot" / "gave")
object <- spaces ("cannon" / "hat" / "mat")

(setf *mygrammar* (cl-peg:create-peg-parser "example.peg"))

(setf *parse-result* (cl-peg:parse *mygrammar* "the cat sat on the mat"))
#S(CL-PEG:PARSE-RESULT
   :MATCHED T
   :MATCHED-WHOLE-INPUT T
   :ROOT-PARSE-NODE NIL
   :ORIGINAL-INPUT "the cat sat on the mat"
   :ORIGINAL-INPUT-OFFSET 0)

 (cl-peg:parse-result-root-parse-node *parse-result*)
#S(PV :PE NT: |sentence| :CHILDREN (#S(PV :PE NT: |article| :CHILDREN #S(PV :PE NT: |spaces| :CHILDREN NIL))
                                    (#S(PV :PE NT: |subject| :CHILDREN #S(PV :PE NT: |spaces| :CHILDREN NIL))
                                     (#S(PV :PE NT: |verb| :CHILDREN #S(PV :PE NT: |spaces| :CHILDREN NIL))
                                      (#S(PV :PE NT: |preposition| :CHILDREN #S(PV :PE NT: |spaces| :CHILDREN NIL))
                                       (#S(PV :PE NT: |article| :CHILDREN #S(PV :PE NT: |spaces| :CHILDREN NIL))
                                        #S(PV :PE NT: |object| :CHILDREN #S(PV :PE NT: |spaces| :CHILDREN NIL))))))))

In Dok, whenever possible, a Dok-like syntax will be used.

class Natural-Lang [PEG|

  class Sentence [
    slots [
      _::" "*, Article?, Subject, Verb, Preposition?, Article?, Object
      complement::[Preposition Article? Object]?
    ]
  ]

  class Spaces -like [
    " "+
  ]

  class Article [
    ignore Spaces
    variant ../"the"
    variant ../"the who"
    variant ../"a"
  ]

  class Preposition [
    ignore Spaces
    variants [ ../"on", ../"at", ../"to", ../"with" ]
  ]

  class Subject [
    ignore Spaces
    variants [ ../"man", ../"men", ../"dog", ../"dogs", ../"cat" ]
  ]

  class Verb [
    ignore Spaces
    variants [ ../"sat", ../"saw", ../"shot", ../"gave" ]
  ]

  class Object [
    ignore Spaces
    variants [ ../"cannon", ../"hat", ../"mat" ]
  ]
|]

var p::~ "the cat sat on the mat with the hat".to(Natural-Lang.Sentence)

assert p.isa(Natural-Lang.Sentence)
assert p.article0.isa(Article/"the")
assert p.article1.isa(Article/"the")
assert p.object0.isa(Object/"mat")
assert p.complement.preposition.isa(Preposition/"with")
assert p.complement.object.isa(Object/"hat")

var p2::Natural-Lang.Verb("sat")
assert p2.isa(Natural-Lang.Verb/"sat")

Another example

class Binary-Number [PEG|
  slot ~::Digit+
  ^:to(Array(~))

  class Digit [
    variant ../"0"
    variant ../"1"
  ]
]

var b::Binary-Number("011")
# this is the default constructor defined by PEG DSL, and it accepts a String as input

var c::~ "011".to(Binary-Number)
# this is another code emitted by PEG DSL.

assert {
  test b == c
  test b.digit+ == c.digit+
  test b.digit+.get(0).isa("0")
  test b.digit+[0] == Binary-Number.Digit/"0"
}

# I want to see `Binary-Number` as an array of digits

class Binary-Number -isa Array(Of::Binary-Number.Digit) -on self.digit+

assert b[0] == b.digit+[0] == b.digit+.get(0)

DSL values

var v1 [HTML|<a href='www.dokmelody.org'>Dokmelody</a>]
assert v1.isa(HTML/a)

var v2 "<p>Hello world!</p>".to(HTML)
assert v2.isa(HTML/p)

DSL references

Some values can be defined in a DSL-A and then used/referenced in another DSL-B. It is probable that they will have a different semantic respect DSL-B, so they are referenced in DSL-B using the $ prefix.

[|Doknil

  roles [
    /responsible
    /customer
    /requester
    /part [
     ../project
     ../feature
     ../task
     ../issue
    ]
    /department
  ]

  entities [
    acme
    company1
    dep1
    proj1
    task1
    task2
  ]

  facts [
    proj1 isa part/project -of dep1
    dep1 isa part/department -of company1
    dep1 isa 'department -of company1
    task1 isa part/project/feature -of proj1
    acme isa customer
    acme isa requester -of task1
    acme isa requester -of task1 :in company1
  ]
|]

assert $acme.isa(Doknil/Entity)
assert $proj1.isa(Entity)
assert $dep1.isa(Entity)
assert $proj1.isa(Doknil/Role/Part/Department)

assert {Doknil| get ?p isa part/project -of ?d }[$proj1][$dep1]

Metainfo

Metainfo is used for adding annotations to the code that can be used from the Dok analysis and compilation tools.

In this example meta-info is used for adding design-by-contracts.

class A [
  :# This is a comment added as meta-info to the class `A`

  slot x::Int
  :^# This is a comment added as meta-info to the previous instruction

  fun f(x::Int)::Int {
    :require x > 0
    ^# the argument `x` must be greather that 0
     # This info is added as metainfo because the test is not executed at run-time,
     # but checked during compile-time analysis and it is a documentation for the user.

    :ensure ~result > 0
    ^# `~result`  is the result of the function
     # This condition must be respected from the code of the function.

    return self.x * x
  }

  :*
  slot y::Int
  slot z::Int

  :only-in-testing
  # this tag is applied to the meta-info scope inside `:* ... *:`
  *:
]

I this example it is used for deriving some boiler-plate code.

class Person [
  slot name::String
  slot surname::String
] -isa JSON [ :derive ]

The previous code can be “compressed” also into

class Person [
  slot name::String
  slot surname::String
] -derive JSON

Chunk-macros

A chunk-macro is a macro generating a piece of code in a rather simple way, without an analysis part. They are mainly used for reducing repetitve code. It can be a function or a constant value. They supportsfeature/chunk-oriented-programming.

data Expr {
  attr value::OneOf(Int, Bool)

  :def-chunk def-xy [
    #-NOTE use `[ .. ]` because it is a chunk of code to substitute at compile-time,
    #      and not code to execute at compile time

    slot x::Expr
    slot y::Expr
    :require self.x.value.isa(Int)
    :require self.y.value.isa(Int)
  ]

  variant ../I [ slot x::Int ]
  variant ../B [ slot x::Bool ]
  variant ../Add [ :chunk def-xy ]
  variant ../Mul [ :chunk def-xy ]
  variant ../Eq [
    slot x::Expr
    slot y::Expr
    :require self.x.value.isa(self.y.value.type)
    :require self.x.value.isa(Eq)
    :require self.x.value.isa(Eq)
  ]

  :def-chunk op-xy(op) [
    set self.value self.x ${op} self.y
  ]

  visit self -when ../I {
    set self.value self.x
  } -when ../B {
    set self.value self.x
  } -when ../Add {
    :chunk op-xy("+")
  } -when ../Mul {
    :chunk op-xy("*")
  } -when ../Eq {
    :chunk op-xy("==")
  }
]