September 2018, rev. October 2018

The operators def and mac have an unusual ability: their parameters don't get evaluated, because these operators are used to define source code. Def defines functions. Mac defines macros. Functions and macros get evaluated when they're called, not when the def or mac operators are called. [1]

This unusual ability to define source code comes with unparalleled freedom compared to the rest of the Lisp operators. Most operators can't be changed because they're hardcoded inside the language core to do low level things with binary code. Since def and mac work with source code, they don't need to work with binary. They don't need to be inside the core. Def and mac can be implemented at runtime.

This difference matters when implementing a rewritable Lisp. Should functions be defined as tagged types? Should macros accept an atom as their only parameter instead of only a list of parameters? Whatever the answer, keeping this logic outside the language core lets the programmer change the language at runtime.

Let's look at an example that defines def and mac as tagged types, as discovered by Arc who implemented def and mac at runtime.

Def is implemented as a tagged type named 'macro whose data is a function that creates source code to assign a variable to a function:

(= def (tag 'macro
            (fn (name params (r body))
                (list '=
                      name
                      (list 'fn
                            params
                            `,@body)))))


Mac is implemented as a tagged type named 'macro whose data is a function that creates source code to assign a variable to a tagged type named 'macro whose data is a function:

(= mac (tag 'macro
            (fn (name params (r body))
                (list '=
                      name
                      (list 'tag
                            ''macro
                            (list 'fn
                                  params
                                  `,@body))))))


This was the definition of the operators. Now how about their execution? What does a rewritable Lisp do when it finds def or mac?

When Lisp sees source code that looks like a function call (be it def, mac, or anything else), first it evaluates the first operator (like def or mac) and treats the operator differently if it's a tagged type with the label 'macro (which both def and mac are). Instead of evaluate the parameters and call the operator as a function, it passes the parameters unevaluated and treats the operator as a macro. Both def and mac are macros.

One surprise here is that unlike the rest of the Lisp operators def and mac call eval twice internally. Once to retrieve their definition, which is passed unevaluated source code as input that returns source code as output, and a second time to execute the output.

All of this requires an agreement with the language core. The core must know that tagged types with the label 'macro should be handled differently and that if it finds one it should call eval twice. But this is a more flexible agreement than requiring the core to handle only def and mac differently.









Notes:

[1]  I generalized this explanation to make it easier to understand. Macro expansion varies per Lisp dialect. Although most Lisp dialects evaluate macros when they're read, some dialects evaluate macros when they're called. Either way, the macro is not evaluated when the mac operator is called.