Posted by kernelbob on January 24, 2011
I’ve been doing some reading this weekend. This post records a little of what I’ve learned.
The focus has been Scheme macros and separate compilation. I’m trying to understand how a production quality Scheme system “should” be organized. “Should” is in scare quotes because every Scheme user has at least one opinion on the topic, and no two seem to agree.
So here are the papers I’ve looked at in the last few days.More fully-featured, modern Lisps, pulleez Manuel Simoni
You have to love a ‘blog called Axis of Eval. This post is a semi-rant requesting that people tackle the hard problems of Lisp language design instead of creating toy subsets. It includes a number of good pointers to other papers on macros, and this is where I found most of these links. I only followed the links on macros; I hope the other links are just as good.Composable and Compilable Macros: You Want It When? Matthew Flatt
This 2002 paper explains why separation of compile-time and run-time phases is a good idea. Then it describes PLT Scheme’s macro and module system.Understanding Hygiene (part 1) Manuel Simoni
Also from the Axis of Eval. This is the simplest explanation of hygiene I’ve ever seen. I didn’t find Part 2; maybe it hasn’t been written yet.SRFI 72: Hygienic Macros André van Tonder
SRFI is pronounced surfy, but it stands for Scheme Request For Implementation. They’re a series of mini-standards proposals for new language features. Similar to Python’s PEPs.
SRFI 72 is a macro system very similar to what ended up in R6RS. In fact, the reference implementation (see below) has been renamed “R6RS macros”. Defines a consistent set of hygiene rules and a set of primitives that are both useful by themselves and a sufficient basis to implement syntax-case, with-syntax, and syntax-rules.
Further links to van Tonder’s macros:Reference Implementation README from the reference implementation Informative summary on r6rs-discuss mailing list.
Those will take a long time to chew through.The Dark Tower of Meta-levels The Adventures of a Pythonista in Schemeland/22 Michele Simionato
If this article is any indication, I should read Simionato’s whole series. Here, he talks about the Tower of Reflection, and he gives an example of how easy it is to write macros that fail by mixing phases. (That is a common theme in these articles. It’s very easy to screw up.)
Then he gives an example where an R6RS library needs to export names at a negative meta level. Which is only surprising until you realize it’s a negative level relative to the library’s own level.Lisp in Small Pieces Chapter 9: Macros, Their Use and Abuse
Christian Queinnec ISBN 0-521-54566-8
Lisp in Small Pieces has been a great text, explaining difficult concepts thoroughly. But the macro chapter was a letdown. I think the problem is that at the time of publication (1994), macros weren’t fully understood yet. The Scheme community certainly hadn’t reached any kind of consensus on them.Executable and Linkable Format (ELF) author not attributed
An introduction to our favorite object file format. I’ve been wondering whether it would be a suitable container for compiled Scheme bytecode files. I haven’t read enough of it to have an opinion yet. I’m also not entirely sure what a compiled Scheme object file needs to contain.The Mystery of the Tower Revealed: A Nonreflective Description of the Reflective Tower Mitchell Wand, Daniel P. Friedman
I haven’t read this one yet. It’s old, written in 1988, but it’s cited a lot. And the title is cool.
So. What have I learned from all of this? I believe I’ve been convinced that the separate phases model of compilation is the way to go. I think that with careful study of SRFI 72 and its reference implementation, I can implement a useful macro expander. I think that a bytecode object file format is attainable, though I don’t have a clear direction to follow yet. And I’m still uncertain how to bootstrap the process. How much of the system do you have to build in the bootstrap language, and how much can you write in Scheme?