I got to work on my Scheme interpreter this weekend.  Real life had interfered for about three weeks straight.

I’m still working on syntax-case, the code transformation workhorse.  I’ve got two separate pieces of Scheme code.  There’s an expander for a subset of Scheme (no macros) that has the framework for α-substitution and is tied into my interpreter.  Then there’s another expander that I wrote in Chicken Scheme which just handles the pattern matching of syntax-case.  That part is debugged and uses the same syntax objects and environments as my Scheme.

Which is kind of useless.  Being written in full Scheme, it won’t run in my Scheme subset.  (My scheme doesn’t have cond, let, etc.)  And it won’t run in Chicken Scheme, because Chicken Scheme uses a different environment format.  But I can use Chicken Scheme to call the pattern match routine with some test input and see whether it finds the pattern variables and binds them correctly.  Which it does, for a small set of tests.

I also puzzled out how syntax-case ties into the macro expander.  Typical use is like this.

(define-syntax my-macro
  (lambda (x)
    (syntax-case x (<literal> ...)
      (<pattern> <output expression>)
      ...)))

Because syntax-case is wrapped in a lambda, it will run when the lambda is called, which is at macro use time.  That’s the right time to bind the pattern variables.  So no extra tricks are needed (I think) to delay its execution.  syntax-case does need to be a special form, so the evaluator doesn’t try to evaluate its arguments before applying it.

So now I’m working on the syntax keyword.  syntax substitutes pattern variables into template expressions, and it expands ellipses.  For example, given this pattern

(my-letrec ((var init) ...) body ...)

and this input form

(my-letrec
  ((a 0) (b #f))
  (display b)
  (display (+ b 3)))

and this syntax template

(syntax ((lambda (var ...) body ...) init ...))

I’m working on generating the right output, which, does to α-substition, is too unreadable to post here.

Actually, I’m not working on it.  I’ve spent the day scanning the literature trying to see how others have done it.  And I haven’t found any actual implementations yet.  So I’m thinking about it…

I’ve written here before about my efforts to write a Scheme interpreter from scratch.  The last update was in May, but I’m still working on it.  I’ve made 60 git commits since then.  The major thing I’m trying to do is implement macros, but I’m having a hard time of it.  No matter.  When they’re done, I’ll understand macros thoroughly.

One area that’s changed is that the interpreter is now featureful enough that I’m writing about 90% of the new code in Scheme instead of C.  So I’m getting some practice writing in Scheme.  It has about 1,500 lines of Scheme now.  Lots of standard procedures, partial reimplementation of the library system, and the aforementioned macro expander, which has a lot of code even if it isn’t complete.

Macros.  R6RS specifies both syntax-rules and syntax-case, and it implies that you can build your own macros out of lower-level routines.  Syntax-case is the most general, so that’s what I’m implementing.  I have repeatedly read three papers by syntax-case’s inventor, Kent Dybvig, and have just about worn out the paper printouts I carry everywhere with me.  I almost understand it… (-:

First, a user’s guide to syntax-case.  Lots of examples ranging from very simple to fairly complex.

http://www.cs.indiana.edu/~dyb/pubs/tr356.pdf

Second, the paper that presents the algorithm.

http://www.cs.indiana.edu/~dyb/pubs/LaSC-5-4-pp295-326.pdf

Third, a chapter from the book, Beautiful Code, where Dybvig presents a toy implementation of his algorithm.  It’s a toy because it implements syntax-case in terms of syntax-case, which is cheating.

http://www.cs.indiana.edu/~dyb/pubs/bc-syntax-case.pdf

So I’m getting closer.  As of this week, I can do α-substitution (see below) on lambda expressions.

Anyway, enough about macros.  Let’s talk about memory.

Very early on, I came up with a pretty good object memory implementation, I thought.  It’s object oriented.  Each object has a header word, and the header word points to what’s basically a virtual function table.  The VF table makes it possible to identify each object’s type, and has virtual method pointers for the operations the GC needs such as getting the object’s size or finding each of its pointers.  There are some abstract classes, fixvec and mixvec, to make writing various fixed-size objects easy.  fixvec is a parameterized type that implements a fixed-size vector of object pointers.  So a pair (aka a cons cell) is a fixvec(2), and the pair implementation doesn’t do much more than map pair_car() to fixvec2_get(0).  Mixvec is similar.  It’s a fixed-size object with M non-pointer words and N pointer words.  A binding, for example, has two pointers (name and value) and one non-pointer word (type flags).

So it’s good.  At this point, there are about 13 object classes, and I haven’t had to redo the core.  But then I realized that I have the world’s only Scheme that doesn’t store simple types  like fixnums, characters, and booleans as immediate values.  I.e., I store a fixnum in an object.  It has one word of header and one word of integer.  The vast majority of Schemes use tagged pointers and store a fixnum directly in the pointer.  Chicken Scheme has a typical implementation.

I was aware of that technique all the time, but didn’t do it that way, for some reason.  Now I’m thinking it would be a good optimization.  After syntax-case works.

Also, let’s talk about anonymous symbols.  A Scheme symbol has the property that it’s unique.  There is only ever one symbol named foo, for example.  Every foo everywhere in the system refers to the same symbol object.  My Scheme stores all its symbols on a long list, and when it tries to create a symbol named foo, it searches the list for an existing symbol by that name.  If it finds one, it returns that.  Otherwise, it creates a new symbol and inserts it at the head of the list.  (N.B., this means that symbols are never garbage collected.  That’s more or less okay, as they have long lifetimes.)

But the macro expander generates lots of symbols.  It uses a technique called α-substitution, where it renames all the local variables as it expands, generating new names for each.  I’d seen, somewhere in my Lisp/Scheme reading, the idea of symbols that don’t have names until they’re printed, and I implemented that.  It worked out really well.  The string object module has a new routine to create a symbol.

obj_t *make_anonymous_symbol(void)
{
    return alloc_symbol(NIL);
}

alloc_symbol() is the usual symbol constructor, and the argument (NIL) is a Scheme string for the symbol’s name.  alloc_symbol() does not link the symbol onto the big list.  So that was easy.  Then we just need a way to give the symbol a name when it’s printed.

The symbol name accessor used to look like this.

obj_t *symbol_name(obj_t *symbol)
{
    return fixvec1_get_ptr(symbol, 0);
}

Now it looks like this.

obj_t *symbol_name(obj_t *symbol)
{
    obj_t *name = fixvec1_get_ptr(symbol, 0);
    if (is_null(name)) {
        wchar_t name_buf[12];
        while (true) {
            ssize_t name_len = swprintf(name_buf,
                                        sizeof name_buf,
                                        L"g%04d",
                                        ++gen_name_counter);
            name = make_string_from_chars(name_buf, name_len);
            if (find_symbol(name))
                continue;
            fixvec1_set_ptr(symbol, 0, name);
            all_symbols_list = make_pair(symbol,
                                         all_symbols_list);
            break;
        }
    }
    return name;
}

That’s a little more complicated, but the idea is that when we need the symbol’s name, we start generating names starting with g0001, g0002, etc., until we find one that isn’t in use.  then we add the symbol to the big list of symbols so it remains unique with its new name.

The “g” in the created name is for “generated”.

This is cool for several reasons.  First, most anonymous symbols never have names.  Second, they can be garbage collected.  Third, they don’t make the big symbol list any longer/slower to search.  Fourth, the path through symbol_name() for regular (named) symbols only slows down by a single test for NIL.

You may have noticed that symbol names are Unicode (wchar_t).  That’s another thing I’ve done since May.  The whole Scheme system is Unicode.  When I started the interpreter, I used libunicode, which was already installed.  That was a bad choice.  When I upgraded to Ubuntu 9.04, libunicode disappeared.  It had been deprecated for the last five years, but I didn’t notice.  In June, after looking long hard at the available Unicode libraries, I wrote my own Unicode support from scratch.  It parses Unicode.txt (part of the spec distributed by the Unicode Consortium) and builds some character class tables.

So that’s a quick overview of the last five months.  Sorry for the lengths – length of time between updates and length of this post.

I found this.

An Incremental Approach to Compiler Construction
Abdulaziz Ghuloum

It’s a paper describing an agile approach to building a compiler.  It starts with a tiny “language” that only contains integer constants, and builds a compiler that emits x86 assembly to return integer constants.  Then it incrementally builds that, in “steps of a single working session”, into a compiler for a fully usable Scheme language.

The author is at Indiana University, so I suppose there’s a class to go with it.  I didn’t find the class notes, but I did find a longer tutorial paper.

http://www.cs.indiana.edu/~aghuloum/compilers-tutorial-2006-09-16.pdf

Last month, I read Clay Shirky’s excellent book, Here Comes Everybody. Shirky’s a smart guy, and I’ve learned something every time I’ve read him. Strongly recommended.

The book’s theme is how the Internet is driving the cost for groups to self-organize to zero, and how that is reshaping our culture. We’re living through it every day, but Shirky looks at it as a big picture.

Anyway, near the beginning of the book, page 14, Shirky wrote,

Human beings are social creatures — not occasionally or by accident but always. Sociability is one of our core capabilities, and it shows up in almost every aspect of our lives as both cause and effect.

As Mr. McMahon, RIP, always said, “I did not know that.” Never thought about it. After reading a bit more, I decided to join Facebook and see for myself. It’s true. People are on there talking to each other all over the place. So I’ve been trying to join the conversation. It’s clumsy, like learning to dance.

Segue.

Anne and I spent three weeks this summer in Europe. The first twelve days we toured with the Eugene Concert Choir, and the last week we were on our own. The choir tour was nonstop. From breakfast through bedtime, we were in a jabbering crowd. I found that very tiring. I needed downtime. A lot.

So we got home. And I found myself staying home as much as possible. I can work from home as long as I don’t need a lot of ‘Net bandwidth, so I did. And the more time I spent by myself (or with Riley), the more contented I got. Now I’m pretty back to normal, but still not in a big hurry to get out.

But the lessons are twofold: 1. People are innately social. 2. I crave solitude. I think most people don’t need it nearly as much as I do.

Want to make the Internet a better place?

Help me out by watching these two tiny video clips (7 seconds each). Then come back here and post a comment and say whether the video worked for you.

Link to video page

Thanks.

I want to see if I can embed a Vimeo video into a post.  Got some travel videos…

Pronunciation of “Bruschetta” from Bob Miller on Vimeo.

That didn’t embed. Let’s try embedding it right in WordPress.

Have to upgrade WordPress for that.

If you know of a better way to publish HD video, leave a comment, thanks.

End Of Trip.

Pics later.  Sleep now.

We’ve had a lot of fun on this trip and done some wonderful stuff.  However, visiting Milan was not one of them.  First, it’s hot here.  32° C (90° F) and most buildings are hotter inside.

Secondly, the city is on vacation this month.  The city is especially on vacation this weekend, because today, 15.8, is a national holiday, celebrating the assumption of the Virgin.  Last night we walked for an hour looking for an open restaurant.  (We finally found one.)

Nonetheless, the National Museum for Science and Technology Leonardo da Vinci was a good find.  We saw models of many of Leonardo’s inventions and some other exhibits, including a recreation of a circa 1950 steelworks.

After about three hours’ sleep, we got up at 03:00 (Athens time).  We’re in the Milano Linate airport now.  Hopping to Frankfurt, then direct to Portland.  Couple of hours in Portland, 2.5 hours driving home.  If all goes well, we’ll get home by 16:00 PDT.  About 21 hours.  If all goes well.

This morning we woke up in Locarno, Switzerland, on the north shore of Lake Maggiore.  We’d planned to hang around Locarno until 1:00, then catch the autostrada to Milan and drop off Mito.  But we’re not so good at following our own plans these days.

Instead, we left Locarno at 9:30, drove along the west shore of Maggiore, then turned right at the town of Intra and headed up into the mountains.  Anne’s guidebook described a road as having “switchbacks not for the timid”, so that’s where we went.  It was a single-lane mountain road that snaked its way up to the town of Aurano.  Aurano is carved into a very steep hillside. The houses are made of stone, and some of them even have stone roofs.  Up the hill from Aurano, we found a promontory that overlooks Lake Maggiore and spent a few minutes there trying to photograph the lake through the clouds.

Then we drove on, and stopped at the first restaurant we’d seen for 90 minutes.  I don’t know exactly where it was.  My first clue that we were in for something special was when the owner/maitre d’ sat down at the table with me to discuss the menu.  He said that he had never served “vegetables” (vegetarians) before, but  when he learned that we eat cheese and eggs, his mood brightened.  We ended up with a full traditional Italian meal, with the best bruschetta we’d ever tasted, slabs of three different kinds of local cheese, polenta, and tortelonis.  And wine.  Just a little, because we still had more switchbacks not for the timid to navigate.  A delightful meal, and totally unexpected.  We thought we were just stopping to use the bathroom. (-:

Anyway, we eventually found our way to the autostrada, then stopped at a truck stop to map our route into Milan in detail.  That sorted, we went in.  We only got a little lost before we found the hotel, but we couldn’t find the car rental place.  It turned out to be a booth inside the train station.  We returned the car at 17:53, seven minutes before it was due.

So we had another long day with Mito.

Our hotel in Milan is being renovated.  It will be quite luxurious in a few months, but right now it’s in “pardon our dust” mode.  The air conditioner works, but the Internet is only in the lobby, not the room.

Tomorrow, we’re planning to go to the science museum and a shoe store.  But you know we can’t follow our plans…

Pics later.