10.times do |i|
  begin
    raise "OH NO!"
    puts "OH YES! #{i}"
    i += 1
  rescue => err
    if i < 5
      err.continue
    else
      raise err
    end
  end
end

Oh, that’s right. This is Ruby. Forty lines later:

class Exception
  class NoContinuation < StandardError
  end
  
  attr_accessor :continuation
  
  def continue
    raise NoContinuation unless continuation.respond_to?(:call)
    continuation.call
  end
end

module NeverGonnaLetYouDown
  def raise(exception = RuntimeError, string = nil, array = caller)
    # With a single String argument, raises a
    # RuntimeError with the string as a message. 
    if exception.is_a?(String)
      string = exception
      exception = RuntimeError
    end
    
    callcc do |cc|
      obj = exception.exception(string)
      obj.set_backtrace(array)
      obj.continuation = cc
      super obj
    end
  end
  
  def fail(exception = RuntimeError, string = nil, array = caller)
    raise(exception, string, array)
  end
end

class Object
  include NeverGonnaLetYouDown
end

Every compilable template can be compiled to ERB.

Well, not ERB-the-syntax, but ERB-the-concept:

Every compilable template consists of three elements:

• Static text
• Dynamic text (pieces of Ruby which are evaluated and sent to the client)
• Blocks (pieces of Ruby which are evaluated and not sent to the client, but might change the control flow).

Nothing revolutionary at all. Just a theory.

Compiling it

There are several ways to render such a template, but the fastest one is to compile it to pure Ruby:

Hello <%= @world %>!
<% if @me.happy? %>
  <%= @greeting %>
<% end %>

Becomes:

_buf = []
_buf << "Hello "
_buf << (@world)
_buf << "!\n"
if @me.happy?
_buf << "\n  "
_buf << (@greeting)
_buf << "\n"
end
_buf.join

Probably you also want to optimize it:

_buf = []
_buf << ("Hello #{@world}!\n")
if @me.happy?
_buf << ("\n  #{@greeting}\n")
end
_buf.join

Or maybe you rather want _buf to be a String? Or print it directly to stdout?

These options are already written and implemented in Erubis, but none of the other template engines can take advantage of that. Which is a shame, because we know that every compilable template can be compiled to ERB. So technically it shouldn’t be a problem to share such code.

Abstractions

The problem with today’s template engines is that they aren’t using enough abstractions. Mustache parses and compiles at the same time¹. Haml does parsing/compiling/optimization in some tightly coupled modules. Parkaby separates parsing and compiling, but could share a lot of code with Haml since they’re both compiling HTML tags. While I haven’t checked it out, I assume the same applies to RuHL. Liquid doesn’t really compile (to Ruby) at all, but rather uses a VM approach.

Every template developers faces the same questions: Should I use an Array or String as buffer? How should I escape it? How can I optimize it? Or, they don’t even compile it at all.

Enter the Temple

Temple attempts to solve these problems. Your goal as a template developer is to end up with an Array like this:

[:multi,
 [:static, "Hello "],
 [:dynamic, "@world"],
 [:static, "!\n"],
 [:block, "if @me.happy?"],
 [:static, "\n  "],
 [:dynamic, "@greeting"],
 [:static, "\n"],
 [:block, "end"]]

Then let Temple take over. You could use Temple::Filters::DynamicInliner to optimize sequential statics/dynamics into a single dynamic. Temple::Filters::Escapable handles escaping, and Temple::Core::ArrayBuffer generates the Ruby code.

The idea is to build an engine based on a chain of compilers. A compiler is simply a class which has a method called #compile which takes one arguments. It’s illegal for a compiler to mutate the argument, and it should be possible to use the same instance several times.

Very much like you have middlewares in Rack, you use compilers in Temple (except everything is a compiler in Temple):

class ERBEngine < Temple::Engine
  use Temple::Parsers::ERB
  use Temple::Filters::DynamicInliner
  use Temple::Core::ArrayBuffer
end  

Step 1: The parser

In Temple, a parser is also a compiler, because a compiler is just something that takes some input and produces some output. A parser is then something that takes a String and returns an Array.

A dead simple ERB parser could look like this:

class ERB
  def compile(src)
    result = [:multi]
    while src =~ /<%(.*?)%>/
      result << [:static, $`]
      case $1[0]
      when ?#
        next
      when ?=
        text = $1[1..-1].strip
        head = :dynamic
      else
        text = $1
        head = :block
      end
      result << [head, text]
      src = $'
    end
    result << [:static, src]
    result
  end
end

It’s important to remember that the parser should be dumb. No optimization, no guesses. It should produce an Array that is as close to the source as possible. That means you’ll probably have to invent your own abstraction, but that is exactly the point!

The Mustache parser compiles to an Array like this:

[:multi,
  [:static, "Hello "],
  [:mustache, :evar, :world],
  [:mustache, :section, :happy?, 
    [:mustache, :evar, :greeting]]]

And that’s fine. The parser should not care about how to compile this further down. That’s the job to a filter.

Step N: Filters

A filter is a compiler which takes an Array and returns an Array. It might turn convert it one step closer to the core-abstraction, it might create a new abstraction, or it might just optimize in the current abstraction. Ultimately, it’s still just a compiler which takes an Array and returns an Array.

Temple::Filters::Mustache takes an Array in the Mustache-abstraction and compiles it down to core. Then it has to be ran through Temple::Filters::Escapable which handles HTML escaping.

You might wonder why we split Mustache into a parser and a filter, and there are several reasons. Basically it’s because the parser shouldn’t need to worry about the code it should generate. Now it’s possible to benchmark, rewrite, test and improve the parser, and the parser only.

It’s also because there isn’t one definite way to go from a Mustache-string to the core-abstraction, but there is only one way to go from a Mustache-string to the Mustache-abstraction. If we want to experiment with another way of ending up at core, we can now write it without duplicating the parsing code.

Anyway, after you’ve run it through a few filters, you probably want to generate some Ruby code, and that’s what a generator does.

Step N+1: The generator

A generator is a compiler which takes an Array and returns a String. Generators, just like parsers, are dumb too. Here’s the ArrayBuffer:

class ArrayBuffer < Generator
  def buffer(str = '')
    '_buf' + str
  end
  
  def preamble;  buffer " = []\n" end
  def postamble; buffer ".join"   end
  
  def on_static(text)
    buffer " << #{text.inspect}\n"
  end
  
  def on_dynamic(code)
    buffer " << (#{code})\n"
  end
  
  def on_block(code)
    code + "\n"
  end
end

Temple includes Array, StringBuffer and Interpolation too, even though the last one works kinda funky with blocks and currently only works as expected with Mustache.

Engines

The user however would only see an engine, which is a chain of compilers:

class ERBEngine < Temple::Engine
  # Here using some helpers, but it's important to remember that
  # it's only sugar around the #use method shown above.
  parser :ERB
  filter :DynamicInliner
  generator :ArrayBuffer
end  

The core of Temple::Engine is simple. So simple I’d like to show it to you (without the helpers):

class Engine
  def self.filters
    @filters ||= []
  end
  
  def self.use(filter, *args, &blk)
    filters << [filter, args, blk]
  end
  
  def initialize
    @chain = self.class.filters.map do |filter, args, blk|
      filter.new(*args, &blk)
    end
  end
  
  def compile(thing)
    @chain.inject(thing) { |prev_thing, compiler| compiler.compile(prev_thing) }
  end
end

No magic at all (as long as you understand how inject works). I really like how Temple contains many small pieces which does one thing, and they all easily stacks up and produce some fairy good Ruby code.

Another abstraction: Haml and HTML

Okay, so I’ve shown you two examples: ERB which compiles directly to core, and Mustache which uses one abstraction. Now let’s have a look at Haml.

Because Haml is so complex, going directly to core can be difficult. Instead it might be smart to introduce an HTML-abstraction:

[:multi,
  [:html, :tag,
    :a,
    [:basicattr, :href, "http://judofyr.net/"],
    [:static, "Magnus Holm's blog"]]]  

There can be several HTML-compilers. One aims for speed (Haml’s ugly option), one that aims for pretty indentation (Haml’s pretty option). And I can use the same compiler in Parkaby. If one of these compilers improves and generates better code, it’s going to improve performance in both Haml and Parkaby. Performance wars are going to be so boring!

Nathan Weizenbaum (the maintainer of Haml) told me targeting Haml could be difficult, but I’m quite optimistic. It’s better to try, fail and learn, than not try at all. And to be honest, I’m willing to bend things around to make Haml a happy citizen in Temple.

I’d also love to see how Liquid can fit into this mix.

The goal

So what do I want with all this? I want to experiment! I want to see if it’s possible to improve on the lowest level of the abstractions, and I’d like to see how it affects the upper layers. I’d like to learn more about different template engines; how they work, how they are parsed, how they perform. I simply want to learn. And I want to share the knowledge.

I want people to experiment and create new concepts in templating. I want to see Domain-Specific Template-Languages. I want people to realize that creating a template engine is just like creating a programming language, just on a smaller scale. Testing frameworks are so boring, why don’t try to create The Perfect Template Language™?

And, of course, I want fast template engines that shares code which each other.

But in the end, it’s just an experiment. Maybe it’s successful, probably not, it doesn’t matter so much as long as I have fun.

Join the fun!

I’m the dumb one here, and if you know anything about template engines I’d love to hear your thoughts on Temple. In fact, I’d love to hear everybody’s thoughts on Temple!

If you’re interested, please join the mailing list. The documentation is pretty non-existing at the moment, if there are any questions at all, please do not hesitate to ask.

The code is available at github.com/judofyr/temple.

• ¹ I wrote that code though.

http://github.com/judofyr/parkaby

~> ruby bench/run.rb simple 10000

                           user     system      total        real
Erubis                 0.030000   0.000000   0.030000 (  0.022264)
Haml                   0.110000   0.000000   0.110000 (  0.117887)
Parkaby (def_method)   0.130000   0.000000   0.130000 (  0.135996)
Parkaby (render)       0.150000   0.010000   0.160000 (  0.150680)
Parkaby (inline)       0.970000   0.000000   0.970000 (  0.988010)
Tagz                   3.250000   0.040000   3.290000 (  3.400699)
Markaby               12.610000   0.140000  12.750000 ( 13.067794)


~> ruby bench/run.rb nasty 500

                           user     system      total        real
Erubis                 0.190000   0.010000   0.200000 (  0.198487)
Parkaby (def_method)   0.350000   0.000000   0.350000 (  0.363106)
Parkaby (render)       0.360000   0.010000   0.370000 (  0.365007)
Parkaby (inline)       0.570000   0.000000   0.570000 (  0.614286)
Haml                   2.490000   0.030000   2.520000 (  2.620025)
Tagz                   5.100000   0.060000   5.160000 (  5.394778)
Markaby                7.220000   0.090000   7.310000 (  7.630588)

Feel free to ask questions in the comments.

# stick in .irbrc
def copy(str)
  IO.popen('pbcopy', 'w') { |f| f << str.to_s }
end

def paste
  `pbpaste`
end

def ep
  eval(paste)
end

Linux with xclip (thanks Bjørn Arild Mæland):

def copy(str)
  IO.popen('xclip -i', 'w') { |f| f << str.to_s }
end
 
def paste
  `xclip -o`
end

A few weeks ago, hakunin ruined my repo-followers-ratio, and in order to restore balance in my soul I now see no other options than to reveal one of the things I’ve been playing with lately: GitHub.js.

It’s not very pretty, not very idiomatic and has a nasty dependency on JS.Class. Please remember that I’m still a novice when it comes to JavaScript, so feel free to fork away and turn this into something usable.

$ irb -rnokogirl
no such file to load -- nokogirl (LoadError)

My head just can’t accept it’s called nokogiri instead of nokogirl. Hopefully, this should teach me:

$ sudo gem install nokogirl
Building native extensions.  This could take a while...

********************************************
It's actually spelled nokogiri, not nokogirl
********************************************

Successfully installed nokogiri-1.2.1
Successfully installed nokogirl-1.0
2 gems installed

$ irb -rnokogirl

********************************************
It's actually spelled nokogiri, not nokogirl
********************************************

>> Nokogirl
=> Nokogiri

I’m also very satisfied with both the design and the concept. You start at the top and read until you reach the bottom. No sidebar. No distractions. The content is king. Black, white and red makes it clean; large fonts and line height makes it readable; narrow width makes the paragraphs look longer.

Still, it’s not perfect. The contrast in the header makes it stand out, almost drowning out the title of the post. The timestamps look like they were just throw on later (they were) and it’s pretty cramped in the “Wall of fame” at the bottom. And I haven’t styled <h4> at all.

Welcome to the new version of judofyr.net! If you’re reading this from your feed reader, please come over and have a look. It’s a lot more cleaner: the title is definitly in focus, the timestamps blend nicely into the rest of the site and it’s a little more colorful with some blue and green.

It’s still not perfect. I’m not totally satisfied with the styling of the sub-headlines and the “Wall of fame” has maybe too low contrast. But hey, I’m a programmer – I’m not supposed to design sites.

So, readers, what do you think of this change? After all, this blog is worthless without you. Better? Worse?

RunAgain = Class.new(Exception)
def fib(i, n = 1, result = 0)
  if i == -1
    result
  else
    raise RunAgain
  end
rescue RunAgain
  i, n, result = i - 1, n + result, n
  retry
end

It was extremely slow (see benchmark at the end of this post) since it has to build backtrace for each raise, but at least it worked! I was proud; very proud. Until I discovered this snippet:

class Class
  # Sweet stuff!
  def tailcall_optimize( *methods )
    methods.each do |meth|
      org = instance_method( meth )
      define_method( meth ) do |*args|
        if Thread.current[ meth ]
          throw( :recurse, args )
        else
          Thread.current[ meth ] = org.bind( self )
          result = catch( :done ) do
            loop do
              args = catch( :recurse ) do
                throw( :done, Thread.current[ meth ].call( *args ) )
              end
            end
          end
          Thread.current[ meth ] = nil
          result
        end
      end
    end
  end
end

class TCOTest
  # tail-recursive factorial
  def fact( n, acc = 1 )
    if n < 2 then acc else fact( n-1, n*acc ) end
  end

  # length of factorial
  def fact_size( n )
    fact( n ).size
  rescue
    $!
  end   
end

t = TCOTest.new

# normal method
puts t.fact_size( 10000 )  # => stack level too deep

# enable tail-call optimization
class TCOTest
  tailcall_optimize :fact
end

# tail-call optimized method
puts t.fact_size( 10000 )  # => 14808                                  

Not only was it faster, you could also drop it in wherever you want. Sweet stuff, indeed. My (failed) attempt was sent to /dev/null immediately…

The Ruby Programming Language arrives

A few months later, I received my copy of The Ruby Programming Language and started reading it. Suddenly, on page 151:

The Ruby Programming Language, page 151, 5.5.4 redo: The redo statement restarts the current iteration of a loop or iterator. This is not the same thing as next. next transfers control to the end of a loop or block so that the next iteration can begin, whereas redo transfers control back to the top of the loop or block so that the iteration can start over. If you come to Ruby from a C-like language, then redo is probably a new control structure for you.

I’ve used Ruby for a long time, still I’ve never heard of redo… Here’s an example from the book:

puts "Please enter the first word you think of"
words = %w(apple banana cherry)
response = words.collect do |word|
  # Control returns here when redo is executed
  print word + "> "               # Prompt the user
  response = gets.chop            # Get a response
  if response.size == 0           # If user entered nothing
    word.upcase!                  # Emphasize the prompt with uppercase
    redo                          # And skip to the top of the block
  end
  response                        # Return the response
end

So I started thinking: The reason I used raise/rescue in my implementation was because I needed the retry-keyword… Maybe… What if?

def fib(i, n = 1, result = 0)
  if i == -1
    result
  else
    i, n, result = i - 1, n + result, n
    redo
  end
end

fib(10000)

Unfortunately, “The redo statement restarts the currentiteration of a loop or iterator”, so it only throws a LocalJumpError. However, don’t forget that we’re dealing with Ruby:

### Everything is possible in Ruby!

define_method(:acc) do |i, n, result|
  if i == -1
    result
  else
    i, n, result = i - 1, n + result, n
    redo
  end
end

def fib(i)
  acc(i, 1, 0)
end

fib(10000)  # Yeah!

The Benchmark

So, how fast is it? Take a look below:

• You can zoom by marking in any of the graphs.
• When you un-tick a graph, mark in the lower graph to zoom in.
• Un-tick everything else than “Redo” and “Iterative” and notice how close they are to each other.
• Look how bad “Rescue” performs,
• and how early “Regular” fails.
• All the code is available at GitHub.

Nifty, eh?

Let’s use Sinatra as an example

Most of them are very similar to Sinatra’s (this is slightly modified for readability, but the idea still applies):

params.inject({}) do |hash, (key, value)|
  if key =~ /\[.*\]/
    parts = key.scan(/(^[^\[]+)|\[([^\]]+)\]/).flatten.compact
    head, last = parts[0..-2], parts[-1]
    head.inject(hash){ |s,v| s[v] ||= {} }[last] = value
  else
    hash[key] = value
  end
  res
end   

Update: As Ryan has written in the comments, I better give some credz to the real author of the snippet above: Sinatra’s nested params implementation was taken from an example posted to the Rack ML by Michael Fellinger (of Ramaze fame). For the background on the patch that went into Sinatra: take a look the ticket over at their bug tracker.

We’re looping through each of the params and doing some stuff if the key includes [something] and therefore is nested. If it’s nested, we need to find the parts:

parts = key.scan(/(^[^\[]+)|\[([^\]]+)\]/).flatten.compact

The first part of the regex matches from the beginning to the first [, and the second matches each of the [parts]. Some cleanup is needed to flatten and remove nils (just try the line in IRB and you’ll see it quickly).

Next up, we’re splitting out the last part, and then comes the inject:

head.inject(hash){ |s,v| s[v] ||= {} }[last] = value  

Here, we’re building a Hash based on the params we’ve already cleaned up and the current param we’re working on. Then, finally, we’re setting the value.

The _why way

All of this makes sense. I wrote approximately the same when trying to cleanup my broken version. However, while I was looking for the way Ramaze did it, I found an excellent link to RedHanded: Injecting a Hash Backwards and the Merge Block. That version is just awesome (this is also slightly modified):

m = proc {|_,o,n|o.merge(n,&m)}
params.inject({}) do |hash, (key, value)|
  parts = key.split(/[\]\[]+/)
  hash.merge(parts.reverse.inject(value) { |x, i| {i => x} }, &m) 
end    

Notice the sweet, micro way to split out the parts: If we got a key like: “first[second][third]” we can simply split by any numbers of [ and ]. Of course, we’re going to fail on stuff like “this]is]really]one]key”, but if you’re writing like that, you deserve it!

Now, the next is what makes this so different and awesome. Let’s look at first part of it:

parts.reverse.inject(value) { |x, i| {i => x} }

We’re reversing it and building it backwards. The inject starts with our original value and then we build our way out by creating Hashes:

parts = ["first", "second", "third"]
value = 123

# first run of inject:
x = 123
i = "third"
return { "third" => 123 }

# second run of inject:
x = { "third" => 123 }
i = "second"
return { "second" => { "third" => 123 } }

# second run of inject:
x = { "second" => { "third" => 123 } }
i = "first"
return { "first" => { "second" => { "third" => 123 } } }

So when we got this little recursive Hash, we need to merge it with the rest. Most of you, including me, would say that it would be a hard task, since we also need to merge it recursively:

params = { "first" => { "second" => { "third" => 123 } } }
current_param = { "first" => { "another" => 456 } }

params.merge(current_param) # fail!
current_param.merge(params) # fail!

Meet the Merge Block

This was the first time I’ve ever heard of the merge block. It’s not even properly documented! But it’s a very simple and very powerful feature: If we get a merge conflict (same keys in both Hashes), it’s calling that block. Easy peasy!

params = { "first" => { "second" => { "third" => 123 } } }
current_param = { "first" => { "another" => 456 } }

params.merge(current_param) do |key, value_from_params, value_from_current_param|
  # key is defined in both params and current_param
  value_from_params.merge(value_from_current_param)
end

# =>
{ "first" => { "second" => { "third" => 123 }, "another" => 456 } }

And in order to merge it recursively, we build the block in advance and pass the block into the inner merge too:

m = proc {|_,o,n|o.merge(n,&m)}
# ...
hash.merge(recursive_hash, &m) 

Shorter? Faster?

I believe the first, natural way is both faster and shorter (if we use the micro splitter), but I still think I’m going to put _why’s version in Camping. Not (only) because it’s _why’s, but also because it’s so wicked awesome and fits perfectly together with the other weird stuff in Camping. Some bytes have to be sacrificed for style; Camping is still far away from exceeding the limit.

If it’s an automatically generated folder, it’s a bit easier, but still quite a few commands:

rake generate_some_docs
git checkout gh-pages
git add .
git commit
git push

It’s even harder if you haven’t created the gh-pages branch yet.

Grancher!

Okey, enter Grancher (yeah, I know, lame name):

# Install:
system('sudo gem install grancher')

# Create a Rakefile:
require 'grancher/task'
Grancher::Task.new do |g|
  g.branch = 'gh-pages'
  g.push_to = 'origin' # automatically push too
  
  g.directory 'website'
end

Do a rake publish and the folder will be copied to the gh-pages branch, committed and pushed. Easy peasy!

Have a look at the documentation which isn’t actually perfect, but should be enough to get started.

Oh, and it’s just a few hours old, so bugs will probably bite you! When they do, please report it at lighthouse.