## Implementing chained comparisons in Ruby

(1..10).each do |x|
  case
  when x < 3
    puts "#{x} is quite low"
  when 3 <= x < 6
    puts "#{x} is quite all right"
  when 6 <= x < 10
    puts "#{x} is quite fine"
  when 10 <= x
    puts "#{x} is quite high"
  end
end


lets
  BEGIN {
        
Object.send(*                                                           
  "ZGVmaW5lX21ldGhvZA=bWV0aG9kX21pc3Npbmc=".unpack("mm"                      )
                                                                          ){|*|}
[[],[61]].inject([]){|_,__|_<<[60,*__]<<[62,*__]}.map{|_|_.pack("c*"        )}.
each { |a| [Float, Fixnum, Comparable].each { |b| b.class_eval               {
c = instance_method(a) and define_method(a) { |d| c.bind(self).call(d) and d }
                                                                      
there                                                                        }
  is                                                                         }
    no              ;FalseClass.class_eval "def %s(*) false end"%a           }
      END                                                                   {}
                                                                            
to                                                                          
  this                                                                      
    madness                                                                  }
                                                                            
this
  is
    RUBY!
      (not perl?)

The gemspec is the README of the bits and the bytes of your code. It can both be understood and changed by both computers and humans. We shouldn’t fear the gemspec; we should embrace it.

Storing the data in the Rakefile and generating the gemspec seems totally backwards to me. Rakefile is for tasks, let’s place the data where it belongs: in the gemspec.

Instead of building tools which generates the gemspec, let’s build tools that uses the gemspec.

Like gem build rails.gemspec.

Like the newest Gemify:

$ gemify
Currently editing gemify.gemspec

Which task would you like to invoke?
1) Change name (required) = gemify
2) Change summary (required) = The lightweight gemspec editor
3) Change version (required) = 0.3
4) Change author = Magnus Holm
5) Change email = judofyr@gmail.com
6) Change homepage = http://dojo.rubyforge.org/
7) Set dependencies

s) Save
r) Reload (discard unsaved changes)
m) Rename
l) List files

x) Exit

It’s not a solved problem though.

Rubygems can read any gemspec with Gem::Specification.load, but can only write it back again with #to_ruby in a normalized version.

Let’s have a look at Bundler’s excellent gemspec:

# -*- encoding: utf-8 -*-
lib = File.expand_path('../lib/', __FILE__)
$:.unshift lib unless $:.include?(lib)

require 'bundler/version'

Gem::Specification.new do |s|
  s.name        = "bundler"
  s.version     = Bundler::VERSION
  s.platform    = Gem::Platform::RUBY
  s.authors     = ["Carl Lerche", "Yehuda Katz", "André Arko"]
  s.email       = ["carlhuda@engineyard.com"]
  s.homepage    = "http://github.com/carlhuda/bundler"
  s.summary     = "The best way to manage your application's dependencies"

  s.required_rubygems_version = ">= 1.3.6"
  s.rubyforge_project         = "bundler"

  s.add_development_dependency "rspec"

  s.files        = Dir.glob("{bin,lib}/**/*") + %w(LICENSE README.md ROADMAP.md CHANGELOG.md)
  s.executables  = ['bundle']
  s.require_path = 'lib'
end

Short and concise. After running it through Gem::Specification.load(file).to_ruby:

# -*- encoding: utf-8 -*-

Gem::Specification.new do |s|
  s.name = %q{bundler}
  s.version = "0.10.pre"

  s.required_rubygems_version = Gem::Requirement.new(">= 1.3.6") if s.respond_to? :required_rubygems_version=
  s.authors = ["Carl Lerche", "Yehuda Katz", "Andr\303\251 Arko"]
  s.date = %q{2010-04-03}
  s.default_executable = %q{bundle}
  s.email = ["carlhuda@engineyard.com"]
  s.executables = ["bundle"]
  s.files = %w[one huge array]
  s.homepage = %q{http://github.com/carlhuda/bundler}
  s.require_paths = ["lib"]
  s.rubyforge_project = %q{bundler}
  s.rubygems_version = %q{1.3.6}
  s.summary = %q{The best way to manage your application's dependencies}

  if s.respond_to? :specification_version then
    current_version = Gem::Specification::CURRENT_SPECIFICATION_VERSION
    s.specification_version = 3

    if Gem::Version.new(Gem::RubyGemsVersion) >= Gem::Version.new('1.2.0') then
      s.add_development_dependency(%q<rspec>, [">= 0"])
    else
      s.add_dependency(%q<rspec>, [">= 0"])
    end
  else
    s.add_dependency(%q<rspec>, [">= 0"])
  end
end

Boom, there goes your nice VERSION constant. And no more Dir globbing for you.

We need to make it possible to write interactive gemspec (heck, that’s why they are written in Ruby!) and make them easily readable by computers and make them easily modified by computers without losing the interactivity.

Anyone?

def process_call(exp)
  # exp[1] => reciever.
  # exp[2] => method
  # exp[3] => (args)
  exp[3] = process(exp[3])
  if text?(exp)
    s(:parkaby, :text, exp[3])
  elsif tag = force_tag_call?(exp) || tag_call?(exp)
    s(:parkaby, :tag, *tag)
  else
    exp
  end
end

I just write down what I want it to look like:

rule :tag_call do
  # Forced tag call
  s(:call,
   s(:call, nil, :tag, s(:arglist)),
   wild % :name,
   args % :args) |

  # or regular tag call
  s(:call,
  nil,
  name % :name,
  args % :args)
end

rule :text do
  s(:call, nil,      :text, s(:arglist, wild % :content)) |
  s(:call, s(:self), :<<,   s(:arglist, wild % :content))
end

Then I implement it.

Okay. The above examples aren’t quite equal, so let’s have a look at what I’m really trying to solve.

Matching complex Sexp in Parkaby

Parkaby is my little experiment to make a super-duper-freaky-fast Markaby replacement by parsing the source and “compiling” it. Ultimately, a template like:

h1 "Hello World!"
p "Welcome #{@user}"

Should be compiled into:

"<h1>Hello World!</h1><p>Welcome #{@user}</p>"

In Parkaby, this involes two steps: a processor which figures out what should be considered HTML-tags and what should be considered regular method calls, and a generator which compiles it into Ruby. An interesting aspect of Parkaby is that the processor actually is quite complex. For instance, it has to figure out that div.post.clearfix.main!(:style => "display:none") should be compiled to <div class="post clearfix" id="main" style => "display:none"></div>.

My original approach was to use SexpProcessor:

def process_call(exp)
  # exp[1] => reciever.
  # exp[2] => method
  # exp[3] => (args)
  exp[3] = process(exp[3])
  if text?(exp)
    s(:parkaby, :text, exp[3])
  elsif tag = force_tag_call?(exp) || tag_call?(exp)
    s(:parkaby, :tag, *tag)
  else
    exp
  end
end

Every time it finds a method call, it checks if it’s a text-node or a HTML-tag and then turns it into a parkaby-sexp, or it just leaves it alone. The code ended up quite messy, and that’s before I even tried implementing CSS-proxy (tag.klass.klass.klass.id!): Parkaby before SexpBuilder.

Just thinking of implementing CSS-proxy got me a light headache, so I quickly realized that something had to be done.

Adam Sanderson to the rescue!

Ka-poof! Adam Sanderson writes SexpPath.

SexpPath is a DSL for matching Sexp. Let’s say we want to match text "Hello" and self << "Hello":

# text "Hello"
sexp1 = s(:call, nil, :text, s(:arglist, s(:str, "Hello")))
# self << "Hello"
sexp2 = s(:call, s(:self), :<<, s(:arglist, s(:str, "Hello")))

# the old approach:
def like_text?(exp)
  rec_meth = exp[1..2].to_a
  rec_meth == [[:self], :<<] || rec_meth == [nil, :text]
end

def text?(exp)
  like_text?(exp) and
  exp[3].length == 2
end

# the SexpPath approach:
query = Q? do
  s(:call, nil,      :text, s(:arglist, wild % :content)) |
  s(:call, s(:self), :<<,   s(:arglist, wild % :content))
end

Isn’t that pretty? The vertical-bar means “or”, `wild` matches everything and the percent sign captures the value in a Hash. Exactly what I want.

SexpBuilder

gem install sexp_builder

Well, SexpPath alone wasn’t enough to solve my problem – I had to introduce a new library. SexpBuilder is a more complete solution to easily match and replace complex Sexp. Very much like a parser, you define matchers and rules, and by combining them with a rewriter it was breeze to implement CSS-proxy.

Have a look at http://dojo.rubyforge.org/ for documentation, and Parkaby::Processor if you want a full example. There’s also Andand.rb for a more traditional example/demo.

What about Parkaby?

Well, at the moment, Parkaby is resting. In order to fully support it on other platforms than MRI, I’ll have to start yet another project. And that’s not happening until I release a few other projects I’m working on (like SexpBuilder).

10.times do |i|
  begin
    raise "OH NO!"
    puts "OH YES! #{i}"
    i += 1
  rescue => err
    if i < 5
      err.continue
    else
      raise err
    end
  end
end

Oh, that’s right. This is Ruby. Forty lines later:

class Exception
  class NoContinuation < StandardError
  end
  
  attr_accessor :continuation
  
  def continue
    raise NoContinuation unless continuation.respond_to?(:call)
    continuation.call
  end
end

module NeverGonnaLetYouDown
  def raise(exception = RuntimeError, string = nil, array = caller)
    # With a single String argument, raises a
    # RuntimeError with the string as a message. 
    if exception.is_a?(String)
      string = exception
      exception = RuntimeError
    end
    
    callcc do |cc|
      obj = exception.exception(string)
      obj.set_backtrace(array)
      obj.continuation = cc
      super obj
    end
  end
  
  def fail(exception = RuntimeError, string = nil, array = caller)
    raise(exception, string, array)
  end
end

class Object
  include NeverGonnaLetYouDown
end

Every compilable template can be compiled to ERB.

Well, not ERB-the-syntax, but ERB-the-concept:

Every compilable template consists of three elements:

• Static text
• Dynamic text (pieces of Ruby which are evaluated and sent to the client)
• Blocks (pieces of Ruby which are evaluated and not sent to the client, but might change the control flow).

Nothing revolutionary at all. Just a theory.

Compiling it

There are several ways to render such a template, but the fastest one is to compile it to pure Ruby:

Hello <%= @world %>!
<% if @me.happy? %>
  <%= @greeting %>
<% end %>

Becomes:

_buf = []
_buf << "Hello "
_buf << (@world)
_buf << "!\n"
if @me.happy?
_buf << "\n  "
_buf << (@greeting)
_buf << "\n"
end
_buf.join

Probably you also want to optimize it:

_buf = []
_buf << ("Hello #{@world}!\n")
if @me.happy?
_buf << ("\n  #{@greeting}\n")
end
_buf.join

Or maybe you rather want _buf to be a String? Or print it directly to stdout?

These options are already written and implemented in Erubis, but none of the other template engines can take advantage of that. Which is a shame, because we know that every compilable template can be compiled to ERB. So technically it shouldn’t be a problem to share such code.

Abstractions

The problem with today’s template engines is that they aren’t using enough abstractions. Mustache parses and compiles at the same time¹. Haml does parsing/compiling/optimization in some tightly coupled modules. Parkaby separates parsing and compiling, but could share a lot of code with Haml since they’re both compiling HTML tags. While I haven’t checked it out, I assume the same applies to RuHL. Liquid doesn’t really compile (to Ruby) at all, but rather uses a VM approach.

Every template developers faces the same questions: Should I use an Array or String as buffer? How should I escape it? How can I optimize it? Or, they don’t even compile it at all.

Enter the Temple

Temple attempts to solve these problems. Your goal as a template developer is to end up with an Array like this:

[:multi,
 [:static, "Hello "],
 [:dynamic, "@world"],
 [:static, "!\n"],
 [:block, "if @me.happy?"],
 [:static, "\n  "],
 [:dynamic, "@greeting"],
 [:static, "\n"],
 [:block, "end"]]

Then let Temple take over. You could use Temple::Filters::DynamicInliner to optimize sequential statics/dynamics into a single dynamic. Temple::Filters::Escapable handles escaping, and Temple::Core::ArrayBuffer generates the Ruby code.

The idea is to build an engine based on a chain of compilers. A compiler is simply a class which has a method called #compile which takes one arguments. It’s illegal for a compiler to mutate the argument, and it should be possible to use the same instance several times.

Very much like you have middlewares in Rack, you use compilers in Temple (except everything is a compiler in Temple):

class ERBEngine < Temple::Engine
  use Temple::Parsers::ERB
  use Temple::Filters::DynamicInliner
  use Temple::Core::ArrayBuffer
end  

Step 1: The parser

In Temple, a parser is also a compiler, because a compiler is just something that takes some input and produces some output. A parser is then something that takes a String and returns an Array.

A dead simple ERB parser could look like this:

class ERB
  def compile(src)
    result = [:multi]
    while src =~ /<%(.*?)%>/
      result << [:static, $`]
      case $1[0]
      when ?#
        next
      when ?=
        text = $1[1..-1].strip
        head = :dynamic
      else
        text = $1
        head = :block
      end
      result << [head, text]
      src = $'
    end
    result << [:static, src]
    result
  end
end

It’s important to remember that the parser should be dumb. No optimization, no guesses. It should produce an Array that is as close to the source as possible. That means you’ll probably have to invent your own abstraction, but that is exactly the point!

The Mustache parser compiles to an Array like this:

[:multi,
  [:static, "Hello "],
  [:mustache, :evar, :world],
  [:mustache, :section, :happy?, 
    [:mustache, :evar, :greeting]]]

And that’s fine. The parser should not care about how to compile this further down. That’s the job to a filter.

Step N: Filters

A filter is a compiler which takes an Array and returns an Array. It might turn convert it one step closer to the core-abstraction, it might create a new abstraction, or it might just optimize in the current abstraction. Ultimately, it’s still just a compiler which takes an Array and returns an Array.

Temple::Filters::Mustache takes an Array in the Mustache-abstraction and compiles it down to core. Then it has to be ran through Temple::Filters::Escapable which handles HTML escaping.

You might wonder why we split Mustache into a parser and a filter, and there are several reasons. Basically it’s because the parser shouldn’t need to worry about the code it should generate. Now it’s possible to benchmark, rewrite, test and improve the parser, and the parser only.

It’s also because there isn’t one definite way to go from a Mustache-string to the core-abstraction, but there is only one way to go from a Mustache-string to the Mustache-abstraction. If we want to experiment with another way of ending up at core, we can now write it without duplicating the parsing code.

Anyway, after you’ve run it through a few filters, you probably want to generate some Ruby code, and that’s what a generator does.

Step N+1: The generator

A generator is a compiler which takes an Array and returns a String. Generators, just like parsers, are dumb too. Here’s the ArrayBuffer:

class ArrayBuffer < Generator
  def buffer(str = '')
    '_buf' + str
  end
  
  def preamble;  buffer " = []\n" end
  def postamble; buffer ".join"   end
  
  def on_static(text)
    buffer " << #{text.inspect}\n"
  end
  
  def on_dynamic(code)
    buffer " << (#{code})\n"
  end
  
  def on_block(code)
    code + "\n"
  end
end

Temple includes Array, StringBuffer and Interpolation too, even though the last one works kinda funky with blocks and currently only works as expected with Mustache.

Engines

The user however would only see an engine, which is a chain of compilers:

class ERBEngine < Temple::Engine
  # Here using some helpers, but it's important to remember that
  # it's only sugar around the #use method shown above.
  parser :ERB
  filter :DynamicInliner
  generator :ArrayBuffer
end  

The core of Temple::Engine is simple. So simple I’d like to show it to you (without the helpers):

class Engine
  def self.filters
    @filters ||= []
  end
  
  def self.use(filter, *args, &blk)
    filters << [filter, args, blk]
  end
  
  def initialize
    @chain = self.class.filters.map do |filter, args, blk|
      filter.new(*args, &blk)
    end
  end
  
  def compile(thing)
    @chain.inject(thing) { |prev_thing, compiler| compiler.compile(prev_thing) }
  end
end

No magic at all (as long as you understand how inject works). I really like how Temple contains many small pieces which does one thing, and they all easily stacks up and produce some fairy good Ruby code.

Another abstraction: Haml and HTML

Okay, so I’ve shown you two examples: ERB which compiles directly to core, and Mustache which uses one abstraction. Now let’s have a look at Haml.

Because Haml is so complex, going directly to core can be difficult. Instead it might be smart to introduce an HTML-abstraction:

[:multi,
  [:html, :tag,
    :a,
    [:basicattr, :href, "http://judofyr.net/"],
    [:static, "Magnus Holm's blog"]]]  

There can be several HTML-compilers. One aims for speed (Haml’s ugly option), one that aims for pretty indentation (Haml’s pretty option). And I can use the same compiler in Parkaby. If one of these compilers improves and generates better code, it’s going to improve performance in both Haml and Parkaby. Performance wars are going to be so boring!

Nathan Weizenbaum (the maintainer of Haml) told me targeting Haml could be difficult, but I’m quite optimistic. It’s better to try, fail and learn, than not try at all. And to be honest, I’m willing to bend things around to make Haml a happy citizen in Temple.

I’d also love to see how Liquid can fit into this mix.

The goal

So what do I want with all this? I want to experiment! I want to see if it’s possible to improve on the lowest level of the abstractions, and I’d like to see how it affects the upper layers. I’d like to learn more about different template engines; how they work, how they are parsed, how they perform. I simply want to learn. And I want to share the knowledge.

I want people to experiment and create new concepts in templating. I want to see Domain-Specific Template-Languages. I want people to realize that creating a template engine is just like creating a programming language, just on a smaller scale. Testing frameworks are so boring, why don’t try to create The Perfect Template Language™?

And, of course, I want fast template engines that shares code which each other.

But in the end, it’s just an experiment. Maybe it’s successful, probably not, it doesn’t matter so much as long as I have fun.

Join the fun!

I’m the dumb one here, and if you know anything about template engines I’d love to hear your thoughts on Temple. In fact, I’d love to hear everybody’s thoughts on Temple!

If you’re interested, please join the mailing list. The documentation is pretty non-existing at the moment, if there are any questions at all, please do not hesitate to ask.

The code is available at github.com/judofyr/temple.

• ¹ I wrote that code though.

http://github.com/judofyr/parkaby

~> ruby bench/run.rb simple 10000

                           user     system      total        real
Erubis                 0.030000   0.000000   0.030000 (  0.022264)
Haml                   0.110000   0.000000   0.110000 (  0.117887)
Parkaby (def_method)   0.130000   0.000000   0.130000 (  0.135996)
Parkaby (render)       0.150000   0.010000   0.160000 (  0.150680)
Parkaby (inline)       0.970000   0.000000   0.970000 (  0.988010)
Tagz                   3.250000   0.040000   3.290000 (  3.400699)
Markaby               12.610000   0.140000  12.750000 ( 13.067794)


~> ruby bench/run.rb nasty 500

                           user     system      total        real
Erubis                 0.190000   0.010000   0.200000 (  0.198487)
Parkaby (def_method)   0.350000   0.000000   0.350000 (  0.363106)
Parkaby (render)       0.360000   0.010000   0.370000 (  0.365007)
Parkaby (inline)       0.570000   0.000000   0.570000 (  0.614286)
Haml                   2.490000   0.030000   2.520000 (  2.620025)
Tagz                   5.100000   0.060000   5.160000 (  5.394778)
Markaby                7.220000   0.090000   7.310000 (  7.630588)

Feel free to ask questions in the comments.

# stick in .irbrc
def copy(str)
  IO.popen('pbcopy', 'w') { |f| f << str.to_s }
end

def paste
  `pbpaste`
end

def ep
  eval(paste)
end

Linux with xclip (thanks Bjørn Arild Mæland):

def copy(str)
  IO.popen('xclip -i', 'w') { |f| f << str.to_s }
end
 
def paste
  `xclip -o`
end

A few weeks ago, hakunin ruined my repo-followers-ratio, and in order to restore balance in my soul I now see no other options than to reveal one of the things I’ve been playing with lately: GitHub.js.

It’s not very pretty, not very idiomatic and has a nasty dependency on JS.Class. Please remember that I’m still a novice when it comes to JavaScript, so feel free to fork away and turn this into something usable.

$ irb -rnokogirl
no such file to load -- nokogirl (LoadError)

My head just can’t accept it’s called nokogiri instead of nokogirl. Hopefully, this should teach me:

$ sudo gem install nokogirl
Building native extensions.  This could take a while...

********************************************
It's actually spelled nokogiri, not nokogirl
********************************************

Successfully installed nokogiri-1.2.1
Successfully installed nokogirl-1.0
2 gems installed

$ irb -rnokogirl

********************************************
It's actually spelled nokogiri, not nokogirl
********************************************

>> Nokogirl
=> Nokogiri

I’m also very satisfied with both the design and the concept. You start at the top and read until you reach the bottom. No sidebar. No distractions. The content is king. Black, white and red makes it clean; large fonts and line height makes it readable; narrow width makes the paragraphs look longer.

Still, it’s not perfect. The contrast in the header makes it stand out, almost drowning out the title of the post. The timestamps look like they were just throw on later (they were) and it’s pretty cramped in the “Wall of fame” at the bottom. And I haven’t styled <h4> at all.

Welcome to the new version of judofyr.net! If you’re reading this from your feed reader, please come over and have a look. It’s a lot more cleaner: the title is definitly in focus, the timestamps blend nicely into the rest of the site and it’s a little more colorful with some blue and green.

It’s still not perfect. I’m not totally satisfied with the styling of the sub-headlines and the “Wall of fame” has maybe too low contrast. But hey, I’m a programmer – I’m not supposed to design sites.

So, readers, what do you think of this change? After all, this blog is worthless without you. Better? Worse?