zerosum dirt(nap)

evolution through a series of accidents

zerosum dirt(nap)

Clone Pastie with Sinatra & DataMapper 0.9

July 02, 2008 by nap · Comments

Waaaay back in November I wrote a DataMapper and Sinatra tutorial. DM was still pretty immature at the time, and it quickly became outdated. I promised I’d update it once 0.9 was released.

Better late than never, right? Anyway, without further ado, here’s Toopaste Tutorial 2.0:

What We’re Building

Today we’re going to learn how to write a dirt simple pastie clone using Sinatra and Datamapper. For those of you unfamiliar with pastie systems, they’re commonly used in IRC to paste bits of code to a globally-visible place outside of the channel. So you copy some code from your IDE, paste it into a web page, submit it, and the Pastie system fancies it up with some syntax highlighting or whatnot. You can then copy the resultant URL and paste it back into the channel so other people can view it.

Examples of this type of service include pastie.caboo.se and Nopaste @ rafb.net. Those sites do a lot of extra cool stuff, like allow you to select the language for different syntax highlighting rules, select themes for viewing, and so on. But the core concept itself isn’t a terribly complex one, and our example is going to be about as barebones as they come.

The interesting thing isn’t the application we’re going to build here so much as it is the tools we’ll use. That is, we’ll be using the pastie example to introduce you to two cool new pieces of Ruby tech: Sinatra, a Ruby web micro-framework, and the DataMapper ORM package.

Sinatra

Sinatra, on the surface, is a lot like Camping, another Ruby web all-in-one-file micro-framework. Camping hasn’t seen much active development lately though, and the syntax can be a bit strange for new users. Sinatra is much more straight-forward and accessible. It’s really a kind of domain-specific language for writing simple web applications, used to define RESTful actions and how they should be handled. This makes it perfect for lightweight mini-apps. It’s also ORM-agnostic, instead of being married to ActiveRecord like both Rails and Camping are.

Sinatra has been making cameo appearances supporting a number of high-profile Ruby-based web apps, including Heroku and GitHub. It runs on top of Rack, which means that it plays nice with a wide variety of Ruby web app servers, including Mongrel, Thin, and Ebb. It also means that you can host your Sinatra applications easily with Apache and Phusion Passenger, as of Passenger 2.0. For more information, check out the ‘official’ Sinatra tutorial (oops dead link). There’s also an open source Sinatra book in the works.

DataMapper

DataMapper is the new ORM package on the block, and an alternative to ActiveRecord (and Og and Sequel and so on). It just moved into town but it’s already sitting at the cool kid lunch table. Whereas AR implements the ActiveRecord Pattern, DataMapper (surprise!) implements the DataMapper pattern.

There are a number of reasons why I prefer this approach, but that’s fodder for an entirely separate blog post, so I won’t get into it here. Besides, the team has already written a great summary of why DataMapper rocks. Read it. It’s good stuff. Oh, and performance kicks ass too.

Take that, non-believers!

Prerequisites

OK, anyway. Tutorial time. Found your plastic hat? Good. Let’s go. First let’s get the gems we’ll need. As of this writing, DM is at v0.9.2 and Sinatra is at v0.2.2. We’re also going to retrieve the Syntaxi gem, which we’ll use for syntax highlighting.

sudo gem install sinatra data_mapper syntaxi

The data_mapper gem is a “meta gem” that includes the most commonly used gems from dm-core, dm-more, and others. We’ll be using a couple of these additional libraries — dm-timestamps and dm-validations — in this tutorial.

Since DM uses the DataObjects.rb (DO) drivers, you’ll want to install them too. For this tutorial we’ll be using Sqlite3 as a data store. If you want to use MySQL or Postgres or any other database supported by DO, well that’s cool too (just make sure to get do\_mysql or do\_postgres).

sudo gem install data_objects do_sqlite3

The Code

Now that our prerequisites are satisfied, let’s get started by creating the file toopaste.rb:

require 'sinatra'
require 'dm-core'
require 'dm-validations'
require 'dm-timestamps'
require 'syntaxi'

DataMapper.setup(:default, "sqlite3://#{Dir.pwd}/toopaste.sqlite3")

class Snippet
  include DataMapper::Resource

  property :id,         Integer, :serial => true    # primary serial key
  property :body,       Text,    :nullable => false # cannot be null
  property :created_at, DateTime
  property :updated_at, DateTime

  # validates_present :body
  # validates_length :body, :minimum => 1

  Syntaxi.line_number_method = 'floating'
  Syntaxi.wrap_at_column = 80

  def formatted_body
    replacer = Time.now.strftime('[code-%d]')
    html = Syntaxi.new("[code lang='ruby']#{self.body.gsub('[/code]', replacer)}[/code]").process
    "<div class=\"syntax syntax_ruby\">#{html.gsub(replacer, '[/code]')}</div>"
  end
end

DataMapper.auto_upgrade!

# new
get '/' do
  erb :new
end

# create
post '/' do
  @snippet = Snippet.new(:body => params[:snippet_body])
  if @snippet.save
    redirect "/#{@snippet.id}"
  else
    redirect '/'
  end
end

# show
get '/:id' do
  @snippet = Snippet.get(params[:id])
  if @snippet
    erb :show
  else
    redirect '/'
  end
end

Next we’ll dissect this code listing to give you a brief look at how DataMapper and Sinatra work. We’ll show the code listings for our views as we get to them.

Code Analysis: DataMapper

The first couple lines require the libraries we’ll be using in this example. dm-core represents the DataMapper core libraries, and dm-validations and dm-timestamps both add extra bits of non-core functionality to DataMapper. In this case, requiring timestamps means that fields such as created\_at and updated\_at automatically get branded with the current Date/Time when a model is created or updated. As for validations, well we’ll see them in use shortly.

Before we create out first DataMapper-backed model we have to set up the database. The following line sets the default database connection and tells it to use the Sqlite3 file-based database in the current working directory. It won’t exist yet but don’t worry, we’ll be creating it automatically when we migrate.

DataMapper.setup(:default, "sqlite3://#{Dir.pwd}/toopaste.sqlite3")
Persistent Models and Properties

Next we define our model, Snippet. Since we want to persist snippets, we include the DataMapper::Resource module in the class definition. From an end user point of view this is the same as inheriting from ActiveRecord::Base in Active Record. It’s a nice bonus that we don’t have to use inheritance to accomplish this any more.

include DataMapper::Resource

The property declarations in our model are DataMapper’s way of specifying attributes, which translate to database table columns. This might look a little bit odd at first if you’re coming over from Active Record…

OMG I don’t have to put comments in my source file to remind me what attributes are available on my models?!”

It’s like a little slice of heaven, isn’t it?

property :id,         Integer, :serial => true    # primary serial key
property :body,       Text,    :nullable => false # cannot be null
property :created_at, DateTime
property :updated_at, DateTime

The only two required parameters are the name of the property and the type, but you can also add other options to mark a property as the primary serial key, prevent it from being null, and so on. Properties get public accessors by default, but you can modify that by setting the :accessor option to :private or :protected (:reader and :writer options are also supported for even more control).

Another interesting DM optimization worth noting: by default, the text field (body) is lazily loaded. Text columns are expensive in databases, and by using lazy loading, we only access them when they’re needed. This speeds things up significantly in most cases. However, if you don’t want to do this for whatever reason, you can just pass a :lazy => false option in the text column property declaration.

Validation

The validators we’re specifying should look relatively familiar to anyone familiar with AR. They’re only available to us because we required the dm-validations library, as mentioned. This is part of DM’s Merb-like minimalist approach; start with just the bare essentials, and allow developers to mix in extra functionality as its desired. The kitchen sink is so fail, right?

# validates_present :body
# validates_length :body, :minimum => 1

So why are these lines commented out then, you ask?

Well, we could explicitly add the validates_present :body to specify that the body property must be present, but that’s already handled for us in the property declaration since we specified :nullable => false. If dm-validations is not required, the nullable option will simply ensure that the database column isn’t null, but if validations are in use it will augment this behavior to assume that we not only mean ‘not null’, but also ‘not empty’.

Likewise, validates_length could be useful in certain situations, but in our case we just want to make sure that the paste body isn’t empty, which is already handled for us by the nullable option on the body property.

There are lots of other validations that we can add in this manner, of course. For more information see the DataMapper API documentation.

Syntax Highlighting with Syntaxi

The only custom method we’ve added to our model, formatted_body, is pretty straightforward. It just takes the body of our snippet (a property on the model) and wraps it in some code that gives us pretty syntax highlighting and line numbering. Syntaxi leverages Jamis Buck’s Syntax gem to mark up the specified code, wrapping CSS span tags around things that should be colored, adding line numbers, and some other goodies too — you’ll see the CSS in our layout shortly.

def formatted_body
  replacer = Time.now.strftime('[code-%d]')
  html = Syntaxi.new("[code lang='ruby']#{self.body.gsub('[/code]', replacer)}[/code]").process
  "<div class=\"syntax syntax_ruby\">#{html.gsub(replacer, '[/code]')}</div>"
end

The replacer stuff just keeps us from stumbling on syntax coloring markup in our output HTML.

Auto-Migrations

Now that we’re done defining our database-backed model, we’re just about ready to move on to the web-serving portion of the tutorial. But first, we need to tell DataMapper to create the appropriate database table if it doesn’t already exist. We do this by using the DataMapper.auto\_upgrade! method, which checks to see if the database table that corresponds to our model(s) need upgrading and then creates or updates them as appropriate. It’s a non-destructive way to do auto-migrations (the destructive equivalent is auto\_migrate!)

Code Analysis: Sinatra

Moving on to the Sinatra portion of our tutorial, you’ll see that routes and actions are intimately married in Sinatra. Although this may not be desirable for a larger application, it’s great for smaller, simpler web apps like this pastie project. In this case we’re talking uber-simple; we only have three actions and they’re only ever available at these three URL patterns.

The Index Action / New Snippet Form

The first of our actions needs to display a form to create a new paste. It should always be available at '/', the application root.

get '/' do
  erb :new
end

This tells Sinatra that when a GET request for '/' comes in, that we should use the erb helper to render the new.erb template, which is stored in the views/ subdirectory by convention and marked up with embedded Ruby (ERb). We can render our responses inline as well, which works for dirt-simple applications, but in most cases you’ll prefer to keep the view templates outside of this file.

In any case, the new.erb template is shown below:

<div class="snippet">
  <form action="/" method="POST">
    <textarea name="snippet_body" id="snippet_body" rows="20"></textarea>
    <br/><input type="submit" value="Save"/>
  </form>
</div>
The Create Action

Our next action is analogous to a #create action in RESTful Rails. The action is requested by a POST to the application root. Sinatra supports the standard GET and POST HTTP methods as well as PUT and DELETE, meaning that you can build 100% RESTful applications with it.

post '/' do
  @snippet = Snippet.new(:body => params[:snippet_body])
  if @snippet.save
    redirect "/#{@snippet.id}"
  else
    redirect '/'
  end
end

Pretty standard stuff, right? We retrieve the parameter passed to us from the new paste form, instantiate a new model and try to save it. If the validations pass, we redirect to the #show action equivalent. If not, we’re just going to dump you back to the #new form again. Since the only way the action will fail is if the body property is empty, we’re not going to bother with any sort of error message at this time. We’re not rendering anything here (merely redirecting), so no template is required.

The Show Action / Show Me The Snippets!

If our post is successful, we’re going to be taken to the #show action, which lives at /:id, where :id is the primary key of the corresponding database record. This action will also get accessed directly when you paste that URL to the chat room, and people click to view your code.

get '/:id' do
  @snippet = Snippet.get(params[:id])
  if @snippet
    erb :show
  else
    redirect '/'
  end
end

In the code listing above, we look up the particular snippet specified in params[:id] and set an instance variable by calling DataMapper’s get method (equivalent to Active Record’s find and one of many ways to do record lookups with DataMapper). If the @snippet is not found we’ll redirect back to the new snippet form. Otherwise we render an ERb template, which of course has access to that instance variable. Here’s the code you’ll want in /views/show.erb:

<div class="snippet">
  <div class="sbody" id="content"><%= @snippet.formatted_body %></div>
  <div class="sdate">Created on <%= @snippet.created_at.strftime("%B %d, %Y at %I:%M %p") %></div>
  <br/><a href="/">New Paste!</a>
</div>
The Layout

We’re pretty much done at this point. However, to dress up our output we’ll use an ERb layout template with some CSS to handle the syntax highlighting that Syntaxi provides for us. If it exists, Sinatra will render a special view template (just like Rails does) named layout.erb in the views/ subdirectory. This layout will be used to wrap the output of the other views rendered when the erb helper method is called.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
  <title><%= @title || 'Toopaste!' %></title>
  <style>
    html {
      background-color: #eee;
    }
    .snippet {
      margin: 5px;
    }
    .snippet textarea, .snippet .sbody {
      border: 5px dotted #eee;
      padding: 5px;
      width: 700px;
      color: #fff;
      background-color: #333;
    }
    .snippet textarea {
      padding: 20px;
    }
    .snippet input, .snippet .sdate {
      margin-top: 5px;
    }

    /* Syntax highlighting */
    #content .syntax_ruby .normal {}
    #content .syntax_ruby .comment { color: #CCC; font-style: italic; border: none; margin: none; }
    #content .syntax_ruby .keyword { color: #C60; font-weight: bold; }
    #content .syntax_ruby .method { color: #9FF; }
    #content .syntax_ruby .class { color: #074; }
    #content .syntax_ruby .module { color: #050; }
    #content .syntax_ruby .punct { color: #0D0; font-weight: bold; }
    #content .syntax_ruby .symbol { color: #099; }
    #content .syntax_ruby .string { color: #C03; }
    #content .syntax_ruby .char { color: #F07; }
    #content .syntax_ruby .ident { color: #0D0; }
    #content .syntax_ruby .constant { color: #07F; }
    #content .syntax_ruby .regex { color: #B66; }
    #content .syntax_ruby .number { color: #FF0; }
    #content .syntax_ruby .attribute { color: #7BB; }
    #content .syntax_ruby .global { color: #7FB; }
    #content .syntax_ruby .expr { color: #909; }
    #content .syntax_ruby .escape { color: #277; }
    #content .syntax {
      background-color: #333;
      padding: 2px;
      margin: 5px;
      margin-left: 1em;
      margin-bottom: 1em;
    }
    #content .syntax .line_number {
      text-align: right;
      font-family: monospace;
      padding-right: 1em;
      color: #999;
    }
  </style>
</head>
<body>
  <%= yield %>
</body>
</html>

Hallo Pastie!

You can fire up your new Sinatra and DataMapper-powered application by issuing the following command:

ruby toopaste.rb
== Sinatra has taken the stage on port 4567!

Sinatra sits on top of <a href="http://mongrel.rubyforge.org/">Mongrel</a> by default, making it super-easy to use (and thread-safe to boot!). If you open up a web browser and point it at <code>http://localhost:4567</code> you'll see the results. Copy and paste away. You now have a fully functional (albeit slightly retarded) pastie clone with  syntax highlighting for Ruby code snippets. And the core logic is all contained in a single file, with a few external ERb templates for cleanliness.

Wrapping Up

As we mentioned earlier, Sinatra is based on top of Rack which means you can deploy it in a number of ways. It’s incredibly useful for lots of quick tasks and is definitely my “micro framework” of choice at the moment. DataMapper is well, just awesome. What else can I say? Lots of Merb developers already know this, of course.

OK enough blathering for now. Longest blog post. Ever. If you like, you can play with the finished app or check out the sources on GitHub. Enjoy, and please comment if you have any problems or suggestions for improvements. This code was written and tested on OS X 10.5 and Debian Etch. Special thanks to Jonathan Stott for the early review and fact checking.

blog comments powered by Disqus