asemanfar - a blog about programming

Rails Plugin: TextMate Syntax Highlighting

April 30, 2008

Have you been envious of my super-pretty code blocks? Well envy no more! I finally released it as a plugin. It's pretty simple, it uses the Ultraviolet gem and provides easy-to-use helpers and then you get the awesomeness you see below.

   1  def create
   2    @post = Post.new(params[:post])
   3    @post.user = current_user
   4    
   5    set_hidden_based_on_commit
   6    
   7    respond_to do |format|
   8      if @post.save
   9        flash[:notice] = "Your post has been successfully created!"
  10        format.html { redirect_to post_path(@post)}
  11      else
  12        format.html { render :action => "new"}
  13      end
  14    end
  15  end

There is a pretty clear README included with the plugin, please do read it if you're going to use it.

A few quick notes:

  1. Ultraviolet and/or TextPow is slow so fragment caching for syntax highlighted blocks is highly recommended.
  2. You need to copy over the CSS files for the themes using the syntax_css generator (described in the README).
  3. Please do contribute if you have any ideas or improvements.

Anyway, here it is: tm_syntax_hightlighting hosted on GitHub.

Comments

posted by Fredrik on 04/30/08 07:40 PM PDT

I want an RSS feed for this blog! In fact I demand it! :P

Can't keep up with blogs unless i have a feed and i really like this one so far.

posted by Arya Asemanfar on 04/30/08 07:56 PM PDT

Hey Fredrik,

Thanks for reading my blog. I actually do have an RSS feed. It's set up as an auto-discovery link so you browser should pick it up. I'll add an RSS icon/link somewhere soon. But here is the link for now:

http://feeds.feedburner.com/UnboundImagination

posted by Peter Haza on 05/01/08 02:29 AM PDT

You might want to run your rss excerpt thru a markdown parser before publishing it, as it currently shows the markdown code.

posted by Fredrik on 05/01/08 04:14 PM PDT

Oh blushes Thank you!

posted by Evan on 05/02/08 07:26 PM PDT

Awesome, I look forward to incorporating this into my site as well, keep up the good posts!

posted by Alistair Holt on 05/03/08 06:17 AM PDT

How slow are we talking?

posted by Arya Asemanfar on 05/03/08 10:17 AM PDT

Alistair,

I did a quick benchmark for you:

   1  require 'rubygems'
   2  require 'benchmark'
   3  require 'uv'
   4  
   5  code = %Q{
   6    def create
   7      @post = Post.new(params[:post])
   8      
   9      respond_to do |format|
  10        if @post.save
  11          format.html { redirect_to @post }
  12        else
  13          format.html { render :action => "new" }
  14        end
  15      end
  16    end   
  17  }
  18  
  19  Uv.init_syntaxes
  20  puts (Benchmark.realtime {
  21    10.times do 
  22      Uv.parse(code, "xhtml", "ruby", true, "sunburst")
  23    end
  24  } / 10)

Came out to 0.06 seconds average but probably even larger for longer code segments. It's not outrageously slow, but it makes a pretty big chunk of the render time if you don't cache it.

posted by universal on 05/05/08 03:10 AM PDT

seems to be an improved(?) version of radiograph? ;)

posted by Ken on 05/05/08 08:52 AM PDT

Hi Fredrik,

Thanks for this! I'm sorry to be so ignorant, but I'm new to plugins - what is the command for installing this particular plugin?

Thanks, Ken

posted by Arya Asemanfar on 05/05/08 02:56 PM PDT

Hi universal,

Yes, radiograph also does syntax highlighting using Ultraviolet. I found it when I was making my blog and had some problems so I decided to write my own that was more customizable.

posted by Arya Asemanfar on 05/05/08 03:06 PM PDT

Hi Ken,

I'm not sure if you meant to be addressing me. Fredrik is just a reader of the blog, not the author. Anyway, to install this plugin you can by downloading the source through git or if you don't use git, you can download the tarball version and extract it into your vendor/plugins folder. Then you can read the README to see how to use it.

posted by Ken on 05/05/08 03:23 PM PDT

Oops - sorry for the confusion on your name. Thanks for the help on the install!!

K

posted by Zach on 05/12/08 06:35 PM PDT

Okay I got it all installed and what not, but how do you implement it on the page? Say I'm using markdown? Or is there something special I gots ta do?

posted by Arya Asemanfar on 05/12/08 08:11 PM PDT

Hi Zach,

To syntax highlight a block of code, you pass a string containing the code to the code view helper and it returns the HTML for syntax highlighted code (assuming you have the respective CSS included). The README's examples cover the basics.

Now if you're using it in your blogging engine like I am and you want to be able to embed code into posts like this:

   1  puts "This is Ruby Code Syntax Highlighted inside of a markdown formatted comment"

You can whip up your own solution for splitting it and parsing them separately or use the one I wrote:

   1  def syntax_highlight_and_markdown(text, options = {})
   2    text_pieces = text.split(/(<c0de>|<c0de lang="[A-Za-z0-9_-]+">|<c0de lang='[A-Za-z0-9_-]+'>|<\/c0de>)/)
   3    in_pre = false
   4    language = nil
   5    text_pieces.collect do |piece|
   6      if piece =~ /^<c0de( lang=(["'])?(.*)\2)?>$/
   7        language = $3
   8        in_pre = true
   9        nil
  10      elsif piece == "</c0de>"
  11        in_pre = false
  12        language = nil
  13        nil
  14      elsif in_pre
  15        code(piece.strip, :lang => language)
  16      else
  17        markdown(piece, options)
  18      end
  19    end
  20  end
  21  
  22  def markdown(text, options = {})
  23    if options[:strip]
  24      BlueCloth.new(strip_tags(text.strip)).to_html
  25    else
  26      BlueCloth.new(text.strip).to_html
  27    end
  28  end

Update: I had to replace "code" with "c0de" in order to for it to parse itself correctly. (Clearly it's flawed, if any readers want to make any suggestions, that would be greatly appreciated.)

It's not exactly the prettiest solution but it gets the job done pretty well. The one issue I have with it right now is that I can't use Markdown's reference links if the reference and the definition is separated by a code block. I intend on fixing that when I have some free time, if you want, I can e-mail you to notify you when I've made those changes.

Anyway, feel free to ask any questions about the above code if it's unclear, I'll be glad to help.

posted by Zach on 05/13/08 11:02 AM PDT

Hey Arya,

Thanks for the fast response and taking the time to whip up some code. I included your code in my application_helper.rb file and I call it like this:

   1  <%=syntax_highlight_and_markdown @article.body %>

The problem I am getting is that the pre tag is not getting the a class appended to it, whether it be sunburst or twilight. Is there something I am missing? I have the CSS properly included, and if I go in there and hardcode the pre tag with the appropriate class the highlighting works correctly.

Thanks for a great plugin and I'd love to receive an email with updates. Also are you on workingwithrails.com?

posted by Zach on 05/13/08 11:08 AM PDT

Sorry to double comment like this. I believe the problem is in my initializer for the defaults, can you explain how you did that?

Thanks again.

posted by Arya Asemanfar on 05/13/08 11:08 AM PDT

Hey Zach,

No problem, I'm glad to help.

There are two ways to specify the theme, you can either pass it when you call the helper, like so:

   1  code(piece.strip, :lang => language, :theme => "sunburst")

Or you can specify it globally so it becomes the default, by doing this is in an initializer:

   1  # config/initializers/tm_syntax_highlighting_config.rb
   2  TmSyntaxHighlighting.defaults = {:theme => "sunburst", :line_numbers => true}

Remember that you'll need to restart your mongrel if you add/change the initializer since those only get executed on boot.

Try that and let me know how it goes.

Also, I am on WorkingWithRails.com. I have the badge on my blog index, but here is a link for your convenience.

http://www.workingwithrails.com/recommendation/new/person/10754-arya-asemanfar

posted by Zach on 05/13/08 11:38 AM PDT

Hey again,

Okay it was a number of factors that were causing the problems. You were right that I needed to restart my server for the initializer, but I went ahead and put it in the helper so that I could use it with a live preview. Which works by the way. I also had to change 'c0de' to 'code', which I think you were getting at in your update. The only thing I don't get is how you made your code areas scrollable, but I suspect that is a CSS problem. Other than that, everything works like a charm. Sweet ass plugin man. Very good job, thanks for being so helpful as well.

posted by Arya Asemanfar on 05/13/08 11:44 AM PDT

Hi Zach,

Yea, the "c0de" (with a zero) was my way of getting around my code parsing itself as a end-of-code block.

As for the scrolling code blocks, I added this CSS to all pre tags:

   1  pre {
   2    margin: 20px 30px;
   3    padding: 15px 10px;
   4    line-height: 14px;
   5    overflow: auto;
   6  }

The overflow is the part that makes it have scroll bars instead of overflowing.

Also, if possible, could you link to your site? I'd like to see it in use on someone else's site :)

posted by Zach on 05/13/08 11:48 AM PDT

Overflow! That was it. And yes, I'll link to it, it's actually in active development, but as soon as it's up (Friday at the latest), I'll link to it.

posted by Jim Gagne on 05/25/08 06:31 PM PDT

The installation of the Ultraviolet gem requires first installing the Onigurama library and gem. I've done all my Unix installs with MacPorts, and it turns out there's a port for Onigurama. Pick the latest: onigurama5.

For newbies like me, first do a "sudo port selfupdate" and then "sudo port install onigurama5"

Then --oops -- no success installing the plugin. Does this plugin require Rails 2.0.2?

I tried "script/plugin install" with each of the following arguments: "tm_syntax_lighlighting" "git://github.com/arya/tm_syntax_highlighting.git" "tm_syntax_highlighting --source http://github.com/arya/tm_syntax_highlighting/tree/master"

Couldn't find it. The last attempt yielded the following error:

HTTP Response 404 fetching http://github.com/arya/tm\_syntax\_highlighting/tree/yaml

Then I downloaded and unpacked the .tar.gz file into the /vendors/plugins directory. But then "script/generate syntax_css list" generated a NoMethodError; apparently it couldn't locate a file. Should I leave the code at the end of the directory name ("arya-tm_syntax_highlighting-e38b5c67f3e05f4cfd5b3128fbf030e1faea1124")? Deleting it didn't help.

posted by Arya Asemanfar on 05/26/08 01:09 AM PDT

Hi Jim,

I'm sorry to hear you're having trouble installing the plugin. I don't think there is anything specific to Rails 2.0.2 in there, it should work with 1.2 as well.

As for running "script/plugin install" on a Git repository, that unfortunately does not work (at least not in the current version of Rails).

You were right to download the tarball and extract it. It should be in a folder named "tm_syntax_highlighting" inside your "vendor/plugins" folder with no suffix or anything. Then you need to make sure you have the following 3 gems installed: oniguruma, textpow, ultraviolet. Note that the oniguruma gem is different (but depends on) the system package you installed through MacPorts.

Lastly, I'm actually not sure if the MacPorts version is compatible with the oniguruma ruby gem. Maybe that has to do with the error you are getting. After you ensure you have the gems correctly installed and if you still get the error, can you paste the NoMethodError you get so I can help debug some more?

posted by Greg on 06/06/08 01:09 AM PDT

I know it's a little off-topic, but I was wondering if you could point me at any resources for actually generating these kinds of syntax-highlighted code samples from TextMate itself. I've looked around everywhere, but Google keeps sending me here! Thanks...

posted by Arya Asemanfar on 06/06/08 05:08 PM PDT

Hi Greg,

If you open up the Bundles menu and go down to TextMate, there should be 3 relevant menu options:

  • Create HTML From Document
  • Create HTML From Document With Line Numbers
  • Create CSS From Current Theme

If TextMate does not appear as a bundle, try going to Bundles > Bundle Editor > Show Bundle Editor and making sure that it's installed and not disabled in the filter list.

If you don't have the TextMate Bundle installed, you can check it out (into Library/Application Support/TextMate/Bundles) using subversion from TextMate's website: http://macromates.com/svn/Bundles/trunk/Bundles/TextMate.tmbundle/

I hope that helps. Let me know if you have any other questions.

posted by combatwombat on 10/08/08 03:44 PM PDT

This is nifty, thank you :)

I too had the problem of using it with Markdown. I scan for code blocks using the original Markdown.pl regex, generate the HTML with your plugin and then pipe the rest through Markdown (RDiscount).
In this way i can just write normal Markdown code blocks, with a optional parameter to choose the language:

Also viewable at http://pastie.org/288223

   1  l: ruby
   2      1.upto(100) do |i|
   3          puts 'ruby code'
   4      end
   5  
   6  normal **markdown** bla foo
   7  
   8      ordinary plain text code block
   9      is the default language
  10  
  11  or
  12      lang: php
  13      function bar($var = 'foo') {
  14         code();
  15      }
   1  def self.text_to_html post_text  
   2    
   3    # 2. Syntax Highlighting
   4  
   5    post_text = self.code_chique(post_text)
   6  
   7    # 3. Markdown
   8    post_html = Markdown.new(post_text).to_html
   9    
  10    post_html
  11  end
  12  
  13  
  14  def self.code_chique post_text
  15    post_text = post_text.gsub(/\r\n/, "\n")
  16    
  17    # From Markdown.pl © John Gruber
  18    regex = /(?:\n\n|\A)((?:(?:[ ]{4} | \t).*\n+)+)((?=^[ ]{0,4}\S)|\Z)/ix
  19  
  20    matches = post_text.scan(regex)  
  21    
  22    for match in matches
  23      c = match[0].gsub(/^    /, '')  # remove 4 leading spaces
  24  
  25      lang = "plain_text"
  26  
  27      lines = c.split("\n")
  28      lang_def = lines[0].split(':')
  29      if lang_def[0].downcase == 'l' || lang_def[0].downcase == 'lang'
  30        if lang_def[1].strip.length > 0
  31          lang = lang_def[1].strip.downcase
  32          lines = lines[1..-1]
  33        end
  34      end    
  35      
  36      code_html = help.code(lines.join("\n"), :lang => lang)
  37      
  38      post_text = post_text.gsub(match[0], "\n" + code_html + "\n\n")
  39  
  40    end
  41  
  42    return post_text
  43  end

Probably not nearly optimal and a bit hackish, but works for me.

I use it in my Model as a :before_save to generate the html and save it in an extra db field alongside the markdown text.

posted by lee on 12/23/08 05:26 AM PST

very appreciate for this plugin.And I just wanna know that how did I develop a web app like this page? I mean the content maybe contains some different codes and text.Is there a demo to show this? thanks again!

posted by Arya Asemanfar on 12/25/08 12:27 AM PST

Hi Lee,

There a few code examples above that show usage of the plugin. You can install the plugin on any Rails application and install the required gems and that'll provide the helper to take code segments and turn them into syntax-highlighted code.


Leave a Comment