Ruby on Rails safari reader like text extraction and boilerplating

I have a digg like web service which briefly explained has a page parser and when people submit stories, the parser returns title and summary based on hpricot and some other small extraction principles I wrote.

I want to take it to the next level and try a content discovery and extraction from the pages people submit to the parser. I want to extract the content of an article for example.

Of course I don't really want to start from scra开发者_StackOverflow中文版tch and write my own boilerplate and extraction algorithms.

Is there a gem or something I can hook to my rails app that can be relevant to some degree to what I want to achieve?

I'm really stuck on this and any help is appreciated.

Best regards.

---

Hpricot is deprecated in ruby in favor of nokogiri. You can use any xpath expression with nokogiri for example

require 'rubygems'
require 'nokogiri'
require 'open-uri'

doc = Nokogiri::HTML(open('http://www.google.com/').read)
puts doc.xpath('//title')[0].text

---

Give the Readability gem a try. It's works very well:

https://github.com/cantino/ruby-readability

require 'rubygems'
require 'readability'
require 'open-uri'

source = open('http://lab.arc90.com/experiments/readability/').read
puts Readability::Document.new(source).content