开发者

How to automatically parse an RSS feed for storage in MySQL database?

开发者 https://www.devze.com 2022-12-21 11:40 出处:网络
I need to somehow automatically update/parse a couple of RSS feeds and place them into a MySQL database almost as soon as the feed is updated, or as close as possible. However, I can\'t work out the b

I need to somehow automatically update/parse a couple of RSS feeds and place them into a MySQL database almost as soon as the feed is updated, or as close as possible. However, I can't work out the best way to do this automatically - I've found tutorials for doing it when a user runs a script - but in this case it all needs to be done in the background. Would a cron job be suitable?

Any ideas? Any advice is g开发者_JAVA百科reatly appreciated thanks.


You should check out Zend_Feed_Reader.
Zend_Feed_Reader provides HTTP Conditional GET Support.
If the Feeds is proper configured your script only has to download and parse the Feed if it has even changed.

You don't need the full Zend Framework. Zend_Feed_Reader has very few dependencies so it can be used standalone.


It can only ever be done in response to something — a cron job just means "In response to it being a certain time." You have to decide what event is best for your particular circumstances.

Assuming you don't control the source of the RSS feeds, doing it periodically via cron makes sense. To have it run "as soon as the feed is updated, or as close as possible" you would have to poll every second, which would make you highly unpopular. Check no more frequently than hourly (unless the feed includes information giving a different check period).


If you do control the source of the RSS Feed, have a look at the other Observer pattern. If not, check if the source feed supports PubSubHubbub:

A simple, open, server-to-server web-hook-based pubsub (publish/subscribe) protocol as an extension to Atom and RSS. Parties (servers) speaking the PubSubHubbub protocol can get near-instant notifications (via webhook callbacks) when a topic (feed URL) they're interested in is updated.

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号