The Technorati story:
How a new Web services product grew out of a research assignment

When Doc and I started doing research for this feature, I was still something of a blogging neophyte. While I was experienced with all the components of the LAMP platform (Linux, Apache, MySQL, PHP, Perl and Python), had an account with Advogato and had even set up my own blog using Moveable Type, I was mostly a blog software consumer. This project gave me a chance to hack.

After we set up our blog software server (kindly loaned to us by Penguin Computing), I installed Moveable Type and immedately looked for ways to improve it. First I wrote an email-to-blog and a jabber-to-blog set of interfaces, so I could post using my favorite email and IM clients. (I've always maintained that the web browser is a lousy web services client -- something we all intuitively know whenever fill out a long form on the web.) Then I began to think about the kinds of Web services that I'd like to see as a regular blogger.

The result, after three weekends of hacking, is Technorati.com, a new site that provides four services:

  1. Link Cosmos shows you what blogs are linking to your blog (or any blog, or any arbitrary URL). Every time another saves a post, it sends out an RSS notification that Technorati receives, puts in a database and lists on request in your current cosmos. This is a major advance over referer logs, which only shows followed links.
  2. Google Rank shows you the top 100 sites on Google for a given search term (including, possibly, yours). Technorati rechecks Google's rankings daily, so can see how the rankings change over time.
  3. Google Juice tells you where your blog ranks in the top 1000 for any search term.
  4. Watchlists keep and track historical information about a given site, allowing you to quickly see new links to your site or historical ranking information not shown in the standard views.

Watchlists are how Technorati answers the question How can you make money with Web services? For $5/year you get a daily email with your latest cosmos listings. For $10/year you get instant access to live watchlist information through an RSS feed.

As I write this, Technorati just is a few days old, and it's already been to the top of both Blogdex and DayPop. It's also made about $250.

Technorati is brand new, a work in progress, and invited into the world by a plethora of open source tools and open protocols, many of which are products of the blogging development community. I used a LAMP combination (see other sidebar) with Linux, Apache, MySQL, and PHP for the live scripting and Perl for the backend web robot and other backend tasks. XML-RPC pings power the activation of the web spider, so Technorati is always full of up-to-date information -- and so it exports RSS feeds for people who want to view output in a more structured format, such as an RSS browser. I set up payment and billing through PayPal's open APIs. These use HTTP POST methods that easily perform credit card processing and billing.

In programming we stand on the shoulders of giants. So I'd like run a bunch of people in Technorati's credits: Dave Winer, who wrote XML-RPC and was a driving force behind SOAP and RSS; Ben and Mena Trott, who wrote Movable Type; Rasmus Lerdorf and many others who developed PHP; Larry Wall and many others who developed Perl; the entire Apache team; Monty Widenius, David Axmark and the MySQL development team; Evan Willams and the folks behind the Blogger API; the Google team behind the Google API; and, of course, Linus Torvalds and the Linux development community.

And hey: If I can do it, so can you!