Friday 13 February 2009

Twitter and collaborative filtering

When Twitter comes up in conversation with my colleagues, the most common response is "I don't get it."  I've said it myself more than once.

This probably has more to do with the age profile of my colleagues than anything else.  They (OK, we) are sufficiently old that we remember the days before email and mobile phones, let alone Twitter, Facebook and MySpace.

Twitter only starts to get interesting when you are following a bunch of people whose tweets are themselves interesting.  And if your friends aren't using Twitter, that reduces its utility.  This is the classic network effect of course.  It's not clear to me how this problem can be overcome.

It does bring my back to a fundamental question.  What is Twitter for?  I'm struck by the high proportion of tweets that contain URLs.  Indeed, this phenomenon is so common that some of the Twitter clients provide tools to automatically convert URLs to a short form that fits more easily into the 140 character limit.

What does this mean?  That Twitter is actually being used to pass around references to things rather than the things themselves.  People are using it as a means of drawing attention to something that they have read (or perhaps would like other people to think they have read) in order to make a point, share some knowledge, start an argument or just make someone smile.

I remember reading a paper in back in the early 1990s that described collaborative filtering [1] and being fascinated by the idea.  I tried and failed to persuade some of my students to build a Usenet news reader that exploited some of these techniques, but a few years later someone did it anyway [2].

Twitter, at least in part, is providing a means for collaborative filtering of the Web.  You follow people you know, or whose views you find interesting and they post links to things you may want to read.  We are filtering the web for each other.

If this is what Twitter is actually being used for, then how "fit for purpose" is it?  Not very, I think.  


First, and it is a minor point, I have to jump through some hoops to post links to Twitter.  Doubtless people will build clients that do this seamlessly, indeed, perhaps they already have.

Second, and I think more serious, there's no easy way to find people that are providing feeds of stuff that I would enjoy.  I'd like a system that would recommend people to follow based on my reading habits.  This sounds a little harder to fix than the first point, but it would be much closer to the spirit of collaborative filtering that was described in the two papers I referenced above.  

Oh, and in case you're wondering, I am an infrequent tweeter and you can follow me here, but I can't guarantee that I will say anything you find interesting.