Why our knowledge society needs a “really good stuff” algorithm

I’m not an expert in search but I do get the feeling that algorithms based on reputation, the number of links pointing to particular content and other mathematical wizardry are not necessarily drawing our attention to the best stuff out there.

Reputation, whilst a clear indicator of knowledge in a particular field, depends precisely on one choosing to specialise in just one area of knowledge.  Is this really what our society needs right now?  It seems to me that specialisms are crowding out our ability to communicate with each other, creating – in a way – Chinese walls that don’t allow us to understand the implications of another area for our own.

In this sense, we may be missing out on many otherwise fruitful connections which could help our complex civilisations out of the holes they are gradually finding themselves sinking in to.

What do I propose then?  A series of search processes which allow us to discover and define the quality and value of a piece of content in absolute, potentially productive and creative terms – not in terms of how many people have linked to it or who the author is or which website has published it.

Yes.  To mention just one kind of latterday endeavour, this would inevitably turn the world of online advertising upside down – but aren’t such distortions of our access to truly imaginative and original work already negatively affecting the futures of our communities?  Don’t we need to turn upside down how “really good stuff” is being ignored?  Isn’t it time we focussed on properly added value – rather than that cleverly massaged SEOed information we generally get these days?

A disclaimer before we finish: I regularly blog over at 21st Century Fix – and I blog on a broad range of subjects.  In a way, it’s a kind of Renaissance Man-style blog – not because of its quality (that, after all, would be for you to judge not me) but simply because I’m prepared to take on any range of subjects.  Yes.  It’s a brainstorming blog, where I follow trains of thoughts to their ultimate consequences.  As such, it’s very difficult to build up a reputation in anything.  Hits-wise, it’s a modest blog.  Ambition-wise, it’s pretty significant.

From the kind of algorithms I’m suggesting we develop, I would be the first to benefit.

Even so, and in the light of such self-interest, don’t you think I might have a point?


  1. saho1989Sam

    Reputation is the bootstrap used by search algorithms as a means to start to understand the world. The problem is, while it ‘works’, there is an arms race because it is easily circumvented. Wikipedia is a good example of where the reputation model can start to fall over – it is incredibly dominating in search results, very often only beaten by corporate websites. Whether Wikipedia’s prime position is naturally or artificially maintained is a different topic, but it certainly doesn’t deserve it just because it’s community-moderated and collaboratively written.

    An interesting quote from Jaron Lanier on this issue:
    “Like Wikipedia, the Bible’s authorship was shared, largely anonymous, and cumulative, and the obscurity of the individual authors serves to create an oracle-like ambiance for the document, as ‘the literal word of God’.”

    So yes, I agree that we need to look elsewhere for information. To search for true authority on a subject and find true, insightful comment is difficult but incredibly rewarding.

    • Mil

      Thanks for your comment. Now that *is* something insightful! :-)

      In relation to your last sentence, do you think this search task still needs to be done by people – or can mathematics get anywhere near? If the latter, is there any real incentive for it to be implemented, given that there’s far more money to be made in billing advertisers for the right to massage content than providing a free service which finds stuff that really deserves to be up there?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s