What makes a search engine biased?

I have been thinking about the sensitivities surrounding China's growing internet culture with regard to censorship and freedom of speech. The Universities of Leiden and Heidelburg archive websites with a view to supporting Chinese studies - see Digital Archive for CHinese Studies (DACHS). A very strict control is kept on who gets to use that collection, no doubt for good reasons. Why? Political control and freedom of speech concerns in China. Might be worth taking a look at Nicolai Volland's PhD thesis The Control of the Media in the People's Republic of China.

This territory of censorship is not new or exclusive to China though - other countries around the world are looking to tighten their laws to 'control' the content viewable by its citizens, including France and Australia. Admittedly the reasons for control in China may well be different to that of other countries; but these governmental interventions beg questions. How much government control over publicly available content should be asserted? How much should people be allowed to determine that at their end, i.e. let the user drive?


What I find more intriguing, possibly because it is less obvious, is the underlying programming in the development of Baidu the search engine and its ability to enable effective information retrieval with Chinese characters and search strings. The fact Baidu indexes websites that search engines like Google don't have access to is an obvious advantage. An article I read in New Scientist (Beyond the great firewall, November 2007) refers to the fact that many Chinese people use Baidu also because it processes Chinese characters more effectively and indexes websites possibly not indexed by Google.

Aside the content bias (or constraints), I began to think a bit about the fact that a great deal of computer coding is done in English, a la, the lingua franca of computer programming is English. It made me wonder about the coding in Google, and how effectively Google handles diverse language content (in its alogrithms) and how much those linguistic calculations are based on an understanding (mostly) of English, or many languages? Is the lingua franca of coding English too with Baidu? Are their linguistic calculations for search different because of the nature of Chinese characters? I don't know, but it pays to look below the surface (i.e. content) too, to see how the internet is or isn't working for information retrieval irrespective of the political controls being asserted over the top of this.

How well have the world's Chinese speakers been served by Google? If they prefer Baidu, what is that about? Is it just better indexing and search results is there something else? How do Baidu and Google compare and what basis are they best compared on? Which is biased in what way and by what?

What changes a mind?

The Annual Question from the Edge website:

When thinking changes your mind, that's philosophy.
When God changes your mind, that's faith.
When facts change your mind, that's science.

At first I liked these three statements as a simple means of distinguishing what seems to affect people's thinking: philosophical thinking, religious thinking and scientific thinking. But.. the more and more I thought about these statements the less satisfactory they became and the more superficial they appeared. Why?

In each of these statements there is a catalyst that effects the way people think.

- Philosophers - reasoning - process of rational argument or debate of ideas
- God - the higher power - ideas codified in ritual, prayer, and text
- Facts - evidence - repeatable, consistently observed and independently confirmed ideas

Without giving greater or lesser weight to any of these three sentences, what they are all talking is about belief, what people believe.

What seemed to be missing in this is any discussion of 'truth' (such a tricky concept that). Truth is examined by:

- who is proffering a truth
- how a truth is arrived at
- how that truth is evaluated and perceived
- who is perceiving this as a truth

Reasoning comes into this equation and so does context. In each of these statements the new belief is arrived at through one's own thought processes and through drawing upon other's thoughts, philosophical, religious or scientific.

Anyway, on reading a popular science book on psychiatric disorders* recently I happened upon various definitions the author offered for delusion and the identification of deluded thought. At one point the author talks about delusion being determined in the psychiatry community by the 'truth-value' and how many people can be convinced of this, i.e. content. This definition of delusion has a weakness, belief of truth may vary depending on the context. So, the next definition was how the idea (deluded or not) is arrived, that is, logical (or probable?) nature of the idea, i.e. form. Who gets to evaluate this, their interpretive skills, assumptions or prejudices perhaps is also a weakness unless there is some ability to gain consensus, with the aim of avoiding 'group think'.

What I found most interesting though was how those that maintain deluded thought often disregard or dispute counter evidence or actively avoid being confronted that evidence. I thought perhaps what could have been added to that was actively undermine the veracity of the counter evidence also. But I think that is a whole different discussion about security of position and defensiveness. So perhaps there are three variables in arriving at a truth-value potentially that all contribute in some way to ascertaining its veracity:

- how the idea is arrived at - reasoning
- how widely embraced and endorsed the idea is - context
- how this idea is negatively tested - fallibility
- how this idea stands up in the face of evidence to the contrary - contestability

*From the Edge of the Couch: Bizarre Psychiatric Cases and What They Teach Us About Ourselves, Bantam Books, Dr Raj Persaud, 2004 - see review by psychiatrist Dr Vaughan Bell.

I'm sure there are lots of far better qualified thinkers spending hours thinking about this and dispute this blog. It is a rough sketch of thoughts - but I do think that the expression of deluded ideas and the behaviours of the deluded do offer insight on thoughts about truth and belief and maybe why such clashes are occurring between religion adherents and scientists over evolution.

Maoist managers

Four key characteristics of a (Maoist) manager from an article in the Economist (19th December 2007, print edition):

*A powerful, mendacious slogan
*Ruthless media manipulation
*Sacrifice of friends and colleagues
*Activity substituting for achievement

I particularly 'enjoyed' the last point. I'm no expert on Mao but the idea that this type of leadership leads to cultural fracture and famine certainly didn't endear me to it.