Smarter than the best brains: IBM builds a new kind of genius

Final Jeopardy: Man vs. Machine and the Quest to Know Everything by Stephen Baker

It’s more than five years since a computer called Watson beat two quiz champs on Jeopardy, the American TV game show. The achievement of that day, witnessed by millions, seems, if anything, more interesting today as developments in artificial intelligence have moved centre-stage.

The spectre of middle-class jobs lost to AI has become part of conventional wisdom. As people spend more and more time exchanging data with distant computer servers, knowing little about what happens between their keystrokes and the results they study onscreen, the systems which control information, whether classified as AI or not, are ever more sophisticated and central to our lives.

The Watson experiment on Jeopardy was both a triumph of scientific and technological research and a kind of homage to the great tradition of computers in the States. Watson was built by IBM’s research team and named after the company founder. Thomas J. Watson and his son, between them, turned the company from a cash register business in Chicago to the epitome of corporate modernity, selling mainframe computers to customers who first had to learn what a computer was.

On a shorter timescale, Watson was the follow-up to another IBM triumph, when its computer Big Blue beat world chess champ Gary Kasparov in 1997. That was an extraordinary feat, but at least chess is a game with a limited number of possible moves – albeit a very large number.

But how could they make a machine that could deal with the natural language used in Jeopardy questions? Especially since the tradition of Jeopardy was to ask witty, punning questions, a bit like crossword clues? To make it more difficult, as a result of the game show scandals of the 1950s, where popular contestants were given the answers to keep them on shows and improve ratings, Jeopardy had been designed to prevent such a possibility by giving the contestant ‘the answer’, and requiring them to formulate the right question.

So that was the challenge IBM’s research team took on, less than four years before the show in which their computer won. It was partly a question of computer speed: even if Watson knew the answer, it had to be able to produce it before the human champions that were its opponents. These winners dealt in split seconds, hitting the buzzer often, apparently, before their conscious minds had an answer. As one put it “you find your thumb pressing the buzzer while the brain races to catch up.”

An early version of the computer was so slow that the programmers would ask it a question and then go to lunch, hoping it might have produced something (usually the wrong answer) by the time they returned.

Stephen Baker, an experienced business and technology journalist, was given privileged access to IBM’s team as they tackled their audacious challenge. The result is a technology thriller, with no shortage of intriguing characters, incidents and, well, jeopardy. The story brings together the East coast world of IBM, and the West coast world of network television – another venerable US institution, harking back to the innocent days when home entertainment meant sitting as a family, choosing between the three networks and a couple of local stations.

Network television, as much as IBM, was on a difficult journey to adapt to the modern world – a world in which TV was one of many choices of screen entertainment beckoning from a variety of devices. To lose Jeopardy’s academic fustiness, its producer Harry Friedman had broadened its agenda. Now, as well as the traditional, fact-based questions, there were many that required a knowledge of pop culture or just ordinary life. When weaved into tricky ‘answers’ by the show’s writers, they made Watson’s life harder. How could a computer possibly get this right?

Answer (question): “Here are the rules: if the soda container stops rotating and faces you, it’s time to pucker up.”
Question (answer): “What is Spin the Bottle?”

Baker’s account gives enough detail to appreciate at least the principles with which the IBM team approached their challenge. For instance, they broke it down into sub-tasks: understanding the question, assembling a massive library of information, creating a list of candidate answers and assigning a level of confidence to each. The latter because a Jeopardy contestant is also required to gamble money on its chance of getting an answer right, and must even take a view on how its opponent will bet.

The story raises the question of how intelligent machines should be presented to human beings. What sort of ‘character’ should Watson be given? After thinking about tones of voice, visual representation and physical form, the team decided to create a screen view of Watson’s brain: activity in the computer would produce a display that showed Watson ‘thinking’. But there would be no attempt to turn the computer into humanoid form, as that might encourage fears of computers taking people’s jobs. It would have a calm male voice and wouldn’t attempt to mimic emotion – triumph, frustration or disappointment. That might produce an unintentionally comic effect. Instead, Watson would remain “relentlessly upbeat”, whatever was going on in the game.

As well as giving IBM some good publicity – and risking the opposite if it had failed – the Watson project had serious business potential. Not only could a Watson-related machine master huge libraries of information, it could also analyse all the online information being produced every second. As Baker puts it:

“A new generation of computers can understand ordinary English, hunt down answers in vast archives of documents, analyse them, and come up with hypotheses. This has the potential to turn entire industries on their heads.”

Medicine might be one of the first fields to benefit, but it won’t just be the limitations of technology that determines how it goes; it will also be human foibles, especially pride. As one doctor put it: “Doctors like the idea of having information available. Where things get more psychologically fraught is when a damned machine tells them what to do.”

The impact of AI is examined in Martin Ford's Rise of the Robots, which I wrote about here. 

View all my reviews

Alibaba: Arab myth for Chinese translation of American dream

Alibaba's World: How a Remarkable Chinese Company is Changing the Face of Global Business by Porter Erisman


The slogans of most internet tycoons suggest the clean lines and aggressive culture of Silicon Valley: “move fast and break things”, says Zuckerberg; “organise the world’s information” say Page and Brin. But “run as fast as a rabbit but be as patient as a turtle” is different. It’s the advice of an entrepreneur who is indisputably in the same league as the founders of Facebook and Google, though not as well known as them outside of China.

Jack Ma, an English teacher, failed his college entrance exams twice because of his poor maths, but was good at English thanks to having chatted up British and American tourists visiting his native Hangzhou when he was a boy.

He wasn’t a Bill Gates, fascinated by computers from childhood. When he first tried one, he was afraid to touch it because it was “such an expensive thing”. His friends encouraged him and he searched for “beer”. He noticed the search engine results didn’t include Chinese beer. That set him off on a journey that led to the building of Alibaba, a company that had the biggest IPO in history two years ago, being valued at $220 billion – worth more than Amazon and eBay combined.

For some of this extraordinary journey, from 2000 to 2008, a young American, Porter Erisman, was within earshot of Ma, taking notes and shooting video – 200 hours of it, which he later turned into a documentary called Crocodile in the Yangtze. In book form, Erisman’s account is satisfyingly intimate and excuseably affectionate. It doesn’t gloss over the traumas of building Alibaba, but more often than not, turns them into character studies of the formidably determined entrepreneur.

Ma’s epiphany about Chinese opportunities on the internet turned into a startup called China Pages. Free enterprise was hardly a key part of local culture, and Ma soon got sucked into a kind of official version of his idea within a trade ministry. But, as he put it afterwards, while the ministry hoped to control small businesses, he wanted to empower them.

So he started again, this time outside government, with Alibaba – the name suggesting the internet could be an “open sesame” to new business opportunities. Working in China as an entrepreneur, relations with government can never be ignored. Ma’s goal, he says, is staying “in love with the government but not marrying it”.

Before joining Alibaba in 2000, Erisman was working for a US ad agency in Beijing. He was recruited by Ma, on the promise of stock options that Ma said would be worth a million dollars when the company went public three months hence.

That didn’t happen, and the path to the eventual IPO in 2014 was strewn with booby traps and elephant holes, many of which Ma failed to negotiate. There was the small matter of Alibaba’s failure to earn any money, resulting in painful layoffs. There was the fight to the death with eBay in China (ultimately successful). And there was the strange deal with Yahoo! that saw Alibaba turned into a Google lookalike search engine, until it quickly changed back to its original Chinese look, having lost a lot of its audience in the process.

Both Ma and Erisman shrug off the mistakes: “Jack used to joke that if he ever wrote a book about his experience, he would call it Alibaba and the 1,001 Mistakes. From watching Jack in action, I realized that two great traits every entrepreneur should possess are resilience and amnesia.”

Part of the interest of Erisman’s story is in the puzzling out by Ma and his colleagues of which aspects of Silicon Valley business models are internet universals and which are simply expressions of American culture. So, for instance, after that disastrous switch to a Google-style search page, new Alibaba sites, such as the retailer Taoboa, did not to imitate American minimalism. As Erisman puts it:

“Compared to the home pages of Western websites, Taobao’s looks busy, with flashing icons and animated cartoon characters promoting special deals. If clicking through eBay is like a walk down Main Street, USA, clicking through Taobao is like a walk down Shanghai’s busy Nanjing Lu, where sights and sounds bombard the shopper. To Western eyes Taobao’s home page might seem too cute or flashy, even distracting, but it is what Chinese users prefer and expect.”

It is Ma’s ability to learn from the Western internet without compromising his instinct for what will work in the Chinese market that’s key to his success. Today Alibaba is a complex web of interlocking businesses including Tmall, through which overseas brands sell in China, AliExpress, through which Chinese manufacturers sell overseas, AliPay, an online payment and financial services business and even Alibaba Pictures, a movie production company.

Today the Alibaba group has annual revenues of almost $16 billion, with net income of $11 billion. There’s still a way to go before those numbers rival Silicon Valley’s biggest, but the profitability is impressive and Ma is still in a fast-growing market, one that Facebook would dearly love to enter. Erisman says that more than half the packages shipped in China are from deals that originated on Alibaba’s websites. Amazon isn’t close to that in its best markets.

One quality that Ma shares with Western entrepreneurs is persistence. Erisman tells us that one of Ma’s favourite sayings is: “Today is tough, tomorrow is tougher, and the day after tomorrow is beautiful. But most companies die tomorrow evening and can’t see the sunshine on the day after tomorrow.” For a man who has come so far so fast, the day after tomorrow is dawning.

View all my reviews

Tech change and society: this time it’s different

Rise of the Robots: Technology and the Threat of a Jobless Future by Martin Ford

People who are interested in the future of technology tend to believe that its impact on jobs and the economy is, and will be, largely benign.

OK, people in ‘disrupted’ industries may suffer: those who work in factories or newspapers or as travel agents. But there’ll be exciting new jobs like Search Engine Optimisers instead. As long as technology oils the wheels of industry, economies will grow and everything will turn out fine.

It’s a convenient view because it means you can sit back and relax, whatever effect technology appears to be having. Amazon may boom while your local bookshop closes. But there are new jobs in Amazon’s ‘Fulfilment Centres’. Isn’t it just snobbery to think they’re not as good as jobs in bookshops?

The theory is getting harder to believe in the light of evidence marshalled by Martin Ford in The Rise of the Robots (2015). Ford has worked in the tech business for decades so he’s not some kind of Luddite – although his conclusions here could easily get him mistaken for that.

Ford forces together economics and technology, acknowledging the conventional wisdom about the replacement of old jobs with new, saying it was once true but ‘this time it’s different’.

Why? Well, it was only true, he says, during a particular era after the Second World War. Technology boosted productivity but wasn’t powerful enough to replace many blue collar jobs. During what Ford calls this “Goldilocks” period, the fruits of increased productivity were shared between business owners, workers and the growing middle class in developed countries.

Everything went well until 1973. But then, somehow, the link between increasing productivity and increasing compensation for workers was broken. Productivity per worker went on rising after 1973 but workers stopped getting better paid. Ford offers a convincing graph to prove it, with the productivity line an unbroken upward trajectory and the compensation line taking an unmistakeable new downward turn since 1973. A worker who earned $767 a week in 1973 would only earn $664 forty years later (with the 1973 figure adjusted to 2013 dollars).

So who is benefitting from those continuing gains in productivity? Predictably enough, it’s the filthy rich. Not only are they getting richer, but as time goes on, the inbalance is increasing: Ford quotes an American study that concluded that “an astonishing 95 percent of total income gains during the years 2009 to 2012 were hoovered up by the wealthiest 1 percent.”

All this could be just circumstantial evidence for blaming technology. Ford’s linking of the two depends on a couple of separate points: first, the richest technology companies today create huge wealth without needing substantial workforces. In real terms, General Motors at its height, in 1979, made 20 per cent less than Google did in 2012. But while Google employs just a few tens of thousand people, General Motors kept more than 800,000 in well-paid jobs. Today, says Ford, it seems unlikely that any profitable new business will be highly labour-intensive.

Second, after factory workers lost out, the much-heralded replacement of middle class jobs with technology is visibly under way. As a result, the spending power of the middle classes is being reduced, and, compounding the effect, the middle-classes may choose to save rather than spend, with a further negative impact on economies.

You need look no further than the IT industry, says Ford, to see what’s happening: where a few years ago there were armies of computer and network specialists servicing the complicated computer systems in every office, those jobs have already disappeared as systems are automated and more data is stored online.

For the users of those computers, the same fate is in store. There’s a Japanese project to write a program to pass the entrance exams into a top university, and thereafter, presumably, to be ready to take on the jobs of university graduates. Already lawyers’ jobs are starting to be automated, with law graduates turned into what seem like little more than pigeons pecking at levers in a 1950s learning experiment:

“Each lawyer sits in front of a monitor where a continuous stream of documents is displayed. Along with the document, there are two buttons: “Relevant” and “Not Relevant.” The law school graduates scan the document on the screen and click the proper button. A new document then appears. They may be expected to categorize up to eighty documents per hour.”

Whatever the particulars of when and how different kinds of jobs are replaced by machines, there’s no mistaking the direction of travel. It’s hard to dispute Ford’s conclusion that the result will be a shifting of wealth towards the owners of businesses, with less and less economic power in the hands of workers. He quotes a study that found that between 1995 and 2002, 22 million factory jobs were lost, but in the same period, manufacturing output actually rose by 30 per cent.

How can civil society survive such an onslaught? How can we avoid a sci-fi nightmare in which “the plutocracy would shut itself away in gated communities or in elite cities, perhaps guarded by autonomous military robots and drones”?

Well, there is way, says Ford. First, it means transferring the burden of taxation from income to capital. Second – and this is the hard one – it means creating a system of universal basic income, or a “citizen’s dividend” as he prefers to call it, in recognition of the fact that the rich have only been able to prosper thanks to the systems put in place by the societies in which they operate.

This isn’t such a radical view any more. Bill Gates, Warren Buffett and other philanthropists talk about “giving back to society” the wealth they’ve accumulated, and about the luck of their birth in providing the social structures and opportunities that they exploited.

But would a universal income create a nation of layabouts (the kind of life enjoyed by the fat, lazy deckchaired citizens in WALL-E)? Not necessarily, says Ford: the security it provided could be a stimulus to entrepreneurship, letting people take risks without risking everything.

There’s an urgency about Ford’s message, echoing the conventional of wisdom in tech circles that we tend to overestimate tech changes in the short term, but underestimate them in the long term. The time of overestimating may be coming to an end: it’s not as if variations on this kind of warning haven’t been heard for years. But Ford’s evidence and our own eyes point to a creeping up on us of really radical change. As he says, “the future may arrive long before we are ready.”

View all my reviews