Old version - please click on "Alucidate Blog" above right

Home/Alucidate Blog/

A Weakness in Artificial Intelligence

Here’s an interesting paradox. Expertise is mostly about better mental representation, say Ericsson and Pool in the book I was discussing earlier.

What sets expert performers apart from everyone else is the quality and quantity of their mental representations. Through years of practice, they develop highly complex and sophisticated representations of the various situations they are likely to encounter in their fields – such as the vast number of arrangements of chess pieces that can appear during games.  These representations allow them to make faster, more accurate decisions and respond more quickly and effectively in a given situation. This, more than anything else, explains the difference in performance between novices and experts.

This is an extension of Newell and Simon’s concept of chunks, which I talked  about before here.  (Ericsson worked with Simon at one point.)

Importantly, such representations are not at all the same as the parsimonious mathematical models beloved by economists, either. An expert may know tens of thousands of specific chunks in their field. But we still often know very little about exactly what that means, or how it works.

Now consider what this means for artificial intelligence.  AI mostly abandoned attempts to develop better mental representations decades ago.  Marvin Minsky of MIT advocated “frames” in 1974, but didn’t follow up with working systems.   To be sure, the 1980s era expert systems had many  simpler “if-then” type encoded rules, but faltered in practice. And today there is a great deal of attention to semantic networks to represent knowledge about the world. One example is DBpedia, which tries to present wikipedia in machine-readable form. A company called Cyc has been trying to hard-code information about the world into a semantic network for twenty years.

But semantic networks are flat and relatively uncomplicated. They are simple webs: graphs and links without much emergent structure.  As for vector models for things like machine language  translation, there’s almost no representation at all. It’s a brute force correlation, reliant on massive amounts of data. Recent advances in machine learning owe nothing to better representations. Chess programs effectively prune search trees efficiently, rather than use representations.

Meanwhile, the main data structures in  machine learning, like ndarrays in the Python Numpy scientific module, or Dataframes in the Pandas module which is commonly used by data scientists, are essentially glorified tables or spreadsheets with better indices. They are not sophisticated at all.

So the most important thing in expertise is something that researchers struggle to reproduce or understand, despite all the hype about machine learning.  Correlation is not the same thing as representation or framing or chunks. That’s not to say AI can’t make much more progress in this area. But there’s an enormous gap here.

Future advances are likely to come from better kinds of data structures.  There’s little sign of that so far.

2017-06-23T09:56:14+00:00 June 23, 2017|Expertise, Quants and Models|

When to dump a leader (Pelosi edition)

Many “leaders” have a tendency to  think that they ought to keep doing the same thing, but with more “passion,” or intensity, or resources. As I said in the post below, however, optimizing is not the same as adapting to a changed situation. There are many situations in which more persistence and determination just get you more trapped in doing the wrong thing. It’s essential tor recognize them. Most people don’t.

The unfortunate consequence is that change then requires a change in leadership as well. Maybe the Democratic Party is realizing that after multiple defeats: “Pelosi’s Democratic Critics Plot to Replace Her.” If things are persistently not working, try someone with a fresh look.

That also means that if you’re a leader it’s better to look for ways to adapt or change your mind before people plot to remove you after a massive setback.  The oldest danger of leadership is woodenheadedness. Yet most leaders hire consultants to put a theoretical or quantitative veneer on what they already think.

2017-06-22T15:33:53+00:00 June 23, 2017|Adaptation, Decisions, Human Error, Organizational Culture and Learning|

Two very different kinds of expertise

Faith in experts has been faltering, as populists attack many established political and academic elites. So it is more important than ever to recognize genuine expertise. The trouble is that is often hard to do, despite how credentials are so important in most areas.

Many studies of expertise don’t help that much. Take Anders Ericcson’s new book, Peak: Secrets from the New Science of Expertise. Ericsson is best known for his interesting research into “deliberate practice.” Malcolm Gladwell popularized the idea by talking about “ten thousand hours”  necessary to pick up any advanced skill, but Ericsson is adamant that the kind of hours matter just as much as the amount. Experience alone is not going to improve your skills unless you push yourself outside your comfort zone, he says.  It’s all interesting stuff, and it’s worth reading.

The trouble is this approach applies almost entirely to fields where knowledge is cumulative, and where there are established teachers and teaching methods. You can practice similar situations over and over again. Examples include playing the piano or violin to a very advanced level,  and many games with set rules like chess, golf or soccer.

Most of the most important fields are not like this. As Ericsson notes in a brief aside,

What areas don’t qualify? Pretty much anything in which there is little or no direct competition, m such as gardening or other hobbies, and many of the jobs in today’s workplace – business manager, teacher, electrician, engineer, consultant and so on.  These are not areas where you are likely to find accumulated knowledge about deliberate practice, simply because there are no objective criteria for superior performance. (p98)

That excludes most of the main areas in which decision-making skill is required. But people have a tendency to ignore the boundary conditions for this kind of research, its limits of applicability.

There’s a deeper problem lurking here as well, if you think about it.  Those other fields are more difficult because the underlying factors that lead to success keep changing, partly because competition in them means people develop new techniques or approaches. What worked in selling computers in 1950 will not work so well now – but the skills required to be a top notch ballerina or violinist are almost the same. The rules of the game stay the same, even if competition grows more intense and training techniques alter over time.

The root of the issue is people generally confuse optimizing with adapting. They are not at all the same thing. Practicing the same skill over and over may optimize but it is much less likely to lead to adaptation.

Where does this leave us? Expertise in an existing field or game is not the same thing as dealing with changes in the rules of the game. That is a wholly different kind of problem.  Indeed, experts may be some of the last people to recognize that the rules are changing, as they have so much invested in existing interpretations. That explains why scientists rarely change their minds, but science slowly evolves without them.

Standard decision science, and most standard economics and economic policy, is about optimizing rather than adapting. But adaptation is necessary to success, and lack of adaptive skill is one reason why even the most credentialed experts often seem out of touch or wrong. To say so is not to deny expertise or science; instead it is to advocate a different kind of expertise and science.

There’s some other points to make, which I will come to shortly.

 

2017-06-22T15:12:35+00:00 June 22, 2017|Adaptation, Expertise|

Another strike against prediction and following formal rules

I was talking in the last post about how inappropriate and outdated some of the economics discussion about monetary policy has become, especially the whole debate about policy rules, credibility and commitment.  Nothing has been learned from the great crisis, and the field is mostly a dusty backwater twenty or thirty years behind most of the rest of the economy,

Idealized planning, prediction and forecasting had its heyday in the 1960s and 1970s in Western business (and before that the Soviet Union put its faith in planning.) It turned out to be a disaster. Formal forecasting didn’t work, and most such economists were shown the door in the private sector in the 1990s. But the approach is still going strong in economic policy, as if the clock stopped twenty years ago.

Here’s a contrast. Take the announcement that Amazon is buying Whole Foods yesterday. Amazon is, of course, wildly successful as a business. But Jeff Bezos does not try to forecast or predict. Instead, according to Farhad Manjoo in today’s NYT:

Yet if there’s one thing I’ve learned about Jeff Bezos, Amazon’s founder and chief executive, after years of watching Amazon, it’s that he doesn’t spend a lot of time predicting future possibilities. He is instead consumed with improving the present reality on the ground, especially as seen through a customer’s eyes. The other thing to know about Mr. Bezos is that he is a committed experimentalist. His main way of deciding what Amazon should do next is to try stuff out, see what works, and do more of that.

If you can’t reliably predict, then you have to think and act very differently.

2017-06-17T14:59:15+00:00 June 17, 2017|Adaptation, Central Banks, Economics, Federal Reserve, Forecasting|

Lack of Fresh Thinking among Academic Economists

The intellectual weakness of much recent thinking on monetary policy is very disappointing. Actually, to call it “recent” thinking is a distortion.  The debate seems stuck somewhere around 1996. Here’s John Taylor complaining about “post-panic” monetary policy at a Congressional hearing in March:

In many ways this whole period can be characterized as a deviation from the more rule- like, systematic, predictable, strategic and limited monetary policy that worked well in the 1980s and 1990s.

It’s as if the last ten years just went down the memory hole as an unfortunate aberration from elegant theory. He just doesn’t seem to occur to Taylor that the consequence of those policies was also massive asset bubbles, volatility and ultimately the most catastrophic financial crisis in seventy years.

That’s because the supposed benefits of policy rules come from oversimplifying the task that policymakers face. There may be a case for comparing policy decisions to a rule, but it elides into pretending that a policy rule can replace policy.

I don’t necessarily hold with the various unconventional monetary policy options that have been used since 2008, or believe that they were as effective as the Fed sometimes claims. However, policymakers had to improvise because nothing else worked. And they at least deserve some credit for that.

At best, the rules school can lead to an appropriate restraint about what policy can achieve. Give policymakers one simple thing to do and maybe they will at least achieve that. But it achieves that simplification by ignoring the interconnections and complexity of the actual economy.

This is much more than the usual discretion versus rules/ time consistency debate. If I had to summarize it in a sentence, I’d say look for balances, not principles or fixed goals.

2017-06-13T08:27:54+00:00 June 13, 2017|Central Banks, Economics, Federal Reserve, Monetary Policy|

Democracy works by suppressing hubris

Another election, another huge surprise. Teresa May’s historic electoral catastrophe last week was actually foreshadowed by plunging opinion polls this time, so the pollsters are not to blame for once. But the exit poll on Thursday night still came as a vast shock, as the scale of the collapse of Tory hopes became clear. Andrew Rawnsley of the Guardian blamed overconfidence and hubris for the disaster:

She conducted a campaign that combined vanity with incompetence and had learned nothing from the now myriad examples from around the democratic world of what happens when a politician behaves as if they are simply entitled to power.

There is something strangely magnificent in this episode beneath the shrieks and journalistic drama and personalities. Democracy is a messy, awkward, short-sighted system of governance. But it has a better error control loop than autocracy or bureaucratic planning.  Democracy is impatient with failure, as I noted before,  and fixes mistakes relatively quickly.  It may be terrible at long-term thinking and elegance, but it is good at throwing overconfident and arrogant planners out.

That is actually a good thing, and it means people are wrong to see all the turmoil of the last two years only as a departure from some idealized global liberal ideal, as a car crash on the way to utopia. The system is messily rebalancing in response to problems that had been ignored. Overconfident elites get punished, and underserved groups get an (occasional )voice.  The process not ideological. It happened to Progressives in the US last November. It just happened to Conservatives in the UK.   If would be worse if the system got stuck – and that may be the case in continental Europe.

 

 

 

2017-06-13T08:06:49+00:00 June 13, 2017|Europe, Politics|

How academics and practitioners think differently

Here is an excellent article at The American Interest on the differences between how policymakers and academics think about international relations in the US. Some of these differences carry very important implication for policy. In general, scholars have (not surprisingly) drifted away from practical concerns which limits their influence, author Hal Brands says.

International relations scholars—particularly political scientists—increasingly emphasize abstruse methodologies and write in impenetrable prose. The professionalization of the disciplines has pushed scholars to focus on filling trivial lacunae in the literature rather than on addressing real-world problems.

But practitioners and scholars also take very different positions on some substantive issues.  Practitioners are more concerned with American interests, while academics think more as “global citizens” or the stability of the system as a whole.  Interestingly, one particular point of difference is attitudes to credibility.

Since the early Cold War, U.S. policymakers have worried that if Washington fails to honor one commitment today, then adversaries and allies will doubt the sanctity of other commitments tomorrow. Such concerns have exerted a profound impact on U.S. policy; America fought major wars in Korea and Vietnam at least in part to avoid undermining the credibility of even more important guarantees in other parts of the globe. Conversely, most scholars argue credibility is a chimera; there is simply no observable connection between a country’s behavior in one crisis and what allies and adversaries expect it will do in the next.

This is clearly extremely important.  I have more sympathy with the scholars on this one: many of the worst policy errors have been caused by “domino theories” of credibility.

It is also interesting that there is a gap at all between practitioners and academics in foreign policy. In economic policy, the academics largely captured policy, certainly in the US, in the last two decades. That naturally carries with it a certain style of thinking – and the outcome has been anything but encouraging, with enormous financial crises and volatility.

2017-06-06T14:26:46+00:00 June 6, 2017|Decisions, Expertise, Foreign Policy|

The right kind of excitement about AI

We are currently at peak excitement about Artificial Intelligence (AI), machine learning and data science. Can the new techniques fix the rotted state of Economics, with its baroque models that failed during the financial crisis? Some people suggest that the economics profession needs to step aside in favor of those are actually have real math skills.  Maybe Artificial Intelligence can finally fix economic policy, and we can replace the Federal Reserve (and most of financial markets) with a black box or three?

 

It’s not that easy, unfortunately. There is a hype cycle to these things. Hopes soar (as expectations for AI did as far back as the late 1960s.) Inflated early expectations get dashed. There is a trough of disillusionment, and people dismiss the whole thing as a fad which entranced the naive and foolish.  But eventually the long term results are substantive and real – and not always what you expected.

 

The developments in data science are very important, but it’s just as important to recognize the limits and the lags involved. The field will be damaged if people overhype it too much, or expect too much too soon.  That has often happened with artificial intelligence (see Nils Nilsson’s history of the field, The Quest for Artificial Intelligence.)

People usually get overconfident about new techniques. As one Stanford expert put it in the MIT Technology Review a few weeks ago,

For the most part, the AI achievements touted in the media aren’t evidence of great improvements in the field. The AI program from Google that won a Go contest last year was not a refined version of the one from IBM that beat the world’s chess champion in 1997; the car feature that beeps when you stray out of your lane works quite differently than the one that plans your route. Instead, the accomplishments so breathlessly reported are often cobbled together from a grab bag of disparate tools and techniques. It might be easy to mistake the drumbeat of stories about machines besting us at tasks as evidence that these tools are growing ever smarter—but that’s not happening.

 

I think if there’s better AI models in economics it won’t be machine learning – like pulling a kernel trick on a support vector machine – so much as a side-effect of new kinds of data becoming cheap and easy to collect. It will be more granular instantaneous data availability, not applying hidden Markov models or acyclic graphs. Nonlinear dynamics is still too hard, such models are inherently unpredictable, and in the case of the economy has a tendency to change if observed (Goodhart’s law with  a miaow from Heisenberg’s cat).

 

None of that new data will help, either,  if people don’t notice the data because they are overcommitted to their existing views. Data notoriously does not help change people’s minds as much as statisticians think.

 

AI has made huge strides on regular repetitive tasks, with millions or billions of repeatable events. I think it’s fascinating stuff. But general AI is as far away as ever. The field has a cycle of getting damaged when hopes run ahead of reality in the near term, which then produces an AI winter. Understanding a multi trillion dollar continental economy is much much harder than machine translation or driving.

 

There’s also a much deeper reason why machine learning isn’t a panacea for understanding the economy.  A modern economy is a complex dynamic system, and such systems are not predictable, even in principle.  Used correctly, machine learning can help you adapt, or change your model. But it is perhaps much more likely to be misused because of overenthusiastic belief it is a failsafe answer. Silicon Valley may be spending billions on machine learning research, with great success in some fields like machine translation, But there’s far less effort to spot gaps and missing assumptions and lack of error control loops – and that’s what interests me.
2017-05-31T15:12:24+00:00 May 31, 2017|AI and Computing, Forecasting, Quants and Models, Systems and Complexity|

Terrorism Response: Crippled by a False Analogy

Generals, it is said, usually plan to fight the last war. The French, for example, built the Maginot Line on their Eastern frontier with Germany. It was a massively fortified replacement for trenches, built as a result of the trauma of trench warfare in the First World War.  Unfortunately, the Germans simply went around the defenses by invading through Belgium in 1940. The French quickly lost the war.

General staffs and security experts often find it very difficult to adapt to new circumstances.

The same applies to most current thinking in Western security circles about terrorism. Officials, military officers and academics try to understand the current threat in terms of previous experience of terrorism, such as the PLO or the IRA.

There is one very important thing to notice, however. Almost all terrorism and irregular warfare before Islamic attacks was connected to wars of national liberation. There were a few exceptions, such as the Baader-Meinhof gang in Germany, but class-based Marxist terrorism has been very rare in western societies. Most major terrorist organizations in the twentieth century were linked to nationalist movements.

In fact, the experience in Iraq, Afghanistan and countries like Somalia (or Vietnam in the 1960s) reinforces the pattern, and it also forms the experience set of many senior military and intelligence officials today. Counterinsurgency is a matter of persuading a potentially hostile and uncooperative majority population to help what those locals see as imperial foreigners of doubtful legitimacy, bent on economic exploitation. Counterinsurgency is imperial thinking without the explicit colony.   The insurgents gain from provoking an overreaction from “occupying” outside forces.  Too much heavy-handed force makes the local population feel oppressed or disrespected (such as the execution of the 1916 rebels in Ireland, the Amritsar massacre in 1919, or internment of IRA suspects in 1971.) Violence forces people to choose one side or another, almost always at the expense of the outside power.

Once the support of the majority population is lost, the position of the foreign or colonial power is grim. It becomes too expensive to maintain a presence. Too many troops are required. Too many of those troops return home in body bags, endangering public support at home.  So the FLN forced the French out of Algeria. The Vietcong forced the Americans out of Vietnam.

The only way to fight such an insurgency is to prevent the alienation of the local majority population. You try to improve economic prospects, or address grievances, to undermine support for the rebels. You are patient, and hope that the inability of the rebels to provide a clear economic alternative will undermine the insurgent cause. Otherwise the administrators and colonels will find themselves on the boat or plane headed back to London, Paris or Langley.

It would be surprising, in fact, if officials and officers did not see current terrorist threats in the light of fifty years of this kind of experience. It is established conventional wisdom.

Think for a minute, however. An attack like the horrific carnage in Manchester a few days ago is not like this at all.  Contemporary Islamic terrorism is not a war of national liberation. It is not like the PLO or IRA or FLN.  In fact, it is the exact reverse in many ways. It is the terrorists who come from a tiny foreign community. Those foreign communities, most often of recent origin, are often seen, perhaps unfairly, as outsiders of doubtful legitimacy bent on economic exploitation of welfare and housing (at least as some locals may see it.)

In other words, the terrorists and their supporters actually occupy the slot of “outsiders” in the war of national liberation model. All the implications run in reverse.  In the event of hostility or escalation, the majority community in Manchester or Nice or Orlando are in their home territory already, and are not going to be driven back to any colonial capital. “Brits go home,” as Irish nationalists put it, doesn’t work very well when the Brits are in Britain.

This turns some of the conventional wisdom based on nationalist terrorism and counterinsurgency on its head. What about the overwhelming necessity to respond to terrorism in a proportional way, to avoid alienating the key target population? In this case a muted response may make the majority local population more alienated and frightened. They may feel they are being left unprotected, much like Iraqi villagers who doubt American troops will turn up to protect them when the local militants come calling.  An inability to protect eight-year old girls going to a concert may undermine the government in the eyes of those who expect protection.

What about the fundamental principle that terrorists win by provoking an overreaction? In national liberation, it is the foreign or colonial community which is most likely to be forced out if things escalate into open conflict. Consider the French settlers in Algeria, the piers noir, almost a million of whom were forced back to France after Algerian independence. But in England or Spain or Germany, it is Muslim communities which are the vulnerable outsiders, the tiny minority of foreign origin.  In other words, escalation and backlash is likely to lead to horrific consequences and departure  for Islamic populations in the west, not the departure of a colonial power back across the oceans. It is completely different to national liberation.

Provoking escalation and backlash is likely to lead to catastrophe for the terrorists, especially in a continent which has been only too willing to indulge in ethnic cleansing within living (and in Bosnia recent) memory. Indeed, it is only the desire of Western governments to prevent such a repeat of history affecting innocent Muslims which stops that Islamic disaster happening.

So why would Islamic terrorists do their best to provoke the authorities into an escalation which their tiny communities would almost certainly end up with a fate like colonial settlers forced to return back from India or Kenya or Indonesia?

It’s because the terrorists don’t think they are fighting a war of national liberation. They are fighting a war of religious conquest, in which they believe their success is divinely ordained because the enemy is decadent and weak.  Dealing with a different enemy requires different tactics.

It’s usually unexamined assumptions which get people into trouble. Western security experts are entrenched in a familiar set of assumptions which simply don’t apply to current terrorist threats.

 

 

 

2017-05-26T07:44:34+00:00 May 26, 2017|Assumptions|

Website Redesign is live

The other pages on the website have been simplified and redesigned. There used to be six different explanatory pages. That has been reduced to two, in order to make it clearer.  There are also other major projects in process, which the redesign will ultimately feed into.

2017-05-24T12:04:08+00:00 May 24, 2017|Uncategorized|