You Must Understand Cognitive Bias and AI

Many organizations are using, or looking for ways to use, AI in their operations. The assumption is that AI provides quality intellectual services that would take humans considerably more time and money to produce. Some executives see intelligence analysis as a service that could be augmented by –or replaced by–AI. To wit, AI can hypothetically review secondary information published online in a fraction of the time it would take human analysts. Some advanced applications of AI can even produce coherent-sounding implications of data fed into it.

AI still has a major weakness—that it is not actually the same as a human. Current AI has no empathy, no ability to predict the behavior of individuals not mentioned in some model, and as such cannot avoid the most obvious mind traps, where effective intelligence analysis goes to die.

The last mile for AI is still a chasm between intelligence products and the decision makers who would use them, and human intelligence professionals are the best hope when it comes to assisting in good decisions.

Cognitive Bias is the major enemy of good decisions assisted by AI

AI software is able to command information in a coherent manner to a computer to be read by a human decision maker.

What it cannot do is evade the cognitive biases of that human. More than any other reason, this is why senior intelligence personnel must be included in the human-machine assistance loop—because they can identify potential pitfalls that result from collection, analysis, and dissemination of intelligence. No computer can match a salty, senior intel professional at this time.

There are several groups of cognitive bias that machines cannot identify when delivering intelligence by way of an LLM.

AI cannot identify information biases from a simple prompt. It cannot warn against anchoring bias, which is when the first piece of intelligence encountered sways our perception of all following information irrespective of its importance. In fact, AI frequently spits out incorrect information, making anchoring bias downright dangerous.

AI may worsen recency bias by referencing the freshest information according to its algorithm. A freshly published falsehood is still bad intelligence.

Illusory truth effect could also be worsened by AI, since repeating a falsehood multiple times might choose the algorithm to preference said information as more reliable.

LLMs are unlikely to be much help with social biases, those cognitive distortions that arise from our evolutionary history as primates who generally do better work together in groups than alone. AI is tailor-made to worsen authority bias, as it looks to algorithmic authority when choosing to select data online in the first place. It certainly is no help with Semmelweis reflex, which is the tendency to reject all information that is from outside of a professional orthodoxy.

Risk biases are nigh impossible for AI as well. LLMs have no ability to screen for sunk cost fallacy, in which a decision maker assesses that changing their mind will put at risk all of the resources already spent on fallacious beliefs. AI offers no path through the thicket of loss aversion in which we must examine our feelings about seeing future risk as potential gain or potential damage.

Currently, AI leads executives straight into the maw of the cognitive biases they must endeavor to defeat.

Executives still need intelligence professionals

At Competitive Futures, we love AI! It’s like a really advanced search engine that gets you more than you thought you wanted.

We even love AI for competitive intelligence! It can give you an example of Porter’s Five Forces analysis of a new market very quickly. Sweet talk the prompt, and you might get some Innovator’s Dilemma analysis that isn’t half bad.

However, AI is still dangerous. It routinely makes basic factual errors–some of them egregious. It cannot tell you about your biases. It has no understanding of risk. Nor memory. Nor ability to understand the people involved…in anything.

But we still think it’s great!

And if you want to integrate AI into your decision making…please give us a call.