The dream of every quantitative trader is to be Jim Simons, the chain-smoking mathematician whose team came up with the Medallion Fund. The Medallion Fund is the greatest hedge fund of all time. It’s so good, Simons and his colleagues threw out all their investors so they could keep Medallion’s returns for themselves.

No outsider knows how it works. But based on how they operate, we know it’s some kind of computer-based trading strategy. The Medallion team has found an inefficiency in the market and they have been mining it for almost 30 years.

Now all of a sudden, everyone has powerful AI technology. AI is particularly good at finding patterns. Might there be many more Medallion funds? Or, with democratised AI, might the days of market inefficiencies be over?

Signal

For AI researchers, the challenge is to separate signal from noise. The signal is the thing you’re trying to perceive. The noise is everything else. 

When you’re driving through a housing estate and a ball rolls in front of your car, that’s signal. When you’re walking down a street and notice €50 stuck in a grate, that’s signal. When you’re crossing a road and hear, from behind you, the hiss of an electric bike, that’s signal.

Human brains are wired specifically to do this job so it’s no big deal to us. But teaching machines to do it has been hard. You may have heard the example of a chair: a person intuitively understands what a chair is. But it’s surprisingly hard to define a chair in such a way that a computer can recognise one.

For the last 50 years, much AI research was stuck in a dead end. It tried to train AIs with elaborate rules: “If an object has four legs and a flat bit which is about two feet off the ground, define it as a chair”.

Large Language Models (LLMs) like ChatGPT are based on a different paradigm. Instead of defining a program to create a desired result, ChatGPT defines the result, and tells the computer to program itself in such a way that it achieves it. 

This involves feeding the AI lots of data. If they want the AI to recognise a chair, they feed it a couple of million pictures of chairs. From the pictures, the AI gets an intuition for what a chair looks like. 

From its very first picture of a chair, the AI will perceive a random pattern of colours. Gradually, picture by picture, it will come to understand that the colour patterns represent different objects and that one particular shape keeps recurring. It’ll learn over time to tune out all the background patterns such as dining room tables, fireplaces, office furniture, green grass, and so on. It’ll learn to define all that as noise.

Getting an AI to this point was hard. It took decades of research, many terabytes of data, and many millions of dollars worth of computing power. 

Now AI is a few steps beyond recognising chair pictures. Trained on the entire Internet, ChatGPT can recognise the subtle differences between a million concepts. It can tell the difference between the morose poetry of Philip Larkin and the melancholic poetry of W.B. Yeats. 

Noise

Observing ChatGPT’s amazingly broad talents, you might wonder what would happen if it were set loose on the financial markets. LLMs are very good at finding patterns in data, and financial markets generate reams of data. The dream would be that an LLM could be fed 100 years of price data, along with data related to the characteristics of securities such as their profitability, leverage, volatility, growth, and anything else you might think of. Having digested all this, the LLM would in theory find patterns and correlations previously undetected by human traders. Step three equals profit.

Though this sounds plausible on the face of it, it’s much harder than it appears. It might be years away. The problem is the signal-to-noise ratio. When it comes to a picture of a chair, the signal (chair) to noise (everything else) ratio is pretty good. After a few hundred thousand attempts, the AI starts to discern signal from noise. 

In financial markets, by contrast, the signal is incredibly faint. The efficient markets hypothesis says stock prices follow a random walk. This random noise makes it hard for AIs to find meaningful patterns. The random noise generates meaningless patterns. Buried somewhere amid all this noise are the meaningful patterns the AI is trying to isolate.

It’s a bit like the Haber process in chemical engineering. The Haber process was invented in 1905. It allows us to extract nitrogen directly from the air. Now we’re worried about greenhouse gases, some have asked whether something like the Haber process couldn’t be used to suck CO2 or methane directly out of the atmosphere. Conceptually, it’s possible. But the problem is that methane and CO2 are much less abundant in the air than is nitrogen. In the air, nitrogen makes up 77 parts in 100; CO2 makes up 38 parts in 100,000; and methane makes up 17 parts in a million. CO2 and methane are harder to efficiently separate from the air because they’re so rare. 

One hard problem with asking an LLM to decode the market is the signal is faint. Another hard problem is that the trainers don’t even know what the signal is. They’re not pointing at a million examples of the signal and saying to the LLM, “You see, that’s what the signal looks like”. The LLM has to find it, firstly, and secondly figure out what it is. In the same way that ChatGPT hallucinates, MarketGPT might confidently tell traders to bet €500 million based on a nonsense correlation. 

Finally, LLMs like ChatGPT aren’t good with numbers. They’re better with words and concepts. An AI that could discern patterns in financial data would need a different architecture. It wouldn’t be a matter of asking ChatGPT to get to work.

To be sure, we’ll get there eventually. It might take a few more years but it’ll happen. What then? 

A world in which AIs can comb financial markets for correlations and inefficiencies will be a bad one for many professional investors. Right now, professional traders and investors get to profit if they find a market inefficiency. They prune the market every day. If they’re good at their job they get paid. They are the reason it is, on aggregate, highly efficient. 

Once the AIs get involved, I’m struggling to see how the same number of traders will be supported. AIs will roam the markets like bloodhounds, sniffing out pockets of inefficiency. They’ll perform services like making markets for illiquid assets, or providing liquidity when it’s needed. At first, they’ll be controlled by a few big funds — like Jim Simons with the Medallion Fund. But in time they’ll be ubiquitous. 

What happens when a new technology gets invented? The naive response is to say it kills jobs. But that’s not usually what happens. Usually the new technology takes over certain tasks, and then people find higher value tasks to do. The canonical example is how, after ATMs were introduced, the number of bank branches and bank tellers increased. Because the ATM freed bank tellers up to do more important stuff. That would be considered an informed answer to the original question. 

But I’m struggling to imagine what most of the active asset managers will all do in a world of ubiquitous AI trading bots. Financial markets aren’t like bank branches. The more efficient the market is, the less traders get paid. There really aren’t more tasks for traders to progress to. 

Sometimes, technologies really do eradicate jobs. Many traders might yet go the way of travel agents.