Markets Need Disagreement

'The knowledge needed for a rational economic order never exists in concentrated form but solely as the dispersed bits of incomplete and frequently contradictory knowledge which all the separate individuals possess.' — F.A. Hayek

I consider myself a techno-optimist. I'm saying that upfront because what follows is doomer-flavored. The economic impact of AI gets discussed constantly, but there's one aspect of it I haven't seen discussed.

Start with markets as distributed compute. A market resolves uncertainty and produces information, and the output of that computation is a price. It works because millions of economic agents (not the AI kind, yet) make relatively independent decisions, and the price is what falls out at the end. Hayek made this argument in 1945: the knowledge an economy runs on never exists in concentrated form, only as dispersed and frequently contradictory bits held by separate people. No central planner can assemble it.

Human intelligence has depth, breadth, and enormous variety. That variety is what makes the distributed part work. As far as I can tell, markets become unbalanced or fail for one of two reasons: too many people get the same idea at the same time, which is a bubble, or too few actors control too much and meddle, which is regulation and interventionism. In both cases the market stops producing useful information.

This is why communism and its friends don't work. You cannot compute centrally, fast enough, what is better computed in a distributed way.

Here is my concern. AI lacks diversity of preference and opinion. A hundred million people using ChatGPT is still just one ChatGPT: one model, carrying its post-training biases and whatever dispositions sit underneath them, which are hard to fathom and test.

The obvious counter is that everyone will have their own preferences, memory, and context, so the outputs will diverge anyway. Maybe. I'm not sure I buy it. Personalization runs on top of a shared base, and the base is where the priors live.

Let's assume AI reaches the adoption level of social media, the level where your weird divorced uncle is sharing questionable shit on Facebook. At that point a large share of economic decisions are being shaped by a handful of models. My worry is that this reduces the information-processing effectiveness of markets. The models don't have to be bad at reasoning for this to be a problem, they only have to reason similarly.

Pull that thread. AI could drive local bubbles in whatever sectors its priors happen to favor. It could produce something that behaves like a monopoly without anyone owning one, where the concentration sits in the reasoning rather than in the ownership.

The standard response is that any large inefficiency gets eaten away by smart money. That's true. Wall Street will make money hand over fist correcting an inefficiency everyone else created by outsourcing judgment. The correction happens. The average person pays for it.

The worst case is a collapse in the number of independent information producers, from something like two hundred million adults down to three models and fifty million holdouts.

I don't think this is likely. I think it's possible, and so it's worth thinking about.

None of this argues for regulation or for pausing development. All problems are opportunities, and we can engineer our way out of most things as long as the incentives point that way.