Here I review the paper “Training Large Language Models to Reason in a Continuous Latent Space”, or “COCONUT.” The main idea is that reasoning only through discrete language tokens may not always be ideal. The authors propose letting models reason directly in a continuous latent space, then mapping the result back to language only when needed.

Key concepts

Continuous latent space reasoning (COCONUT)

Instead of forcing the model to reason step by step through language tokens, like standard Chain-of-Thought methods, COCONUT allows the model to reason directly within continuous hidden states. These hidden states, called “continuous thoughts,” are fed back into the model as inputs for later reasoning steps, avoiding unnecessary token generation.

Emergent breadth-first search (BFS) behavior

One observation is that continuous latent reasoning encourages the model to keep multiple possible next reasoning steps alive at the same time. In practice, the model implicitly explores alternative reasoning paths in parallel, behaving somewhat like breadth-first search (BFS). This differs from standard CoT, which is linear and can be short-sighted.

Node heights and evaluation complexity

The paper measures the difficulty of reasoning steps by defining the “height” of nodes in a reasoning tree: the minimum distance to any leaf node. Nodes closer to leaves (lower height) are easier for the model to evaluate accurately. This metric showed that latent-space reasoning is especially good at rejecting clearly incorrect reasoning steps early in planning.

Emergent parallelism and uncertainty management

By examining how probabilities are distributed across potential reasoning steps, the authors noticed that the model initially keeps multiple possibilities open. This uncertainty gradually narrows as reasoning progresses. That is one of the more interesting parts of continuous latent reasoning to me.

What I learned

Why continuous latent reasoning can work

At first, reasoning without explicit language tokens felt odd. Language models, by nature, predict discrete tokens. But this paper argues that reasoning in latent space lets the model bypass some of the noise and redundancy of natural language. Many tokens in CoT seem to serve fluency more than actual logic or planning, so latent reasoning may avoid unnecessary overhead.

Emergent BFS-like reasoning behavior

Continuous latent reasoning seems to let the model explore multiple reasoning paths at once, somewhat like BFS. Multiple potential reasoning steps are encoded into the hidden state, so the model can delay commitment until more information is available. This contrasts with linear token-by-token CoT, where a wrong early decision can cascade.

Shortcomings of purely language-based reasoning

I realized that traditional CoT can be quite inefficient, as it forces the model to articulate every reasoning step explicitly. Many of these tokens provide minimal reasoning value but consume the same computational resources as critical reasoning tokens. COCONUT sidesteps this by handling reasoning implicitly, reserving explicit decoding into language only when necessary.

Potential issues and practical concerns

COCONUT is appealing, but it also creates practical problems. Continuous latent states are not stable across different models, or even across weight updates, so this kind of reasoning is tightly coupled to the model architecture and parameters. That makes generalization and scalability nontrivial. Training also becomes trickier because the model has to handle both discrete language tokens and continuous embeddings.

Alternative approach: distribution-based inputs

One alternative I thought about was feeding distributions over next tokens (softmax outputs) as inputs for later reasoning steps, rather than using raw hidden states. This would still allow implicit parallel reasoning and uncertainty handling without introducing a separate embedding space. It might not capture the full richness of continuous thoughts, but it could avoid some practical inefficiencies of handling embeddings directly.

Interpretability challenge

A limitation I foresee with COCONUT is interpretability. Human-readable reasoning steps provide some transparency, making it easier to debug and understand the model’s process. Latent reasoning, by contrast, is much more opaque. Future work would need tools to interpret what these latent states represent and how reasoning happens inside them.

Summary

COCONUT is an interesting alternative to standard token-based reasoning. Letting language models reason inside hidden continuous representations may improve efficiency and allow more parallel uncertainty management. The tradeoff is interpretability and training complexity.