The Merging
of Modalities.

Every previous expansion of the media stack arrived with the institutions it needed already built. The printing press inherited copyright and publishing. Photography inherited likeness rights and the studio. Cinema inherited unions, guilds, and broadcast regulators. The multimodal frontier is the first time the production of media has expanded without the institutions to absorb it — and that absence, not the capability itself, is the story this issue is tracking.

I. What the merging actually does.

The technical story is simple to describe and easy to under-read. A frontier multimodal model is a single set of weights that ingests text, image, audio, and video, and emits any of the four. That is the surface change. The deeper change is that, for the first time, the production of media is no longer separated from its understanding. A model that can read a frame can write one. A model that can listen to a take can produce one. The pipeline collapses, and so does the rationale for having separate tools, separate teams, and — over a longer horizon — separate institutions to govern them.

The merge also breaks a working assumption that has been quietly load-bearing for two centuries: that each modality is its own object of regulation, its own labor market, and its own asset class. JPEGs were governed differently from MP3s not because the bits were different, but because the institutions producing them were. When the same model produces both, the institutions producing them are the same. That is the merger this issue is actually tracking.

The more you're building these general purpose models, they benefit everyone. Nano Banana benefits a lot from our image understanding capabilities, our video understanding capabilities. — Nicole, Google DeepMind Nanobanana team

The lift effect Nicole describes is not a marketing line. It is the empirical claim that capability in one modality raises capability in the others once the base model is shared. If true at scale, it inverts the procurement logic of the last cycle: instead of picking a best-in-class vendor per modality, the dominant strategy becomes consolidating on whichever vendor's base model is improving fastest across all four. The lab-to-product transmission belt narrows. So does the moat.

II. The institutional history this merger is colliding with.

Each modality acquired its institutional infrastructure at a different historical moment, in response to a different production technology, and around a different labor structure. Reading the merger against that backdrop is what tells you which institutions will absorb it and which will simply break.

Three observations follow from reading the diagram literally. First, every regime in the chart is older than the digital-era institutions written on top of it — publishing and copyright are five and a half centuries old, photography and likeness rights almost two, broadcast roughly one. Second, every regime is structured around a labor force, not a product: writers, photographers, musicians, performers. Third, none of them anticipated a single producer that traverses all four.

That third observation is where the institutional vacuum lives. The merged-output row at the bottom of the chart has no inherited regime to absorb it, because no prior production technology produced anything that crossed those boundaries in a single artifact. The closest analog is the silent-to-talkie transition in cinema, which collapsed image and audio production into a single shoot — and which took a decade of labor and copyright realignment to settle. Multimodal compresses an analogous reshuffling across four lanes at once.

III. Where value accrues during an institutional vacuum.

Institutional vacuums are not neutral. Whoever sets the de facto rules during the gap usually keeps them. Three layers are doing the rule-setting right now, and the order in which they congeal is the most predictable thing about the next eighteen months.

Platform-level policy is being set by the frontier labs and their distribution partners. Watermarking standards, training-data opt-outs, prompt-disclosure conventions, age-gating — these are being decided by Google, Anthropic, OpenAI, and the cloud platforms that distribute them, in dialogue with a small number of large customers. By the time the EU AI Act's implementing rules and the US Copyright Office's guidance arrive, the practical defaults will already be in place. Regulation will codify what the labs negotiated.

Litigation infrastructure is consolidating. The performer-rights settlement at landmark scale this quarter (see Landscaping) is the early signal: a small number of plaintiffs' firms are positioning themselves as the de facto enforcement layer for likeness, voice, and now multimodal-output cases. Their settlements set the licensing structure other producers will adopt, not the statutes that follow.

It's a combination of science and art. Combination of rationality and feeling. Everyone in our company watches a lot of videos. — Yang Tang, Opus Clip

The taste-and-curation layer is the most under-discussed and most economically interesting. Once any team can produce technically competent output across four modalities, the binding constraint stops being capability and starts being judgment. Yang Tang's observation about Opus Clip is the company-scale version of it. The same dynamic governs which Veo-rendered shot makes it into a feature, which Nano-Banana output ships into a product surface, which voice clone reads which line. That layer is human, low-throughput, and very hard to commoditize. Its economic value is rising in inverse proportion to the falling cost of the underlying model output.

IV. Where specialization survives the merger.

Not everything merges. The lesson from the cinema-to-television transition is that specialization survives where the human production capital — domain expertise, taste, on-set craft — does not transfer to the new medium for free. A general-purpose model lifts the floor; it does not raise the ceiling on judgment. Three places where this is already visible.

Vertical reasoning systems. Mustafa Bhuiyan's observation from §02 — that visual reasoning has to be conditioned on the deployment stack, not bought off the shelf — is the enterprise version of this. Long-tail physical environments, regulated industries, and sovereign-data domains do not absorb a frontier multimodal model cleanly. They absorb a specialized layer on top of one.

Realtime and on-device. David Chen's robotics-streaming thesis applies here too. The frontier multimodal model is moving toward cloud inference; the realtime layer between that model and the world is becoming its own specialization. Whoever owns that seam — voice, robotics, AR overlays, telepresence — owns a moat the foundation model cannot eat.

Taste, curation, brand. The Coframe interview and the Veo Hollywood pipelines are both bets that the human curation layer is where durable margin will live. As output volume goes vertical, the scarce resource is the judgment that says this one ships, that one doesn't. That is not a capability you scale by adding parameters.

V. Four predictions and the closing observation.

One. The 2026–2027 procurement pattern in enterprise will be explicitly multi-vendor and modality-agnostic. The Anthropic Economic Index's diversification finding will hold; single-vendor multimodal lock-in will not be the dominant pattern, regardless of which lab leads on benchmarks. Procurement teams should model two-to-three frontier-vendor stacks and a specialization layer on top.

Two. The first coherent legal regime to land on multimodal output will be a California statute, not a federal one and not an EU one. The combination of the existing performer-rights framework (AB 2602, AB 1836), the active disclosure bill, and the concentration of frontier labs in the state means California will set the de facto US baseline. Other states will harmonize within 18 months.

Three. By Q4 2026 the labor disputes around multimodal output will look less like SAG-AFTRA strikes and more like guild-licensing structures — voluntary opt-in pools that license likeness, voice, and motion to specific named uses, administered by a small number of intermediaries. The settlement structure of the landmark voice-clone case is the template.

Four. Within the labs, the org-structure consolidation Rocky names in The Foyer continues. The teams that ship coherent multimodal product will be teams whose research and product reporting lines already converged. The teams that ship incoherent product will be teams that did not. Expect at least one major lab to publicly restructure around this thesis in the next two issues of this report.

The closing observation is the one this issue exists to make: the merging of modalities is not, primarily, a model-quality story. It is a story about which institutions, labor structures, and legal categories survive contact with a production technology that does not respect the boundaries they were built around. The labs are not waiting for those institutions to catch up, and neither is the capital. The next eighteen months will be decided by who sets the defaults during the gap — and on that, the smart bet is on whoever is shipping product at the seams, not whoever is filing briefs about it.

— KN