Meta put 400B-class open weights on one H100 at a 17B inference cost — then spent the week defending how it measured them

WEEK OF APR 5 – 11, 2025

Meta put 400B-class open weights on one H100 at a 17B inference cost — then spent the week defending how it measured them

Llama 4 activates 17B of 400B parameters and fits a single H100 host, the most aggressive cost-floor move of the quarter. Within 48 hours Meta was denying it trained on test sets and explaining a leaderboard score that came from a model nobody can download.

Llama 4 arrived on Saturday: Meta's first mixture-of-experts and first natively multimodal generation. Scout activates 17B of 109B total parameters across 16 experts with an advertised 10M-token context and fits one H100 under int4; Maverick activates 17B of 400B across 128 experts with a 1M-token window and ships in an FP8 build that fits a single H100 DGX host. Serving a 400B-class model at a 17B-class per-token cost on hardware you can rent by the hour resets the build-versus-buy arithmetic.

The license is the catch. Organizations above 700M monthly active users need a separate agreement, and the multimodal weights are withheld from individuals domiciled in and companies headquartered in the European Union. Then the story turned within 48 hours: the Maverick ranked second on LMArena turned out to be an unreleased experimental chat variant rather than the checkpoint developers can download, and an exec had to deny training on benchmark test sets while conceding mixed quality across hosting providers.

Two other threads ran underneath. An unauthenticated remote-code-execution flaw in Langflow made the agent builder itself the attack surface, and the White House put a planned Nvidia H20 export crackdown on hold after a Mar-a-Lago dinner — right up until a license requirement and a $5.5B charge landed four days later.

THU · Apr 10, 2025memoryagentschatgptcontext-engineeringprivacy

ChatGPT memory starts referencing every past chat — persistent context becomes a product feature

On Thursday OpenAI upgraded ChatGPT memory so it can reference all of a user's past chats, not only facts explicitly saved; Sam Altman called it a surprisingly great feature and pointed at AI systems that get to know you over your life. The interface now carries two independent switches — Reference saved memories and Reference chat history — and the chat-history layer, unlike saved memories, is not itemized or individually editable in settings because it evolves as context changes. Rollout started with Plus and Pro, with Team, Enterprise and Education following in a few weeks; Temporary Chat still neither reads nor writes memory, and users already opted out of memory are opted out by default.

OpenAI positioned the change as drawing on tone, goals, interests and recurring topics across prior conversations rather than retrieving them per query. That distinction matters operationally: there is no storage cap to reason about and no auditable list to prune, so the documented controls are the setting toggles, deletion of saved memories, and Temporary Chat.

ChatGPT memory starts referencing every past chat — persistent context becomes a product feature — cdn.searchenginejournal.com

WHY IT MATTERS

Cross-session memory is the step that turns a stateless chat app into an agent with a model of its user, and it is now a baseline expectation in anything you ship against ChatGPT. Architect it the way this release does — separate explicitly stored facts from inferred, non-inspectable context — because the inferred layer cannot be listed or selectively deleted, which is a compliance problem the moment your product has enterprise or EU users. Give users a switch per layer and a session type that touches nothing, or you will be retrofitting both under audit.

OpenAI Help Center: Memory in ChatGPT Search Engine Journal: ChatGPT memory now remembers info across all chats

WED · Apr 9, 2025export-controlsnvidiah20infrastructurepolicy

White House puts the Nvidia H20 export crackdown on hold after a Mar-a-Lago dinner

NPR reported Wednesday that the Trump administration reversed course and put planned export restrictions on Nvidia's H20 — the most capable AI chip US companies may legally sell to China — on hold, after CEO Jensen Huang attended a $1M-a-head dinner at Mar-a-Lago the previous week. Two sources said the controls had been in the works for months and were ready to be implemented as soon as this week; the change of course followed Nvidia promising the administration new US AI data-center investment. The White House and Commerce Department did not return NPR's requests for comment, and Nvidia declined to comment.

The H20 is deliberately de-rated relative to the H100 but still ahead of most Chinese domestic accelerators, and China cannot yet produce the volumes its AI build-out needs — which is why the planned curbs had drawn pressure from Congress. Sens. Elizabeth Warren and Josh Hawley jointly called for H20 export controls in February, after DeepSeek's R1 release stunned the market in January.

White House puts the Nvidia H20 export crackdown on hold after a Mar-a-Lago dinner — TechCrunch

WHY IT MATTERS

Accelerator supply into China just became a week-to-week policy variable negotiated at dinners rather than a settled rule, which means any China-dependent inference capacity in your critical path — directly owned, or via a China-hosted API — carries repricing and availability risk. If you architect around H20-class dense inference for cost reasons, keep a documented swap path to alternative SKUs or vendors, and track export-control changes as a dependency in your capacity plan rather than a news item. Four days later the crackdown landed anyway.

NPR: Trump administration backs off Nvidia H20 chip crackdown after Mar-a-Lago dinner

MON · Apr 7, 2025llama-4benchmarkslmarenaevalsopen-weights

Two days after the open-weight launch, Meta is defending Llama 4's benchmark numbers

TechCrunch reported on Sunday that the Llama 4 Maverick sitting second on LMArena (ELO 1417) was an unreleased experimental chat version tuned for conversationality, not the checkpoint developers can download, and researchers on X documented visibly different behavior between the two — the hosted variant tending toward emojis and long-winded answers. On Monday Meta VP of generative AI Ahmad Al-Dahle denied the accompanying rumor that Meta trained the models on benchmark test sets, calling it simply not true, while conceding that users were seeing mixed quality across the cloud providers hosting Scout and Maverick and that public implementations needed several days to get dialled in. Ars Technica recorded the community verdict from Simon Willison — the vibes around Llama 4 so far are decidedly mid — and noted that using even a fraction of Scout's advertised 10M-token window was already running into memory limits.

The contrast is the story: an open-weight release celebrated for putting frontier-adjacent capability in downloadable form, undercut within 48 hours by a leaderboard score that came from a model nobody can download. Meta's own launch blog had disclosed the ELO figure and described the variant as an experimental chat version, so the disclosure existed — it just did not survive the trip into coverage.

WHY IT MATTERS

Leaderboard positions earned by a withheld, fine-tuned variant do not transfer to the artifact you pull from Hugging Face — evaluate the exact checkpoint and the exact inference stack you will run, not a hosted demo. Dropped as soon as it was ready with implementations still stabilizing is a direct instruction to pin provider, version and sampling config, and to expect quality deltas across Groq, Together, Fireworks and local vLLM. This is also the failure mode to avoid in your own evals: report numbers for the build you ship.

TechCrunch: Meta's benchmarks for its new AI models are a bit misleading TechCrunch: Meta exec denies artificially boosting benchmark scores The Register: Meta accused of Llama 4 bait-and-switch to juice AI benchmark rank Ars Technica: Meta's surprise Llama 4 drop exposes the gap between AI ambition and reality

MON · Apr 7, 2025securityagent-harnesscvelangflowsandboxing

CVE-2025-3248: unauthenticated RCE in Langflow, the visual agent builder

NVD published CVE-2025-3248 on Monday: Langflow versions before 1.3.0 are susceptible to code injection in the /api/v1/validate/code endpoint, letting a remote, unauthenticated attacker send crafted HTTP requests that execute arbitrary code on the server. Horizon3, which found and reported the flaw, published a full technical walkthrough on Wednesday, noting it had chosen to release details early because an exploit was already public. Langflow is an open-source visual builder for agentic AI workflows with 50K+ GitHub stars, backed by DataStax and IBM, and Horizon3's framing is blunt: it offers remote code execution as a feature to authenticated users and ships no sandbox for code execution by design.

The vulnerable endpoint is the code-validation call behind the drag-and-drop editor, meaning the exposed surface is the tooling developers use to assemble agents, reachable before authentication on internet-facing instances. The fix landed in Langflow 1.3.0, and the standard advice applies: upgrade, and do not leave an agent-builder UI on a host that holds credentials or reaches production data.

WHY IT MATTERS

If you build agents, your harness is a more attractive target than your model: code-validation and tool-execution endpoints that eval model-authored Python in-process are pre-auth attack surface, not convenience features. Patch Langflow to 1.3.0 or later, put agent-builders behind authentication and network isolation, and treat the sandbox as a hard requirement — the same pattern applies to any run-this-snippet tool in your stack. Rank this above the week's corporate breach noise, because it is the architecture you are copying.

NVD: CVE-2025-3248 Horizon3: Unsafe at Any Speed — Abusing Python Exec for Unauth RCE in Langflow AI Langflow 1.3.0 release notes

SAT · Apr 5, 2025open-weightsllama-4mixture-of-expertslicensingmultimodal

Meta releases Llama 4: 400B-class open weights at a 17B-per-token inference cost

Meta published Llama 4 Scout and Llama 4 Maverick on Saturday, its first mixture-of-experts and first natively multimodal Llama generation, with checkpoints on llama.com and Hugging Face under the Llama 4 Community License. Scout activates 17B of 109B total parameters across 16 experts with an advertised 10M-token context window and fits on a single H100 under int4 quantization; Maverick activates 17B of 400B total parameters across 128 experts with a 1M-token window and ships in BF16 and FP8, the FP8 build fitting a single H100 DGX host. Llama 4 Behemoth — 288B active, roughly 2T total — was held back as a still-training teacher model, not released. Meta said Meta AI across WhatsApp, Messenger and Instagram was switched to Llama 4 in 40 countries, with multimodal features limited to the US in English.

The license is not open source in the OSI sense: organizations above 700M monthly active users on release day need a separate agreement with Meta, and the rights to the multimodal weights are withheld from individuals domiciled in, and companies with their principal place of business in, the European Union — a carve-out carried over from Llama 3.2 Vision. The Hugging Face model card lists training data of roughly 40T tokens for Scout and 22T for Maverick, with a knowledge cutoff of August 2024. The cost-floor consequence is the point: 400B-class weights served at 17B-class per-token cost, on hardware you can rent by the hour.

WHY IT MATTERS

You can now self-host a 400B-parameter multimodal model on one H100 host, which resets build-versus-buy maths for any team with per-token cost or data-residency constraints — and pressure on closed-frontier API pricing follows. Read the license before you commit: the EU multimodal carve-out and the 700M-MAU threshold are shipping blockers, not paperwork. Treat Scout's 10M-token window as a theoretical ceiling bounded by KV-cache memory, and budget your context strategy accordingly.

Meta AI blog: The Llama 4 herd Hugging Face: Llama-4-Maverick-17B-128E-Instruct model card Llama 4 Acceptable Use Policy (EU multimodal restriction)TechCrunch: Meta releases Llama 4