SemiAnalysis

Memory suppliers are finally acknowledging our HBM de-spec call. SK Hynix publicly said 16-hi is difficult, and the market is now prioritizing bandwidth over capacity. Hybrid bonding’s case in HBM was always stack height, and at 4-hi and 8-hi there is no height problem to solve. D2W hybrid bonding for HBM has gone from a question of when to a question of if. (1/3)🧵

Few weeks ago, Samsung is reportedly building D0a, its first sub-10nm-class DRAM node, around a wafer-to-wafer (W2W) hybrid bonding process. That is a device layer process on a different tool set, and commercialization is still roughly two years out. It does nothing for D2W, and it is not a BESI story. (2/3) t.co/H1RSZwHlSA

When does OpenAI launch an agent swarm for a lean proof on their actual annualized recurring revenue?

Big respects big

Open-source models are already taking the easier tasks. The bull case for frontier labs rests on one question. "A lot of software engineering, white-collar work in general, does not need Fable 5.1 or Astra 6 level intelligence." "A lot of businesses, especially lower gross margin businesses, are very rationally making the decision to start offloading some of the easier tasks to these increasingly capable open-source models." "So whether the frontier labs are a durable business model really comes down to your belief on whether the TAM of new things you can do with increasingly smarter frontier intelligence will outgrow the portion of tasks being offloaded to these cheaper models." "If you were to magically spawn 100 million superintelligent PhDs who are experts in every field and don't need to eat or sleep, do you think the economy would be able to absorb that relatively quickly and still get high ROI?"

Vídeo

Get the Full Podcast on Spotify: open.spotify.com/episode/7EedkN…

A $200 AI subscription can be worth $12,000 in tokens. Which plan delivers it is not close. "They're selling to willing buyers at the fair market price. This is capitalism." "You can pay $200 for a Claude plan and get $12,000 of Opus 5.5 tokens at API pricing. So that's obviously a good deal." "You have to say the $200 plan, when running a particular workload on a particular model, gives you some dollar amount. But then you also have 5.5, which is a beast. $6K or $5,000 worth of value." "And when you do that comparison, it's not even a question of who's providing more value. ChatGPT versus Anthropic subscriptions." @maxkan and Andrew Megalaa join @JordanNanos this week

Vídeo

Watch Now: youtu.be/2NviLP2SwZI?si…

Congrats to @AnjneyMidha for launching National Compute! Accessible & subsidized on-demand MI355X with availability for only $3/hr/GPU on Crusoe & Crusoe B300 for $4/hr/GPU. It is so cheap, we are worried that someone might start signing up with tons of .edu emails and start reselling their capacity for higher prices at, like, a 25% margin and ruin it for everyone. (1/2)🧵

@AnjneyMidha From our testing, it seems like a lot of the National Compute Public Research GPUs are located at Crusoe's Denver data center. (2/2)

IREN continues to come up in our conversations with neocloud customers, and usually for the wrong reasons. In Prince George and Mackenzie in Northern BC, Canada, we’ve heard the stories: multi-day power outages, network upgrades, storage failures, air quality controls leading to link flaps, XIDs everywhere. The #1 worst site in the industry, according to users. And we are some of their users, as we’ve gotten to test IREN GPUs from multiple providers that resell their capacity. A lot of our users we are hearing from who aren’t having a good time aren’t renting from IREN resellers, but from IREN themselves. (1/3)🧵

With that said, the new builds in Childress and Sweetwater look much better (read: not N+0 power, cooling, and ISPs!). We’ve covered plenty about these sites in our Datacenter Model; suffice it to say that despite all the technical shortcomings in their cloud services, we expect the $2.1B investment from NVIDIA will help them cut fewer corners this time around. (2/3)

InferenceX, the leading open-source ML systems performance dashboard, now has a GTA theme, in addition to CSGO and Minecraft. What themes should we add next? (1/2)🧵

Check the dashboard out yourself👇️ (2/2) inferencex.com

Every transistor needs power. Traditionally, it comes down from the chip’s frontside: package contact → wide upper wires → finer wires/vias → local rails → transistors. The wires are largely copper, built layer by layer in insulating material alongside signal wiring. (1/4)🧵

Power rails and via stacks crowd routes near the transistors. As wires shrink, resistance and voltage drop rise. Backside power delivery separates the jobs: signals stay frontside, while shorter, wider backside wires bring power to the devices. (2/4)

Optical backplanes are gaining traction and so is the ecosystem surrounding it. Senko showcased the expanded beam backplane connector at CIOE and we estimate that there are ~1500 strands or more of fiber per connector. Optical backplanes will become relevant when hyperscalers and neoclouds eventually implement in-rack optics to connect GPUs and switches within the rack for scale-up. Granted, in-rack optics for scale-up is still rather far out in the roadmap but we think tracking the ecosystem and evaluating its maturity would be indicative of the rate of adoption of optical scale-up.

ClusterMAX Underperform: We couldn’t make much progress in our testing of FarmGPU. The Slurm layer did not advertise GPU resources properly, while Kubernetes did not expose any RDMA devices for scale-out networking. With that said, we very much appreciate FarmGPU’s open development culture, including their solid Grafana monitoring experience and detailed notes on provisioning. We find the technical team to be trustworthy and solid to work with, even if they are stretched too thin just trying to get a few small clusters deployed correctly.

It doesn't make sense to say a subscription plan is worth $ X in isolation. The way to think about subscriptions is that your monthly payment grants you some numbers of “credits”. Each (model, token type) combo consumes a different amount of credits. Because credit cost ratios can differ dramatically from API price ratios, the “value” of the same plan changes depending on what model and workload you’re running. Here’s how the API-equivalent value of the same $200/month Claude plan changes depending on the model and workload. (1/6)🧵

With this nuance, you're able to make much more interesting comparisons across subscriptions. For example, assuming a typical agentic workload, you get similar Fable/Astra limits across the equivalent Anthropic/OpenAI plans. (2/6)

Anthropic Subscriptions Offer 5x+ More Value Than OpenAI Limit testing every AI subscription plan from Anthropic, OpenAI, Meta, SpaceXAI, MiniMax, Moonshot, Zdotai, Cursor, and Cognition newsletter.semianalysis.com/p/anthropic-su…

A classifier maps an input to a fixed set of labels.  The different kinds of classifiers include: 🟠 Binary Classification: email ∈ {spam, not spam} 🟠 Multiclass Classification: text ∈ {positive, neutral, negative} 🟠 Multilabel Classification: movie ⊆ {action, horror, comedy, romance, fantasy, thriller} It works by encoding the input into a vector either through hand-build features like logistic regression or a learned encoder like CNN or BERT.  It then projects that vector to K scores (logits) with a linear layer and applies a softmax/sigmoid function to turn the scores into probabilities and takes the vector with the highest probability. (1/3)🧵

An LLM generates text autoregressively. It predicts the next token, then repeats: 🟠 <start> -> "The" -> "The capital" -> "The capital of" -> "The capital of New" -> "The capital of New York" -> "The capital of New York is" -> "The capital of New York is Albany" -> "The capital of New York is Albany." 1 word = 1 token for simplicity sake but actual tokenizers may split words into multiple sub-word pieces.  The last layer of an LLM is technically a classifier over the vocabulary set but it runs that classification step over and over again.  At each step it outputs a probability distribution over every token and samples from it. This is why output tokens are always 3-5x more expensive than input ones.  Input tokens are processed in parallel.  Output tokens are generated one at a time during decode, which is memory bound because it must re-read all the model weights and KV cache at each step. (2/3)

What four weeks of SemiAnalysis consulting can do to a company “There are a lot of things that I can’t say publicly.” @AbhilashJain17 tells @JordanNanos to discuss neocloud DDs, SLA insurance, 100MW in 6 months, and how client asks became the AI Cloud TCO model.

Vídeo

@AbhilashJain17 @JordanNanos Watch Now: youtu.be/u4qyDSJuYEk

Ever since NVIDIA acquired SLURM/SchedMD, their support for non-NVIDIA chips has gotten worse, such that AMD created their own popular competitor called "spur" that is way better than SLURM in many ways. Remember, NVIDIA claimed to keep SLURM hardware neutral when they acquired SchedMD, but so far that has not happened. Will the same thing happen to @huggingface after NVIDIA acquired them? We shall see... we love Hugging Face and hope we are wrong, but NVIDIA historically has not been hardware neutral at all.

Recently, Alibaba T-Head unveiled the Zhenwu V900 during the Apsara Conference 2026. Zhenwu V900 features 216 GB of memory capacity and 1,200 GB/s of interconnect bandwidth. The V900 is slated to be "3x" the performance of the Zhenwu M890 and will start shipments in Q1 2027. (1/3)🧵

We previously covered the Zhenwu M890 during our WAIC 2026 takeaways, where we took a closer look at the chip's floorplan. The Zhenwu M890 chip features 4 stacks of HBM3E 12-Hi, with 2 compute dies. (2/3)

Meta unveiled Muse Charm at Connect. Tamagotchi for 2026. Our Snapdragon Summit note sees personal AI devices as a new growth market. We expect Snapdragon inside Muse Charm, though Meta has not disclosed the chip. (1/4)🧵

The iPhone is still the device to beat. Humane’s Ai Pin shut down. Rabbit stopped making the R1. Friend faced a backlash. Muse is the next challenger, but it still needs to give you a reason to carry another device. (2/4)

ClusterMAX Participation Medal (Below Bronze): Vultr advertises itself as “the world’s largest privately-held cloud infrastructure company,” which they are not, and “the world’s largest privately held hyperscaler,” which they wouldn’t be even if they were. Still, they are a relevant player in neocloud land, with modern GB300s and MI355X, including a claimed 50 MW AMD site in Ohio. In ClusterMAX 2.0, Vultr’s cluster was delivered with a handful of basic errors. Unfortunately, the pattern is the same almost a year later. (1/3)🧵

The images that Vultr built for our testing had many libraries with applicable CVEs, including CUDA, DCGM, Docker, runc, and ConnectX firmware. Our Slinky login pod did not mount `/shared` and lacked the Pyxis plugstack config, so we had to orchestrate from a worker pod until the Vultr support team reprovisioned the login pod mid-campaign, fixing both issues. There was another basic problem with the storage visible from the K8s layer: the default storage configuration was incompatible with our bare-metal cluster. The default StorageClass, `vultr-block-storage`, provisioned and bound without complaint, then the pod sat in `ContainerCreating` forever. However, the Lustre tier, which came attached to the worker nodes, was excellent, achieving 91.7 GB/s aggregate read across 32 clients. Grafana was configured but only partially correct, as NVLink read 0 due to unconfigured `DCGM_FI_PROF_NVLINK_*` fields. (2/3)

BIG MOMENT FOR AMD🚨THIS WEEK: AMD HAS REACHED ABOVE 90% PARITY ON UPSTREAM vLLM GATING TEST GROUPS! This was after months of hard work from AMD maintainers, specifically Andreas, and also vLLM CI Lead Kevin, along with SemiAnalysis getting vLLM enough CI AMD GPUs after months of upstream maintainer complaints about not having enough AMD GPUs on vLLM and AMD CI fleet-wide stability issues. (1/3)🧵

Gating tests in vLLM mark one of the highest bars of testing in vLLM. That being said, there are still over 11 full test groups that AMD does not yet have parity on, but parity should be expected soon. (2/3)

🚨Custom HBM frees up die space for compute🚨 On Rubin, HBM controllers and PHYs take up roughly 16% of the GPU die. On Feynman with NVHBM, we estimate that falls to ~4%. How: the memory controller moves off the XPU and onto the HBM base die, and the wide standard PHY is replaced by Nvidia's compact NV-HBI die-to-die link. Samsung's custom HBM4 interface is ~60% smaller than its standard PHY. Nvidia claims up to 25% more die area for compute, up to 30% more bandwidth, and 15% lower HBM power vs standard HBM4E. (1/2)🧵

Who else goes custom, who stays standard and why, and what it means for memory vendors and foundries: all in our latest Memory Model note, All About That Base: sales@semianalysis.com (2/2) semianalysis.com/memory-model/

Are GPUs money printers? The InferenceX team breaks down Engram memory offloading, AgentX’s profit calculator, TPU v7, Vera Rubin vs. Blackwell, and whether faster tokens are worth the premium. Plus: TileRT is pushing NVIDIA GPUs further. What does that mean for specialized chips? Jordan Nanos (@JordanNanos) is joined by Cam Quilici (@noslawextratost), Bryan Shan, and Alec Ibarra for a new SemiAnalysis Weekly.

Vídeo

Watch the full conversation: youtube.com/watch?v=NcqHQm…

Overall, the Google GPU Cluster experience is Gold tier! Great work from the Google team, going from Bronze to Gold in less than 2 years! That being said, the GCP GPU experience is less refined than that of our Platinum providers, CoreWeave and Nebius. The console feels like the DMV compared to the streamlined UIs of the younger, more focused neoclouds. Access required a Google Cloud CLI that was occasionally uncooperative, sometimes forcing us to switch to the browser-based connection in Google’s console, which was also unreliable. (1/2)🧵

One thing that is really amazing now about the Google GPU cluster is that NVIDIA and Google have set up auto-activation for Google’s ConnectX-7/8 NCCL plugin. Previously, users would need to mess around with the correct library load paths and env vars to get it set up correctly and optimize performance on ConnectX NICs on GCP NVIDIA GPU machines. We gave feedback a couple of quarters ago, and now it is fully automated! (2/2)

Ruthenium has risen from about $560 an ounce in early 2025 to dealer quotes near $1,950 this September. It is a key material in hard drives, with new use cases also emerging. (1/5)🧵

As chip wiring shrinks, the layers that stop copper from spreading into surrounding materials take up more of the wire. Ruthenium can work without copper's diffusion-barrier stack, and its resistivity rises less sharply as the wire narrows. At widths around 17nm and below, it can compete with copper. It also melts more than 1,000°C above copper and expands less than half as much when heated, reducing the mismatch with silicon. These properties make it a leading candidate to replace copper in the narrowest wires in chips beyond 2nm. Ruthenium already helps copper wiring scale through its use in liners. Researchers have also demonstrated ruthenium-filled nano-TSVs, or through-silicon vias: tiny vertical connections that can deliver power from the back of a chip. Imec has separately demonstrated ruthenium buried power rails. (2/5)

Real estate on the compute die is extremely valuable, yet a surprising share of it is spent just interfacing with memory. We estimate that HBM4 controllers and PHYs take up roughly 16% of Nvidia's Rubin compute die. That is a lot of leading-edge silicon spent talking to memory instead of performing computations. (1/3)🧵 Source: Nvidia

Custom HBM tackles this issue head on. First, the wide standard PHY gets replaced with a compact die-to-die link. Samsung shows that its standard HBM4 PHY occupies 8mm x 4mm, while the custom D2D interface needs just 8.5mm x 1.5mm, roughly 60% less area. The better D2D interface also delivers more bandwidth per unit of interface area and simplifies interposer routing. Second, the memory controller moves off the compute die entirely and onto the HBM base die. Together, these free up a meaningful chunk of the compute die to be handed back for compute or cache. (2/3) Source: Samsung Hot Chips 2026

We criticized Lambda in ClusterMAX 2.0 for their reliability; in this round of testing, it was one of Lambda’s points of emphasis. Their passive health checks cover the relevant conditions and ultimately autoremediated three errors we simulated during testing. Two synthetic XIDs were autoremediated in less than 15 minutes, and there were connected dashboards to monitor node health. Our test that triggered a genuine XID 79 by performing a PCIe secondary bus reset brought the node into `NotReady` with no `GpuXid`, no cordon, and no monitoring visibility on the Kubernetes layer, but it ultimately rejoined the fleet after 2 hours. Our node-reboot tests on the Kubernetes layer also completed without incident. Great work by the Lambda team on the ClusterMAX Sliver!

While Qualcomm announced a month ago that it would implement FlexCache in its Snapdragon 8 Elite Gen 6 processors, Apple has already done the same in their M6 chip without much fanfare. For the first time, Apple combines heterogeneous cores in the same shared L2 cache domain, allowing them to introduce a 3rd core type (branded P but identified as M) without adding a separate M cluster with its own L2.

🚨 IMPORTANT THREAD FOR GPU RENTERS 🚨 GPU cluster reliability is measured when something breaks. During ClusterMAX assessments, we inject failures and follow the path from detection to restored capacity. Providers differ sharply in how well they handle that path. (1/6)🧵

A dashboard should show the failed component, affected jobs, scheduler state, and when each check last ran. A stale green result is not evidence that the cluster is healthy. (2/6)

We assessed GLM-5.3 cyber capabilities on ExploitGym and analyzed the traces. GLM-5.3 spent much of its execution budget testing whether hidden runtime conditions changed its conclusion. In problem arvo5665, GLM-5.3 explored more of the surrounding program through sanitizer builds, corpus tests, and target fuzzing. (1/3)🧵 t.co/YAcLMweMqo

We present our full analysis of GLM-5.3, including cyber capabilities, inference performance, model architecture, and post-training pipeline in our newsletter (2/3) newsletter.semianalysis.com/p/sparse-savin…

AMD Helios (partial co-design): AMD revealed the TH6 scale up switch design publicly for the first time. If you look close enough, you can spot all the Broadcom ethernet retimers next to the flyover cables coming from the backplane into the switch ASIC. This design demonstrates AMD's lack of complete silicon product portfolio, particularly a scale up switch, which is necessary for a scale up rack. In the case of Helios, AMD relies on Broadcom for the scale up switch and retimers alone the scale up path. In the future, AMD are also working with UALink switch vendors in their pursuit of a true UALink scale up system. t.co/pWC888azm6. In our advancing AI 2026 article, we talked about the challenges with these retimers and flyover cables towards manufacturing and signal performance. The retimers also add extra latency alone the scale up path. t.co/4Myxl9Uukb

AI is making papers cheaper to produce. ICLR submissions: 4,938 (2023), 7,262 (2024), 11,603 (2025), 19,525 (2026). Reported 2027 IDs exceed 62K, above roughly 56K paper submissions in all previous years COMBINED. Can reviewers keep up? (1/6)🧵

Why the surge? AI tools make drafting, coding and revision cheaper. The effort of producing a polished paper can speed up; checking whether its claim is new and true still needs scarce expert time. (2/6)

ALERT 🚨🚨OpenAI's latest GPT6.1 Sol Ultrafast is NOT running on Cerebras but is instead running at a low batch size on NVIDIA GPUs. What does this say about Cerebras? Will Cerebras be serving GPT6.1 Sol Ultrafast in the future?

"6.1 Astra"

In Semicon Taiwan, EVG had described the use of Nano-Imprint Lithography (NIL) in the mass manufacture of Photonic Integrated Circuits and for laser manufacturing. NIL is especially useful to define periodic structures within a photonic chip such as grating couplers or microlens arrays for microemitter arrays and potentially for DFB laser gratings. (1/3)🧵

Currently, most photonic chip fabrication uses e-beam lithography. In this process, an electron beam manually scans the wafer to inscribe the individual chip features. Electron beam lithography allows high precision and allows for the flexibility to inscribe arbitrary patterns, at the cost of very low throughput since the ebeam needs to manually scan across the wafer unlike optical lithography which uses a mask. Since, PICs did not traditionally require manufacturing at high-volume, there wasn’t an immediate need to switch over to optical lithography (or nanoimprint lithography) since manufacturing a mask for a low volume chip is impractical. (2/3)

Apple paid $1B for Intel's modem unit and spent 6+ years building the C1 to cut its dependence on Qualcomm. Last week's agreement is a reminder that Apple still can't escape the $2.5B+ in patent royalties it pays Qualcomm every year just for the iPhone's right to work on cellular networks.

The most basic test SemiAnalysis applies to a GPU cluster health check is whether it could possibly work on paper. Several could not. "We've been through a few of these clusters where there's no way it could possibly work. These are health checks so bad they're worse than no health checks, because they're actively interfering with jobs." "On Amazon HyperPod Slurm, the first time we tested, there was a health check that needed the node to be healthy before it could run. It could only run once the node was back in the fleet, and it was needed to bring the node back into the fleet." "There are probably five or ten examples of this throughout testing where the health check never had a chance. It makes you wonder whether these people are really putting it through the paces before they hand it off to customers."

Vídeo

Watch Now: youtu.be/gO7oczGh9qE?si…

Singapore, you won't want to miss this: we're hosting the SemiAnalysis × Nanyang Capital Stock Pitch Challenge. Pitch an AI or semis stock for a shot at S$5,000 in prizes and a fast track into SemiAnalysis. Register by 11 Oct: nanyangcapitalspc.com/register

How GLM5.3 Sparse Attention Affects HBM Memory Usage GLM-5.3, KV Cache Offloading, HiSparse, AgentX TileRT, InferenceX DeepSeek Sparse Attention, IndexShare, Single-rollout Asynchronous Optimization, Cybersecurity newsletter.semianalysis.com/p/sparse-savin…

SpaceX is building Terafab at full speed. Program launched in March, equipment orders placed by April, and concrete is already being poured on site. (1/5)🧵

We believe Terafab Grimes Phase 1 will be built as fast as, if not faster than, TSMC Arizona Fab 21 P1, the fastest US leading-edge fab so far. We expect tool move-in mid-2028 and volume production mid-2030. (2/5)

Micro VCSELs are currently in serious contention for scale-up interconnects. A Vertical Cavity Surface Emitting Laser (VCSEL) always emits perpendicular to the wafer. The device consists of the substrate, and active region with many quantum wells (acting as the gain medium) and an n-doped and p-doped Distributed Bragg Reflector (DBR) mirrors surrounding the active region (acting as a cavity). (1/4)🧵

Conventional datacom microVCSELs are top-emitting, with an emission wavelength at 850 nm, suitable for multimode fibers. Lumentum and Coherent primarily lead in top emitting VCSELs. Light leaves through the p-DBR and it intentionally has less reflectivity to let light exit through the top. The p-metal has to be a ring around the light exit aperture, forcing current to flow sideways into the aperture. This design quirk creates current crowding and extra resistance leading to increased heat generation. These designs usually have thick substrates, however newer device progression trends towards thinner GaAs substrates. This microVCSEL grid is then assembled epitaxial side facing upward and connected through wire bonding. (2/4)