Running Kimi K3 on MI355X costs less per token than running it on Nvidia's B300, even though the B300 pushes more raw throughput. AMD's chip wins on performance per dollar because it's roughly 2.4x cheaper per GPU-hour, and that price gap outruns the B300's speed advantage.
Short answer: Running Kimi K3 on MI355X beats B300 on performance per dollar — about 48 tok/s per $/GPU-hour on MI355X versus 33 tok/s on B300, per Wafer AI's July 31, 2026 benchmark. B300 still wins on raw throughput (up to 1,568 tok/s aggregate versus 952 on MI355X), but MI355X's $2.50/hr pricing against B300's $6.00/hr makes it the cheaper way to serve Moonshot AI's 2.8-trillion-parameter model.
Last updated August 2, 2026.
Kimi K3 is Moonshot AI's 2.8-trillion-parameter mixture-of-experts model, and it's big enough that the hardware bill is the whole story. Loading it needs over 1.5TB of VRAM before you've allocated a single token of KV cache for long context, which rules out anything smaller than a full 8-GPU node. When I ran the cost-per-token math from Wafer AI's numbers myself, the MI355X's advantage held up: it's not close on raw speed, but it's not supposed to be — it's a value play, the same way a Honda Civic beats a Ferrari on cost per mile without pretending to beat it on the track.
What you'll need
An 8-GPU node of either chip — MI355X or B300 — since Kimi K3's memory footprint doesn't fit on fewer. Both chips ship with 288GB of VRAM per GPU, so the node math is the same either way; a B200 node needs 16 GPUs across two nodes to hit the same capacity, since each B200 only carries 192GB. You'll also want vLLM (day-0 support landed July 27, 2026) or another inference server with FP4/MXFP4 support, since that's the precision both benchmarks below ran at. Rented capacity from a GPU cloud is the realistic path for almost everyone — buying either chip outright isn't something most teams do for one model.
Step-by-step: running Kimi K3 on MI355X
1. Rent an 8x MI355X node
Look for a GPU cloud offering AMD Instinct MI355X instances with all 8 GPUs on one node — that's the minimum to hold Kimi K3's weights plus working KV cache. Confirm the listed price is close to the $2.50 per GPU-hour figure Wafer AI benchmarked; markups vary a lot by provider.
2. Pull vLLM with Kimi K3 support
Use a recent vLLM Docker image — the vLLM team's own post flags that Kimi K3 support currently ships in Docker only, because of pre-release dependencies. Trying to build it from a plain pip install is the fastest way to lose an afternoon.
3. Turn on prefix caching explicitly
Kimi K3 doesn't get this by default. Add --enable-prefix-caching to your vLLM launch flags, or you're leaving repeated-prompt performance on the table for no reason.
4. Enable speculative decoding if you want single-user speed
Baseline single-stream decode on vLLM is around 118 tok/s. Turning on DSpark speculative decoding pushes that toward 370 tok/s for a single user — about 3.1x faster — using a launch flag like --speculative-config '{"model":"Inferact/Kimi-K3-DSpark","method":"dspark"}'. This matters more for a chat-style single user than for a busy multi-tenant server.
5. Set the AMD-specific environment variables
VLLM_USE_RUST_FRONTEND=1 and VLLM_ALLREDUCE_USE_FLASHINFER=1 are the two flags called out in vLLM's own K3 launch notes. Skipping them doesn't break the deployment, but it leaves throughput below what the benchmark numbers assume.
6. Load-test at your real concurrency, not just one user
Wafer AI's numbers move a lot with concurrency: at low concurrency MI355X's tok/s-per-dollar lead is largest, and it narrows as more simultaneous users pile onto the node. Test at the user count you'll actually run in production before you commit to a chip.
Example prompts you can copy
These are useful once Kimi K3 is up and you're checking that the deployment behaves the way the model card says it should — not benchmark prompts, just sanity checks:
Summarize this 40,000-word document in 5 bullet points, then list any numbers that appear more than once.— a long-context check, since Kimi K3 supports up to a 1M-token window.Here is a 300-line Python file. Find the bug that causes the off-by-one error and explain your reasoning before showing the fix.— checks reasoning-before-answer behavior on a coding task.Describe what's happening in this image, then write three follow-up questions a user might ask about it.— tests the native vision support Moonshot AI built into K3.Plan a 4-step research task, then execute the first step and report what you'd need to continue.— a light agentic check, since this is the kind of workload the MI355X-vs-B300 comparison is actually about serving at scale.
Common mistakes to avoid
Treating this as a "which chip is faster" decision instead of a "which chip is cheaper for my traffic pattern" one is the mistake I'd flag first — B300 wins on raw throughput at every concurrency level Wafer AI tested, so if your bottleneck is p99 latency for a small number of demanding users, the "better performance per dollar" framing doesn't automatically apply to you. Skipping the Docker requirement is a close second: pre-release dependencies mean a plain pip install of vLLM won't have working Kimi K3 support yet, and you'll burn hours chasing an import error that a container would have avoided. Forgetting --enable-prefix-caching is easy to miss because most other models in vLLM cache prefixes by default — Kimi K3 doesn't, and running without it quietly costs you the repeated-prompt savings the whole cost comparison assumes you're getting. Sizing for one B200-class node instead of two is another real trap, since B200's 192GB per GPU means you need 16 of them across two nodes to match what a single 8x MI355X or 8x B300 node holds. And comparing sticker GPU-hour prices without checking whether your specific cloud provider's markup holds up is the last one — Wafer AI's $2.50/$4.25/$6.00 figures are a snapshot from one benchmark, not a guaranteed rate from every reseller.
Tools that make this easier
Renting raw GPU nodes and hand-tuning vLLM flags is the DIY end of the spectrum; my guide to open-weight AI’s Kubernetes moment covers the managed layer above it — tools like Ollama and hosted inference platforms that trade some cost efficiency for not having to babysit prefix-caching flags yourself. If you're weighing Kimi K3 against other open-weight options on price and benchmark scores rather than raw hardware cost, my DeepSeek V4 Flash 0731 analysis and best AI models roundup are the next stops. Kimi K3 also has a safety dimension worth knowing before you deploy it at scale — see the UK AISI/CAISI assessment of Kimi K3’s cyber capabilities for what regulators actually tested. If the GPU-hour math in this piece makes AI infrastructure feel expensive regardless of which chip you pick, why AI is getting way too expensive covers the broader cost trend. And for the opposite extreme of hardware — proof that not everything needs a data-center node — running a 28.9M-parameter LLM on an $8 microcontroller is a fun contrast to the 1.5TB-of-VRAM reality of Kimi K3.
How the three chips compare running Kimi K3
| GPU | VRAM per GPU | Price per GPU-hour | Peak aggregate throughput (node) | Throughput per $/GPU-hour |
|---|---|---|---|---|
| AMD MI355X | 288GB | $2.50 | 952 tok/s | ~48 tok/s |
| Nvidia B300 | 288GB | $6.00 | 1,568 tok/s | ~33 tok/s |
| Nvidia B200 | 192GB | $4.25 | 498 tok/s (2-node, TP16) | ~7 tok/s |
Source: Wafer AI’s Kimi K3 benchmark, published July 31, 2026, using an 8x MI355X (TP8) node against a B300 (TP8+DCP8) node and a 16x B200 (TP16, 2-node) configuration, 1,024-token input / 400-token output. B200 needs twice the GPU count to match the memory of a single MI355X or B300 node, which is most of why its per-dollar number lags so far behind the other two.
MI355X is the pick if you're optimizing for cost per token served at scale. B300 is the pick if raw single-node throughput or lowest possible latency matters more than the bill — it moves 1.65x more aggregate tokens per node than MI355X, just at 2.4x the hourly price. Neither is a bad choice; they're built for different constraints.
Frequently Asked Questions
Is running Kimi K3 on MI355X free?
No. You're renting GPU-hours from a cloud provider — Wafer AI's benchmark used $2.50 per GPU-hour for MI355X, so an 8-GPU node runs about $20/hour before any provider markup, plus storage and networking. Kimi K3's weights themselves are open, so there's no model licensing fee on top of the compute.
How long does it take to get Kimi K3 running on MI355X?
If you're using a pre-built vLLM Docker image with the right flags, expect an hour or two from renting the node to a working endpoint, most of it spent pulling the 2.8-trillion-parameter weights. Building support from scratch instead of using the Docker image the vLLM team ships can turn that into a day or more.
What's the easiest way to compare MI355X and B300 for my own workload?
Start from Wafer AI's published numbers as a baseline, then load-test both at the concurrency you actually expect — the cost advantage for MI355X is largest at low concurrency and narrows as more users share the node, so your traffic pattern matters more than the headline number.
Do I need a different setup for B300 versus MI355X?
Mostly the same vLLM flags apply to both, but the parallelism strategy differs — Wafer AI's benchmark ran MI355X at plain tensor-parallel-8 and B300 at tensor-parallel-8 plus data-parallel-8, so don't assume a config tuned for one chip is optimal on the other without retesting.
Does Kimi K3 run on Nvidia B200 instead?
Yes, but expect to double your GPU count — B200's 192GB per GPU means you need 16 GPUs across two nodes to hold what a single 8-GPU MI355X or B300 node holds, and Wafer AI's benchmark found that configuration the worst of the three on cost per token.