@GDCPresoReviews "Why don't you just make them bigger then?"
Broadly, they're as big now as they have to be. It's up to several megabytes now in the big GPUs, that's a lot of dedicated memory for this, and they absolutely don't want to grow it by another 4-6x or whatever to deal with crappy unshared verts everywhere.
Plus they'd need ~4-6x the BW in and out as well! It's all bad.
@GDCPresoReviews Broadly, increasing usage by a double-digit percentage because you lose cache hits from resetting the vertex cache every 128 or 256 tris or whatever, when you shade in big batches: you can absorb that.
Absorb "suddenly everything is un-shared": not gonna happen at reasonable cost.
Finally, the thing I've been dodging: it doesn't have to be a single global thing. You can, and people do, split them a bit.
@GDCPresoReviews The thing you see in bigger GPUs is that they are sometimes effectively built as like 2 smaller GPUs in a trenchcoat with a shared command processor and memory interface, and then you sometimes have a setup where there's not a full crossbar and not every vertex shader can generate data used by every fragment shader. Lets you build more, smaller caches (better for BW, contention), limits work redistribution.
@GDCPresoReviews i.e. say you have 64 shader cores total, you might build that in 2 "hemispheres" where batches are cut up very early on runs of something like 128-256 prims or so, get assigned to a partition (with 32 shader cores), and then it just can't leave that partition whatsoever until it all re-joins at the blend/ROP/output stage.
Simpler to build, more scalable, but costs you work efficiency, so a delicate balancing act.
(end infodump!)
@GDCPresoReviews bonus note: if you're looking at AMD HW, you might have noticed that a while back, they added support for making vertex attributes half-floats.
A lot of attrs really don't need to be float32 (e.g. most normals/tangents, some UVs), and making them float16 takes up half the space in the parameter cache, which as noted is often a bottleneck.
(It also has savings elsewhere, but the parameter cache is usually the most significant one.)
@GDCPresoReviews In the other direction, bonus win from mesh shaders:
"Classic" vertex shaders don't know what tri they belong to. Things like back face culling happen _after_ storage for vertices is committed, and at that point you're stuck even if all of the tris referencing a vertex end up culled.
Mesh shaders can cull _before_ allocating output space, and actually remove unreferenced verts entirely.
@rygorous This was really interesting. Thank you Fabian ❤️
@GDCPresoReviews
@rygorous @GDCPresoReviews though the output space allocation strategies also vary by hardware. NVIDIA's current architectures do use a pre allocation (think shared memory) before the mesh shader is run. This is worst case outputs. (And also true for vertex shaders). As result it can limit occupancy.
From that, the surviving subset is then passed through pipeline.
NV doesn't support non 32-bit outputs, but one can use shader barycentrics and interpolate custom packed attributes.
@rygorous @GDCPresoReviews on tiny triangles it can be more efficient to move attribute loads to pixel shader. One can leverage quad thread setup that pixel shader has to distribute attribute shading work across 3 of the 4 quad threads/lanes.
Have experimented with this over here https://github.com/nvpro-samples/gl_vk_meshlet_cadscene too many experiments made some of the code a bit messy
@pixeljetstream @GDCPresoReviews we did some experiments with this in the experimental LRB tess pipe; from underlying dynamics it was clear that below a certain number of quads per tri, the plane equation setup step (which we also had to do on "shader cores" since that was all we had!) wasn't worth the cost.
We tried an alternative interpolation setup for PSOs (well, the internal equivalent) used for high tess where we did full 3D I,J,K barycentrics, K=1-I-J, and interp as
K*v0+I*v1+J*v2
@pixeljetstream @GDCPresoReviews slightly more expensive interp (mul-mad-mad instead of mad-mad, essentially) and the pixel shader had to gather the tri attrs instead of a dedicated setup stage doing it, but a PS was in a better position to hide the gather latency than the dedicated setup did, because it had some reasonable other work to interleave it with.
Dedicated setup was basically all gathers and then just a sub or two, then back to gathers, so constantly waiting on gathers.
@pixeljetstream @GDCPresoReviews it wasn't a huge win, but it helped!
And a fun reminder of how all this stuff is tied together in strange ways.
So it sounds like: if you’re using the GS to:
1. Generate a many-vertex triangle strip for every input primitive
2. And every triangle strip has (roughly) the same number of vertices in it
Then you might be able to get up to half the hit rate of the vertex cache and vertex/parameter buffer utilization/bandwidth as you would get for a traditional fully-connected indexed mesh (because a tri-strip has avg. connectivity = 3, compared to 6)
And if you’re generating 1 triangle for every input triangle, then you’ll have the same problem that tessellation has with low tess factors
And if you’re generating 0, you’re no worse than tessellation, but you are (probably) worse than mesh, because mesh shaders generally (at least) cull entire meshlets, rather than individual triangles, meaning the vertex/parameter buffer you’re leaving empty is 6x the size for GS compared to MS
So it sounds like the key to mesh shaders being more effective is changing the output topology to use indexed meshlets rather than individual triangles or triangle strips.
(And I also think that, once you’re generating meshlets, the only sensible programming model is to have one be generated cooperatively by a whole workgroup, rather than the old single-thread programming model)
That’s an interesting explanation, thank you very much for it.
Hang on, does this mean that you need an index buffer to get good perf, even for traditional draw calls, so the cache can use it as a deduplication key?
Like if I implement an index buffer myself by doing ‘vertex pulling’ in software in the VS, the draw call itself isn’t an indexed draw call, which means I fall off that same 6x perf cliff, right?
@GDCPresoReviews @rygorous See https://schmalstieg.github.io/pdf/Schmalstieg_351.pdf, Revisiting The Vertex Cache: Understanding and Optimizing Vertex Processing on the modern GPU (HPG 2018), for a bunch of experimental measurements and reverse engineering of different GPUs. Apropos, [Giesen 2011] is a reference in there to Fabian's graphics pipeline blog series. :)
❤️
Spoiler alert: After I finish reviewing all the GDC presentations, I'm going to move on to reviewing all the HPG presentations
https://www.youtube.com/@HighPerformanceGraphics/playlists 👀👀👀
@GDCPresoReviews @rygorous Graphics people can confirm or deny, but I believe AMD since the first RDNA generation changed how some things work even for the non-meshlet legacy vertex pipeline; I either heard that in private or from a public presentation. But I don't know of any comparable paper for the post-meshlet era, so you'd have to update it with information from AMD's public docs if you don't want to deal with NDA'd information.
@GDCPresoReviews Normally if you do vertex pulling in the VS you use SV_VertexID. That ID is the unique cache for the formation of vertex shade warps/wavefronts.
Non-indexed draws just auto-generate an incrementing counter here, so yes, every vertex ends up treated as unique, and there is no sharing.
@GDCPresoReviews There is some uniquely cursed behavior here where non-indexed drawing is specified (in the D3D spec) as generating an incrementing index sequence (as if you'd bound a 0 1 2 .... index buffer).
I have confirmed, on actual HW, that if you do a non-indexed draw of a triangle strip while the GPU is in 16-bit index mode (it's a toggle in some register on the HW in question), the triangle containing vertex 65535 is skipped, because that is the Primitive Cut Index for 16-bit inds.
@GDCPresoReviews Presumably, you would see the same thing with 0xffffffff and 32-bit inds, and this is actually required by the D3D spec (yes, I checked when I was implementing this myself), but that one is not actually a problem because a 4-billion-triangle draw would almost certainly just trip a timeout on any actual HW anyhow.
@GDCPresoReviews Anyway, to translate some old-school terminology into what is happening these days:
"Pre-transform vertex cache": identifies duplicate verts by just their indices. Effectively disabled for non-indexed draws. Used when building groups of verts to be shaded.
"Post-transform vertex cache": this is what the position/parameter caches in AMD GPUs are derived from - the storage you assemble primitives out of.
And then there's regular cache for things like vertex data fetches.
@GDCPresoReviews As in, loads you do from vertex buffers and such are just memory accesses and go through your regular shader memory hierarchy.
Vertex data reads in shaders used to have somewhat dedicated HW (the "IA" in D3D terminology) but these days usually don't. Even if you don't declare it, a normal VS is launched with the equivalent of SV_VertexID in some register, and vertex fetches from a VB end up compiling to some variant of `VertexBuf[SV_VertexId]`.
@GDCPresoReviews sometimes there's a bit of "minor magic" there to introduce a layer of indirection for old-school shaders (pre-PSO) where VS code was defined independent from the vertex format/input layout and the same shader could be used with multiple vertex layouts.
I've seen it both ways. Sometimes that remapping HW is still there, some HW does it by compiling the fetch portion into an independent shader fragment with the VS getting a function pointer to it and doing a subroutine call.
@GDCPresoReviews and, of course, in the post-PSO world, you know the input layout when you're compiling the VS so you can just hard-wire it all in, at which point it really does just turn into buffer loads indexed by SV_VertexID (or SV_InstanceID for per-instance data).
@rygorous @GDCPresoReviews NV still has dedicated hardware for vertex attribute fetch to hide latency. Though it may be less impactful these days in theory can schedule vertex warps "later", rather than schedule them as shader, which then immediately wait for their loads and doing so already eats into occupancy. Though because of easier GPU driven rendering doing custom pulling, those benefits tend to outweigh doing it manually.
Hahaha that is amazing
Oct 5, 2026, 12:51 AM@rygorous @GDCPresoReviews Oooh - nice bug! One of those where it's puzzling as hell until you figure out the mechanism.
@GDCPresoReviews GS has a ton of complexity around the whole emit flow. (Likewise, the internal decomposition of hull/domain shaders in D3D into fork/join etc phases is bizarre. Don't know what other GPUs did with it, in the compiler I worked on first thing I did was just flatten it back into a normal control flow, because I had no use for any of it in the form it was presented.)
@GDCPresoReviews If you're generating 0 somewhere, you're probably still bad, because the amount of output space allocated can be proportional to the actual vertex count (asymptotically - often there's some allocation granularity, like groups of 4 or 16 or something), but it's also often just based on the maximum output space estimate you provided. When that's there, it's generally for other reasons, to do with the other Big Problem in GPU design other than data logistics: deadlock avoidance
@GDCPresoReviews Allocating as late as possible uses memory most efficiently but can lead to deadlocks. When you allocate all resources up front before you even allow a job to spawn, you can statically avoid deadlocks (and other types of ordering problems), but waste a lot.
A compromise you see sometimes is that you allocate up front for the worst case (guaranteeing forward progress is possible after) but can realloc to shrink post-launch. (Returning unused space to pools.)
@GDCPresoReviews Yup!
@rygorous Wow, thank you for this thread!
Oct 4, 2026, 8:29 AM@rygorous @GDCPresoReviews Programmers aren't used to worrying about internal register bandwidth, because in many cases the HW architects go to huge lengths to either make "enough" bandwidth, or to hide those bandwidths with various cunning mini-caches and bypasses, but it's hugely important in cases like this, and it's why too many interpolants going from VS to PS can be such a surprising perf hit.