Posted by: eygneph | 2010/02/26

The complexity in shader combination

Shader explosion or shader combination is always a headache to modern game development.

With the power of high level shading languages, people can be more productive to define how a surface should looks like, under a certain lighting environment. But flexibility comes with a price. For example, you have one animation guy who carefully crafted a 28 bones hardware skinning shader, another shadow guy write his stunning penumbra correct soft shadow shader, and you write some tone mapped HDR lighting from image light sources. We’re all happy to see those individual effects shown in FX composer or in Max viewport. But, once you gonna integrate those effects into your editor, the problem appears: some game objects needs receive soft shadow and animation, other game objects needs HDR lighting and animation, and most of them can be affected by arbitrary number of lights, each can be of {directional, spotlight, omni}, and should be affected by fog. So you comes to sooooo many shaders called:

HardwareSkinning_ShadowReceiver_Omni_Fog()
HardwareSkinning_ShadowReceiver_Directional_Fog()
HardwareSkinning_HDRLighting_Spotlight_Fog()
HardwareSkinning_ShadowReceiver_HDRLighting_Spotlight()
MorphAnimation_ShadowReceiver_HDRLighting_Omni()
MorphAnimation_HDRLighting_Directional_Fog()
… etc etc

As you can see, this is an combinatorial  increase in number of shaders. And this is hard to solve just by brute force. Shawn Hargreaves of Microsoft XNA team stated it as an unresolved problem in today’s graphics world. Many people have come to an solution towards this issue, but all has its advantage and disadvantage.

So instead put my naive opinion here, let’s take a look how other people address this issue:

Uber-shader or SuperShader has the idea to put *all* shading techniques in one huge shader, and use static branching/dynamic branching/conditional compilation to rip off the unnecessary code. This is simple for design time and tool development, but comes with an cost at runtime and performance.

Direct3D 11 introduce dynamic shader linkage feature to address this problem. See the Dynamic Shader Linkage 11 sample from DXSDK. Dynamic shader linkage is a bit like “standardlized” #ifdef uber-shader. User declare base material/light shader and derive concrete ones. Client needs to specify concrete material/light shaders on C++ side and put them into the right shader variable via D3D calls. Then the pixel shader just uses the abstract base class to shade the surface. This gives me an impression of dynamic branching(or more specifically dynamic linking + static compilation). That’s why I call it standardlized uber-shader/super shader.

NVLink is an tool trying to assemble shader assembly code to solve the combination problem. I didn’t take too deep into NVLink, because it looks like only support assembly shader code.

ID3DXFragmentLinker is a similar interface to address the issue. Still I didn’t take too much time on it because I read from many source on the web, say it’s not practical for serious development, even XNA doesn’t use it!

Unreal Engine 3 and Mental images are using node based editor(UnrealEd and mental mill) to weave shader graph, and compile into target shader automatically. This is an novel solution but requires a lot of programming and tool efforts, even inventing a new shading language like MetaSL and .usf. BTW 3ds Max 2010 has come up with mental mill artist edition for “free”, and UDK is also available for free download, so check it out.

Shawn has compiled a thorough list of techniques to address the issue. Uber-shader, micro shader, and his approach, using HLSL fragment and code emits.

Yann used an Effect-like system to handle the pass and dependency of individual shaders, but with more finer control, including shader cache and priority shader selection.

Here’re some reference I considered be userful:

Why shader cache/combination important and what’s the problem:
http://www.gamedev.in/showthread.php?p=602
http://blogs.msdn.com/shawnhar/archive/2009/08/17/combining-shaders.aspx

Yann’s approach:
http://www.gamedev.net/community/forums/topic.asp?topic_id=169710

Shawn’s collection and his approach:
http://www.talula.demon.co.uk/hlsl_fragments/hlsl_fragments.html

Autodesk talking about using MetaSL to facilitate game production:
http://www.gamedev.net/columns/events/gdc2009/article.asp?id=1746

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: