Graphics Blitting in Unity Part 1

Gas Giant Planet Shader Using Ping Pong Buffers

Gas Giant Web Player

A very powerful feature in Unity is the ability to blit or render a new texture from an existing set of texture using a custom shader.  This is how many post process effects are done such as bloom, screen space ambient occlusion, and god rays.  There are many more uses for blitting than just post process effects and I'm going to go over a few of them in the next few posts starting with uses for standard separable blur and moving on to ping pong buffers to create cool effects like the gas giant planet shown above.

First example is a separable blur shader and how to use it to put a glow around a character icon.  This idea came from some one in the Unity3D sub-reddit who was looking for a way of automating glows around their character icons.  Get the package here!

So separable blur, separable means that the blurring is separated into 2 passes, horizontal and vertical.  It's 2 passes but we can use the same shader for both passes by telling it what direction to blur the image in.

First lets have a look at the shader.

Shader "Hidden/SeperableBlur" {
Properties {
_MainTex ("Base (RGB)", 2D) = "black" {}
}

CGINCLUDE

#include "UnityCG.cginc"
#pragma glsl

Starts out pretty simple, only one exposed property, but wait, whats that CGINCLUDE?  And no Subshader or Pass?  CGINCLUDE is used instead of CGPROGRAM when you want to have a shader that can do lots of things.  You make the include section first and then put your subshader section below with multiple passes that reference the vertex and fragment programs you write in the the include section.

struct v2f {
float4 pos : POSITION;
float2 uv : TEXCOORD0;
};

We don't need to pass much information the the fragment shader.  Position and uv is all we need.

//Common Vertex Shader
v2f vert( appdata_img v )
{
v2f o;
o.pos = mul (UNITY_MATRIX_MVP, v.vertex);
o.uv = v.texcoord.xy;
return o;
}

appdata_img is defined in UnityCG.cginc as being the standard information that you need for blitting stuff.  The rest is straight forward, just pass the uv to the fragment.

half4 frag(v2f IN) : COLOR
{
half2 ScreenUV = IN.uv;

float2 blurDir = _BlurDir.xy;
float2 pixelSize = float2( 1.0 / _SizeX, 1.0 / _SizeY );

I know it says half4 but the textures we are using only store 1 channel.  Save the UV's to a variable for convenience.  Same with the blurDir (blur direction), I'll talk about this more later but this variable is passed in from script.  And pixelSize is the normalized size of a pixel in uv space.  The size of the image is passed to the shader from script as well.

float4 Scene = tex2D( _MainTex, ScreenUV ) * 0.1438749;

Scene += tex2D( _MainTex, ScreenUV + ( blurDir * pixelSize * _BlurSpread ) ) * 0.1367508;
Scene += tex2D( _MainTex, ScreenUV + ( blurDir * pixelSize * 2.0 * _BlurSpread ) ) * 0.1167897;
Scene += tex2D( _MainTex, ScreenUV + ( blurDir * pixelSize * 3.0 * _BlurSpread ) ) * 0.08794503;
Scene += tex2D( _MainTex, ScreenUV + ( blurDir * pixelSize * 4.0 * _BlurSpread ) ) * 0.05592986;
Scene += tex2D( _MainTex, ScreenUV + ( blurDir * pixelSize * 5.0 * _BlurSpread ) ) * 0.02708518;
Scene += tex2D( _MainTex, ScreenUV + ( blurDir * pixelSize * 6.0 * _BlurSpread ) ) * 0.007124048;

Scene += tex2D( _MainTex, ScreenUV - ( blurDir * pixelSize * _BlurSpread ) ) * 0.1367508;
Scene += tex2D( _MainTex, ScreenUV - ( blurDir * pixelSize * 2.0 * _BlurSpread ) ) * 0.1167897;
Scene += tex2D( _MainTex, ScreenUV - ( blurDir * pixelSize * 3.0 * _BlurSpread ) ) * 0.08794503;
Scene += tex2D( _MainTex, ScreenUV - ( blurDir * pixelSize * 4.0 * _BlurSpread ) ) * 0.05592986;
Scene += tex2D( _MainTex, ScreenUV - ( blurDir * pixelSize * 5.0 * _BlurSpread ) ) * 0.02708518;
Scene += tex2D( _MainTex, ScreenUV - ( blurDir * pixelSize * 6.0 * _BlurSpread ) ) * 0.007124048;

Ohhhhh Jesus, Look at all that stuff.  This is a 13 tap blur, that means we will sample the source image 13 times, weight each sample (that long number on the end of each line), and add the result of all the samples together.  Lets just deconstruct one of these lines:

Scene += tex2D( _MainTex, ScreenUV + ( blurDir * pixelSize * 4.0 * _BlurSpread ) ) * 0.05592986;

Sample the _MainTex using the uv's but then add the blur direction (either (0,1) horizontal or (1,0) vertical ), multiplied by the size of one of the pixels, multiplied by how many pixels over we are (in this case it's the 4th tap over), multiplied by an overarching blur spread variable to change the tightness of the blur.  Then multiply the sampled texture by a gaussian distribution weight (in this case 0.05592986).  Think of gaussian distribution like a bell curve with more weight being given to values closer to the center.  If you add up all the numbers on the end they will come out to 1.007124136, or pretty darn close to 1.  You will notice that half of the samples add the blur direction and half subract the blur direction.  This is because we are sampling left AND right of the center pixel.

Scene *= _ChannelWeight;
float final = Scene.x + Scene.y + Scene.z + Scene.w;

return float4( final,0,0,0 );

now we multiply the final value by the _ChannelWeight variable which is passed in from script to isolate the channel we want.  Add each channel together and return it in the first channel of a float4, the rest of the channels don't matter because the render target will only be one channel.

Subshader {

ZTest Off
Cull Off
ZWrite Off
Fog { Mode off }

//Pass 0 Blur
Pass
{
Name "Blur"

CGPROGRAM
#pragma fragmentoption ARB_precision_hint_fastest
#pragma vertex vert
#pragma fragment frag
ENDCG
}
}

After the include portion goes the subshader and passes.  In each pass you just need to tell it what vertex and fragment program to use.  #pragma fragmentoption ARB_precision_hint_fastest us used to automatically optimize the shader if possible... or something like that.

Now lets check out the IconGlow.cs script.

public Texture icon;
public RenderTexture iconGlowPing;
public RenderTexture iconGlowPong;

private Material blitMaterial;

We are going to need a texture for the icon, and 2 render textures; one the hold the horizontal blur result and one to hold the vertical blur result.  I've made the 2 render textures public just so they can be viewed from the inspector. blitMaterial is going to be the material we create that uses the blur shader.

Next check out the start function.  This is where everything happens.

blitMaterial = new Material (Shader.Find ("Hidden/SeperableBlur"));

This makes a new material that uses the shader named "Hidden/SeperableBlur".  Make sure you don't have another shader with the same name, it can cause some headaches.

int width = icon.width / 2;
int height = icon.height / 2;

The resolution of the blurred image doesn't need to be as high as the original icon.  I'm going to knock it down by a factor of 2.  This will make the blurred images take up 1/4 the amount of memory which is important because render textures aren't compressed like imported textures.

iconGlowPing = new RenderTexture( width, height, 0 );
iconGlowPing.format = RenderTextureFormat.R8;
iconGlowPing.wrapMode = TextureWrapMode.Clamp;

iconGlowPong = new RenderTexture( width, height, 0 );
iconGlowPong.format = RenderTextureFormat.R8;
iconGlowPong.wrapMode = TextureWrapMode.Clamp;

Now we create the render textures.  Width, Height, and the 0 for the number of bits to use for the depth buffer; the textures don't need a depth buffer, hence the 0.  Setting the format to R8 means the texture will be a single channel (R/red) and 8 bits or 256 color grey scale image.  The memory footprint of these images clock in at double the footprint of a DXT compressed full color image of the same size, so it's important to consider the size of the image when working with render textures.  Setting the wrap mode to clap ensures that pixels from the left side of the image don't bleed into the right and vice versa, same with top to bottom.

blitMaterial.SetFloat ("_SizeX", width);
blitMaterial.SetFloat ("_SizeY", height);
blitMaterial.SetFloat ("_BlurSpread", 1.0f);

Now we start setting material values.  These are variables we will have access to in the shader.  _SizeX and Y should be self explanatory.  The shader needs to know how big the image is to precisely sample the next pixel over.  _BlurSpread will be used to scale how far the image is blurred, setting it smaller will yield a tighter blur and setting it larger will blur the image more but will also introduce artifacts at too high a value.

blitMaterial.SetVector ("_ChannelWeight", new Vector4 (0,0,0,1));
blitMaterial.SetVector ("_BlurDir", new Vector4 (0,1,0,0));
Graphics.Blit (icon, iconGlowPing, blitMaterial, 0 );

The next 2 variables being set are specific to the first pass.  _ChannelWeight is like selecting which channel you want to output.  Since we are using the shame shader for both passes we need a way to specify which channel we want to return in the end.  I'm setting it to the alpha channel for the first pass because we want to blur the character icon's alpha.  _BlurDir is where the "separable" part comes in.  think of it like the channel weight but for directions, here the direction is weighted for vertical blur because the first value is 0 (X) and the second value is 1 (Y).  The last 2 numbers aren't used.

Finally it's time to blit an image.  icon being the first variable passed in will automatically be mapped to the _MainTex variable in the shader.  iconGlowPing is the texture where we want to store the result.  blitMaterial is the material with the shader we are using to do the work, and 0 is the pass to use in said shader; 0 is the first pass.  This shader only has one pass but blit shaders can have many passes to break up large amounts of work or to pre-process data for other passes.

blitMaterial.SetVector ("_ChannelWeight", new Vector4 (1,0,0,0));
blitMaterial.SetVector ("_BlurDir", new Vector4(1,0,0,0));
Graphics.Blit (iconGlowPing, iconGlowPong, blitMaterial, 0 );

Now the vertical blur is done and saved!  We now need to change the _ChannelWeight to use the first/red channel.  The render textures we are using only store the red channel so the alpha from the icon image is now in the red channel of iconGlowPing.  We also need to change the _BlurDir variable to horizontal; 1 (X) and 0 (Y).  Now we take the vertically blurred image and blur it horizontally, saving it to iconGlowPong.

Material thisMaterial = this.GetComponent<Renderer>().sharedMaterial;
thisMaterial.SetTexture ("_GlowTex", iconGlowPong);

Now we just get the material that is rendering the icon and tell it about the blurred image we just made.

Finally let's look at the shader the icon actually uses.

half4 frag ( v2f IN ) : COLOR {

half4 icon = tex2D (_MainTex, IN.uv) * _Color;
half glow = tex2D (_GlowTex, IN.uv).x;
glow = saturate( glow * _GlowAlpha );

icon.xyz = lerp( _GlowColor.xyz, icon.xyz, icon.w );
icon.w = saturate( icon.w + glow * _GlowColor.w );

return icon;
}

This is a pretty simple shader and I have gone over some other shaders before so I am skipping right to the meat of it.  Look up the main texture and tint it with a color if you want.  look up the red (only) channel of the glow texture.  Multiply the glow by an parameter to expand it out and saturate it so no values go out of 0-1 range.  Lerp the color of the icon with the color of the glow (set by a parameter)  using the icons alpha as a mask.  This will put the glow color "behind" the icon.  Add the glow (multiplied by the alpha of the _GlowColor parameter) to the alpha of the icon and saturate the result.  Outputting alpha values outside of 0-1 range will have weird effects when using hdr.  And that's it!  There is a nice glow with the intensity and color of your choice around the icon.  To actually use this shader in a gui menu you should probably copy one of the built in gui shaders and extend that.

Now that this simple primer is out of the way the next post will be about ping pong buffers!