OSL Programming: Let’s write a Reoriented Normal Blend map

In this tutorial, we will look at creating a – high quality – Normal Blend map in OSL.
We focus on building an external function and a shader and how to tie it all up and build a small user interface with a couple of practical entries, such as MIX and a switch, pr map.

Youtube video, from scratch to working shader, rendered with Arnold in 3DS MAX 2020.3

Introduction

Sometime ago, I stumbled over an interesting article about various normal blend modes.

Colin Barré-Brisebois and Stephen Hill, the authors of the article, illustrates various technics an artist can use, in the attempts to blend 2 normal maps, while preserving details.

We are going to look at their own proposal – reoriented blending – ( which is a superior blend method compared to almost any other technic ) and write a small OSL utility shader around the function.

The Important part

Copy to Clipboard
  • This is the part we are interested in, from the linked article.
  • HLSL code, and we need to do something about it so it plays nice in OSL.
Copy to Clipboard
  • We changed float3 to vector, since OSL pr default has no concept of the naming of float3.
  • Renamed vector t and u, to a and b.
  • Fixed the wrong syntax in line 5, t.z became a[2], since we renamed t to a and channel call is different. HLSL is .x .y .z osl is [0] [1] [2] in this case.
  • We scrapped the first value in a and b, the tex2D() is HLSL texture calls, we want vector input instead, since our bitmaps can carry through a vector in a very simple and efficient way.
Copy to Clipboard

TIP: If you import larger constructs with some predefine names, you can easily change any OSL type that matches it with a #define newname construct.

Wrapping the content into a function

Copy to Clipboard
  • Next, we need to create a function wrap for the code so it can compile.
  • vector is the output, yes funny, output stands first.
  • We name the function rnm.
  • We add ( ) which will contain the functions inputs
  • We add { } which will contain the functions actual calculations
Copy to Clipboard
  • The code is now wrapped into the vector function called rnm.
  • We inserted the 2 inputs, vector A, vector B into the functions “loader” between ( )
  • We load in A and B and multiply these 2 inputs and add another on line 4 & 5 with static vectors to move the input into a a larger space.
  • Line 6 is where the gold is, dot product of the 2 vectors a and b divided by the hight of a minus b. This is the guts of the shader.
  • Line 8, we pack the guts into a 0-1 space again and return it as a vector.

Adding the shader function

Copy to Clipboard
  • Let’s add an actual shader function to the code beneath our function, we call it RNMNormalBlend
  • We can now compile the shader and the console should return OK.
Copy to Clipboard
  • We add 3 input entries pr map. The color, the switch, the mix value.
  • We also add an output called out, the output is a color and the initial value is 0.
  • Remember to press “7” on the keyboard often, the compiler should return OK at all times by now.
Copy to Clipboard
  • Adding in Meta data on the switch and the Mix for both input maps.
  • We also add a shader meta data area with a custom category placement and shader UI text header.

Done and Done, what now?

OK, so we now have a function and we build a user interface.
Now we want to connect everything. Lets move to the shader functions body and try set up a system that uses all the instruments we just made.
Copy to Clipboard
  • See line 45. We change the output Out so it picks up the function called rnm and sticks our 2 bitmaps through it.
  • If we compile we will see the normal map working.
Copy to Clipboard
  • Let’s start to integrate the switch check box.
  • We can make a small “if something happens then do one thing else do something else.”
Copy to Clipboard
  • We need to create a base Normal Color for when the switch is off
  • We also need to create 2 new color channels because we need to do a selection between the map or the static base normal
  • This is done by the check on the Enable_NormalA condition set to either 0 or 1
  • We then set pipeA and pipeB to either one of the 2 inputs
  • Finally we make sure that out output in line 32 is changed to recive input from pipeA and pipeB instead of NormalA and NormalB which were just static bitmap hooks with no analytic connection to anything.
Copy to Clipboard
  • Integrate the MixA and MixB float UI entry, by using the mix() feature
  • Notice that there is a 1- in front of the Mix names, this is because pr default that a value of 1 would make the map disapear and a value of 0 would print it full effect. Using this little trick with a 1- infront of a value inside the range of 0-1 will flip it around.
Copy to Clipboard
  • The final constuct.
  • Notice there is another switcher called FlipOrder, it flips the Maps on/off states around if you want to check 1 then the other then back, then this is the switch to use.