Updated 6/6/99 The fiftyfifth anniversary of D-Day

Things the RAY DREAM Manual Doesn't Tell You

USING A MASK IN SHADER CHANNELS

INTRODUCTION

This is something a little bit different...

The Ray Dream Manual actually does provide the hint necessary to solve the central problem of this writeup. But it doesn't provide any concrete examples or procedures and you won't find the subject referenced in the index. So, the hint and its application would probably be obscure to all but the more experienced user or most diligent reader of the manual.

THE PROBLEM

While working on my B17 model, a number of comments suggested the addition of nose art--the often carnal illustrations which lonely, and very brave, young combat crew would paint onto the fuselage of their aircraft during WWII.

This seemed straightforward--simply apply the nose art as a paint shape over the existing fuselage shader.

The fuselage shader, itself, contains data in the color, highlight, shininess, bump and reflection channels.

For the nose art, I created a texture map in my favorite bitmap editor, Paint Shop Pro (PSP). It consisted of text and a figure on a white background, solely in the color channel.

Despite having checked "white is Invisible," the first effort produced an oddly tinted border around the central figure and text. As can be seen in the first illustration (below), the background to the artwork may be transparent, but is obviously not invisible:

Illustration 1
Illustration 2

This was not a problem of the background being other than pure white--the RGB values were uniformly 255. [Go here if you want to know why it won't work.]

I then tried matching the original fuselage color and using that as the background color of the nose art texture map. This was fairly successful (see Illustration 2, above). But, Adrian van der Park pointed out that the figure and text appeared washed-out.

He was right, of course, and the explanation can be found on page 183 in the RD5 manual. The paint shape used only the color channel. The data in the underlying fuselage shader's remaining channels was being applied to the entire nose art paint shape--the highlight, shininess and reflection values of the fuselage shader were effecting the nose art's figure and text.

Attempts to reproduce the characteristics of the other channels within the paint shape were unsuccessful. Painting the nose art directly onto the fuselage was not an option; I was interested in finding a more elegant solution.

THE SOLUTION

What was needed was some way of simulating a mask--blocking the underlying highlight, shininess and reflection channels within the area of the figure and text, while allowing them to show within the background.

It was a short reference within the Ray Dream Handbook that led to the ultimate solution. Rick Greco's "Use Image Maps as Mask" on pp. 201-2, suggests that texture maps can be used within the Mix operator, but provides only a simple illustration and no elaboration. After looking at that illustration repeatedly over the course of a couple days, it dawned on me what was happening.

It was only after finishing the project that I stumbled across the critical information in the RD5 Manual which confirmed what was implied in that illustration: The white areas of a texture map used as a function within the Mix operator pick up the left subshader data, while the black areas of the map pick up the right subshader's data (RD5 Manual, p. 211).

Using this feature, it is possible to simulate the effects of a mask.

WHAT A MASK DOES

A texture map in the Mix operator let's you assign different values to different areas within one shader channel. Of course, you could use a grayscale map all by itself within a shader channel, but if you want the values to be anything more than all on or all off it would be nearly impossible to achieve precise values.

Let's say you have a decal (paint shape) with an irregularly shaped illustration that you want to apply to a surface with a high reflectivity value, such as a metal. Like a decal for the plastic models you used to play with, the paint shape is rectangular overall and you've got two areas within the paint shape: the illustration (the printed part), and the area outside the illustration (the plastic substrate on which it's printed).

The illustration needs a very low reflectivity, because you want it to look like a painted surface.

The area outside the illustration should have a higher reflectivity, because in order for it to disappear it will have to match the metal surface on which the decal is to be applied. You will, in addition, want this area outside the illustration to be transparent. (Think about it: Using the Transparency channel alone won't make it disappear because you still have to make the Reflectivity, Shininess and Highlight of both the metal and the unpainted part of the decal match. To illustrate: Glass is transparent, but the sun reflecting off it would make you notice a piece of it lying on a slab of concrete.)

The function of the map in the Mix operator is to allow you to specify two different areas of reflectivity within the one paint shape: one high to match the metal surface, one low to simulate a painted surface.

The next illustration shows the use of a map in the Mix operator within the Reflection channel, to produce the decal-on-metal under discussion.

Remember: The white areas of a texture map used as a function within the Mix operator pick up the left subshader data, while the black areas pick up the right.

In this illustration, the white area of the map in the Function box causes the data in the left subshader--set to a value of 10 percent (which is the reflection value of the metal shader)--to be assigned to the unpainted areas of the decal. The black areas of the map are assigned the value from the right subshader. [Note: I used black in the right subshader just to experiment. I could just as easily have used a value slider set to zero. RDS interprets black as zero in this function].

The result is that the illustration has no reflectivity, and the unpainted part will exactly match the underlying metal surface.

porno pic The result, when combined with appropriate maps within the Highlight, Shininess and Transparency channels, and the illustration itself in the Color channel, is a decal that merges perfectly with the object to which it is applied.

The easiest way to explain the procedure is by example:

PROCEDURE

NOTE: The following Steps 1 through 41 produce a rendering with a washed-out paint shape image. This section is included to demonstrate the problem and provide instruction in basic techniques to new users. For more experienced users, I recommend reviewing Steps 20 through 28, and then beginning the procedure at Step 42.

Create the Color and Mask Texture Maps

You'll need two maps. You can either copy the following two simple illustrations, or create maps of your own. If you want to copy these, right-click and Save As...*.bmp files (These are .gif files).

If you're creating your own maps, make one color and the other white and black. You can use any color as the background, but white is usually efficient. In your bitmap editor, create the mask from the color map, using whatever means is most appropriate. I used the Create Mask function within PSP, and then went back and saved the resulting "*.msk" as a .bmp file, but you could just as easily select the color portion and use a paint bucket tool to fill it with black.

Color Map
Mask Map

Create the Object to be Shaded

We'll need a rectangular object to shade. If you know how to create a Free Form object in RD5, you can skip to Step 12, and we'll continue in the Perspective Window.

  1. Start Ray Dream Studio.
  2. Select Create Empty Scene. The Perspective Window will appear.
  3. Drag-and-drop the Free Form icon into the Perspective Window.
  4. In the Set Name window, type 'Mask' or any descriptor.
  5. Select View/Preset Position/Front.
  6. On the drawing plane, use the Draw Rectangle tool to create a rectangle of the desired sizeómake it relatively large, so that it will be easy to work with later.
  7. Select Sections/Center.
  8. Select View/Preset Position/Left.
  9. Click on the Selection Tool, then the sweep path, and shorten the sweep path to something reasonable.
  10. Select Edit/Jump Out. You will be back in the Perspective Window.
  11. Select Arrange/Send to Origin. Then use the Zoom tool to view the rectangle object. You may also want to select Windows/Properties/Transform/Size & Scaling/% Overall to make the object a workable size.

    Set the Lighting

  12. Move the light in the scene to a position somewhere in front of the object. Use the Left and Right views to center the light in front of the object.
  13. Select the Light, then Shift-click on the rectangle object to select it, also.
  14. Select Arrange/Point At, and the object should show some evidence of the light reflecting off its surface.

  15. Double-click on the light and the Properties window should open.
  16. Select the Light tab and set the Brightness to 100% and the Half Angle to 90%. If Auto is not checked, press Apply.

    Apply the Base Shader

  17. Select Better Preview Quality.
  18. Select Windows/Browser/Shaders.
  19. In the Shaders Browser, go to the Metals1 directory and then drag-and-drop the Chrome shader onto the object in the Perspective window. (We could use any shader. The chrome shader just happens to have data in several channels that give obvious effects.) Leave the Browser open.

    Record the Surface Settings

  20. Select Windows/Current Shader Editor. Make sure the Auto box is un-checked. If the chrome shader data is showing in the channels, fine. If it isn't, select the Eyedropper tool and click on the shaded object.
  21. Go to the Highlight, Shininess and Reflection channels and write down the values of each of the sliders. Put that information aside for later use.

    Empty the Shader Channels

  22. When you've done that, go to all active channels (including the Color tab) and empty them of data by clicking on the functions (including the color chip) and pressing the Delete key. The words "Unmodified Channel -- Drop your shader here" should appear in each channel. A red sphere should be visible in the Shader Preview window (top left corner of the Current Shader Editor).

    Load and Save the Mask

    There's a trick to loading shader data into operator functions, which we'll be doing in a minute. In the Handbook, Richard Bucci wrote a section titled "Use the Browser to Copy Channels," p. 198. The following three steps use his suggestion:

  23. In the Current Shader Editor, select View/Flat Preview.
  24. Click on the Highlight tab. Select Insert/Texture Map, and then browse until you locate the mask .bmp map created above. The image selected should appear in the Shader Preview and in the Highlight channel.
  25. Drag the Shader Preview from the Current Shader Editor to the Browser and drop it in any convenient directory. You will be prompted for a name; call it "testmask."

    Load the Color Texture Map

  26. Back in the Current Shader Editor, click on the Highlight tab, again. Click on and delete the mask map inserted in Step 24 (above).

  27. Click on the Color tab. Select Insert/Texture Map, and then browse until you locate the color .bmp map you created at the outset. The image selected should appear in the Shader Preview and in the Color tab.
  28. Check "white is Invisible."

    Apply the Paint Shape

  29. If you're not there already, select View/Preset Position Front
  30. Select the Rectangular Paint Shape tool.
  31. Go to the object, start near the top left corner and click-and-drag the mouse to near the lower right of the object. The paint shape rectangle should follow your cursor, but don't be concerned if no detail appears within it. Remember that this is a highly reflective object, and we're looking at it straight on.
  32. Select View/Preset Position/Reference. After a couple seconds of drawing, the shader should appear on the object, faintly.

  33. Select View/Production Frame, and resize the frame to include most of the rectangle.

    Your screen should now appear something like this:

    Prior to Render

    Render the First Picture

  34. Select Render/Current Scene Settings.
  35. In the Scene Settings Window, click on the Renderer tab. Select the Adaptive Renderer, turn off Bump and Transparency, set the Antialiasing to Best and Silhouette Quality to 100%.
  36. In the Scene Settings Window, click on the Output tab. Expand the Image Size window by clicking on the "+" sign, then select "Keep Proportions" and set your output to a comfortable image size.
  37. Click on the Effects tab.
  38. Expand the "background" window, select "Color" and choose a dark background color.
  39. Close the Scene Settings Window.
  40. Close the Current Shader Editor and the Browser.
  41. Select Render/Use Current Settings.

    The rendered image should appear something like the following illustration--the paint shape shows some color, but is washed-out.

    Washed-Out Paint Shape

    Load the Maps into the Shader Channels

    Now, we're going to use the maps we created to make the colored areas of the paint shape more vivid.

  42. Select Windows/Current Shader Editor. The Color channel with the color map should be the only active channel. Make sure that "white is Invisible" is still checked; otherwise, leave it alone; leave the color map in the Color channel.
  43. Click on the Highlight tab. The channel should be empty.
  44. In the Current Shader Editor, select Insert/Operators/Mix. You'll see a three-part box--two adjacent subshader boxes and, below them, a "drop function here" box:

  45. Select Windows/Browser/Shaders, again.
  46. Drag-and-drop the testmask shader you created in Step 24 (above) from the Browser to the "drop function here" holder in the Highlight channel. The image in the Shader Preview probably won't appear any different.
  47. In the Highlight channel, click in the "Drop first shader here" box.
  48. In the Current Shader Editor, select Insert/Value. The Value slider, set to zero, should appear in the left subshader box.
  49. Click in the "Drop second shader here" box.
  50. In the Current Shader Editor, select Insert/Color. A red color chip should appear in the right subshader box.
  51. Double-click on that red color chip. The Color Picker should appear. Set all the RGB values in the Color Picker to zero. The color chip in the right subshader should turn black. Close the Color Picker.
  52. Close the Shader Browser.

    Your screen should look something like this:

    We've taken care of the Highlight channel. Now, we're going to load the mask into the shininess and reflection channels. This goes much quicker:

  53. Click on the Highlight tab, somewhat below the word "Highlight." The entire Mix operator, all three boxes, should be outlined in gray.
  54. Drag-and-drop the Mix operator box from the Highlight tab over to the Shininess tab, until the word "Shininess" is highlighted in bold text. The Mix operator box should appear complete within the Shininess channel.
  55. Do the same drag-and-drop maneuver from Shininess to Reflection.

  56. Click again on the Highlight tab.
  57. Set the value of the slider in the left subshader to the number you recorded in Step 21 (above). Oh, all right: it's '84.'
  58. Set the values of the other two masked channels, as well. (Shininess is '89' and Reflection is '60.')

    Delete the Washed-Out Paint Shape

    Before applying our revised shader, we need to get rid of that previously-applied washed-out paint shape:

  59. Select View/Preset Position/Front.
  60. Click on the Paint Shape Selection tool, use that to click on the object within the outline of the existing paint shape, and press Delete.

    Apply the Masked Paint Shape

    Now we apply the revised, masked paint shape. The procedure is exactly the same as Steps 29-31.

    When you've applied the new paint shape, render the image as before (Steps 34-41).

    Compare this picture to the render made with the original map. You'll see how much more vivid the artwork is, and that the surface characteristics of the chrome shader are reproduced in the unpainted portions of the paint shape.

    By using a map with the Mix operator, we have given the white area of the map values that match the original chrome shader, while the black area (the part that represents the art) has no such data assigned.


    Tutorial Index


    All constructive comments, suggestions, and corrections are welcomed.

    All contents copyright 1999-2000 bilagaana. All rights reserved. Except as noted, the images may not be copied in any form without the permission of the author. Digitally watermarked.