CS 428 - Fall 2005
Project 3: Adventures with Textures & Fragment Shaders

Due electronically: Friday, December 2nd, 11:59 pm (EDT)


Check this page frequently for updates and clarifications
(Significant changes or clarifications are marked with [Update])


Normal Map from http://members.shaw.ca/jimht03/normal.html

Description

We will build on the previous work we did for project 2. The new code base is very similar to proj2. First, we will introduce basic texturing. Then we will write the following programs:

Program

The skeleton code for this project can be found on the soup machines in the directory:
~sueda/cs428/proj3.zip

Copy everything in the folder to your account.

        cd ~/cs428/
        mkdir proj3
        cd proj3
        cp ~sueda/cs428/proj3.zip .
        unzip proj3.zip

Then, to compile the program: 

        make
If you get a compilation error, try running the setup script. 
        source ~sueda/cs428/setup
To run the program: 
        make run

Basic Texturing

We will first apply basic texturing using the fixed-pipeline. We will load an earth texture and apply it to the objects in the scene. The texturing code is already written for you. In fact, you should see that the teapot is already textured. This is because the teapot already comes with its texture coordinates, just like its positions and normals are pre-defined. To make the sphere and plane textured as well, you need to define the texture coordinates for each object. Recall that in proj2, you defined these objects by specifying the vertex positions and normals. Now, you have to specify the texture coordinates. First start with the plane. Modify PlaneNode.draw() to include the texture coordinates. You may use your own code from proj2, if you don't like the one provided. You can use glTexCoord2d(s, t) to specify the texture coordinates. At each vertex, you should have: 

    glTexCoord2d(s, t);
    glNormal3d(nx, ny, nz);
    glVertex3d(x, y, z); // Put this line last!
First try to get the whole earth to show up once on the plane. You can do this by linearly varying s and t from 0 to 1. Make sure that the earth isn't upside-down or flipped.

The texture coordinates are not limited to be between 0 and 1. To see the effects of this, modify the code you have just written so that s goes from 0 to maxs and t goes from 0 to maxt (maxs and maxt are member variables of PlaneNode). If you do this correctly, you should see the picture of the earth shrink and expand as you adjust the maxs and maxt sliders for the plane. Make sure that if maxs and maxt are both 2, you see 4 pictures of the earth.

Now try to get the earth texture to show up on the sphere as well, by modifying SphereNode.draw().


(Image from Mathworld)

s should go from 0 to 1 as theta goes from 0 to 2PI, and t should go from 1 to 0 as phi goes from 0 to PI (note that phi starts from the north pole). Once you get the earth in proper orientation, incorporate the member variables maxs and maxt here as well. Again, if both maxs and maxt are 2, you should see 4 pictures of earth on the sphere.

Textures & Shaders

We will now write a shader that uses a single texture. Take a look at simpleTexture.vp and simpleTexture.fp. Here is the vertex shader: 
1    void main()
2    {
3        gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;
4        gl_FrontColor.rgb = vec3(1, 1, 1);
5        gl_TexCoord[0] = gl_MultiTexCoord0;
6    }
In line 3, the eye-space vertex position is computed. In line 4, WHITE is assigned to the vertex output colour. In line 5, the texture coordinates are simply passed through to the fragment shader. Here is the fragment shader: 
1    void main()
2    {
3        gl_FragColor = gl_Color * texture2D(texture, gl_TexCoord[0].st);
4    }
In line 3, the fragment colour is computed as the multiplication of the texture colour and the input colour. If you modify line 4 of the vertex shader so that the fragment input colour is red, then you should see that the the whole earth becomes red. This is because by multiplying the texture colour by (1, 0, 0), we are throwing away the green and blue channels of the texture.

Instead of loading a texture from file, we can also create one procedurally. Modify SimpleTextureProgram.findTexture() to load the grid texture instead of the earth texture. Then implement GridTexture2D.loadData(). You need to fill in data byte by byte. data is declared as a 1-dimensional array of bytes of length (size * size * 3). The information is stored as: 

    data[0] = red colour of texel at row 0, col 0.
    data[1] = green colour of texel at row 0, col 0.
    data[2] = blue colour of texel at row 0, col 0.
    data[3] = red colour of texel at row 0, col 1.
    data[4] = green colour of texel at row 0, col 1.
    data[5] = blue colour of texel at row 0, col 1.
    ...
    data[3 * j + 0] = red colour of texel at row 0, col j.
    data[3 * j + 1] = green colour of texel at row 0, col j.
    data[3 * j + 2] = blue colour of texel at row 0, col j.
    ...
    data[3 * size + 0] = red colour of texel at row 1, col 0.
    data[3 * size + 1] = green colour of texel at row 1, col 0.
    data[3 * size + 2] = blue colour of texel at row 1, col 0.
    data[3 * size + 3] = red colour of texel at row 1, col 1.
    data[3 * size + 4] = green colour of texel at row 1, col 1.
    data[3 * size + 5] = blue colour of texel at row 1, col 1.
    ...
    data[3 * size * 2 + 0] = red colour of texel at row 2, col 0.
    data[3 * size * 2 + 1] = green colour of texel at row 2, col 0.
    data[3 * size * 2 + 2] = blue colour of texel at row 2, col 0.
    data[3 * size * 2 + 3] = red colour of texel at row 2, col 2.
    data[3 * size * 2 + 4] = green colour of texel at row 2, col 2.
    data[3 * size * 2 + 5] = blue colour of texel at row 2, col 2.
    ...
    data[3 * size * i + 3 * j + 0] = red colour of texel at row i, col j
    data[3 * size * i + 3 * j + 1] = green colour of texel at row i, col j
    data[3 * size * i + 3 * j + 2] = blue colour of texel at row i, col j
Right now, the texture is white, because the whole array is set to 255 (maximum value of a byte). If you change it to 0, the texture will be black, and if you change it to 127, it will be grey. Make a grid texture by setting the texel be black every "tileSize" texels in both i and j. (Use the mod, "%", operator.) Once you get this done, by running "Simple Texture" program, you should see a grid pattern on all the objects in the scene.

Multi-texturing

We can access more than one texture at a time through a function called "multi-texturing." We will create a shader that uses both the earth and grid textures. Open multiTexture.vp and multiTexture.fp. You'll see that currently, the vertex shader looks the same as simpleTexture.vp, and the fragment shader simply assigns the colour looked up from the texture into the final fragment output colour. First modify the fragment shader to use two textures instead of one. Add the line, 
    vec3 grid = texture2D(gridTexture, gl_TexCoord[0].st).rgb;
And then modify the line that assigns the final fragment colour so that you get this:

We just used the same texture coordinates to look up the two textures. We are free to use other texture coordinates as well. We can do this by:

Now we can animate the grid texture by playing around with gl_TexCoord[1] in the vertex shader. Make the grid rotate by multiplying gl_MultiTexCoord0 by a rotation matrix before assigning it to gl_TexCoord[1]. If you do this correctly, you should see that the grid rotates about the origin of the texture space (which is the lower left corner for the plane). Once you have that done, make the grid rotate about the middle of the plane instead (which is (0.5, 0.5) in the texture space). You'll need to apply both translation and rotation matrices (in some order) to gl_MultiTexCoord0. Note that in order for the translation matrix to work, the vector that is getting multiplied need to be in homogeneous coordinates. You can define a 3x3 matrix in GLSLang like this: 

    // Create a 3x3 identity matrix (column major)
    mat3 M;
    M[0] = vec3(1.0, 0.0, 0.0); // First COLUMN
    M[1] = vec3(0.0, 1.0, 0.0); // Second COLUMN
    M[2] = vec3(0.0, 0.0, 1.0); // Third COLUMN

    // Multiply in homogeneous coords
    gl_TexCoord[1].stp = M * vec3(gl_MultiTexCoord0.st, 1.0);

Reflection Shader

We will now learn how to use cube maps. Recall the 2D textures are looked up using 2D coordinates. Instead, 3 values are used to look up a cube map. Imagine placing a cube around the origin. Now for each vector pointing away from the origin, there is exactly one place where the vector (or its extension) intersects the cube. This is how a cube map texture look up is done. Given a 3d texture coordinate (s, t, p), a cube map returns the colour of the portion of the cube that the vector (s, t, p) points towards.


http://www.cg.tuwien.ac.at/studentwork/CESCG/CESCG-2002/GSchroecker/img32.png

By storing the environment in the 6 sides of the cube, we can achieve some nice effects.


http://www.developer.com/img/articles/2003/03/24/EnvMapTecIm01.gif

We're going to write a shader that's known as an environment map shader. The objects rendered using this shader will look like they are made up of a mirror-like material, and will reflect the environment. To implement the reflection shader, implement the following:

You can use the built-in function "reflect()" to calculate the reflection vector. (In fact, you can use any function you can find in the GLSLang specs.)

Refraction Shader

Instead of reflecting the light, we can make the objects refract the light. Using Snell's Law, we can compute the refraction vector as: 
    // I = incident direction
    // N = normal
    // eta = relative index of refraction
    vec3 refract(vec3 I, vec3 N, float eta)
    {
        float IdotN = dot(I, N);
        float k = 1.0 - eta * eta * (1.0 - IdotN * IdotN);
        return eta * I - (eta * IdotN + sqrt(k)) * N;
    }

[Update]
Don't worry about the thick border you get near the edge of the objects. The default values of eta should be less than 1, not greater than 1. You may want to change the default values to be something like 0.66. And if you do this, things inside the sphere should be upside down.
[/Update]

Once you get the basic refraction working, we can add some diffraction. Diffraction occurs when the different frequencies of light are refracted at different rates. We can simulate this effect by individually refracting the red, green, and blue components, using etaRed, etaGreen, and etaBlue respectively.

Finally, combine reflection and refraction so that as the surface normal becomes more perpendicular to the eye vector, you get more reflection, and as the surface normal becomes more parallel to the eye vector, you get more refraction. This is similar to the silhouette shader you wrote for proj2.

[Update]
The varying variable "beta" in refract.vp is there so that you can use it as 

    gl_FragColor.rgb = (1 - beta) * reflection.rgb + beta * refraction.rgb;
You'll have to come up with a value of beta in the vertex shader and use it to "mix" the reflection and refraction values as above in the fragment shader.
[/Update]

Just Reflection Just Refraction Both Reflection and Refraction

Notice that in the right-most image, near the edge of the sphere, we get just reflection, whereas in the middle of the sphere, we get just refraction.

Per-Fragment Lighting

In proj2, you should have noticed that the objects looked bad with vertex lighting if there weren't enough vertices. With fragment shading, we can make things look pretty nice even with only a few vertices. Implement fragLighting.vp and fragLighting.fp. You could use any space you want to do the lighting calculation in the fragment shader, but eye-space is probably the easiest one to use. This means that you have to send in the eye-space vertex position (which becomes the fragment position as it gets interpolated across the face) and vertex normal to the fragment shader. Note that varying parameters "position" and "normal" have already been declared for you. In the vertex shader, you'll need to set them to the appropriate values, and in the fragment shader, you can use them to perform the lighting computations. Implement the Blinn shading model as before, but don't worry about attenuation.

Bump Mapping

Bump mapping allows us to perturb the surface normal at each fragment before we perform the lighting calculations. One way to do this is to look up a perturbation value in a texture map. Such a texture map is called a normal map, because it encodes the surface normal at each texel.

The (r, g, b) values of a texel stores the (x, y, z) components of the normal vector. But note that the RGB values can't be negative. Therefore, the normals are compressed into the [0, 1] range before they are stored. For example, the normal (x = 0, y = 0, z = 1) would be stored as (r = 0.5, g, = 0.5, b = 1.0), and (x = -1, y = 0, z = 0) would be stored as (r = 0, g = 0.5, b = 0.5). An unperturbed normal is by convention defined to be (x = 0, y = 0, z = 1) = (r = 127, g = 127, b = 255).

The tricky part about bump mapping is that there is an extra change of coordinate space involved in the lighting calculation. Imagine that we have a uniform normal map that has (x = 0, y = 0, z = 1) = (r = 127, g = 127, b = 255) at all texels. When this texture is looked up, we always get the normal (0, 0, 1), regardless of what the texture coordinates are. But what does (0, 0, 1) mean? Which space is this normal in?

The returned normal (0, 0, 1) is in a space called "tangent" space (a.k.a. texture or surface-local space). The tangent space is a space that varies over the surface of the object, and assumes that the origin is the surface point itself and the surface normal is (0, 0, 1). An example of a tangent space for the sphere and the plane is shown below.

At each surface point, there is a coordinate space with the origin at that point, and the z-axis pointing along the surface normal. The x- and y-axes are chosen appropriately to complete the coordinate space. At each fragment, we can look up the normal map in tangent space for the normal at that fragment. If we have the light and eye vectors in tangent space as well, we can compute the phong lighting model as before.

For the plane, the tangent space is very simple. The z-axis, as always, points along the normal. The x-axis and y-axis can be chosen to point towards +x and -z axes respectively. (Note that if we fix z to be the normal, the other two axes can be chosen in more than one way.) If we assume that the tangent space is orthogonal, we only need to specify the z and x axes; the y axis can be taken to be the cross(z, x). In PlaneNode.draw(), we currently only send the vertex position, normal, and texture coordinates. Modify it to send the x-axis of the tangent space as well. You can use the gl_MultiTexCoord1 attribute to send in the tangent space information. At each vertex, your calls should be: 

    glTexCoord2d(s, t); // Texture coords
    glNormal3d(nx, ny, nz); // Normal
    glMultiTexCoord3d(GL_TEXTURE1, tx, ty, tz); // Tangent space x-axis
    glVertex3d(x, y, z); // Put this line last!

Do the same thing for SphereNode.draw().

We are going to do the lighting computations in the tangent space in the fragment shader. The job of the vertex shader is to send the light and eye vectors in tangent space to the fragment shader. The transformation matrix that takes a vector from object space to tangent space can be constructed from the normal and the tangent space x-axis as follows: 

    [Tx Ty Tz]
    [Bx By Bz]
    [Nx Ny Nz]
where 
    T = tangent space x-axis vector
    N = normal
    B = cross(N, T)
The eye-vector and the light-vector can be obtained in eye-space as before. To transform them into tangent space, you'll need to first transform them into object space and then into tangent space. For example, if L is the light vector in eye space and you want to transform it into tangent space,

Both of the matrices in the last expression can easily be constructed. The first matrix is constructed from the T, N, and B vectors, and the second matrix is just the transpose of the upper 3x3 portion of the modelview matrix. Remember that as stated earlier, matrices in glslang are column major. Once you have the tangent space light and eye vectors in the fragment shader, the lighting computation is exactly the same as before.


With bump maps, you should see stunning lighting effects, even with only a few vertices.

You are free to generate your own proceduarl normal map, or download one from the web (mention the source in the code and the readme).

Extra Credit

You must do step 1 before doing step 2.
  1. Combine bump-mapping and reflection to create a bumpy-reflection shader.
  2. Create your own shader. You are free to search the web for ideas, as long as you give the credit where it's due.

Handing In

We will be using an online handin program as described on the info page. The project name for this assignment is "Project 3". Note that you can only hand in under this project name if you hand in before the deadline. After the deadline, there will be another project listed - "Project 3 - LATE" to which you will be able to hand in in the same way. If you hand in to BOTH "Project 3" and "Project 3 - LATE", your on-time assignment ("Project 3") will be IGNORED.

What to hand in?

As mentioned on the info page, you will be handing in a single file - an archive of all the files that you need to hand in. This file should contain:

To run your application, we should be able to type exactly the following: 

        tar xvf archiveName
        javac *.java
        java ShaderApp

Suggestions and Tips

428 Home