Passing a second texture to a shader, and rotating the second texture separately

[Jamin Grey]

Hi, I'm new to OpenGL and shaders. Also, my math is poor so matrixes are throwing me for a loop. 
I'm trying to understand them while working through this task, and bits and pieces are coming together, but it's too many things coming at once so I'm getting confused.

I'm trying to take one grayscale sf::Texture, and use it as the alpha channel for another sf::Texture.
Okay, easy enough:

gl_FragColor = (texture2D(texture, gl_TexCoord[0].xy) * gl_Color);
gl_FragColor.a = texture2D(mask, gl_TextCoord[0].xy).a;

(where 'mask' is uniform sampler2D mask; passed into the shader)

My difficulties arise because:
A) I want the textures to be able to be mirrored horizontally and/or vertically.
B) I want the textures to be able to be rotated (the mask rotated relative to the main texture, and the mask only rotated in 90 degree increments)
C) I want the mask to stretch itself to conform to the size of the texture if the mask happens to be larger or smaller or of unequal width/height.
D) I want the two textures to be rotated seperately from each other.

For the sake of convenience and experimentation, I'm doing this in a C++ function called 'SfmlDrawTextureWithMask()', where I pass in the textures separately with their rotations and other settings.

Right now, I'm playing with a shader like this: (SFML's basic vertex shader example)

uniform mat4 textureOrientation;
//uniform mat4 maskOrientation;

void main()
{
    // transform the vertex position
    gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;

    // transform the texture coordinates
    gl_TexCoord[0] = gl_TextureMatrix[0] * textureOrientation * gl_MultiTexCoord0;

    // forward the vertex color
    gl_FrontColor = gl_Color;
}

My problems so far, are that:
A) 'textureOrientation' isn't stretching to fill the texture, it's sampling outside of the texture and just has the usual bands of the border pixels:


I realize that this is because I'm rotating the texture coordinates and not the vertices themselves, and that I can use sf::Transform to rotate the primary sf::Texture by passing it into the secondary parameter of sf::RenderTarget::draw() - still, since I'm wanting to rotate the mask texture separately, I'm not sure exactly how to compensate for the rotation - I guess I could pass in the mask texture's size, use it to normalize the mask's texture coordinates, and then re-adjust with width and height swapped?

B) I don't know how gl_TexCoord[0] is created or passed to the fragment shader. It looks like an array (because of my C++ background), but I don't know the size of the array, so I don't know if that means I can just start using gl_TexCoord[1] to hold my mask texture's texture coordinates or if that'll overflow the buffer.

Why does gl_TexCoord[] and gl_MultiTexCoord0 exist? That seems to me like OpenGL supports binding multiple textures at once, but SFML only publicly exposes binding one texture at a time. Am I way off track here?
Is gl_TexCoord[1] available for me to use, and if so, do I have to do anything special to 'enable' it? I mean, I can't just use gl_TexCoord[999] can I?

Or to clarify my question: How do I pass my mask's separate texture coordinates from the vertex shader to the fragment shader (keeping in mind that they need to be interpolated for each fragment)?

And how do I get my mask's separate texture coordinates into the vertex shader in the first place? Especially considering that the mask might be only a subrect of a large texture container multiple masks. The first texture's tex coords are stored in gl_MultiTexCoord0, right? And they are passed into SFML using the attributes of the vertices - but a secondary mask texture wouldn't have vertices. How can I use SFML (or failing that, OpenGL) to put my mask texture's texcoords into gl_MultiTexCoord1?

Please keep your answers simple! I'm new to GLSL and OpenGL. 

[Jamin Grey]

Okay, I got alot of it working now. My biggest challenge remaining is I have no clue how to pass in a second sf::FloatRect for the Vertex shader to use for the mask. Is this possible with SFML?

[fallahn]

Although glsl supports vec4 (a rectangle is just 4 values after all) I don't think SFML provides a setter for it in the API as there's no such thing as sf::Vector4<T>. Without modifying SFML or using OpenGL directly your best bet is probably to split the rectangle into 2 parts, position and size. So in your shader:

uniform vec2 maskPos;
uniform vec2 maskSize;
 

and then set it in your code via:

shader.setParameter("maskPos", sf::Vector2f(rect.left, rect.top));
shader.setParameter("maskSize", sf::Vector2f(rect.width, rect.height));
 

Not ideal, but will get you where you want to go.

[Jamin Grey]

Not ideal, but will get you where you want to go.

Thanks for responding!

How would I write the vertex shader to use that?

I guess part of my confusion is a lack of familiarity with shaders:

void main()
{
    // transform the vertex position
    gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;

    // transform the texture coordinates
    gl_TexCoord[0] = gl_TextureMatrix[0] * gl_MultiTexCoord0;

    // forward the vertex color
    gl_FrontColor = gl_Color;
}

What am I supposed to assign to gl_TextCoord[1]?

[Jamin Grey]

I got it to work using OpenGL functions intermixed with SFML draw calls:

//Set the currently selected texture to TEXTURE-1, for OpenGL functions to operate on.
glCheck(glClientActiveTextureARB(GL_TEXTURE1_ARB));
//Tell OpenGL that this texture (TEXTURE-1) is going to be used in the next draw call.
glEnable(GL_TEXTURE_2D);

//Give OpenGL (bypassing SFML) the second pair of texture coords.
const char* data = reinterpret_cast<const char*>(&maskTexCoords[0]);
glTexCoordPointer(2, GL_FLOAT, sizeof(sf::Vector2f), data);

//Tell OpenGL that we want to use texture coordinates for TEXTURE-1 in the next draw call.
glEnableClientState(GL_TEXTURE_COORD_ARRAY);

//WARNING: Commented out as an optimization because SFML _currently_ doesn't get confused (SFML remembers to set the OpenGL active texture itself).
//If it later does, just uncomment this.
/*
        //Switch back to TEXTURE 0 being the selected OpenGL texture for functions to operate on, so SFML doesn't get confused.
        glClientActiveTextureARB(GL_TEXTURE0_ARB);
*/

//Use the sf::RenderTarget to draw the vertices using the sf::RenderStates we set up.
destination.draw(vertices, 4, sf::Quads, states);

//Disable texture coordinates for TEXTURE 1.
glClientActiveTextureARB(GL_TEXTURE1_ARB);
glDisableClientState(GL_TEXTURE_COORD_ARRAY);

//WARNING: Commented out as an optimization because SFML _currently_ doesn't get confused (SFML remembers to set the OpenGL active texture itself).
//If it later does, just uncomment this.
/*
        //Switch back to TEXTURE 0 being the selected OpenGL texture for functions to operate on, so SFML doesn't get confused.
        glClientActiveTextureARB(GL_TEXTURE0_ARB);
*/

//VERTEX SHADER

void main()
{
    //Transform the vertex position.
    gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;

    //Transform the texture's TexCoords.
    gl_TexCoord[0] = gl_TextureMatrix[0] * gl_MultiTexCoord0;

    //Transform the mask's TexCoords.
    gl_TexCoord[1] = gl_TextureMatrix[0] * gl_MultiTexCoord1;

    //Forward the vertex color.
    gl_FrontColor = gl_Color;
}

//FRAGMENT SHADER

uniform sampler2D texture;
uniform sampler2D mask;

//gl_Color
void main()
{
    vec4 textureColor = texture2D(texture, gl_TexCoord[0].xy);
    vec4 maskColor    = texture2D(mask, gl_TexCoord[1].xy);

    //Undo the alpha change by any algorithm, and instead use the alpha passed in as the 'Coloration' of the image.
    gl_FragColor = (textureColor * gl_Color);

    //Just use the red color channel of the mask for the alpha color.
    //But multiply it against the existing alpha instead of setting it, just incase the texture has alpha parts in it.
    gl_FragColor.a = (gl_FragColor.a * maskColor.r);
}
 

This kinda tricks SFML and isn't very stable since it depends on implementation details of SFML.



I'd definitely like a more native SFML method if it could be done.

I understand what you are saying about passing in GLSL uniforms using SFML, but I don't get how to use that to replace gl_MultiTexCoord1, since the vertex shader would make gl_MultiTexCoord1 a different value depending on which of the four vertices are being passed to the shader.