Welcome, Guest. Please login or register. Did you miss your activation email?

Author Topic: Clipping mask implementation proposal  (Read 11081 times)

0 Members and 1 Guest are viewing this topic.

Kipernal

  • Newbie
  • *
  • Posts: 29
    • View Profile
Clipping mask implementation proposal
« on: April 29, 2015, 06:19:06 am »
So apparently clipping masks have been a requested feature for a while.  I don't know if I have the solution, but I think I might have a solution.

But first, pretty picture time!


As you can see, the background is only being drawn over certain parts of the screen.  First we drew a sprite and a circle as stencils, then we drew the background "tracing" over that stencil, then we drew the sprite and circle again in different locations, completely ignoring the existence of stencils.  In this implementation there's only one new class, the StencilSettings class, that is passed along with sf::RenderStates.  This implementation also works on anything that can be drawn, including textured sprites that use the alpha channel and arbitrarily complex polygons, and is respected by view transformations.

Basically, my implementation takes inspiration from the implementation of blending through sf::BlendMode.  In the same way you simply pass an sf::BlendMode to a sf::RenderStates' constructor to change how that object is drawn, you can pass an sf::StencilSettings to change how this object affects the stencil buffer or is affected by the stencil buffer.

The two main StencilSettings defined are sf::StencilCreate and sf::StencilTrace (of course you can make your own like you can with sf::BlendMode). Basically:
window.draw(someObject, sf::RenderStates(sf::StencilCreate)) will draw someObject as a stencil.
window.draw(anotherObject, sf::RenderStates(sf::StencilTrace)) will then draw anotherObject only where it overlaps the area someObject drew.

More specifically, sf::StencilCreate draws this object to the stencil buffer instead of the color buffer.  sf::StencilTrace makes it so that the object is only drawn when it overlaps an object drawn with sf::StencilTrace.  (There is also a third pre-defined sf::StencilDisable that is just constructed with the default sf::StencilSettings--it's only there for completeness and is not strictly necessary.)

Here's the example code that generated the above image.  Nothing of note happens until the event loop.

int main(int argc, char* argv[])
{
        sf::RenderWindow window;
        window.create(sf::VideoMode(640, 480), "Title", sf::Style::Default);
        window.setFramerateLimit(60);

        // Boring initialization stuff.
        // Nothing interesting here--you can just skip past it.
        sf::Sprite sprite;
        sf::Sprite backgroundSprite;
        sf::Texture texture;
        sf::Texture backgroundTexture;
        sf::CircleShape shape;

        texture.loadFromFile("TestSprite.png");
        sprite.setTexture(texture);

        backgroundTexture.loadFromFile("Background.png");
        backgroundSprite.setTexture(backgroundTexture);

        shape.setOutlineColor(sf::Color::Black);
        shape.setOutlineThickness(3.5f);
        shape.setFillColor(sf::Color::White);
        shape.setRadius(40);
        shape.setOrigin(40, 40);
       


        int animationCounter = 0;

        while (window.isOpen())
        {
                handleInput(window);
               
                animationCounter++;
                window.clear(sf::Color::Blue);

                // Move the sprite and circle around a bit
                sprite.setPosition(0 + std::sinf(animationCounter / 100.f) * 100, 0);
                shape.setPosition(0 + std::cosf(animationCounter / 100.f) * 100 + 100, 256.f);

                // Draw our sprite as a stencil.
                window.draw(sprite, sf::RenderStates(sf::StencilCreate));

                // Draw our circle shape as a stencil.
                window.draw(shape, sf::RenderStates(sf::StencilCreate));

                // Draw our background only where we've previously drawn a stencil
                window.draw(backgroundSprite, sf::RenderStates(sf::StencilTrace));

                // Draw normal sprites and shapes unrelated to the stencil.
                sprite.move(static_cast<sf::Vector2f>(texture.getSize()));
                shape.move({ 0.f, shape.getRadius() * 2.f });
                window.draw(sprite);
                window.draw(shape);

                window.display();
        }
        return 0;
}


For reference, three's already a pull request on GitHub that offers an alternative solution.  I'm mostly offering this as a different take on the same idea as the current pull request takes a completely different design approach than I took here.  Competition's always good, right?  ;D


The implementation fork is here.  The changes are:

  • Added extra constructors to RenderStates to take StencilSettings, as well as a stencilSettings member variable.
  • Added void applyStencilSettings(const StencilSettings&) to RenderTarget
  • Added a lastStencilSettings member variable to RenderTarget
  • Changed ContextSettings' default constructor to request one stencil bit
  • Changed RenderStates::Default to include the default StencilSettings
  • Changed RenderTarget::clear to clear the stencil buffer as well
  • RenderTarget::draw calls applyStencilSettings
  • RenderTarget::resetGLStates calls applyStencilSettings(StencilSettings())

These should all be backwards-compatible unless you were doing stuff with the stencil buffer before this without calling pushGLStates/popGLStates.


So I'm just going to finish this off by saying that I'm totally more than welcome to comments/criticisms/concerns! ...but I don't really know OpenGL.  I basically made this through sheer unbridled determination to get my unmoving plaid effect working after trying and failing to implement it with shaders and then with fancy blending abuse.  In other words, I'll do my best to answer any questions, but at the same time if it gets too technical in terms of OpenGL-related jargon it's probably going to fly right over my head.   :P

(Incidentally I wasn't sure if it would have been better to simply do a pull request or not instead of creating this post, but I sort of got the impression from here that it'd be better to create a forum post first.)

Jesper Juhl

  • Hero Member
  • *****
  • Posts: 1405
    • View Profile
    • Email
Re: Clipping mask implementation proposal
« Reply #1 on: April 29, 2015, 05:20:44 pm »
I haven't looked at your implementation of the feature (yet), but the end result (pretty picture) and code example of use looks really nice and simple.

Kipernal

  • Newbie
  • *
  • Posts: 29
    • View Profile
Re: Clipping mask implementation proposal
« Reply #2 on: May 01, 2015, 06:36:11 am »
Thank you--the end goal was to try to keep it as simple and easy-to-understand as possible.  You can still customize it to basically any degree that OpenGL allows, but I feel like just passing some extra objects to draw's RenderStates that say "this is a stencil, not an actual object" or "only draw this where we've already drawn a stencil" is at least reasonably intuitive.  But this is totally subjective of course--as I mentioned before zsbzsb has already implemented this with the alternate idea of making a clipping mask an array that you add Drawables to and then applying it by calling target.setClippingMask(mask) (it also adds the ability to use scissor testing to more quickly clip out whole rectangles, which I don't do).  If you haven't seen that as well you should definitely take a look.

Anyway, I went and made a couple of small changes, including adding clearStencilBuffer to sf::RenderTarget and, based on zsbzsb's implementation added the necessary code to get RenderTextures in on the clipping fun (I gotta admit that I didn't even know they'd need special handling, so all the credit for that one goes to zsbzsb).  I also made the default alpha test for the StencilCreate object GreaterEqual instead of Equal, which I figured would make more sense if you lowered the default alpha threshold (makes way more sense to test for >= 128 than == 128, for example).  Finally, I added the StencilInverseTrace object, which just does the opposite of StencilTrace (draws only where a stencil hasn't been drawn).

So...any other comments/suggestions?  If not I'll probably do a pull request so this and #846 can battle it out, but if anyone has anything to say in the meantime please speak up!

Hiura

  • SFML Team
  • Hero Member
  • *****
  • Posts: 4321
    • View Profile
    • Email
Re: Clipping mask implementation proposal
« Reply #3 on: May 01, 2015, 02:03:24 pm »
Nice to see people motivated to implement this.  :D

Having never used clipping masks before, tell me if I'm wrong in the following:

From the user perspective, it looks like your implementation is similar to how SFML currently handle drawable order (i.e. let the user manage the order himself) while zsbzsb's implementation handle it more like a view (i.e. the drawing order doesn't matter since they use the current clipping mask).

Are those observations correct?

Regarding inheritance, your method seems more flexible: one could propagate clipping information to sub-drawable elements. (However, I've no experience if this make any sens in practice.) Does your implementation actually support that?
SFML / OS X developer

Kipernal

  • Newbie
  • *
  • Posts: 29
    • View Profile
Re: Clipping mask implementation proposal
« Reply #4 on: May 01, 2015, 10:35:53 pm »
Having never used clipping masks before, tell me if I'm wrong in the following:

From the user perspective, it looks like your implementation is similar to how SFML currently handle drawable order (i.e. let the user manage the order himself) while zsbzsb's implementation handle it more like a view (i.e. the drawing order doesn't matter since they use the current clipping mask).

Are those observations correct?

I think so.  If zsbzsb's reading this they'll have to correct me if I'm wrong but from what I understand their implementation basically says "once I apply this clipping mask, it stays on until someone explicitly turns it off or uses a different one".  Mine is like BlendMode in that unless you specify it specifically the default stencil settings are just "totally ignore the existence of stencils".  If you want to draw a stencil or trace around a stencil you have to explicitly specify it, or use the RenderStates someone else passed you that explicitly specified it.

Regarding inheritance, your method seems more flexible: one could propagate clipping information to sub-drawable elements. (However, I've no experience if this make any sens in practice.) Does your implementation actually support that?

Yes, in the same way that you can propagate anything else that's stored in a RenderState object; if some object says it wants to be a stencil or only be drawn over a stencil, then if it sets its RenderState to do that and passes that state down to its children, then they'll "inherit" the stencil settings the parent used.

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32498
    • View Profile
    • SFML's website
    • Email
Re: Clipping mask implementation proposal
« Reply #5 on: May 01, 2015, 11:03:23 pm »
I haven't looked at the implementation yet, but I like the API. I would just replace all these technical terms with friendlier names: from a user point-of-view, it would be more intuitive to talk about masks, and adding to/drawing inside/drawing outside/ignoring them instead of these stencil buffer operations.
Laurent Gomila - SFML developer

Kipernal

  • Newbie
  • *
  • Posts: 29
    • View Profile
Re: Clipping mask implementation proposal
« Reply #6 on: May 02, 2015, 01:37:28 am »
That's definitely a good point.  I mentioned a couple of posts ago that I added a function named clearStencilBuffer to RenderTexture--I knew that probably wasn't a good name for it but couldn't really come up with anything satisfactory.  But if we're going with calling things masks instead of stencils, maybe just RenderTarget::clearMask() would work?

Going along with that we could rename StencilSettings to MaskSettings and change the default objects as well.  Naming is pretty subjective so these are a few ideas:

// For drawing to a mask
window.draw(shape, sf::RenderStates(sf::MaskAdd));
window.draw(shape, sf::RenderStates(sf::MaskAddTo));
window.draw(shape, sf::RenderStates(sf::MaskCompose));
window.draw(shape, sf::RenderStates(sf::MaskAppend));        // I don't particularly care for this one as it implies a data structure

       
// For only drawing where we've drawn a mask
window.draw(shape, sf::RenderStates(sf::MaskInside));
window.draw(shape, sf::RenderStates(sf::MaskInsideOnly));
window.draw(shape, sf::RenderStates(sf::MaskDrawInside));
window.draw(shape, sf::RenderStates(sf::DrawInside));
window.draw(shape, sf::RenderStates(sf::DrawInsideMask));
window.draw(shape, sf::RenderStates(sf::Mask::DrawInside));

// For only drawing where we haven't drawn a mask
window.draw(shape, sf::RenderStates(sf::MaskOutside));
window.draw(shape, sf::RenderStates(sf::MaskOutsideOnly));
window.draw(shape, sf::RenderStates(sf::MaskDrawOutside));
window.draw(shape, sf::RenderStates(sf::DrawOutside));
window.draw(shape, sf::RenderStates(sf::DrawOutsideMask));
window.draw(shape, sf::RenderStates(sf::Mask::DrawOutside));
       
// For ignoring mask operations (same as the default constructor so not strictly necessary)
window.draw(shape, sf::RenderStates(sf::MaskIgnore));
window.draw(shape, sf::RenderStates(sf::MaskDisable));
   


But there's something else beyond that--StencilSettings (or probably MaskSettings now) has some members with names like stencilOperation and stencilFunction (like how BlendMode has members with names like colorSrcFactor and alphaEquation).  If you're just using the default objects you'll never need to worry about them, but at the same time they do (obviously) make some rather technical references to OpenGL stencil operations.  Do we change those as well, or leave them as-is?

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32498
    • View Profile
    • SFML's website
    • Email
Re: Clipping mask implementation proposal
« Reply #7 on: May 02, 2015, 10:18:18 am »
Without looking at the implementation, I didn't notice that you had a structure similar to BlendMode, to have full control over stencil buffer operations. Maybe we could do the same as for blending modes: start with the most common modes (as a simple enum), and make it evolve to a structure that exposes all the underlying OpenGL stuff later, only if it is really needed.

But yes, in any case, finding the correct naming now is important if we don't want to break the public API later.
Laurent Gomila - SFML developer

Nexus

  • SFML Team
  • Hero Member
  • *****
  • Posts: 6287
  • Thor Developer
    • View Profile
    • Bromeon
Re: Clipping mask implementation proposal
« Reply #8 on: May 02, 2015, 01:49:52 pm »
A spontaneous thought: since these clipping mask are essentially set operations, maybe some terms could be taken from there? E.g. "union" instead of "add".

But it only makes sense if the relation to sets is relevant for the user. Is it possible to support other operations, such as the intersection between two masks, yielding a third mask? I'm referring to the combination of masks themselves, not rendering an object using a mask.
Zloxx II: action platformer
Thor Library: particle systems, animations, dot products, ...
SFML Game Development:

Kipernal

  • Newbie
  • *
  • Posts: 29
    • View Profile
Re: Clipping mask implementation proposal
« Reply #9 on: May 02, 2015, 04:16:42 pm »
Without looking at the implementation, I didn't notice that you had a structure similar to BlendMode, to have full control over stencil buffer operations. Maybe we could do the same as for blending modes: start with the most common modes (as a simple enum), and make it evolve to a structure that exposes all the underlying OpenGL stuff later, only if it is really needed.

I'm not 100% sure what you mean--if you're talking about providing "simple" objects that don't require users to dig deep into the documentation that's what (the currently named) StencilCreate, StencilTrace, etc. are for.  They serve the same purpose as, for example, BlendAlpha and BlendMultipy.

A spontaneous thought: since these clipping mask are essentially set operations, maybe some terms could be taken from there? E.g. "union" instead of "add".

I like the idea of using "union" somewhere in there as it makes a lot of sense, but just calling something "MaskUnion" (not that you were implying that be the full name) doesn't make it clear which broad operation you're performing (adding to the mask versus tracing the mask).  We need another word in there, and it's finding that word that's been causing me trouble.  Personally I'm leaning towards the "MaskCompose" I suggested last time, as it doesn't necessarily imply creating a new mask (like the current "StencilCreate"), and it doesn't imply that the user needs to have an already-existing mask (like "MaskAdd" or "MaskAddTo").

But it only makes sense if the relation to sets is relevant for the user. Is it possible to support other operations, such as the intersection between two masks, yielding a third mask? I'm referring to the combination of masks themselves, not rendering an object using a mask.

I haven't actually tried it, but as far as I know you sort of can.  OpenGL stencil operations are arithmetic, not bitwise, so you can add or subtract 1 from the current pixel's stencil value but not set or clear a specific bit.  Basically you can count how many times there's been an overlap in drawing masks but not specifically pinpoint which masks caused it, if that makes sense.  That being said if you're only using the default objects, which only write 1s to the buffer, then this doesn't matter much. :P

zsbzsb

  • Hero Member
  • *****
  • Posts: 1409
  • Active Maintainer of CSFML/SFML.NET
    • View Profile
    • My little corner...
    • Email
Re: Clipping mask implementation proposal
« Reply #10 on: May 02, 2015, 05:09:44 pm »
First off, let me get clear my personal feelings about this. I am really pissed (hence why I haven't replied yet), no one takes any time for years to try and do something about clipping masks so I spend a good amount of time doing an implementation and then < 2 months later someone decides to basically rip off my PR with their own design without even bothering to give feedback on my own design (which I ask for multiple times).

Quote
I'm mostly offering this as a different take on the same idea as the current pull request takes a completely different design approach than I took here.  Competition's always good, right?
Quote
So...any other comments/suggestions?  If not I'll probably do a pull request so this and #846 can battle it out, but if anyone has anything to say in the meantime please speak up!

No, not always good - when it comes to open source people should be working together rather than competing and battling with each other. Its generally accepted if someone has done something and they ask for design feedback you give that to them rather trying to push them out of the way. I would gladly make changes based on feedback that people agreed on.

If you really thought you had a much better idea then why didn't you just comment? As I said before, I asked for feedback many times, but no - you decide to just do your own thing without mentioning it to anyone. Oh, and to everyone else - why has more people commented on this design in 3 days than what I got in 2 months?

Not to mention the fact that since everyone loves to discuss design around here we probably won't see an accepted design for 2.4 now....

Anyways, besides the ethical/personal issues, your design is still has issues.

What about when people just need simple clipping for GUIs? You offer no way to utilize glScissors for simple rectangle clipping.

What if someone wants to clear the mask and apply a different one? There doesn't appear to be any way to do that with your design.

Without looking at the implementation, I didn't notice that you had a structure similar to BlendMode, to have full control over stencil buffer operations. Maybe we could do the same as for blending modes: start with the most common modes (as a simple enum), and make it evolve to a structure that exposes all the underlying OpenGL stuff later, only if it is really needed.

In the case of blend modes there isn't any better option but to expose the entire enums simply because there is no better way to wrap it. But when it comes to stencils I don't think we need to expose the entire interface. I mean, what exactly is the point of even giving away that the clipping API uses the stencil buffer? Isn't this kind like of exposing that the graphics module uses libjpeg?

In reality, the stencil buffer is used mainly for clipping and not much else (and if someone wants something more specific they aren't going to be using SFML). Isn't the idea anyways to expose clipping masks? Not just the stencil buffer? Because as far as the current design goes, it basically gives you limited access to just the stencil buffer and then requires you to use it only for clipping.

If we wanted to give access to the stencil buffer then we should, or we should implement clipping masks (as my design does). Just to say, what if we want to port SFML to another backend or platform that doesn't support the stencil buffer or there is a better way to implement clipping? But you can't now because you shackled one foot and one leg to the stencil buffer.

Quote
If you want to draw a stencil or trace around a stencil you have to explicitly specify it
Doesn't this totally defeat the purpose of what one of the biggest reasons is for implementing clipping masks? What if you want to limit your GUI to an area of the window but that GUI will not explicitly use the clipping mask?

one could propagate clipping information to sub-drawable elements. (However, I've no experience if this make any sens in practice.) Does your implementation actually support that?

No it doesn't in the sense that you might be thinking of sf::Transformable. Sure the sub-drawable will be using the mask (same in my design) and the sub-drawable may even extend the mask. But that sub-drawable will have no way to erase the changes it made to the mask. Because once it draws what it needs to the mask it has to way of clearing just what it drew or knowing what was already drawn to the stencil buffer.

So just say a sub-drawable decides it wants its own mask - so it clears the stencil buffer (actually it needs the glClear function because this design doesn't expose that). Then it draws what it wants and returns from its draw function. Now the caller may not even know that the mask was changed (or that it may need to entirely redraw the mask, but what if it changed the mask itself?).

Is it possible to support other operations, such as the intersection between two masks, yielding a third mask? I'm referring to the combination of masks themselves, not rendering an object using a mask.

This is very much possible with my design, and see my comments directly above. It would even be possible to revert the changes a sub-drawable makes to the mask.

https://github.com/SFML/SFML/compare/master...Kipernal:master#diff-ee21bf2821ee1b97f677cabb9bb964c5R33
This isn't how you are supposed to add gl functions.
« Last Edit: May 02, 2015, 05:18:58 pm by zsbzsb »
Motion / MotionNET - Complete video / audio playback for SFML / SFML.NET

NetEXT - An SFML.NET Extension Library based on Thor

Kipernal

  • Newbie
  • *
  • Posts: 29
    • View Profile
Re: Clipping mask implementation proposal
« Reply #11 on: May 03, 2015, 01:41:42 am »
Alright, so I should probably get this out of the way first: I didn't do this with any sort of intention of "fighting" with your implementation--when I said "battle it out" I didn't think you'd take it so literally.  I meant let the people decide which API they like better, and that's really the whole thing.  This is just about APIs, and in fact I couldn't care less whose makes it in.  If someone else pops up tomorrow with a third design that everyone accepts immediately I'd be just as happy.

This, then, is why I didn't instead submit it as a comment on your pull request--the fundamental design was so different that it'd basically just be saying "change the very foundations of your API" which wouldn't have accomplished anything.  I figured that by designing my own API separate from yours we could have the community decide which they liked better, and then focus on the implementation details.  Implementing clipping masks isn't hard, as evidenced by the fact that someone like me with very little OpenGL knowledge could put something together.  So the fact that it's taken so long for anything to come up suggests that the difficulty of the creation of this feature comes more from how to build an easy-to-understand API rather than the nitty-gritty OpenGL calls.

That is why I did this and why I said "competition's always good".  I made this because I thought up an API for clipping masks that I liked while experimenting with the OpenGL stencil buffer, and thought maybe the SFML community might have some opinion on it.  And that's it.

What about when people just need simple clipping for GUIs? You offer no way to utilize glScissors for simple rectangle clipping.

As I mentioned before that's one thing that your implementation provides that mine doesn't, pretty much exclusively because I was just focusing on how to get stencil tests under control.

What if someone wants to clear the mask and apply a different one? There doesn't appear to be any way to do that with your design.
There is, under the currently-named RenderTarget::clearStencilBuffer function.  There's also an alternative I mention below.

In the case of blend modes there isn't any better option but to expose the entire enums simply because there is no better way to wrap it. But when it comes to stencils I don't think we need to expose the entire interface. I mean, what exactly is the point of even giving away that the clipping API uses the stencil buffer? Isn't this kind like of exposing that the graphics module uses libjpeg?

This is not wrong, and was something I struggled with.  If I'm being honest I'm not sure I'm 100% happy with the idea that StencilSettings serves the double duty of both creating and applying masks, but I felt that with the enums 90% of users wouldn't need to care from an API standpoint, and the other 10% would be able to take advantage of it in order to do advanced stencil operations if they desired.  Again, like with BlendMode.

If we wanted to give access to the stencil buffer then we should, or we should implement clipping masks (as my design does). Just to say, what if we want to port SFML to another backend or platform that doesn't support the stencil buffer or there is a better way to implement clipping? But you can't now because you shackled one foot and one leg to the stencil buffer.

Again, more than valid.  I chose the route of giving the user more flexibility if they really needed it and then hiding that power from users that don't, but of course this means the ability to use clipping masks is more tied to OpenGL stencil buffers specifically than might be otherwise.

Doesn't this totally defeat the purpose of what one of the biggest reasons is for implementing clipping masks? What if you want to limit your GUI to an area of the window but that GUI will not explicitly use the clipping mask?

If you wanted to limit a GUI to an area of the window then you'd draw a mask onto the window, then draw the GUI with StencilTrace, ensuring that the RenderStates are properly propagated to the GUI's children elements.

So just say a sub-drawable decides it wants its own mask - so it clears the stencil buffer (actually it needs the glClear function because this design doesn't expose that). Then it draws what it wants and returns from its draw function. Now the caller may not even know that the mask was changed (or that it may need to entirely redraw the mask, but what if it changed the mask itself?).

If a sub-drawable wants its own clipping mask, it can increment the stencilReference member variable of the StencilSettings object it got in its draw call.  This causes it to write different values to the stencil buffer than its parent, which can then be traced over without affecting the parent's "portion" of the screen.  This is somewhat complicated, however--I will fully admit that.

https://github.com/SFML/SFML/compare/master...Kipernal:master#diff-ee21bf2821ee1b97f677cabb9bb964c5R33
This isn't how you are supposed to add gl functions.

Of course not. But as I mentioned in the comment above that line, it's not in GLLoader.hpp and I didn't want to mess with that file.  I figured I'd let whoever was in charge of knowing exactly how to handle it take care of that one single function.