Issue with Retina Display and gl_FragCoord on OS X

[thomas9459]

So I was playing around with shaders and I encountered this bug. Basically, when a window is switched between a Retina display and a non-high DPI display, gl_FragCoord (in a fragment shader) begins reporting incorrect values. In this gist there is some example code that can be used to reproduce this issue. Here are some pictures that demonstrate this issue in action.

This first one is what the program looks like when launched on the non-DPI screen:


And this is what it looks like on the Retina display:


My guess as to what is occurring here is that when the application switches into "high-DPI" mode, the screen size doubles, but SFML attempts to hid this in its own API (see here, here, here, etc.), but OpenGL is still aware of the change and changes gl_FragCoord accordingly, so that they now differ from what SFML would report. This explains why above the range of values reported by gl_FragCoord double, as evident by the original gradient being shrunk down to a quarter of the size in the second image. Note that the problem is the opposite if the window is originally launched on the Retina display: then the image displays normally on the Retina display, but if the image is moved to the low-DPI display, then it appears "zoomed in" by a factor of 2.

I have narrowed down the bug to this commit (which makes sense, as it's the one that added support for high-DPI displays). I don't really know how to fix this issue, or if it is even possible. Any ideas?

[eXpl0it3r]

SFML currently doesn't support multi-monitor setups. It's not aware that there are different monitors in use and thus can't really adjust itself.
But maybe it still can be fixed...

[Hiura]

Are you using SFML 2.2 or the last master version?

It was further improved by https://github.com/SFML/SFML/commit/7d4235a38fdda3804a706946d8de6bc192d3fdc2

The scaling factor should be updated when the window is moved to another screen but I never could test it since I don't have an external display. Maybe there is also some issue with the context...

[thomas9459]

Yes, this problem persists into 2.2, and the scaling factor is updated just fine. I believe I figured out what is causing this issue though. When a window is moved from a low-DPI display to a Retina display, the number of pixels doubles, but the number of "points" stays the same ([urlhttps://developer.apple.com/library/mac/documentation/GraphicsAnimation/Conceptual/HighResolutionOSX/Explained/Explained.html#//apple_ref/doc/uid/TP40012302-CH4-SW1]this[/url] explains the difference). However, since OpenGL only works with pixels, it thinks the rendering area has doubled and then adjusts gl_FragCoord and other values accordingly. In face, SFML itself has no concept of "points" either, as evident by the fact that value of window.getSize() also doubles once the window switches displays.

So, this seems not to be a bug after all, but rather just some subtleties in how OpenGL and SFML handle the situation. While everything is technically okay as it currently is, I would like to make two suggestion to improve this for the future:

• Trigger a resize event when the window changes scale factor, as that actually changes the size of the window measure in pixels
• Eventually, add full support in SFML for distinguishing between pixels and points. This may be overkill, especially if this only affects OS X (I'm not sure how other systems handle high-DPI displays), but this is needed for "complete" DPI independence.

Obviously, the second suggestion would have to wait until SFML 3.0 because of API changes, but the first one could be implemented now without any issues, and would allow the app to become aware of the change (whereas right now one has to constantly check window.setSize() to detect the change).

[Hiura]

I'll try to dig up an old monitor to test that but it won't happen before mid January... In the meantime, can you create an issue? I think, even though SFML doesn't provide (yet) any API for multi monitor setup, we might be able to fix it with your first solution.

And if you want to implement it, you're welcome, of course! 

[thomas9459]

So implementing that first solution was rather easy, so I went ahead and did it (here is the pull request), although you will definitely want to look over it to make sure I didn't do something stupid, as those 7 lines of code represent half of all my experience with Objective-C. I also updated the gist so that the problem and solution are more prominent.

As for supporting points vs pixels natively in SFML, do you think I should create an issue or thread to discuss it, or is it not something that is appropriate for SFML?

[Hiura]

As for supporting points vs pixels natively in SFML, do you think I should create an issue or thread to discuss it, or is it not something that is appropriate for SFML?

Another thread would be better since this should be OS agnostic. 

[BlueCobold]

• Eventually, add full support in SFML for distinguishing between pixels and points. This may be overkill, especially if this only affects OS X

It also affects iOS and I think such a function is somehow needed. If not provided by SFML, the OS X and iOS developers have to write some themselves - which imo sounds more plausible.

The situation is this: A non-highres-application launched on retina screens will result in having the double size (each pixel gets doubled in each direction) - an 800x600 window will actually be 1600x1200 pixels large. The fragment shader however doesn't know about this and only sees every fragment - which means it sees double as many pixels than on a non-retina-screen. Launching the same app *with* high-res-capability stops doubling the pixels. The result is though that a 800x600 window really is only 800x600 pixels in size which is much smaller. Apple added this system because apps which run only at low resolutions are stupidly tiny on retina displays. So Apple simply doubled their size virtually. The graphics-card and fragment-shader notices that of course.

To solve this, you would have to send the scale-factor of a non-highres-app on retina screens to your shader (which means either SFML or you yourself will have to provide this factor) *or* use only high-res-apps ("High Resolution Capable" in info.plist).