Java integration features

[pdinklag]

While the first release of JSFML will be more like a direct binding to SFML standing by itself, in later versions I would like to build up a range of features to integrate JSFML into Java environments.

Generally, to be able to tell apart between pure SFML bindings and Java integration features, new classes for this task will get a different root package. I am still split about whether to call it org.sfml.java or simply org.jsfml, but I'm leaning towards the latter.

What follows below is a list of goals I want to achieve, a todo list so to speak, which I would like to discuss and possibly extend using this discussion thread.

• RenderCanvas
  This feature will be a major milestone. It is a Java canvas that can be used as an SFML render window, thus making it possible to have OpenGL contexts in any Java AWT / Swing user interface, including browser applets.
  
  
• Type conversions
  I will introduce several conversion methods between common SFML and Java AWT types. This includes:
  • org.sfml.graphics.Color ↔ java.awt.Color
    
  • org.sfml.graphics.FloatRect ↔ java.awt.Rectangle
    
  • org.sfml.graphics.Image ↔ java.awt.image.BufferedImage
    
  • org.sfml.graphics.IntRect ↔ java.awt.Rectangle
    
  • org.sfml.system.Vector2i ↔ java.awt.Point
    
  • org.sfml.system.Vector2f ↔ java.awt.Point
    
  The conversion methods will come as special copy constructors and toXYZ() methods in the SFML classes.
  
  
• Implementable Drawable class
  This will be an abstract subclass of org.sfml.gaphics.Drawable, allowing custom drawables to be implemented in Java. This can be used to create simple extensions of existing drawables such as compounds, but also to create a possible link between JSFML and LWJGL, which would also allow 3D rendering via JSFML.
  
  A note on that: JSFML is not to be seen as some sort of competitor to LWJGL. LWJGL focuses on providing direct access to OpenGL, OpenAL and OpenCL functions for Java, while JSFML focuses on providing a binding to the straightforward SFML library. There are some aspects that are covered by both libraries (especially concerning input), but in such cases, the user has to decide what to do.
  
  
• Implementable SoundStream class
  This will be an abstract subclass of org.sfml.audio.SoundStream, allowing custom sound streams to be implemented in Java. This will make it possible to write MP3 decoders in Java, for example, and makes JSFML less dependent of C++ source code. A subgoal of this should be to attempt and connect JSFML to Java's sound API (which I do not know too much about yet). That way, any already existing sound decoder library available for Java could also be used for JSFML.
  
  
• AWT / Swing component rendering
  This is still more of a brainstorming, I am not entirely sure how well this will be possible. The idea is to make it possible to render and possibly even manage AWT / Swing components within an SFML render window. This would be the perfect way to create complex GUIs within JSFML applications, using a toolset every Java developer should be perfectly used to. Since it would also support Java LookAndFeels, there would be practically no limits concerning styles either.
  

[Laurent]

RenderCanvas
This feature will be a major milestone. It is a Java canvas that can be used as an SFML render window, thus making it possible to have OpenGL contexts in any Java AWT / Swing user interface, including browser applets.

SFML, in C++ and other languages, provide a Window/RenderWindow constructor which can take a handle to an external canvas. This is usually enough. Why do you need to provide a dedicated class in Java? Does it require more complicated code?
A potential problem that I can see, is that you will have to provide two versions: a canvas that is a Window, for OpenGL only, and one that is a RenderWindow, for using the graphics module.

The idea is to make it possible to render and possibly even manage AWT / Swing components within an SFML render window.

Good luck
It is usually not allowed to draw on top of an OpenGL area with non-OpenGL functions.

[pdinklag]

SFML, in C++ and other languages, provide a Window/RenderWindow constructor which can take a handle to an external canvas. This is usually enough. Why do you need to provide a dedicated class in Java? Does it require more complicated code?
A potential problem that I can see, is that you will have to provide two versions: a canvas that is a Window, for OpenGL only, and one that is a RenderWindow, for using the graphics module.

On a Java level, there is no way to get a window's handle. Also, not every Swing window or panel has a window handle. It's what I had worked on before, if you remember. The problem was getting the actual handle, and then there were a good amount of issues concerning event handling (because both SFML and AWT override GWLP_USERDATA, I was able to fix this and it already worked fine on Windows).

Good luck
It is usually not allowed to draw on top of an OpenGL area with non-OpenGL functions.

The only idea I have so far is to let Swing render its controls and then convert the result to a SFML texture. Rendering them to an image is perfectly doable, so there should be nothing speaking against this idea other than possible performance issues. I will definitely give this a try.

[Laurent]

Also, not every Swing window or panel has a window handle. It's what I had worked on before, if you remember. The problem was getting the actual handle, and then there were a good amount of issues concerning event handling (because both SFML and AWT override GWLP_USERDATA, I was able to fix this and it already worked fine on Windows).

Yeah, I remember now

The only idea I have so far is to let Swing render its controls and then convert the result to a SFML texture. Rendering them to an image is perfectly doable, so there should be nothing speaking against this idea other than possible performance issues. I will definitely give this a try.

Sounds good. But will user inputs still work if the widgets are rendered to an image instead of being directly integrated to the application window? How will focus/keyboard/mouse things work?

[pdinklag]

How will focus/keyboard/mouse things work?

I guess the only way to accomplish that is to eat SFML input events and somehow emulate AWT events and give them to the components. This will need a good amount of experimentation, I'm not sure how well this will work. But I can't really imagine that it isn't doable.

[pdinklag]

I just did a little experimenting with synthesizing mouse events and passing them to a root component. It works perfectly! Even focus traversal seems to work correctly (that is, focus within the Swing focus manager, I am not yet sure what happens to the actual OS focus). I believe keyboard events (in the end it only comes down to mouse and keyboard events) will not be any more problematic.

Translating SFML input events to AWT events should be easy and not performance expensive. So I think this feature might become reality. All I need to do is find a quick way to convert AWT images to SFML textures.

[pdinklag]

I've been doing some more concrete experiments with having Swing GUIs rendered by SFML, and as expected, I hit on a bunch of problems.

The awesome news is, the concept works and it will be possible! But there is quite some work ahead.

Event dispatching will probably require me to implement my own simple focus manager, or get the existing Java one to work with JSFML somehow. The latter will be extremely tough, especially if you consider that you can add Swing GUIs to SFML windows inside another Swing GUI...

Another problem is painting. Apparently, after dispatching events (such as mouse enter events), control repainting does not have an effect immediately. I can't quite explain this behaviour yet, but I know that repainting the GUI every frame seems to be quite a performance killer and should not be the way to accomplish things.

I also failed to get the contents of JScrollPanes (and probably any other pane with embedded views) to render, but I'm fairly certain that there's some tricks to get them to work.