-
Hi all, is it possible with SFML to capture a portion of the users screen (what's being displayed on their monitor as well as hidden views (when have multiple monitors)) and quickly display it in real-time inside the SFML app?
If so how should it be done? Should I use sprites or textures?
If SFML doesn't have such a feature, does anyone know how I can achieve this and then use SFML to display it?
Thanks in advance 8)
-
Are you creating an invisibility cloak? lol!
I'm not sure its trivial to do, but you will probably be using the OS API to fetch that bitmap data.. The problem is that buffer you get from the OS will usually include your own window too.. Not sure how to solve the problem..
Assuming you could do that and get the screen bitmap data you could easily render it to the entire window and achieve your invisibility cloak :P (don't forget to make the window borderless and without title bar)
-
OS will usually include your own window too.. Not sure how to solve the problem..
No no... it's okay if it captures the apps window too. I just wanted to emphasize the fact that I wanted to capture the screen, as in, 'the users whole screen', not the 'apps screen'. :P
-
You almost definitely need the API of OS and to watch out which frame you capture (one with your window in it or not etc.) but you can hide own window with setVisible(false); if you need to.
-
:'(
Is there anyone experience in this that can make a working code? I have looked at almost a dozen codes through google and haven't found anything that works, let alone integrate it into SFML.
-
http://stackoverflow.com/questions/3291167/how-to-make-screen-screenshot-with-win32-in-c
After googling for 30 seconds I found that, it seems to do what you want (hopefully). You just do that and then use the HBITMAP structure to extract the pixel data, check the windows API documentation on how to do that.
From there, you can easily convert to a sf::Image and after that to a regular sf::Texture to use with a sprite. Be careful with pixel formats tho, might get tricky if you are not paying attention :)
-
Yeah, I already saw that one. I cannot get the code to build however.
-
Then don't stop at these errors and google (or whatever is your favorite search engine) them, take a look at the documentations of the function you use, etc. ???
-
Ill give you a tip on how to solve the first error, you search the rest.
Many windows API functions come in two variants, when they deal with strings, to receive ansi and wide strings. They are identified by a A or W suffix, respectively.
In your case, you are calling CreateDC, which is redirected automatically to CreateDCW, as it is assuming wide string by default. Try explicitly calling CreateDCA OR actually send a wide string to CreateDC by prepending an L before the string (L"MyWideString")
-
Here is an update of the code with some help from other people in other forums who know a bit more about windows functions.
#include <Windows.h>
#include <SFML/Graphics.hpp>
int main()
{
// (1) get the device context of the screen
// Definition: The CreateDC function creates a device context (DC)
// ..for a device using the specified name.
HDC hScreenDC = CreateDC(L"DISPLAY", NULL, NULL, NULL);
// (2) and a device context to put it in
// Definition: The CreateCompatibleDC function creates a memory
// ..device context (DC) compatible with the specified device.
HDC hMemoryDC = CreateCompatibleDC(hScreenDC);
// (3) Gets the users screen size in x and y dimensions
// Definition: The GetDeviceCaps function retrieves device-specific
// ..information for the specified device.
int x = GetDeviceCaps(hScreenDC, HORZRES);
int y = GetDeviceCaps(hScreenDC, VERTRES);
// (4) maybe worth checking these are positive values
// Definition: The CreateCompatibleBitmap function creates a bitmap
// ..compatible with the device that is associated with the specified
// ..device context.
HBITMAP hBitmap = CreateCompatibleBitmap(hScreenDC, x, y);
// (5) get a new bitmap
// Definition: The SelectObject function selects an object into the
// ..specified device context (DC). The new object replaces the previous
// ..object of the same type.
HBITMAP hOldBitmap = (HBITMAP)SelectObject(hMemoryDC, hBitmap);
// Definition: The BitBlt function performs a bit-block transfer of the
// ..color data corresponding to a rectangle of pixels from the specified
// ..source device context into a destination device context.
BitBlt(hMemoryDC, 0, 0, x, y, hScreenDC, 0, 0, SRCCOPY);
// Don't need this apparently
//hBitmap = (HBITMAP)SelectObject(hMemoryDC, hOldBitmap);
//////////////////////////////////////////////////////////////////////////////////
// Create the SFML Window
sf::RenderWindow window(sf::VideoMode(800, 600), "SFML works!");
// Load the bitmap from memory into texture
sf::Texture Texture;
Texture.loadFromMemory(hOldBitmap,sizeof(hOldBitmap));
// Load the texture into a sprite
sf::Sprite Sprite;
Sprite.setTexture(Texture);
while (window.isOpen())
{
sf::Event event;
while (window.pollEvent(event))
{
if (event.type == sf::Event::Closed)
window.close();
}
window.clear();
window.draw(Sprite);
window.display();
}
// clean up
DeleteDC(hMemoryDC);
DeleteDC(hScreenDC);
return 0;
}
Still crashes however when run... Though I think it is an SFML issue. However, I'm not confident that the Windows part is completely right either..
-
Here is an update of the code with some help from other people in other forums who know a bit more about windows functions.
...
Still crashes however when run... Though I think it is an SFML issue.
"Hey, I went to a forum where people know more than you guys. They are so awesome that they fixed my 2 missing symbols, which solution was already given by you guys previously. But besides all the smartness, it doesn't run at all, because I am just doing something random when uploading my bitmap data, something which ALSO been warned about before in this forum of people that don't know enough about windows functions."
Good luck, I give up
-
You misinterpreted/exaggerated what I was trying to say. Don't take it personally. ::)
What I meant to say was that I asked some people who understand what I'm trying to do for the Windows part and then update my thread here... (so if the code changed you didn't think I was just randomly changing stuff - for the windows part, anyway)
This is an SFML forum after all, I can't expect anyone here to know Windows stuff, let alone specifically what I'm trying to do. It's hard to find people who know both SFML and Windows to a high degree, and I cannot figure out the Windows stuff out on my own since it's completely new to me.
-
You don't need to know anything to a high degree when you have good documentation. That's the whole point of writing documentation, so everyone has access to the API. Speaking of API, if you took some time to google how to extract pixel data from a HBITMAP and then put it in a sf::Image like we told you, your problem would be solved already.
http://msdn.microsoft.com/en-us/library/windows/desktop/dd144850(v=vs.85).aspx
Speaking of high degree knowledge, that's not what this is.. This is a trivial task you were able to just copy paste from stack overflow.. You didn't need to be einstein to prepend an L in your strings without getting to winAPI tech support forums :p
-
I'm sure you are like some people, very good at reading APIs and Wikipedia articles, even if its about things you know nothing about. Not everyone is like you. Most of the time when I read APIs I get completely lost and have no idea what they're on about.
If I had a bit more knowledge in these things maybe it would be easier...
Anyway, it's hardly my fault it's so damn difficult! That you need to type so much code, read so much documentation to do something this simple... And then when you finally have a 'bitmap' stored in memory you still can't use it, because it's a different type of 'bitmap'... >:(
-
Okay mate. I am sorry I was rude to you. I still think your behavior wasn't the most appropriate but I want to apologize for mine.
Give me some seconds and ill find this out for you and paste the code here.
-
It's okay, I don't want anyone to be unhappy here. :-*
Maybe I should have pointed out that I am building my first C++ application (if I exclude 'cout << Hello world' and 'SFML Works!'). ;D
It's not that I want people to do everything for me and I just lay back. I would just like someone to explain in layman terms how to do things so I can both get the job done as well as understand what I've done.
-
I have it rendering the screen but it seems to be glitchy, aspect-wise. Let me fix this and I give you the code.
It's not that the concepts present in this problem are hard, but there are tons of details you need to understand first, so you can understand the whole thing. Windows API is like that. It will have a function ready for anything you want to do, but sometimes it will just tricky to get it working because there are little details to using it that are, to say the least, obscure.
-
// (1) get the device context of the screen
// Definition: The CreateDC function creates a device context (DC)
// ..for a device using the specified name.
HDC hScreenDC = CreateDCA("DISPLAY", NULL, NULL, NULL);
// (2) and a device context to put it in
// Definition: The CreateCompatibleDC function creates a memory
// ..device context (DC) compatible with the specified device.
HDC hMemoryDC = CreateCompatibleDC(hScreenDC);
// (3) Gets the users screen size in x and y dimensions
// Definition: The GetDeviceCaps function retrieves device-specific
// ..information for the specified device.
int x = GetDeviceCaps(hScreenDC, HORZRES);
int y = GetDeviceCaps(hScreenDC, VERTRES);
// (4) maybe worth checking these are positive values
// Definition: The CreateCompatibleBitmap function creates a bitmap
// ..compatible with the device that is associated with the specified
// ..device context.
HBITMAP hBitmap = CreateCompatibleBitmap(hScreenDC, x, y);
// (5) get a new bitmap
// Definition: The SelectObject function selects an object into the
// ..specified device context (DC). The new object replaces the previous
// ..object of the same type.
HBITMAP hOldBitmap = (HBITMAP)SelectObject(hMemoryDC, hBitmap);
// Definition: The BitBlt function performs a bit-block transfer of the
// ..color data corresponding to a rectangle of pixels from the specified
// ..source device context into a destination device context.
BitBlt(hMemoryDC, 0, 0, x, y, hScreenDC, 0, 0, SRCCOPY);
BITMAP bm;
::GetObject( hBitmap , sizeof(bm) , &bm );
BITMAPINFO bmpInfo;
bmpInfo.bmiHeader.biSize = sizeof(BITMAPINFOHEADER);
bmpInfo.bmiHeader.biWidth = bm.bmWidth;
bmpInfo.bmiHeader.biHeight = -bm.bmHeight;
bmpInfo.bmiHeader.biPlanes = 1;
bmpInfo.bmiHeader.biBitCount = 32;
bmpInfo.bmiHeader.biCompression = BI_RGB;
bmpInfo.bmiHeader.biSizeImage = 0;
bmpInfo.bmiHeader.biClrImportant = 0;
COLORREF* pixel = new COLORREF [ bm.bmWidth * bm.bmHeight ];
GetDIBits( hMemoryDC , hBitmap , 0 , bm.bmHeight , pixel , &bmpInfo , DIB_RGB_COLORS );
Image captureImage;
captureImage.create(bm.bmWidth, bm.bmHeight, Color::Black);
int j = 0;
for(int y = 0; y < bm.bmHeight; ++y)
{
for(int x = 0; x < bm.bmWidth; ++x)
{
COLORREF thisColor = pixel[j++];
captureImage.setPixel(x,y, Color(GetBValue(thisColor), GetGValue(thisColor), GetRValue(thisColor)));
}
}
Texture captureTexture;
captureTexture.loadFromImage(captureImage);
And then you may render a sprite normally with captureTexture set. This solution worked flawlessly in my computer. Hope it works in yours too.
-
YAAAY! :D Thanks very much Grimshaw :-* :-* :-*
It works on mine too! Now I just have to get it streaming - though I should hopefully be able to figure that one out on my own.
I do wonder however how efficient it is to load bitmap into an image, then load the image into a texture, then load the texture into a sprite before drawing? (not saying it's wrong... but is this okay?)
-
YAY I got it streaming!!!!!!
Could you please explain how the code works?
EDIT: Updated code. Actually runs better now. Still a tiny bit choppy, but I'm not sure if that's a code problem or maybe I'm maxing out my computer (the capture is after all 1600x900).
Seems like there's many lines of code that I don't need in the stream. Here's my stream code:
/////////// OUR LOOP TO STREAM ///////////////////////////////
BitBlt(hMemoryDC, 0, 0, x, y, hScreenDC, 0, 0, SRCCOPY);
GetDIBits( hMemoryDC , hBitmap , 0 , bm.bmHeight , pixel , &bmpInfo , DIB_RGB_COLORS );
j = 0;
for(y = 0; y < bm.bmHeight; ++y){
for(x = 0; x < bm.bmWidth; ++x){
COLORREF thisColor = pixel[j++];
captureImage.setPixel(x,y, sf::Color(GetBValue(thisColor), GetGValue(thisColor), GetRValue(thisColor)));
}
}
captureTexture.loadFromImage(captureImage);
//////////////////////////////////////////////////////////////
EDIT: Checked the frame-rate, I get about 15FPS. I was hoping to get at least 30-60FPS. Does anyone have any suggestions in how I could speed it up?
-
Screen capture is a rather advanced topic and I don't think you can just call an OS screenshot function and hope to stream that at 30-60 fps. If I'd wanted to stream a screen capture on Windows, I'd most likely start by looking at the source code of Open Broadcaster Software (https://github.com/jp9000/OBS), it's the only software I know that gets a good screen capture going (and it's open source).
-
Why does it slow down? For example...
If I disable updating the texture, I get 30FPS then gradually decreases to about 20FPS
If I enable updating the texture, I get 20FPS > 15FPS.
I think perhaps the way I am streaming it is wrong? (FPS should be stable IMO)
Also, updating the texture seems to drop FPS by 5-10FPS. Maybe there's an alternative? Can I set the texture to some kind of dynamic/streaming mode?
(and yes, this is in 'Release' mode.)
-
Why does it slow down?
Looking at the outcome doesn't help much, instead you should just run a profiler and find out where the time is lost.
Also, updating the texture seems to drop FPS by 5-10FPS. Maybe there's an alternative? Can I set the texture to some kind of dynamic/streaming mode?
Updating the texture is a heavy operation and you can't do anything else with SFML directly. What you actually want to do, to get max performance is locating the "texture" of the screen and copying parts of that to your own texture. I've no idea if it's possible and even less how. I just know it's possible for game capturing, where you inject the game with your code and capture the hook yourself into the back buffer updating. ;)
-
In other words, instead of loading image into texture, actually write the pixel info directly into the already existing texture?
I don't see why it wouldn't be possible... Surely the texture is made up of pixel information..? Just need to find the address in memory and write it in.
EDIT: There's two lines that severely drop the FPS (from a base of 1,300 FPS)
(When commenting out all other code in the loop...)
Drop to about 30FPS
BitBlt(hMemoryDC, 0, 0, x, y, hScreenDC, 0, 0, SRCCOPY);
Drop to about 65FPS
captureImage.setPixel(x,y, sf::Color(GetBValue(thisColor), GetGValue(thisColor), GetRValue(thisColor)));
(this one is in two for loops scanning every pixel)
Drop to about 250FPS
captureTexture.loadFromImage(captureImage);
EDIT: I managed to increase the FPS of the streaming from 15FPS to about 40FPS by disabling the Windows Aero theme!? I suppose this means bitblt has less work to do. But I still must use bitblt otherwise it won't work.
-
To get maximum performances you should do the pixel conversions into your own array (std::vector<sf::Uint8>) and then use Texture::update. In other words, drop sf::Image, which is not suitable for realtime image manipulation.
-
In other words, instead of loading image into texture, actually write the pixel info directly into the already existing texture?
I can only speak for Windows and even then I've very limited knowledge. With Windows Vista and thus Aero the whole "desktop" is hardware-accelerated which means that the stuff you see is actually rendered somewhere on the graphics card and thus it should be possible to hook directly into DWM and grab the texture information, but since Microsoft obviously uses DirectX stuff, I've no idea if the textures can get converted on the fly, otherwise you'd still have to go via CPU/RAM, which then would still have a big impact on the performance.
Drop to about 30FPS
Drop to or by?
BitBlt(hMemoryDC, 0, 0, x, y, hScreenDC, 0, 0, SRCCOPY);
BitBlt is known to be slow, especially with Aero being active (see here (https://stackoverflow.com/questions/7154574/bitblt-performance-with-aero-enabled)).
captureTexture.loadFromImage(captureImage);
As Laurent said, you should use your own vector/array of pixels and update the texture with update();
-
To get maximum performances you should do the pixel conversions into your own array (std::vector<sf::Uint8>) and then use Texture::update. In other words, drop sf::Image, which is not suitable for realtime image manipulation.
Do you mean create my own array, then pass a pointer to the texture update function? Is the formatting top left to bottom right? RGBARGBARGBA? What if I don't need alpha channel?
Drop to or by?
Drop to. (Yes, I know it's a very significant drop from 1,300!)
-
Do you mean create my own array, then pass a pointer to the texture update function? Is the formatting top left to bottom right? RGBARGBARGBA? What if I don't need alpha channel?
Yes. Yes. Yes. Set alpha components to 255.
-
Glad you got it working. Your major bottleneck for that application should be the streaming part. Sending over the network such a huge amount of data in realtime is nearly impossible. That's why so much work is put into creating amazing decoders to stream video even in the worst environments. Its all about making the frames as small as possible to send via network, while still being able to decompress them quickly.
Play with it, but don't expect to achieve a top-notch application for this, unless you use good middleware.
-
Your major bottleneck for that application should be the streaming part. Sending over the network such a huge amount of data in realtime is nearly impossible.
Maybe I got the wrong impression, but with streaming I think he rather meant local "streaming", from screen grab, via SFML to the GPU and the to the screen. ;)
-
By streaming I mean just constantly executing the capture code over and over again, which is just GDI capture then loading the data into SFML for drawing.
Anyway, today I'm gonna get working on that vecor/update method and get back to you guys on how it went.
EDIT: I can't get texture update to work :(
I'm just trying to test out how to update one pixel to get an idea of how to do it.
sf::Texture myTexture;
myTexture.create(1,1);
std::vector<sf::Uint8> myVector;
myVector.push_back(255); // RED
myVector.push_back(0); // GREEN
myVector.push_back(0); // BLUE
myVector.push_back(0); // ALPHA
myTexture.update(myVector);
:-\
Also, should I be using push_back for this??
EDIT: I managed to get arrays to work, but not vectors. Why should I use vectors? (won't it be faster to use arrays - or were you thinking that I would be changing the size of the capture?)
sf::Texture myTexture;
myTexture.create(1,1);
sf::Uint8 myArray[] = {255,0,0,255};
myTexture.update(myArray);
-
Use std::vector to be safe on the memory management side (even if the size is constant, you won't be able to declare an array of this size on the stack). It won't be slower if you use it correctly (avoid dynamic allocations after init).
std::vector<sf::Uint8> pixels(width * height * 4);
pixels[(x + y * width) * 4 + 0] = r;
pixels[(x + y * width) * 4 + 1] = g;
pixels[(x + y * width) * 4 + 2] = b;
pixels[(x + y * width) * 4 + 3] = a;
texture.update(&pixels[0]);
-
80FPS!!!
(http://4.bp.blogspot.com/-JoFrc3NVvOI/T3ak-hEOHFI/AAAAAAAACzg/87bZeeiqTMY/s1600/excited.jpg)
I guess that solves my problem. :)
Thanks very much everyone for your help. I strongly appreciate it. :-*
Especially a big thanks to Grimshaw who took the time to write most of the Win code. ;)
-
Would be nice if you could put the final code somewhere (wiki/your GitHub/etc). ;)