C#/C++ build comparison

[Bobbo]

I wrote a 500 line game in C#, simple space shooter, one player, one AI opponent, scrolling backgrounds, text on screen.

I then ported it to C++ and setup both builds to run a series of programmed sequences for 30 seconds and report the amount of frames produced.

The C++ build produced on average 1% fewer frames than the C# build.

Does this kind of result make sense? I would have assumed (clearly wrongly) that C++ would outperform C# given the library is natively C++.

[ChronicRat]

M.b. you are just bad C++ programmer? =)

[zsbzsb]

Really? You are comparing two languages by how many frames they render in a set period of time? Just..... 

Does this kind of result make sense?

No, not at all because you can't even statistically compare languages this way.

[Bobbo]

M.b. you are just bad C++ programmer? =)

That may be

Really? You are comparing two languages by how many frames they render in a set period of time? Just..... 

Does this kind of result make sense?

No, not at all because you can't even statistically compare languages this way.

I'm not comparing languages, I'm comparing the performance of builds.

[SuperV1234]

How about posting the code for both C# and C++ implementations before drawing conclusions?

[Bobbo]

How about posting the code for both C# and C++ implementations before drawing conclusions?

I haven't drawn a conclusion, nor did I state one.

[SuperV1234]

That's true, sorry if it sounded like I was attacking you.
But it looks like you're trying to say (even if this is not your intention): "Aha! I finally proved that C++ is not faster than C#!" ... without even posting the code of your benchmarks.

I should have chosen better words.
Anyway, could you post the code of your benchmarks?

[zsbzsb]

I'm not comparing languages, I'm comparing the performance of builds.

Then your title is severely misleading. Even if you claim to only be comparing builds, the test is still extremely flawed.

[Bobbo]

But it looks like you're trying to say (even if this is not your intention): "Aha! I finally proved that C++ is not faster than C#!" ... without even posting the code of your benchmarks.

If anything, I express confusion at my results.

Then your title is severely misleading. Even if you claim to only be comparing builds, the test is still extremely flawed.

The content of my post clearly talks about build performance. I have changed the subject to hopefully encourage people to read the content more closely.

It might also be useful to explain in what way the test is extremely flawed.








I cannot share the code involved in my test. I will create a new test and see if I get similar results. If so, I'll happily post the code.

[Jesper Juhl]

As I see it, the test is flawed in that measuring frames per second is a bad metric. It probably has less to do with the programming language used than whether or not vsync (or some other frame rate limiting mechanism) is in play, and the quality/performance of your graphics drivers and their OpenGL implementation, than anything else.
And a measured 1% difference is not something I'd call significant regardless of what you measured - it's likely in the statistical noise and likely to fluctuate from run to run.

[Bobbo]

As I see it, the test is flawed in that measuring frames per second is a bad metric. It probably has less to do with the programming language used than whether or not vsync (or some other frame rate limiting mechanism) is in play, and the quality/performance of your graphics drivers and their OpenGL implementation, than anything else.
And a measured 1% difference is not something I'd call significant regardless of what you measured - it's likely in the statistical noise and likely to fluctuate from run to run.

I'm not measuring frames per second, I'm measuring total frames generated over a time period, using the same system for each build's run (each built for x64 release), where most work done in the tight loop is in the SFML library (mostly draws). Also, the frames measured over my 30 second time frame is in the tens of thousands, so vsync is not in effect if I understand this correctly.

Would the above still mean the test is not effective as a measure of each environment's raw performance? If so, what is a more effective way to test perfomance of the library in each environment?

[Laurent]

I'm not measuring frames per second, I'm measuring total frames generated over a time period

You're not measuring frames per second, but a number of frames over a number of seconds?

where most work done in the tight loop is in the SFML library (mostly draws)

So you're comparing SFML to itself? What's the point? If it's to measure the overhead of the C# -> CSFML -> SFML bridge, I can tell you for sure, without any benchmark, that it is negligible compared to the overhead of the draw call (mostly spent in the OpenGL driver).

Would the above still mean the test is not effective as a measure of each environment's raw performance? If so, what is a more effective way to test perfomance of the library in each environment?

You don't need to do that. Since you're testing a binding, and not a port, you'll always be comparing SFML to itself.

And next time you want our feedback on a benchmark, the first thing to do is to post the code. Just saying that you did something doesn't mean that it is done right.

[Bobbo]

So you're comparing SFML to itself? What's the point? If it's to measure the overhead of the C# -> CSFML -> SFML bridge, I can tell you for sure, without any benchmark, that it is negligible compared to the overhead of the draw call (mostly spent in the OpenGL driver).

The point is exactly that, to see how much time is lost in the translation. I took the sample hello world program from the tutorial and got the same result. The C# build calls draw ~260K times in 30 seconds on my system. The C++ only manages ~250K (both X64 release builds). As you rightly say, there should be some, albeit negligible, overhead in the C# build, but I don't see it, and it's slightly baffling to me.

[Jesper Juhl]

Without seeing the code it's all just random guessing.. And my crystal ball is not being helpful atm.
If you want to see where time is spent, try a profiler...

[StormWingDelta]

Next challenge.  Do the comparison without a graphics lib only in the console and see what you get.


That aside the way you are comparing these two versions of SFML will yield invalid data.  You're comparing FPS to compare the performance of two languages and the way you are isn't going to give you anything useful.

Also since we can't see your code we can't tell if you messed up somewhere so unless you are working on something that the code has to be hidden on why not just show us already?