Unit Testing

[MorleyDev]

I like to explain it as, whenever I write code I want to see it work and see it work immediately. Testing something inside the running program can take a lot of effort and time, firing up the program and getting it to the exact scenario and point that triggers the code, and so is done when the feature is all written to make sure it's actually working.

But along the way, I'm a serial recompiler. I want to see everything compile and work. Now, to do that quickly I want to isolate the code as I'm writing it and call the functions to make sure they behave as expected. So pre-encountering TDD I would be keeping a separate main function that let me run what I was working on and see it work.

Effectively, I was constantly writing and throwing away unit tests. Post-TDD I have a framework that helps me write and keep those tests, so I can keep them to document the expected behaviour of those functions and even run them when I make further changes to make sure I haven't broken any past behaviour (and naturally I can modify them when I want to break past behaviour).

It also helps that practising stricter TDD tends to self-enforce a lot of SOLID principles. The real trick comes in learning to 'read the tests', namely accepting that if testing is becoming difficult there's a reason for that. Testing becoming difficult is a code smell, and often is indicative of code hitting a point where it must be refactored and evolved.

And whilst very useful for business applications, client/server applications especially, for programs like games ATDD (automated testing of your acceptance criteria/high level specifications) is dark witchcraft and may actually be more effort than it's worth. I have heard there is a whole team at Microsoft dedicated to writing AIs to test people's games via friend-of-a-colleague, but that's really all I know on the subject.


The tricky stuff for me comes in how testing integration with 3rd party libraries is something of an art-form in and of itself. I've yet to find a way I'm happy with, especially for C++.

In languages like C# and Java there are libraries that (ab)use the reflective features of the language to let you mock out real classes instead of interfaces, and C++ has a couple of libraries that abuse the low level memory access and inherent understandings of how specific compilers work to achieve a similar result but it's not cross-platform and can break from compiler version to compiler version (especially if you like to work with the latest and greatest of everything).

Typcially the solution involves a facade later around the 3rd party library that must be extensively tested manually, or in passing delegates/individual functions (via std::function) that are easier to mock and test. Neither of these are ideal but they are the lesser evils in my mind, at least for C++.

[Tank]

Nice writing, and full ACK.

[wintertime]

int sum( int a, int b ) {
  return 3;
}

Oh yeah, I read you should only add the simplest thing to satisfy the test.

That's true, but why on earth do you want to trick yourself? ;-) That's what a lot of people get wrong as well: Unit testing doesn't mean to find ways to make everything fail, it's there to make sure that the stuff you write works as expected.

Returning a constant 3 in a function named "sum()" is clearly a logic error. If that's what you intended, then it's 100% bug-free. Again, illogical stuff is not a bug!

Thats not about deceiving oneself. From what I read about TDD its good to remind oneself when the test is not specific enough to warrant the full real implementation if you can cheat it that easy, then add one or two more tests to improve coverage (like with bigger or negative numbers in that example) and only then add the real thing.

But the things you would gain most from having tests for are also the most difficult to write the tests for (thats what I'm still struggling with).

Can you give an example? I'm willing to help.

I dont have a concrete example atm. One thing is when its something that needs to call other functions with sideeffects or methods on classes which are non-virtual or the class did not derive from an abstract baseclass or like mentioned above interacting with a third party library.
I guess most can be wrapped up in a new class and then some baseclass and a stub/mockup written and calls checked, but that seems like very tedious tripling of the needed code. Or maybe adding extraneous callbacks everywhere just to make it easier to stub something out could help, but that feels so wrong.

[Tank]

I see. Well, if a function is used wrong, I tend to catch that with assert()s, and not by writing test cases that check for errors. I only check for success.

Doing integrations tests can be indeed time-consuming, but it's often (not always) a good sign of the need of refactoring if the task is going to be extremely massive. It's different with 3rd party libraries, but to be honest I haven't had to do much stuff in that regard yet.

[Lo-X]

Hi Tank,

As a user of unit tests with PHP I wanted to use them as well with my C++ jukebox project, even if it is already started.

But the project as a major GUI aspect, so I don't see very well what kind of unit test I can do. As a matter of fact, I've only planned tests on model classes that play with the SQL database. I don't see well other tests I could make.

Anyway, thanks for the links you provided.

[MorleyDev]

But the project as a major GUI aspect, so I don't see very well what kind of unit test I can do. As a matter of fact, I've only planned tests on model classes that play with the SQL database. I don't see well other tests I could make.

Testing the GUI can be automated through a few tools. First of all, you can test everything after an action with the standard methods of dependency injection. Your GUI code should be as shallow as possible anyway (bolded for emphasis). The GUI layer is only responsible telling your underlying system what to do or being told what to do by your underlying system, and those things can and should be testable.

As for testing actual GUI actions (i.e a click), usually that is preserved for more end-to-end tests (like what one would do with Selenium) and depends on the system. I'm not familiar with how to do so on Windows, but this wikipedia page is probably a start.

On Linux, however, this is actually surprisingly simple. You can expose your GUI to the accessibility layer and drive your tests through that. People should be doing this anyway for blind users. By making it testable you also increase the number of people who can use your software. Win win.

For testing SQL interaction, there are a few ways to do it. End-to-end and integration tests often involve using a local SQL server you control, building the server up to contain the information you want at the start of the test and cleaning it at the end. There are probably some SQLite based libraries for this, but if you can't find any SQLite alone should cover that need.

For unit level tests, you can mock out the SQL server functions and just verify they are called.

In case you can't tell, I'm a huge fan of fractal TDD. End-To-End acceptance tests at the top, and drill down to specification tests one step below and then all the way to individual classes and functions.  Always mocking/stubbing out dependencies as you go to the degree that level calls for.

At the highest level, you mock nothing. These tests make sure it all fits together. At the specification level, you stub externals such as servers and databases. Your specification calls for you to interact with them in certain ways, you aren't testing they behave as expected at this level. At the unit level, you mock out anything that isn't inside the class/function being tested. Your classes specify they behave in certain ways, if it's not the class you are currently unit testing you don't care if they actually do or not because their unit tests will guarantee that.

[Lo-X]

Thanks, I'm on linux so I'll look at it =)