How does 3.051758E-05 instead of 0 break your collision detection?
I first move objects in the X axis and check for collision (and fix their position if there's overlap), then do it in the Y axis. Which means that if the position is moved, there should be no overlap. Because of the rounding errors, there's still an overlap, so for example, there could be a collision on the X axis, the position is fixed but because of a rounding error there's still a minuscule overlap, which then potentially breaks the Y axis fix.
EDIT: I've fixed it now by rounding the Position down to two decimals after every collision, which seems to work well enough. Still wondering if anyone knows of any best practice or similar to minimize the risk of such issues.. These problems are of course quite hard to debug so it would be nice to know if there's some standard way of dealing with it.