I'm not in a position to try the code right now but after a cursory glance the only thing which stands out is that you're not using the unit conversion functions when creating the the player body.
b2PolygonShape playerShape; playerShape.SetAsBox(32, 32);
This is creating a 32 metre by 32 metre box which, in SFML units, is 3200px * 3200px with the default scale, which is pretty big. Box2D is probably trying to force the giant body out when it collides with the rest of the map. If you want your player to be 32px * 32px try
playerShape.SetAsBox(tmx::sfToBoxVec({32.f, 32.f}))
Alternatively if you want a more 'prebaked' solution my game framework
xygine has support for tmx maps with box2D physics.