And yes, I need physics -- if I remove it, it will take too much of the game experience away.
Are you sure you really need a full blown physics engine? Or if all your after is collision detection (running, jumping, falling, colliding into walls, basic platformer stuff) you might want to consider implementing your own system.
Right now I'm using Box2d, but it is very.. "cluncky" and "sloppy". I want more have more control over it. It feels like it takes like hours to just change the maximum movespeed.
Sounds like you need to adjust the velocity that your applying to your character. Yes getting it fine tuned can take some work, but I have never experienced Box2D feeling clunky.
And that is after I've written nearly 800 lines of "special physics rules" just for the player actor
I'm not a Box2D guru, but I think you should show some of this code. As far as I know in Box2D you can define collision callbacks, adjust speed, velocity, and other properties, but I never heard of writing "physics rules". Unless your talking about callbacks that determine if objects collide or pass through each other which would not make your player feel clunky.