Well, after struggling with this for a bit I thought maybe I should re-install the library from scratch and make sure I set the same configurations to all platforms from the beginning and see if that fixes the problem, but on the download page it says it's recommended that you compile for 32-bit machines to reach the biggest possible audience, since a 64-bit machine can perfectly run a 32-bit app, so I guess I don't have to do any of this after all, should've done my research instead of assuming compiling to 64-bit would have some hidden benefit.
Well, thanks either way, hope this helps someone else in the future.