If you're directly aiming at 3D games, a 3D engine such as Irrlicht or Ogre might also be an idea. I wouldn't recommend this however, starting with 2D is definitely more reasonable.
In any case, I think you underestimate this endeavour massively. If you are new to game development, I suggest to get used to the basics first (game loop, rendering, vector algebra, collision detection/response, animations, particles, ...). SFML is an ideal library to begin with, because it offers a lot of high-level features. There are also some interesting extensions to SFML that make your life easier, see
here.
Also, make sure you have a decent amount of knowledge about C++; one of the most often made mistakes is trying to learn game development without really knowing the programming language. People believe they save time by skipping not so interesting topics in the beginning, but eventually they need to tediously fill the gaps and produce much worse code that results in wasted time for development and debugging.