Spatial partitioning allows one to skip checking certain objects entirely, since they are grouped in such a way that the algorithm can simply skip the group. In the game I am currently developing, I created a Quad Tree to sort all of my lights and shadow casters in such away that I can easily detect which lights are visible and which hulls its perimeter can reach using some region queries.
The Quad Tree works by grouping objects into groups which are in turn grouped as well, where every group has 1 parent group and 4 child groups.
For more information, read here:
http://en.wikipedia.org/wiki/QuadtreeThe example shows it storing points, but you can store anything you want.
The KD-Tree that Box2D uses works differently from the Quad Tree, it turns the game world into a sort of 2D binary search tree. It has advantages and disadvantages over a quad tree, it is faster for region queries but can be slower if re-balancing is required often (the entities in the game work move around allot).
For more on that, go here:
http://en.wikipedia.org/wiki/Kd-treeOther methods exist too, such as BSP Trees, Octrees (3D quad tree, Minecraft uses it to store chunks), Bins, R-Trees, etc. All have advantages and disadvantages.
Unless you want to ignore parts of your game world completely if too far away (e.g. Minecraft), you will still have to perform some logic on all entities, but at least you do not have to render them all and do note have to do things like collision checks on them all (trees speed it up by only testing against other objects in the vicinity, not all other objects).