How, for example, does the compiler know where to look for dlls like "opengl32" without my giving it any kind of path to that file, or having that file on my path (to my knowledge. Maybe I do? / same with all of the other dependencies like gdi32, winmm, jpeg, etc. etc.?)
The
compiler doesn't have to know anything, except the location of headers that are included in the compiled code.
The
linker searches for the openal32 import library in all its known paths, and finds it because it is located in its own libraries (yes, OpenGL libraries are shipped with the compiler -- in your case it means that you'll find opengl32.a in your mingw/lib folder).
The
operating system searches for the openal32 DLL (when you run the app that linked it) in locations contained in the PATH environment variable, as well as in the application directory. And it finds it in windows/system32 because your graphics driver installed it there (yes, although the OpenGL library is shipped with the compiler, the DLL is part of the graphics driver -- which makes sense if you think about it).
At first I was actually looking for these files on my system and trying to find/link to the full url and extension
If it's not written in the tutorial, then don't do it. Also keep in mind that the "getting started" tutorials are there for convenience for beginners, a library is not supposed to teach its users how to use their C++ environment. So don't expect every little detail to be written there, although we do our best so that beginners can start with SFML without first reading the full theory of including/linking/executing (but they should definitely do it!).