and checking the memory in windows task manager.
That's not how one accurately messures memory changes and mostly never tells anything about the actual memory usage. The OS can assign as much memory as it wants and release whenever it wants.
How would I go about checking the GPU memory?
Probably with some GPU tools, I don't know.
Do you already have something running or even run into performance issues, or are you just paranoid about memory usage?
As for the plain calucaltion of an image size: width*height*4*8 bits
4 because we have 4 Uint8 values for each channel in RGBA.