Hmm, I suspect it has something to do with the syncing, so I went and check, well, it does go out of sync for a while (bout a second or less) when you seek, so I thought maybe it has something to do with that.
I fixed that and uploaded it as 1.2.3, tested it with 2 videos and there's no audio stuttering, however, when I tried testing it with 1.2.2, there's also no audio stuttering, it really occurs very randomly for me. Please try downloading 1.2.3 and see if it still occurs.
One more thing, try replacing the openal32.dll and libsndfile-1.dll with the new one (In the SFMLTheora 1.2.3 folder or the sfml2 extlibs folder). I forgotten to replace them when I switched over to sfml2, since the 2 dlls changed since sfml 1.6 (there's a big difference in file size).