IIRC you need to re-setFont on the text objects if you change the font.
Additionally loading the font in a separate thread isn't really adviced. OpenGL isn't multithreaded and you need to synchronize the font object, making the threaded loading pretty much pointless.
If you do run into performance issues due to loading, I recommend you just load the file from disk into memory in a separate thread and the call loadFromMemory in the main thread.