You can hack a faster solution with an OpenGL call:
sf::Texture::bind(&texture);
glTexSubImage2D(GL_TEXTURE_2D, 0, x, y, width, height, GL_BGRA, GL_UNSIGNED_BYTE, bytes);
Depending on how the driver implements the conversion, the speed difference may be more or less noticeable.
However this may break the internal cache of SFML. You can make it almost safe by preserving the texture binding:
GLint textureBinding;
glGetIntegerv(GL_TEXTURE_BINDING_2D, &textureBinding);
sf::Texture::bind(&texture);
glTexSubImage2D(GL_TEXTURE_2D, 0, x, y, width, height, GL_BGRA, GL_UNSIGNED_BYTE, bytes);
glBindTexture(GL_TEXTURE_2D, textureBinding);
Try with caution