Hi,
That sounds like a cache related issue. You probably have asset/page caching activated. For logged in users, usually cache is not active (especially for administrators), thus everything is fine. Once logged out, an older version of the cache is served, when the plugin styles were non-existent – resulting in an unstyled box.
Clearing all layers of cache and refreshing everything should resolve the issue.