Ran into the same situation. Just couldn't get Sitefinity working properly on our load balancer, which runs on a round-robin configuration.
What we ended up doing was selecting one of our 4 servers and assigning that one the task of Sitefinity administration. So we created a separate web site on that server, and set up redirects from the 4 "public" sites so that if anyone hit up the /sitefinity path, it would redirect to this sitefinity site running only on the third server. Then we set up local hosts file entries to allow people to map directly to this web site, bypassing the load balancer.
It might be a little wonky from an architecture standpoint, but we don't need loadbalancing for the back end, just the front-end.