Thank you for posting your question.
By default, Sitefinity checks the cache state, which is in this case stored in the database, on every 1000ms. This means that there could be a slowdown on updating all the servers which is equal to 1000ms + the time for executing the db queries. I do not think this is the particular case though.
Can you please check both servers computer clocks? Since we are comparing the time, both servers should have approximately the same time.
Other than that, the sitemap cache could not be disabled, but we do not have problem there, reported so far.
the Telerik team