We were able to reproduce the issue on some computers today. Still not sure what is causing it, but here is what you can. When you add a tag that you want to delete, just save the content right after you add the tag. Then edit the content item again, and click on the delete icon of the tag. This time, the tag should be deleted as expected.
We tested the issue with the current internal build, and everything is fine with it. In fact I tracked down the tags handling, and saw this area optimized. We are going to release 3.7SP1 next week.
I have updated your Telerik account for the cooperation you have provided though.
the Telerik team