When you are not logged in you should not see the browse and edit buttons at all. When logged in, there's a cookie saving your selection (whether Browse and Edit is on or off) So if on a certain page you've selected browse and edit to be on, an if you close this page, next time you open it, the links will be shown, if you switch it off then next time you visit this page they'll be off - that's the default behavior of the widget. Of course, if you do not need this functionality, or you don't want ti displayed anymore, you can just set EnableBrowseAndEdit to False (from ~/Sitefinity/Administration/Settings/Advanced ->Pages).
Please provide some more information if the issue you are reporting differs from what I've described above (if you could include a video snapshot describing how the problem reproduces it would be very useful).
the Telerik team