You cannot remove the LABEL tag but you can hide it. To do so click Edit link of the particular field, go to Appearance tab and set a CSS class. After that add a CSS rule like the one below in your public site theme. Also, you can use " " (space) for label text.
As for the button, you can add CSS rule in your public site theme to make the button an image.
background: transparent url("") no-repeat 0 0;
Let us know if you need further help.
the Telerik team