Adding a mail-to link to the link selector is pretty straightforward and you should follow the steps in the How to extend the custom RadEditor dialogs
KB article. However, it seems that it is a good idea to add another article which explains how to achieve this case in details. So once I am ready with it, I'll get back to you. Thanks for pointing this out.
Actually, the option to make image links to some other address via the editor dialogs is already implemented in the SP2 of 3.2 version. You'll notice that once you select an already added image to the editor and click on the Insert a link
dialog, the Title text box would not be visible, but only the address field. So in case you are not using this version, you would need to upgrade to it.
the Telerik team