Try removing the WebResource handler mapping. Then restart the application, add it again together with DialogHandler
<add verb="*" path="Telerik.Web.UI.WebResource.axd" type="Telerik.Web.UI.WebResource" validate="false" />
<add path="Telerik.Web.UI.DialogHandler.aspx" verb="*" type="Telerik.Web.UI.DialogHandler" validate="false" />
You should also have the following lines in the 4.x based website.
the Telerik team
Do you want to have your say in the Sitefinity development roadmap? Do you want to know when a feature you requested is added or when a bug fixed? Explore the Telerik Public Issue Tracking
system and vote to affect the priority of the items