I am not sure how you have implemented your custom editor, but if the dialog you want to resize is like ImageUrl I suppose you can add a CSS class and set new width and height in ~/Sitefinity/Admin/Themes/Default/PropertyDlg.css (see screenshot).
width: 710px !important;
height: 450px !important;
left: 20px !important;
top: 10px !important;
the Telerik team
Do you want to have your say in the Sitefinity development roadmap? Do you want to know when a feature you requested is added or when a bug fixed? Explore the Telerik Public Issue Tracking
system and vote to affect the priority of the items