After removing the project from the Project Manager List, you need to delete the site directory manually. Actually, the physical folder is not deleted because SQL Server database is kept intact in case you need to restore the project. Removing projects is a potentially harmful operation and the idea is to have an opportunity for restoring them.
You can find more information about removing and restoring projects from the project manager list in the User Manual > Working with Web Site Projects.
the Telerik team