Yes, stupid that you can't delete themes now. What potato came up with that idea?
To manually remove them. FTP in and delete the themes from the wp-content/themes folder
When I delete them from wp admin they stay deleted. Not sure what you are doing different.
It may be the "auto upgrade" option causing it to rewrite the old files.