You can delete a public or private key whenever required.
To delete a key:
Click on the Manage GnuPG keys link in the Services menu area.
Click on the button next to the required key.