An invoice payment can be deleted either from the invoice it was assigned to, or in the payments area.
Here's how:
Find the invoice and click on the payment. This will bring up the payment modal.
Go to Accounts - Payments , select the payment and "Delete". This will not delete the Invoice, only the Payment and the invoice will change to "unpaid".