Dear Sir,
The law does not require that the agreement to sell has to be writing, signed, stamped, registered etc. Those requirements are for a sale deed.
As of now you have entered into a valid contract and if you terminate it without any sufficient cause, the other side can sue you for breach of contract and recover reasonable damages.
Coming to the solution, please do the following:
1. Transfer the amount back to the buyer in his bank account.
2. The dealer was acting as an agent of the buyer therefore issue a email/whatsapp/letter/legal notice to the dealer stating the following:
"Dear Mr......
This is pursuant to our conversation dated........ It is unfortunate that your client Mr.............(buyer) is unable to proceed in accordance with the terms agreed upon by us. I understand that these are uncertain and testing times and it is difficult to arrange funds. Although I was hoping we could negotiate, but since the buyer has made his mind not to continue with the deal, I have refunded the earnest money to his bank account number provided by you. Details of transaction: (Account no., Amount, date and transaction number). I have not deducted any amount as a goodwill gesture. Please convey it to Mr................. (buyer).
Thank you.
............................"
This way you can make it look that oral agreement was orally terminated by them.
Hope this helps!
Best wishes.