Cancellation of agreement of sale

Dear sir, We've signed an Agreement of sale for our house yesterday and were given 11.5 L as advance. Due to sudden health concern for our mother, who is a sr citizen we can no longer move away and so do not wish to sell the property. We requested buyer to nullify the agreement on humanitarian grounds but they refused. The amount was partially given in check and partially cash, both kept AS IS. The agreement was notarized and he took our signatures and thumb imprints on stamps he stuck at the end of the agreement. It is not court registered. The house is jointly owned by my late father and living mother. The contract had a clause for cancellation that seller will have to pay double amount of the advance money in case of cancellation and the date to vacate the property was 27 Dec 2022. So kindly advice on 1. Can we cancel the agreement? And if we can, then kindly explain what shall we do, the process. Is there any legal recourse available to us. 2. Can the other party go to court regarding the issue? The buyer claims he has partners and we are bound with the agreement.