Builder asking interest for delay in disbursement.

Hi, I bought a flat in Pune. The following are the payment breakup done by me. 1. 26th Jun 2020 (Rs. 200000) as a booking paid 2. 28th Jul 2020 (Rs. 365000) Stamp duty registration paid 3. 28th Jul 2020 (Rs. 377660) part of down payment paid 4. 06th Aug 2020 (Rs. 265000) part of down payment paid Only loan disbursement was pending. Sanctioned letter is already generated and provided to the builder. The sales agreement was done on 07th Aug 2020. This is a ready to move property. The contact person (Mr. X) from the builder side told me that there is initial work is pending in the flat. This will be done withing 2 weeks of time. We accepted this and asked and pointed some of defects in the property (like color, broken tiles, gapes, broken parts, loos fitted windows, AC holes etc…). The Mr. X accepted these defects and promised me to fix them. As usual some defects are solved immediately withing 2 weeks. But most of the issues are not resolved. I had to follow-up for every small-small thing. This lost my confidence on them. It took almost 2 and half months. During this time Mr. X kept asking me to disburse the loan amount. But I already lost hope on them and asked them to finish the work and I will initiate the disbursement. I have already paid all processing fees and only mortgage activity is pending. Today is 10th Oct 2020. According to Mr. X all work is done now. But we have already traveled to hometown for some family emergency. I told them that I can only come back to Pune by one week later which is 19th Oct 2020 due to (Covid+ home quarantine) difficult situation. On 19th Oct 2020 I can go to Pune and Initiate the process. As per bank person it will take at lease 2 to 3 days for disbursement. Now the Mr. X is saying that you are unnecessarily delaying disbursement from day 1. And for that you will have to pay the interest for the balance amount or cancel the flat (I guess, at this time canceling the flat will also cause me some fine/penalty). At this situation I am not willing to cancel the flat. Not willing to pay any interest or penalty. And due to situation cannot go to Pune till 19th Oct 2020. Kindly advise.