Wrong penalty by builder due to delay in payment

I have bought a flat from a builder in kolkata. The purchase is on the basis of agreement executed between me and builder.However, it is not registered.Out of total Flat price, 10% was paid by me & 90% through bank loan. However, the builder delayed the process of providing documents required for home loan to me substantially and provided me documents in pieces and installments despite many reminders , resulting in delay in processing of the same by my bank/lawyer. Due to this there was delay in payment to builder. Builder had sent me mail on 05.02.16 to make payment within timelines to avoid penalty. In response, I replied him that the delay is due to receipt in documents from builders end. Hence, I requested HIM to count the payment schedule from date of receipt of all documents from him and also requested them to not to charge any penalty. However, the builder did not reply to the mail. I made the payment after some days and that time in demand notice, the builder did not charge any penalty. However, a small line was there in notice that total due amount is excluding interest for late payment. Now, after 3+ months at the time of final payment, builder is asking for penalty for old delay and interest thereon.As per clause of agreement, the builder shall provide to me all available documents for the buyer availaing loan from banks to finance the purchase of flat. As there was substantial delay from builder's end to provide documents, can it be treated as deficiency in service and can the payment schEdule be treated as extended by no. of days builder delayed the submission of documents. I already made 100% flat cost payment to builder & requested him to waive penalty . However, builder was not replying initially and after rigorous follow up, they replied that my case can be considered for part waiver. On calculation, I found that builder has inflated penalty by more than double amount. Can I go to consumer forum/court against builder for waiver of penalty and deficiency in service? Please guide.