Contractor sent legal notice for payment of dues

Hello Sir, Our house was constructed by a local contractor between 2012-2013. The contractor has been made all due payment as per the agreement signed. The agreement had clear lines stating that payment will be made in various phases of construction. We made the due payments at relevant time though cash. We moved into house in 2013 itself, but at the end the contractor claimed that the total construction cost is 40 lacs INR. As per the rules in the agreement and the price of construction per sq ft., the price came out to be around 32 lacs. We had made all this payment of 32 lacs during construction phase(cash). Now in 2016, just few days back this contractor has sent us a legal notice mentioning that we have cheated him and have to pay due to INR 20 lacs. He has mentioned wrong things in the notice claiming that "we have not paid him any dues as always we told him we will pay in the future". This has caused grave mental problem to my mother and father who were involved with this contractor. we are not sure how to handle this situation now. We have no clue so the first step is that i am asking for some advice from to solve this problem.