Fraud

My father wanted to build a house. So we hired a contracter and gave him the house contract .He said he will complete the work within 6 months . He asked advance payment so we paid the money. At starting he would do the works properly after sometime he would state silly reasons and delay the work .He started to ask money very frequently by saying he wanted to give advance for materials. Now recently he had a stroke and now recovering. So we asked him about our house construction and he says he has no money to start the construction so we need to pay again. We have paid 19 lacks total to him. He has only spent around 5 lacks for house construction. When we asked about what he did with 13 lacks he has no answer and hasn't even paid the advance for materials he told that he would buy. Now when I talked to his wife when will they give our money they told they will sell their property and give but when I asked "when" they started shouting at me that they don't know when they will give. Then I talked to their daughter and she said that they can't sell their property for us and they recorded the audio call.i don't understand why they recorded it because I haven't said anything wrong and I asked very politely . We are very shocked to hear that they recorded my audio call now they are acting as if I threatened them we have all the proof and agreement. Should we file a case against them for fraud since they used our money and for audio recording a conversation without consent.