Cancellation agreement

Dear sirs I booked a flat in a multi storied apartment complex by paying earnest deposit of 1lakh in November,2017. Thereafter I paid 10% of flat cost, close to 5lakhs which includes 1lakh which already paid. We were supposed to sign agreement of sale during January 2018, but the doc was entirely biased towards the builder. We tried negotiating with the builder concerning clauses such as quantum of fine, if he delay the project, from paltry 3rs/sft., And payment terms, but to no avail. Couple of days ago,they started calling to apply for home loan. We have not yet signed agreement. We have decided to cancel allotment as we think they won't be able to meet the deadline. They said they will refund entire 5lakhs in 3 trenches over 5months, which I doubt seeing their attitude now. My question is as we have not signed any agreement, will they refund us the entire money including earnest deposit of 1 lakh in 5months? If they don't or delay refund, what options do we have legally? Hope I explained the scenario. Requesting assistance.