Can loan be sanctioned without signing any sale agreement ?

Hello, We have ran into a situation where we have not been before, because we are selling house for first time. We had appointed a broker to sell this house, his brokerage is about 1% of selling amount. Now we agreed with him verbally that we would not sell home less then "x" amount, and we would need 2 months period before we transfer to our any new house(at that time,like a year ago, we did not have a new house decided ) Now we got one buyer that has agreed to these terms, and he paid 51000/- as token. What we understood is, this token is for to provide him all papers to be verified from his owner, and Not as a sell agreement. Now they have done all research, that they can get these much pan amounts, and all, and they can purchase this house in 15 days Max. But here, they say that after they pay, they won't be able to give even a day to transfer, and we need to have atleast a month for that.. So as nothing can go further, we want to step back, will give buyer back 51000+ interest(it's been about one month ) if he insists, and to broker , we will give him 21000 as compensation for his expenses of advertisements. Real pain: Now broker is blackmailing us, saying that we can't step backwards , or he and his broker association would come down to our home. We haven't signed on any agreement. And we don't want any mess around at home. Please advice,if broker can do this, sincere accepted the token , and accepting the token means we are agrees to all terms verbally ( as he is saying now.). (P.s. We don't have any high reach to police or politicians. A real common person. :! ) Kindly please respond.. Hoping with a helping advice, Kuntal.