Property sale

I have a situation that is I had a flat which I needed to sell.. So had made a contract with the buyer taking a minimal booking amount.. The contract stated that the buyer will avail loan and buy the house within the next 2 and a half month and pay the remaining amount.. In case the situation changes I am liable to return the booking amount. Now due to an urgent situation I had to sell the flat to a third person because it was an emergency situation and that person was willing to pay much more at a day notice.. After that I have returned the money due to the previous buyer but he is now demanding much more than he has booked it with as compensation. My question is what kind of legal trouble I might fall into in such a case and as I am a single woman with an aged mother they have started harassing us in different ways.. Do I need to pay the buyer whatever amount he demands as compensation? And if even after paying that he continues these harassment then what should I do?? Kindly guide me if possible