Keys handed over by seller on his own will and own accord

A broker has shown us a flat and when we met with the owner it came to light that the property is on the joint name of owner and his wife, and it has many loans against that property. I just booked the property by paying rupees one lakh only, for which broker has given me the receipt. Later owner met me personally and decided to get rid off from the broker and deal with me directly. He even mentioned that his wife is in advance stage and registration will not be possible before February 2023. I agreed on it. Later he on his own accord and own will he handed over his flat keys to me saying that I can take the possession and reside. Later he will clear all his bank loans through me i e. On behalf of him I will clear his loans. Accordingly I taken over the flat and got it painted, clear it's outstanding dues for IGL, Electricity bill and maintenance. Now broker came to know this. He started chasing the seller. Now seller decided that let us make a rent agreement till Feb 23 and my one lakh will be adjusted for 5 months rent. After that we will go for sale procedure later, and by this time he will settle all his loans. I have a sale agreement made and notorized with his signature since his wife is out station it was not signed by her. Now my question is can he later ask me to vacate the house after Feb 23 or he has to sale the flat to me only.