Unregistered Agreement to Sale and Registered GPA

I purchased a plot in year 2014 and got it registered for Tehsil/Sub-registrar office in Punjab. Recently some property dealer approached me and get it purchased from me on the basis of Agreement to Sale (Which is unregistered) and General Power of Attorney (Which he got registered in Sub-registrar office). He did so because the plot cannot be Registered due to Closed Registeries in my property area. He paid me all the deal amount via bank transfer which was safety received by me. All documents draft were prepared by him only. The Agreement to Sale was done on a 100 ₹ estamp paper. I was aware of all the terms and conditions written on Agreement to Sale and agreed to them. On my request there was a clause added that I have received the full payment and handed over all the required documents to the buyer and today onwards I am not responsible for this property. Problem starts after making full payment he made a pressure and rush saying that I have made the payment so please immediately on me sign the Agreement to Sale and I did it but he didn't signed this document. Then he took me to the Sub registrar office and get properly executed the GPA document. He also took previous original registry copy from me along with coloured copies of my PAN Card and Aadhar Card. He also handed over me photo copies of both (partially and Fully executed) documents after my request. As both documents (ATS and GPA) were in Punjabi language and I am not so much familiar with this language therefore taking advantage of it he cleverly added 1 point in GPA that I am allowing him that he can apply for Bank loan against this property and he can sign on behalf of me. 2 days after when I came to know this I immediately asked him to sign and share with me Agreement to Sale copy. But he is not responding to my requests. I am having intuition that he is taking loan in my name on the basis of GPA and my old original registry. I checked with few banks the possibility of loan, but have received mixed response. Few small banks said that he can get loan in such cases. I am now completely confused about future liabilities may occur in such case. Please advise.