Kumki land conflict6

Hello, as i understand from my father that about 30 years back my father used to work for someone who actually claimed some kumki land as his own. So my father took permission from that person to build a small house for us to stay in some 1 acre of land.. we got house number and have been paying electricity bill for that 30 years as well. Now we thought we could build a little bigger house in same place. So we asked the same person but his son answered yes(initially) because his father has passed away few years back. Even though he said we can build house initially, after 50% of house was ready he comes to us and was asking for money for the land where we built house or leave. As i understand kumki cant be sold or bought. but due to pressure we ended up paying him some money for only the 2 gunta(out of 1 acre) of land where house is built. 4 years later he again comes back asking us to giveup the remaining land(1acre-2gunta) where my father spent 30 years working on agriculture. He again says we should pay for complete 1 acre of give up the remaining land. I am confused as that agriculture is a part of my family income which would affect our life a lot. My father has worked hard on that land for 30 years. Please help me how i can handle this situation.