The plot is in your parents name. You contributed 20 % towards it. Same with the house. Your parents want to have the house for their sons.just because you don't have good relations with your brother that does not mean that they will cut off all ties with him. Emotions do take precedence over reason.therefore you should take the money and settle elsewhere if goubwant yo settle this Amicably and your bro unbearable.
But if you can compromise then let them do what they want. After all they are your parents everything is yours after them.
This is not a legal issue as people and relations are involved. Talk to them. But there can be no legal solution to this kinda problem.
In case they were not your parents I could have told you that you could sue them but only if They had promised that they would not let your bro enter the place but they did.