The situation in your side is really a sad and pathetic.
Generally it is women who take the legal terrorism in hands to trouble the husband and the in laws to satisfy their selfish and greed motives/intentions but here your brother i.e., her hen pecked husband joins her in the cruel activities which is a curse on Indian families.
Well, since the house (Ground floor) was inherited by your father as his share through a Will or a partition, this property belongs to him and he is an absolute owner of the same.
If he want he can issue notice to your brother asking him to vacate the house and file an eviction suit if he is not complying with the demands.
Your father can even execute a settlement or gift deed in favor of anyone including you ignoring your brother, because this property now belongs absolutely to your father alone.
This cannot be termed as ancestral property.
So neither your brother nor you can legally claim a share in this property.
If it is feared that your brother may occupy the entire house if your father leaves the house, then first issue a legal notice to vacate and then initiate eviction suit for evicting him through due process of law.
Your father can file this suit through his power of attorney too, and you too can be your father's power of attorney agent to conduct and prosecute the case.
If the tortures are beyond tolerance you also can lodge a criminal complaint with the police against the violent brother and cruel sister in aw with the police. Your father also can join you in the complaint against them or he can lode a separate complaint.
Consult a local lawyer on further issues and proceed as per his prudent advice