1. Who is the owner of the house? If the owner is either of your parents then he/she may file a suit for eviction of your brother's wife from the house. She has no right to reside in the property of her in-laws without their consent.
2. Furthermore, you say that your mother is also being 'tortured'. She may file a case under Section 12 of DV Act to seek immediate relief of protection order from the court to bring an immediate end to acts of domestic which she is suffering.