1. If your brother's wife can lay hand on any evidence that your brother and his paramour had physical relationship (like having evidence of hotel register showing both of their signatures therein for the purpiose of night stay, then your brother will be charged with the allegation of adultery u/s497 of IPC punishable with jail term for 5 years.
2. Your brother should collect evidence of her cruel behavoiur with himself and all his family members by audio/video recording her conversations and acts and then file a divorce suit on the ground of cruelty.
3. Your brother should now take up a rented house nearby leaving your mother's house duly informing it to his wife and wait for her going to her mother's place.
4.Your mother should then lodge a police complaint against her said daughter in law alleging physical assault and also file a DV case against her. Thereafter she shall have to file an application before the Court under Order 39 Rule 1 & 2 praying for a stay on her restraining her entry in to your mother's house.
5. Once her entry in to your house is restrained and the divorce case is filed, your brother shall have to pursue with the case fittingly.