In laws not letting me live peacefully with husband

I am a slalaried emplyee. married for over 6 years, have a 1.5 yrs old daughter. in laws are very orthodox and doesnt want their son to be with me/in my control (in their words). husband loves me but do not say anything against my inlaws (father, mother, sister and brother). he believes watever they say against me and so he fights with me. his brother misbehaved with me long back and now his mother and sister abused me and spoke offensively for me when he was not at home. sister in law used very abusive lang for me and also said k "isse jalaa k maar dena chahiye" and such other bad things. my MiL instead of stopping her, supported her that I am a bitch and used such bad language which aggrevated my sis in law's bad words for me. when I told all this to my husband, he slapped me (this is not the first time he slapped me). he do not want to say anything to anyone at home. n when now I am taking stand that I will not stay with such people, he said he will stay seperately with his parents n family and would divorce me. everytime my mother in law or sis in law fights with me, he say he will divorce me. he himself is an advocate. so threatens me that he will keep my daughter with him. I need advice, plz guide. do i need to fear that he will take away my daughter? he wants divorce, I will give as now I am fed up of all this and dont want to live with them anymore, I can bring up my daughter in better way than him as I earn good. do I need to worry?