Child custody in case of adultry

My husband is accusing me of adultry with a man i work with. My colleuge and i have known each other since college days and are very good friends. My husband however, does not approve of being in touch with him post our marriage. we got in touch with each other after a long time at work again and hit it off really well...my husband started having doubts and started to look for his emails on my phone and my chats with other friends without my permission. He has some private pictures ( not nude) of mine which i have sent to my girl friends and is accusing me of having sent to him...he also hired some thugs to scare him. now he is threating me to take away child custody from me. We have two children, i am not considering the divorce also, i am ready to put up with him for the sake of my children. however, i am also at fault. i did become emotionally attached to my friend. But we have no physical relationship whatsoever. if push comes to shove, will i lose my children,in a custody battle? i am a good mother...i care for my children unlike my husband. He was previously married and has a daughter too. I helped him file a custody case so that he gets visitation rights to his daughter, which he is no longer interested in...He earlier divorced his first wife (ex-wife) ...she had filed curelty and mental torture against him. He has also accused her of infidelty.