Boundary fence on the common easement of me & my neighbour

Good after noon sir, Recently i constructed a common easement to my home which is legally the easement for me and my neighbor(my uncle).Before we were walking commonly through their property without making that easement which is legally allowed,but later on one day they told that we don't have easement to our property.After that i went to the village office & made the easement legally. Now i am actually constructing the easement & the work is almost over. in the easement i kept fence with bricks 3 bricks height.i have a fence on the right side of the easement which was constructed by temple people long back . Now my neighbor is telling i should construct by fence in the same height of that temple fence.his intention is clearly to make me jam between those high fence i already built-ed with three bricks height,for me that is enough. i don't want to increase the height further. He is telling if i didn't increase the height , he will increase it. its clearly to make me in trouble. my question is can he built on the bricks fence i already made. or can he destroy my bricks fence & make a new one without my consent.I am living abroad & my mother is alone in my home & they are making threatening dialogues form their home to my mother.