Pathway extension to my property

My father brought a land 13 years ago. When the land was purchased, the owner gave us a pathway from main road to our land which was 20 feet (touching south-west corner). At the time of our purchase, he did not sell any other part of his property. The 20 feet pathway was the only accessible to us and that pathway stops at the begining of our land and does not goes further. After few years, he divided his land which was behind our land and sold them to other parties. Furthermore he has extended that 20 feet pathway till the end of his land, so that the other parties can use that pathway. Now the problem is my land at the south is 25 meters length. The pathway that is given to us is only 6 meters south-west. From the 7th meter till 25th meter he has built a wall so that i don't build a gate or use that path. We requested him to remove the wall, since outside our land is the common path which he has extended for the parties behind. However he refuses. Please advise. Is it legal for me to ask him to remove that wall which he has built in the common path, though in document its not mentioned?