Dispute over private path way

Before asking the question I would like to give a brief case history of our property dispute with our neighbor. We have a residential house of our own in which we stay and it has open space around it. The total area is around 12000 square feet. We had put up a wired fence around it. There was a private pathway which went through the center of our property to the neighbor's house. It is not a public pathway but we had allowed them to use it out of good will. Now they claim it is their right to use that path citing they have been using it for the past many years. We wanted to block the path way and use the entire land which belongs to us by constructing a compound wall. Our bitter neighbor objected and forced us to make two compound walls on the eastern side where their house is. The two walls are separated by a width of 3 feet and runs around 12 feet long. This was done to create another pathway for them within our property. They have another pathway which they use now, which goes through another person's private property. We want to remove the inner compound wall of that 3 feet path and use the entire space for ourselves. But our neighbor is objecting to it even though they do not use that pathway much. We would like to ascertain our legal right to retrieve the land. What are the legal options available to us and what are our chances of success? Kindly advise.