1. the owners around 3 sides have got every to build their houses and you have no legal right to restrain them.
2. At the time of purchasing the proeprty one is expected to be due diligent as regards everything including the access road.
3. Now as you says there is only one approach way, you can use the same and if it is public thoroughfare or a common space then no one can block it anymore.
4. However if you had any easementary rights over other sides out of ling usage as rad then you could have restrained them from blocking the passage but since they have raised constructions now nothing could be done anymore.