Ancestral property dispute

My father has an ancestral property in Mangalore. The property was in our great great fore fathers name till the early 1990's. My Father had two brothers, one elder (Bachelor) and one small (married and has a family). When it was realised that the property was in our very old ancetors name my father's elder brother took the initiative to bring the property in the name of the small brother by finding the family tree. My fathers elder brother obtaining the power of attorney (for release of property in the name of younger brother) from all who were legally entitled to a share. My father was also approached and was promised that for the time being give the power of attorney for release as it was important to bring the property in the name of one among us and my father can claim his share later on when the need arise. As my father was working in kuwait he gave his POA thinking that he can claim his share later on and not make any issue right now. The property was made in the small brothers name. in the early 2000 my father suffered a stroke and had to leave his job and come to India. We stay in Mumbai but my father visited Mangalore frequently (three months in Mangalore Three months in Mumbai) and stayed there for sometime. During his stay in Mangalore, looking at his brothers behaviour my father realised that he had been falsely influenced to give POA for release. my father wanted to build a house of his own in the ancestral property but my father passed away in 2004 and his wish to build a house remained unfulfilled. The fact was known to everyone that my father wanted to build the house hence of late i approached both my uncles (fathers brother) asking for my fathers share to build his dream house. The elder brother totally agreed to it but the small brother is making excuse to delay in giving my fathers share. Also my fathers elder brother has said that if the younger brother does not give us our fathers share he will ask for his share and then make it in our name. Please advice can i legally claim a right on my fathers share in the property. what is the course of action i should take to claim my fathers share from his small brother. Also can my fathers elder brother take legal action to claim his share in the ancestral property