Co-applicant right in Joint Property and Loan

I and my brother brought a independent house in Hyderabad in Dec 2012 at a cost of 22 Lakhs with mutual understanding that I live in Ground Floor and he will be in First Floor. We took 15 Lakhs from the Bank as joint irrespective of any individual eligibilities and after registration and loan sanction he dropped his first floor construction and I completed ground floor by paying difference amount 7 Lakhs to Builder (by way of cheques from by brother’s account as elder and main applicant). Sale Deed and Loan Agreement on both of us and nowhere mentioned the property share and loan share and EMI shares either in Sale Deed or in Loan Agreement. The Ground Floor construction completed and its under my possession since July 2014.. I have been transferring my EMI share to his account as EMI is scheduled to pay from my brother account. Now, after 2 1/2 years he started 1st floor construction and availed another 10 lakhs loan as top-up and 70% works completed. Now he is saying the loan is because of him and i have to pay some 3-4 lakhs as he lost personal loan eligibility due to housing loan and paying construction cost more to the builder for his 1st floor (comparing my construction cost) and paid rent for all the 2 -3 years along with EMI share.. Also threatening me as he only paying EMI as ECS scheduled to pay from his account and I am dummy as co-applicant / co-borrower.. PL HELP ME ON THIS ISSUE..