1. It is illegal to add the previous tenant's unpaid utility bill to your monthly rental bill.
You can object to that and withhold the payment of entire rental bill for want of clarification in this regard and be firm to refuse to pay a third person's charges that are levied on you which is against the law as well as against natural justice.
2. If there is no clause towards any charges to be paid to the caretaker in the rental agreement then you can refuse to pay the same and can question their legality towards this claim.
3. If you are paying the water charges to the caretaker separately, then you can object to this charges as well and refuse to pay.
4. You may vacate the premise observing the conditions as per the agreement, they cannot stop you from doing this.
5. You go through the rental agreement clause once more properly and confirm if there is no such clause present in the agreement, if yes, then you can refuse to pay and can issue a legal notice venting out all your grievances, with instructions to clear all those unjustified and illegal claims made against you and to relieve you to enable you to deliver vacant possession of the premise as per the date given in your notice.