Dear Querist,
since you have not entered into any formal agreement which is in a written form, but the agreements can be formed orally and by conduct. since you have worked in the organisation holding a position over there. Better recourse will be send a notice stating the reason and intimation as to your quitting the company. Since, agreements in restrain of trade are not enforceable anyhow as per the law of contract. You cant be forced to work there if you have sufficient reason for not working there anymore. My advice for you is to send a notice of intimation through the lawyer stating the reason for not joining the office.
Hope this solves your query.
You can contact me for consultation and sending the notice of intimation on your behalf.
Regards,
YUGANSHU SHARMA
ADVOCATE