Since, the marriage was never solemnised at all and this registration of marriage was done in advance just to expedite the visa documentation, it wouldn't imply that you two are married.
You are not required to file for annulment or dissolution of marriage.
Move a representation before the Registrar of Marriages, apprising him of the situation. Ask him to expunge the relevant entry of your marriage from the Register of Marriages.
If the needful isn't done despite your representation, approach the High Court and seek necessary directions.