As you wish to back out from the agreement, the builder can forfeit your token money
Ask the builder to show you the stamp duty payment challan paid in your name
If he has already paid the duty and if agreement is being cancelled then he can apply for refund to the collector of stamps
The costs associated for making application for refund can be met from the token forfeited by builder
The reason of builder that he has already paid registration fee, is wholly incorrect
Stamp duty can be paid prior to signing of agreement and registration
However registration fee is paid only on date of registration of agreement
As your agreement itself is not registered, there is no question of already having paid the registration fee by builder
You can do nothing against the forfeit of your booking amount by builder except requesting him not to forfeit full token money but only a part of it
Going legally for recovering your token money would prove to be more expensive than the quantum of token money itself. So it would be a futile exercise