1.If there was forfeiture clause then the advance money can be forfeited.However on this the builder can raise legal dispute and can bring injunction order of further constructions of the building.Hence it i s advisable that you resolve the dispute amicably.
2. It is usual for the new builder to remove the mess first so he can enter in to the fresh agreement with you.
3.that is the reason i am suggesting to talk to the old builder and meet his coenen.Else The whole project would be jeopardised for endless time.