Destroying the enemy castle enables you to hit their foot soldiers from further away, becomes a big advantage when you are trying to get the flag and instead of having to shoot at the flag, you can hold them at bay while your guys get it.
The problem is that the AI to retreat is too strong, makes it almost impossible to actually gain any ground at all. Would be better if the tanks couldn't recharge.