Walls and defensive towers give a large bonus to defending troops. If you have no one to man the walls, then they are not very helpful... but if you have defenders, they help a lot!
I just defeated the Normans (playing as the Vikings) on Hard. These are the pertinent stats: Vikings Dead: 2142, Normans Dead: 1567; Vikings Fled: 239, Normans Fled: 3617. Thank you for a strategy game that involves actual strategy!
Walls and defensive towers give a large bonus to defending troops. If you have no one to man the walls, then they are not very helpful... but if you have defenders, they help a lot!