The stats it gives you when you mouse-over the XP bar are only valid for the next level. Damage for example only gets increased every few levels (this is different for each class) :). We could increase raider damage by a fraction of 1 each level to have the same effect (the engine actually supports this), but chose to go this way instead because whole numbers seemed more intuitive. What do you think about it?
Just let them die ... haha kidding^^. Yeah, nice idea.