I'm okay with the quests and production taking time, but once you send out you sworn sword and start production, there's really nothing to do but wait, so you pretty much just have to go do something else.
It'd be nice if there were something to do, like some sort of mini game at least while you're waiting with even a little bit of a reward like xp for a sworn sword, a resource, silver, food, something.
or maybe make the quests a bit more engaging, rather than just pick a guy and then come back a few hours later to see if you won.