I feel like I've lost more survivors to that church of the chosen than anything else in this game combined. Plus the whole town "celebrates" and does nothing for several turns straight
I think having a building that increases crafting speed by 5% each level, kind of like how the monastery increases resource gain, would actually be a balanced way of handling it without being too OP. Having it be a late game building, maybe with a requirement that all the other buildings be built and at least level 5 will make it a worthwhile investment? Capped at level 5 for an overall crafting bonus of 25%.