There is a connection between environment and stress on both ends, with excessive clutter and excessive attention to detail both holding the power to distract us from our ability to love fully, work productively and relax effectively. So, what makes sense to me is for each of us to think this through on a few fronts: what constitutes a comfortable environment for us, how much effort we're willing to put into it relative to other priorities, and how well-matched we need our partners' preferences to be to ours.
The only answer that has any chance against against the information saturation kids face these days is to talk openly with kids, early enough and often enough and unflinchingly enough that you set the precedent of being the safe place they can go to ask their difficult questions. It has to happen starting when they're 2 or 3, and they ask you where babies come from and instead of freaking out and deflecting, you give facts commensurate with their ability to understand.
There has been, for some reason (or more likely an unfortunate accumulation of reasons) a trend over the past several decades for parents to do the work of parenting in the isolation of their own homes - and not only that, this trend has overlapped with the other trend of much deeper parent involvement in raising kids. That you also represent trend No. 3, more people raising kids solo, has only exacerbated a close-to-no-win situation.
Plan your own vacations when you want to, and plan a suitable combined vacation with this other family when you want to. If they freak out at your planning your own vacations as you see fit, then let them. Bowing to unreasonable demands because someone will make you pay emotionally if you don't is not a healthy option.