Thought I'd reply to your post as some of my friends have told me about it this week!
About four years ago, my husband told me that he wanted to go away for a few weeks and drive across India. My children were 7, 5 and 3 at the time. I said "no".
Having not realised it at the time, this has now come back to haunt me. I hadn't realised his "need" to have space and how important it was for him to have some "time out". We are now getting divorced.
He constantly throws my decision to not let him have "space" back in my face and sites it as one reason, albeit random in my opinion, that we are now separated.
I guess the moral of this story is, therefore, that perhaps a compromise can be made and don't be too hasty to say no. My (now ex) husband has recently told my children that he now intends to travel the world for six months in the near future... Mid life crisis or not, it's definitely worth being "open minded" at this stage rather than denying him the chance to go.