What a difficult one Thirdchild. I haven't been in your position but maybe I can give a different POV.
My husband and I had our first child nearly 18 months ago. We both thought we wanted two. Even though our child is relatively easy, we've both realised we think we're happy to stick with one child. I'm one of four, he's one of two, and both of us have siblings with three children. So we're used to big families. However, we like our life as it is. Granted this may change in time but this is where we've been since our son was born.
We don't want to go back to the newborn stage. We don't want to change our unit to one that is forever chasing children. We listen to friends with multiple kids tell us how they love how their kids play together while we watch them constantly breaking up fights and stopping bickering. We think back to our family dynamics as kids and while one of four, me and my siblings didn't all get on, and because of their age gap my husband wasn't close to his sibling.
We like that we can take our son to lovely hotels and restaurants, on lovely holidays, and he enjoys experiences that we couldn't do with more than one (I take him on train rides and to galleries and places that I couldn't manage alone if we had another child). He's super socialised through play dates and nursery and he loves to play alone, too.
Some people might call it selfish, but it's equally easy to call having more children in a world that needs fewer selfish. What I mean by that is that the choice is always personal and subjective. There's always an argument to defend both decisions. I don't think there is a perfect number because it's different for everyone. Some people would hate to have one, as we're likely to choose, we would hate the opposite. Years ago I read research in the Economist that non-parents are as happy as parents, they just have different pleasures and pains. I imagine that's true of parents of different numbers of children, too.
But I really don't buy the argument that 'people regret not having them, they don't regret having them'. It's easy to say you regret someone who doesn't exist (especially when that someone will be idealised and not real). Which parent is honestly going to say they regret the realised child they've nurtured and loved? It's not a fair comparison. The 'regretted child' might have been an absolute nightmare in real life, versus the idealised little cherub that is imagined (because who imagines that their kid will be anything less than perfect?)
I have a similar situation in that I'd have loved a girl. As much as I chastise myself for being so bloody sexist I can't help myself, because I imagine the incredible, smart, funny little girl power champion I'd have had who'd change the world while wearing the gorgeous clothes you can get for girls. And then I reign it in because I'm not having another child on the punt she's a girl, and likely she wouldn't be what I imagine because that's not how the world works.
Similarly, as one of four I enjoy having siblings now as a grown adult, and I wonder what we're potentially depriving our son of as he grows up. But I ask single child friends and they don't feel that their worlds are lacking for not having someone who has never existed. Besides, not all adult siblings get on.
However, I wanted to share the other side because I dislike the idea that it's selfish men who don't want more children because they're selfish. I'm female and I don't want more. And as long as any of us are following our personal desires above those of others, that makes us selfish, whatever those desires are - more children or no more children.
I think as someone else mentioned above, the danger is that this causes a rift between you and your husband and negatively affects you both and your two children. Maybe you need to fully work out what you think is missing and why you think a third will be the answer, and he can explain his side. But to be a broken record, I think it's really important to remember that both arguments will be full of valid reasons and personal preferences of how you both want to live your lives, because we are more than parents, we're still people who have other needs beyond children.