If you'd 'love to have a fourth', go for it! I am one of 4, and now we are all older, it is amazing to have three siblings-growing up we did fight (of course!) but were all incredibly close, and we still are now, really, despite the odd falling out with my very spoilt youngest sibling!
We currently have two kids (nearly 4, and 4 months), and my husband would really love 2 more... Financially and emotionally I'm not sure we personally can take it, but you never know. We'll definitely try for 1 more though.
BUT do take the finances into consideration-my husband (who definitely earns less than most of the main breadwinners around here!) doesn't see that is a huge issue-he would be happy to dress the kids head to toe in charity shop stuff, never go on holiday, never re-do the house or buy anything 'nice' in order to have 4 kids-whereas I, while not particularly bothered about expensive clothes or anything for me, would like to be able to afford to do the odd nice thing, and not worry that our kids were unable to do things there friends could (not EVERY little thing, I don't want spoilt kids!) but some of the things... and for us as it stands, we couldn't do that if we had 4. Still, I (fortunately!) have a bit of time on my side...
Sorry, starting to ramble, but if you'd like a fourth, and have the funds, and think you can get through another pregnancy/birth/tough few years initially-go for it-I loved being one of 4. You never know, I may well join you in the 4 kids camp one day.