Venue/guests

I've been engaged 1 month, 1 MONTH and already I'm stressing! We have always said we want a small low key affair but it seems there are very few venues who accommodate this in our area. We there for started writing s guest list with more people on bit it's getting out of hand.....if we invite one set of cousins (who we are closer to) we should invite the other set and so on. Guest list is over 120 people now, some of which we really dont have a preference as to whether they are there or not. Is this just how wedding planning goes? X