Well, what are you carrying and how?
What are the children like? Are they very active like to jump all over you and wrestle types?
I have found that when I have to be in the company of young children, a quality holster that offers sufficient retention for all day carry is adequate, plus I just keep it out of their sight.
Also, if the children are trained properly, that helps a lot too. My 8 year old cousin knows I have a gun on me, just like her daddy has one and her mom and her uncle and some of her aunts do. She knows guns are for grownups and big kids and she'll be in big trouble if she touches one. For her a gun is normal, and she's even fired a gun herself. When you take the mystery out of a thing like a gun, the temptation to "play" with it goes away.
Plus, as I alluded to before, out of sight out of mind. Younger children have a lack of abstract thinking faculties. Take something out of their immediate sight and they're likely to forget about it quickly when something else stimulating comes along.
There are some considerations to make... off body carry is a big no no, and I don't pocket carry around small children either. Take reasonable precautions and keep it concealed.
I've had exactly one opportunity to mind several small children for about an hour in a very crowded place while carrying. Honestly, your vigilance can only go so far. If someone were to try something, a firearm is a tool you'd be grateful you had. I personally would never let anyone take away a child of any age... I'd take the shot and if I was unarmed I'd still be on the kidnapper like stink on poo. I was glad to be armed.
Just as important here is the situational awareness and tactics that could apply.
Do the kids know what to do if there's a stranger around, etc.? Do they know how to dial 911? Do they have a plan in place in case of fire, etc.? Have you surveyed the area and thought about what to do if you had a situation?
And at the end of the day, it all comes down to this: You're the adult, and that means doing things that are for the best even if it's inconvenient or uncomfortable. Take some basic precautions and all should be well I would imagine.
If these people trust you with their precious children, the fact you are armed is immaterial.