Continuing the family tradition, she's taken her young daughters to the Dominican Republic, the Czech Republic, Italy, Ireland, Costa Rica and Brazil. This summer, her older daughter will accompany her to China, then the whole family is going to Scotland and Wales.

"Vacation time is often the only time when families get a good solid chunk of time together," Frommer added. "So what are parents with wanderlust to do? Just stick with the tried-and-true theme park and cruise vacations, squelching their own desires to see the world? For many, leaving the kids at home over vacation just isn't doable, financially or emotionally."

Elisa Bernick, author of "The Family Sabbatical Handbook: The Budget Guide to Living Abroad with Your Family," said traveling with children has advantages for grown-ups. Her family lived in Mexico for 18 months, starting when her kids were 2 and 7. "One of the primary benefits was that they were the best little diplomats. Kids were our passports into that world," she said.

One memorable occasion was being invited to dinner by a little girl from her daughter's class. "The dinner turned out to be cactus tortillas — nopales — from the back of their pickup truck," she said. "We all hung around, watching the kids run around. We never would have been invited into this kind of situation without the children."

Itineraries also make a difference. The first big trip Sunny Kobe Cook took with her stepsons and husband was an African safari.

But she added that "not only do people waste a lot of money by bringing children too young to appreciate it — they waste our money and take away from the enjoyment of others by the behavior and coaxing the rest of us have to endure."

Her advice: "Stick to the Disney and Carnival cruises until your kids are old enough to appreciate what they are seeing."