With the summer in full swing and families getting together for barbecues, weddings, and other gatherings, we’re thinking of one of America’s most famous families: The Kennedys. The political dynasty has long held the fascination of the American public and people around the world, who have followed not only their political moves but their personal lives, and how they spend their leisure time. The Kennedys are known for their large family gatherings on holidays and in the summertime, including their storied Fourth of July parties at the Kennedy Compound in Hyannis Port, Massachusetts. (And of course, their celebrity guests: In 2012, Taylor Swift reportedly attended the family’s Independence Day festivities).

Described as the “Summer White House” during the presidency of John F. Kennedy, the compound—which is made up of three houses on the water—was featured this year in the FX television series Love Story, renewing interest in a retreat that endures as a symbol of Camelot.

Below, see photos of scenes of the Kennedy family’s vacations, from sailing to golfing and more.