Growing up in Florida, Jim Brady started sailing as a teenager. He won his first national title in Laser class in 1983, at age 20. Brady attended the College of Charleston, where he was named All-American in sailing and in 1985 was named Most Valuable Sailor by the Intercollegiate Yacht Racing Association. After racing Lasers, Brady often crewed on J/24s. He later earned World (1990) and European Championships in J/24 Class and a World title in J/22. He also won the title in J/24 at Kiel Week. Brady was named 1990 Rolex Yachtsman of the Year.
At the 1992 Barcelona Olympics Brady won a silver medal as a crew member in Soling. At those Olympics he met fellow US sailor Julia Trotman, and they later married. After the Olympics, Brady worked as a crew member for Dennis Conner, participating in the Challenger Trials aboard Stars & Stripes for the 1995 America’s Cup. He and Trotman later settled near Portland, Maine where he worked in real estate development.