Tore Milsett took up competitive cycling at a young age, becoming the Norwegian individual and team time trial champion in the junior races in 1961. Six years later Milsett won his first national road race title as a senior, retaining his title the following year. His performances earned him a place on the Olympic team for the 1968 Mexico City Games, where he finished 14th in the road race and fifth in the team time trial. Four years later Milsett competed in the road race at the München Olympics, this time finishing 17th. He also rode in two editions of the Peace Race and competed in the road race at the 1966 UCI Road World Championships.