Willoughby's early years were spent in Colorado. He left to join the Navy where he quickly rose to become a Marine Corps officer and pilot by the age of 21. Following his active duty career, he received a degree in Engineering from the University of Colorado.
Upon graduation he went to work for NASA as a mission controller for the Apollo program and was integral in the success of the moon landings.
Willoughby and his family later moved to Alaska where he had a successful career with the FAA.