Nathan Daggett Colby was born in Emden, Maine, in 1820, the son of Ebenezer Colby. He accompanied his parents to Beloit in 1849. His wife was Sybil Willard Pease. The children of this couple were Charles, Alvin, William, Ida, Nathan, Frank and Emily. Nathan, Sr., became a trucker and also transported passengers to the lake ports. He would also haul a load of wheat to the lake port and trade it for a load of stoves. These he sold to settlers living along the way. He also contracted to haul the railway ties for the Racine division of the C.M.Ct.P and P.RR.

The Colby homestead was on south Race Street on the site of the Rockford, Beloit, Janesville railway power station. Colby was a lifelong Republican and abolitionist. His home was used numerous times as a refuge for escaped slaves on their way north. He was the leader of the Union League which was very active in this cause. The charter of the secret organization was kept on the back of a picture showing a Civil War soldier in uniform bidding goodbye to his wife while little children played on the floor with their father’s sword. Dissenters of the draft and those who lacked enthusiasm for Lincoln were subjected to rather arbitrary treatment at the hands of the Union League members. Nathan Colby died in 1891 and is buried near his father’s grave, Ebenezer Colby, an 1812 veteran.