The Milton House, a National Historic Landmark, is a hexagonal stagecoach inn, built in 1844 by Joseph Goodrich, Milton's founder. The inn, constructed of "grout," is considered the oldest building of its type in the United States. The inn was a stop along the Underground Railroad. Goodrich, an abolitionist, provided a safe haven for runaway slaves by hiding them in the basement of the inn. A tunnel connects the basement of the inn to the 1837 cabin included on the tour. The Milton House is part of the National Underground Railroad Network to Freedom.