Madison House traces its roots to the 1856 founding of the University of Virginia Young Men's Christian Association, the first collegiate YMCA in the United States. The YMCA focused on creating campus-based spiritual communities, which in turn devoted their time and energy to volunteer service. In 1968-69, in response to a changing University and declining membership, the YMCA was reincorporated as the Masters and Fellows of Madison Hall. Madison House remained an independent non-profit corporation responsive to the needs of the community, the University, and the students.
To sustain its rapid growth and independent status, the organization sold the YMCA property to the University in 1971. In 1975, funds from this sale were used to construct a smaller but more useful building and to begin an endowment to ensure the long-term stability of the corporation. With the creation of a new home, the corporation changed its name to Madison House.