320 First St. NW was built in 1927 by the Acacia Mutual Life Insurance Company. Acacia sold it to the U.S. government in 1936 and the building was extended. The building became the headquarters of the Federal Home Loan Bank Board, which was part of the New Deal. The building is now used by the Federal Bureau of Prisons.
The building is ten stories, with a limestone exterior with brick trim. The architect was George E. Mathews, who designed the building in the Classical Revival style. The building is on the National Register of Historic Places.