The property where the house stands was first purchased around 1680 by a Quaker named Judah Allen. In 1754, Josiah Halstead bought the property and turned the existing dwelling on the site into the Blue Ball Tavern which operated for almost sixty years at that busy intersection. Over the years, several major improvements were made to the structure, including construction of an attached kitchen with a large brick fireplace, and conversion to the present gambrel roof so that the attic space could be turned into more usable rooms. The first floor provided a tap room where patrons could enjoy a drink, discuss local politics and engage in a card game by the fireplace. The second floor contained a large meeting room, and a smaller chamber that might have accommodated overnight patrons. The rich history of the Allen House includes, among other events, a Revolutionary War massacre (1779) and a visit from President and Mrs. Woodrow Wilson (1916), who stopped in for goods during a period when the property acted as a local store.