The house is positioned just north of the original redoubt, or fortification, built before 1685 in accordance with New York Governor Sir Edmund Andros’s requirement for colonists to protect their settlements against the possibility of a Native American attack. 

After patentee Isaac DuBois’s death in 1690, his widow Maria took over his estate. Tree-ring dating confirmed that a two-room stone house was built on the homestead fifteen years later, in 1705. 

Maria’s great-great-grandson, Daniel I. DuBois, extensively renovated the house in the 1830s, transforming it into a two-story structure with an upper porch, a garret, and a kitchen wing. Its current bright yellow hue was a popular house color during this period. 

Since no records survive that indicate the home was ever used for community defense, scholars have questioned whether the portholes are original to the house. Brick fill around the holes in the east and north walls indicate that the ports may have been created during the house’s extensive renovations in the 1830s. 

Several generations of the DuBois family enslaved people of African descent. Wele was named in a 1752 legal agreement outlining property given by Daniel DuBois (1684–1752) to his son Simon. A later Daniel DuBois (1750-1817) enslaved Dean and her daughter Isabel, children named Esibal and Sellaw, as well as men named Charles and Graus. According to an 1810 Bounty Hunters’ Agreement, Graus was considered likely to attempt self-emancipation. Daniel DuBois’s 1810 last will and testament stated, “It is … My Will that My Negro Man Grause be a freeman directly after my decease,” but it is unknown if this wish was honored.  

The house has long served as a community gathering place since the late eighteenth century. After inheriting the house in 1799, Daniel DuBois (Maria DuBois’s great-grandson) established a tavern on the property and served as its innkeeper. It was home to a tearoom in the 1920s, and by the 1930s it became known as the “Old Fort Restaurant,” originally run by Elsie Hanna Oates.  In 1976, longtime New Paltz resident Alice Crans assumed ownership of the restaurant; it operated until her retirement in 1990.