Tobytown is a predominantly African-American community located in Potomac, Maryland, United States. History. Tobytown was established in the 1860s. According to oral tradition, it was founded by three emanicpated slaves: William Davis, Elsie Martin, and Emory Genies (or Genus). The community was named after Tobias Martin, who was Elsie Martin’s husband, son, and grandson. However, the details are not clear. The land the community was built on was first purchased in March 1875, by William Davis, who bought 4 acres of land for $80 from John and Susan Rowzee. According to the 1870 US census suggests that Tobytown, as a distinctly Black community was older. The census showed that there were seven African American households, including those of William Davis, Tobias Martin, and Emory Genies. According to the oral tradition, these early Tobytown residents already lived on land that they bought in the mid-1870s. Governance. The community of Tobytown is owned and managed by the Housing Opportunities Commission of Montgomery County, which is Montgomery County's public housing agency.