From the mostly archaic sense of office as a "duty" or "function" and hence unmentionable "bodily functions".
house of office (plural houses of office)
- (obsolete, euphemistic) An outhouse: an outbuilding used as a lavatory.
- 1764 August 5, David Garrick, letter:
- I never, since I left England, till now, have regal'd Myself with a good house of Office... the holes in Germany are... too round, chiefly owing... to the broader bottoms of the Germans.
- 1823, Lord Byron, Don Juan, Canto XI, §xl, ll. 123 f.:
- The very clerks—those somewhat dirty springs
Of office, or the House of Office.
- Oxford English Dictionary. "office, n."