Those of us who are familiar with Landseer’s painting, “The Stag at Bay,” know the meaning of the phrase “to hold at bay”; when facing a desperate situation, to hold it at a standstill.
Despite the picture, the fact that the dogs are obviously barking or baying is just a chance double meaning of the word “bay.”
Our phrase seems to come instead from the French phrase, tenir a bay, which really means “to hold in a state of suspense or inaction; to hold in abeyance,” or, literally, “to hold agape.”