“Board,” in nautical language, is the side of a ship.
Thus “overboard,” for example, means over the side of a ship; hence, out of the ship, into the sea, and “by the board” has the same meaning, i.e., down the ship’s side, overboard.
Accordingly, “to go by the board,” in its literal sense, is to go down the ship’s side, to fall overboard and to be carried away; hence, to be lost for good.
These several literal meanings date back at least three centuries, and some are older. But the figurative sense of our present phrase, meaning, to be utterly lost, as if carried away by the sea, is scarcely more than a hundred years old.
The earliest literary usage thus reported occurs in The Autobiography of a Beggar Boy (1855) by James D. Burn: “Every instinct and feeling of humanity goes by the board.”