The expression “to ride the goat” means: To be initiated or inducted into an organization, especially into a secret society.
In all probability, although no facts are ever likely to be disclosed, this expression actually did arise from the practice in some college Greek-letter fraternity of introducing a goat into the hazing of prospective candidates for membership.
But the earliest record of the phrase occurs in Peck’s Bad Boy and His Pa (1883), by George Wilbur Peck.
In Chapter XIX, “His Pa Is ‘Nishiated,” the bad boy and his chum train a goat to butt a bock-beer sign, borrowed from a neighboring saloon, and ask “Pa” if he would like to be “‘nishiated” into their lodge and take “the bumper degree.”
When “Pa” agrees, he is told to “come up pretty soon and give three distinct raps, and when we asked him who come there must say ‘A pilgrim who wants to join your ancient order and ride the goat.'”
The goat, as the bad boy says, is “loaded for bear,” and when “Pa” repeats the order, “Bring forth the Royal Bumper and let him Bump,” the goat, seeing the bock-beer sign pinned to “Pa’s” back side, lets him have it with the best “bump” of which he is capable.