A ship’s colours are raised or lowered to signal the ship’s status. “All flags flying” signals great pride, but flags could also indicate degrees between failure and conquest.
These flags were once held in place by a system of pegs, so lowering them was done by taking down a peg.
This was a shame to the ship and its crew and gave us the expression for humiliation: to be taken down a peg.
We say that someone who is arrogant needs to be “taken down a peg”.