In order to produce the harmony desired by the composer of an orchestral piece, someone must be willing to play the violin of lower tone, or second violin, while another plays the first violin and the leading part.
Hence, metaphorically, we speak of anyone who occupies a subordinate position, especially of a person who steps from a leading position into the lesser status, as one who plays second fiddle.
And we use the expression also to describe a suitor who, though still smiled upon by the girl of his heart, is not her favorite but is her second choice should the first fail her.
Both the academic “violin” and the colloquial “fiddle” have been used in the expression for some two centuries or more, but the latter is more common in America.