To be “turned down” comes from an antiquated courting custom followed by our very proper ancestors.
When all meetings between young men and women required chaperones, and because aggressive romantic suggestions were forbidden, a man carried a courting mirror, which, at a discreet moment, he would place face up on a table between them.
If the woman favored his advances, the mirror went untouched, but if she had no interest she would turn down the mirror, and the suitor.
So being “turned down” means being rejected.