When a lady fair, an American one, rejects her suitor, she gives him the mitten; he gets it.
This has been in American usage for at least a hundred years.
The Knickerbocker, a New York periodical, back in July of 1847 wondered how the expression had originated, but it seems clear that the intent of the phrase was that the suitor, requesting the hand of his lady, received instead only it empty, fingerless and insensate covering.
The saying apparently has no relationship at all to the medieval custom popularized by Sir Walter Scott. Then, when a knight exhibited his courage and skill in combat upon the jousting field, he might wear or display the glove of the lady of his choice.
The meaning then, however, was the reverse of our present phrase.