Just what these spots were, which when knocked off gave one a victory, is a matter of guess.
Apparently they were New England spots, and possibly localized still further in Vermont.
At least the first printed reference leads to the latter inference.
It was in an article on the breeding of Morgan horses in the publication, Porter’s Spirit of the Times, November 22, 1856: “Addison County leads the van (or ‘knocks the spots off,’ as we say here) in Vermont and is celebrated over the world for its fine horses.”
We have a hunch that the “spots” were the prominent members of the countenance of one’s adversary, the eyes, the nose, at which one would be most likely to aim one’s fists.
