The sense of the phrase “to speak by the card” is nicely shown by Shakespeare’s use of it in Hamlet.
There, in Act V, scene 1, he has Hamlet himself use it in his conversation with the grave digging clown who takes everything that Hamlet says in its most literal meaning.
Finally Hamlet turns to his friend, Horatio, and says: “How absolute the knave is! we must speak by the card, or equivocation will undo us.”
He realized, that is, that he would have to express himself precisely, or he would get some such answer as Gracie Allen gives when she takes any remark literally.
Shakespeare, the first on record to use the expression, undoubtedly alluded to the mariner’s card, which may have been either the sea-chart or card, the nautical map indicating the position of rocks, sandbars, capes, and so on along a coast, or the circular card of stiff paper with the points of the compass upon it.
In either case, “by the card” would denote absolute precision.