It is the American practice to add point or mark; the British are content with “exclamation,” or occasionally with the older terms, ecphonesis or epiphonema.
The mark itself(!), however, came into English use about three hundred and fifty years ago, borrowed from earlier Italian usage.