Related forms of the word yesterday are to be found throughout the entire group of the Teutonic languages, and, indeed, more distant relatives are known in many of the languages of the large Indo-European family of which the Teutonic and Romance are two important branches.
In English, the word goes back to the Anglo-Saxon geostrandæg or gestordæg (sometimes as two words, geostran dæg).
It is uniquely in English, though, that the two elements always go together (some writers have experimentally written yester or yestern alone, but the practice has never won wide acceptance).
In German, for example, “yesterday” is expressed by the simple word gestern.
Interestingly, in some languages the related term has also had the sense of “tomorrow,” so it is presumed that the original sense was “a day either before or after today.”