This phrase and others like it, for example “For Pete’s sake”, are euphemisms for the phrases “For the love of God/Christ” or “For God’s/ Christ’s sake” and hail from a time when those phases were considered blasphemous.
Nowadays phrases like “For the love of God” are commonly used, but the euphemisms are still used as well.
Why Pete? Most likely it is a reference to the catholic Saint Peter.
Other phrases with similar origins are “Zounds”, archaic British slang, a contraction of “Christ’s wounds”; “Oh my goodness” and “Oh my gosh” for “Oh my God; and “Gosh darn it” for “God damn it.”
