The phrase “gosh all hemlock” is just one of various minced oaths or exclamations by which a man who thinks he is non-profane actually says “God Almighty!”
Gosh has served as a substitute for “God”, usually in “By gosh”, for more than two hundred years.
Some other similar substitutes that have appeared on the American literary scene are, gosh-amighty, gosh awful, gosh burned, and even, back in 1857, gosh all Potomac.