During the time when the entire family, beginning with the eldest, used the same bathwater, you had to be careful that a child wasn’t still inside when it came time to throw out the dirty water.
But the phrase was introduced in 1909 by George Bernard Shaw, who wrote,
“Like all reactionaries, he usually empties the baby out with the bathwater.”
When something valuable is destroyed while eliminating waste, we say they’ve “thrown the baby out with the bathwater”.