The expression “to go to rack and ruin” means: To go to destruction; go to pot; go haywire; go to the dogs.
In the sixteenth century, and later, men spelled by ear, rather than through knowledge of the historical background of words.
In fact, many of our present-day spellings are still affected by that custom.
Rack, in this phrase, is one of them.
It should have been, and should now be, wrack, which in turn was another spelling of wreck and with the same meaning.
But we have had “rack and ruin” almost four hundred years and in the works of the best writers, even to the present day, and a change would be a slow process.