Back in 16th century England, before homes had indoor plumbing such as toilets and sinks, chamber pots were used as receptacles for dirty water and wastes. People had the habit of emptying these chamber pots by throwing them out of upper-story windows of their homes into the gutters.
Therefore, a stroll along the city streets was often, somewhat hazardous for a couple. So the gentleman showed his gallantry by taking the side nearest the road when walking with a lady. That way, he put himself in position to be splashed if a chamber pot should be emptied.
To this day and in most countries of the world, gentlemen have taken this position when out walking with women, probably without really knowing why they were doing so. Fortunately, they are no longer in danger of a shower from chamber pots.