Pep Guardiola's side took an early lead through Alfie Dorrington's own goal, but they made hard work of seeing off their fourth-tier opponents at the Etihad Stadium.
There were only nine minutes left when City defender Guehi delivered the decisive blow with his maiden goal since signing from Crystal Palace in January.
"They defended really well, so tight and we didn't attack the spaces the way we should. The game was flat until we scored the second goal," Guardiola said.
"It has been a really tough three weeks with a lot of games and it's demanding."
Owned by Manchester United legends Gary Neville and David Beckham, Salford were playing in the fourth round for the first time in their history.
City beat Salford 8-0 in the FA Cup third round last season and it took just six minutes to breach the minnows' defence again.
Rayan Ait-Nouri's cross was arrowing towards Omar Marmoush until Salford's Dorrington stuck a leg out and deflected it into his own net.
City fans expecting that to be the trigger for a deluge of goals were disappointed.
Guardiola's men struggled to find any momentum and needed a James Trafford save from Salford striker Ben Woodburn to preserve their lead.
Brandon Cooper wasted a golden opportunity to equalise when he headed wide from Luke Garbutt's corner on the stroke of half-time.
City were unable to muster a single shot on target in the first half and Salford nearly drew level after the interval when Kelly N'Mai's fierce strike was parried by Trafford.
Guehi finally ended Salford's resistance in the 81st minute, tapping in after Matt Young palmed Rayan Cherki's pass into his path.
Mansfield dump out Burnley
At Turf Moor, Mansfield's 2-1 win over Premier League strugglers Burnley took them into the fifth round for the first time since 1975.
Burnley went into the game bolstered by their first Premier League win since October over Crystal Palace in midweek, but boss Scott Parker made nine changes and the gamble backfired.
Burnley took the lead through Josh Laurent's 21st-minute goal, but Nigel Clough's team, who sit in mid-table in League One, staged an impressive fightback in the second half.
Rhys Oates headed in the equaliser in the 53rd minute and Louis Reed capped a fine individual performance with a brilliant free-kick 10 minutes from full-time.