Scotland had the first chance of the game after just three minutes when Ben Gannon-Doak forced a low save from Johny Placide after a tricky run.
Louicius Deedson was unable to test Angus Gunn with his shot at the other end, but Steve Clarke’s men had to weather a spell of Haitian pressure in the opening quarter-hour.
Having done so successfully, Scott McTominay came a post’s width away from an opener on 17 minutes after a sweetly struck first-time effort from the edge of the box.
Scotland’s last goal at a WC came almost 28 years ago to the day from Craig Burley against Norway, but just before the half-hour, McGinn stepped up to end that wait.
His deflected effort found the net after Ché Adams’ close-range effort was blocked by Placide, making the Aston Villa man Scotland’s oldest scorer at a WC finals in the process.
The swathes of Tartan Army inside Gillette Stadium were in raptures and it almost got better moments later when Lawrence Shankland headed just wide after a cross from the dangerous Gannon-Doak.
However, it was not all one-way traffic and Scottish hearts were in mouths soon after when Gunn spilt a fierce strike from Ruben Providence, but his defence mopped up the rebound to preserve Scotland’s one-goal lead.
Ten minutes after the restart, Andy Robertson delivered an inviting ball from the left that just evaded Shankland, but not Gannon-Doak at the back post.
However, he was denied by a crucial block by Martin Expérience. Clear-cut chances were few and far between in the second half, but McGinn had a big opportunity to double his tally and extend the Scottish lead just after 70 minutes, but he dragged wide under pressure from Ricardo Adé.
Sébastien Migné’s men would not lie down, and Sunderland forward Wilson Isidor came inches away from connecting with Providence’s ball across the face of goal.
Almere City forward Providence was a thorn in Scotland’s side all night and he robbed Jack Hendry in a dangerous position, but saw his shot blocked behind for an ultimately inconsequential corner by Nathan Patterson.
Frantzdy Pierrot glanced a header just wide as Les Grenadiers continued to press for what would have been a first-ever WC point in their second appearance at the finals, but Scotland held on for a crucial win - their first at the WC since 1990 - ahead of upcoming clashes with Morocco and Brazil.
Flashscore Man of the Match: Aaron Hickey (Scotland)