Both sides scored six tries before France scrum-half Maxime Lucu missed a relatively straightforward late conversion which would have claimed victory for the hosts.
Quins will play record five-time champions Toulouse or Exeter, who meet in France on Sunday, on the weekend of May 3-5 in the last four.
Quins raced into a surprise 14-0 lead after just nine minutes thanks to a try from scrum-half Will Porter, standing in for the experienced Danny Care, and a penalty try. Referee Andrea Piardi awarded the second touchdown after Bordeaux-Begles' fly-half Mateo Garcia knocked down a pass from Marcus Smith in the build-up.
The 29 degrees Celsius heat in south-west France was highlighted as the hosts' substitutes brought on umbrellas to shadow players on the field from the blazing sun during a break in play after a quarter of an hour.
By the interval the English outfit deservedly led 28-12 as Porter claimed his second try and flanker Will Evans crashed over following efforts from Lucu and Romain Buros for the home side.
Quins went into the break on a high, just 40 minutes away from a historic win, before Lucu moved across to fly-half with Garcia off with an injury. The French side's positional change paid off three minutes into the second half as France centre Nicolas Depoortere broke free and Lucu's conversion cut the deficit to 28-19.
Lucu, looking comfortable at outside-half, kicked a long-range penalty to make it 28-22 with less than half an hour to play.
By the 60-minute mark there was still six points in it at 35-29 after No 8 Alex Dombrandt crossed for the visitors before winger Louis Bielle-Biarrey followed suit for Bordeaux-Begles who had stunned Saracens in the last 16.
During a frantic final quarter the lead changed hands twice as Congolese winger Madosh Tambwe strolled over for the French club and full-back Tyrone Green crossed for the Quins, who led 42-36.
With four minutes to play, Tambwe claimed his double, setting up Lucu for a match-winning conversion but the France playmaker missed and Quins held on for a famous win.
Later Saturday, there is a repeat of the 2022 and 2023 finals as Leinster host two-time champions La Rochelle in front of a sold-out Landsdowne Road before an in-form Northampton welcome a second-string Bulls.
