Harlequins won 39-29 at the Stoop in Twickenham to qualify while the Sharks collected a late try bonus point that should give them a home tie in Durban.
Lukhanyo Am put the visitors, already sure of a last-16 place, ahead with a try in the first minute.
Quins hit back with two rapid tries.
Danny Care touched down in the corner in the seventh minute on his 350th appearance for Quins. Alex Dombrandt crossed in the ninth minute.
Cadan Murley cantered over in the corner and after 20 minutes Quins led 21-8.
Jaden Hendrikse struck back after 28 minutes.
Just before half time, Eben Etzebeth surprised Care, who was assessing his passing options at the back of a ruck.
The veteran lock picked up the ball and scampered half the length of the field with a posse of Quins in pursuit to cut the lead to two points at the half.
Quins opened the second half with another surge. Stephan Lewies powered over against his former team after catching the ball at a 42nd minute lineout.
Nick David then caught a Sharks kick in his own 22, booted ahead, chased and collected the ball before sprinting away to score.
Marcus Smith put the hosts 17 points ahead with a penalty.
Sharks grabbed a potentially crucial try bonus point when Aphelele Fassi scored their third try with two minutes left.
"We've been struggling for four or five weeks. To come out before a packed out Stoop against the Sharks and put on a performance like that means the world," said David.
La Rochelle thrash Saints
In Northampton, defending champions La Rochelle completed a sweep of their four group games as they won 31-13.
They will play at home in the round of 16.
Levani Botia scored the opening try.
After Fraser Dingwall was sent off for a dangerous tackle just before the break, the visitors ran in four tries in the second half.
Quentin Lespiaucq scored twice and Gregory Alldritt and UJ Seuteni once each.
Tom James scored the sole try for the home team who ended with 13 men Lukhan Salakaia-Loto was sent off in the last minute for throwing an elbow.
Saints ended the group stage with four defeats and a solitary point.
Leinster claimed a fourth successive win in the Champions Cup on Saturday, a 36-10 victory over Racing 92 securing top seeding in the last 16.
The four-time European champions scored five second-half tries through Hugo Keenan, who crossed twice, Josh van der Flier, Jimmy O'Brien and captain Garry Ringrose in the last minute.
Janick Tarrit and Christian Wade scored tries for the French side who were eliminated from the tournament.
