Bournemouth did not disclose financial details of the deal, with British media reporting that the Brazilian Serie A club will receive an initial fee of £24.7 million plus a potential £5.6 million in add-ons.
Sky Sports reported the deal included a release clause worth 100 million euros (£87m).
Rayan, 19, made 34 appearances in Serie A for Vasco da Gama last season, netting 14 goals as they finished 14th.
He has played 11 matches for Brazil's Under-20 team, scoring twice, and will bolster Bournemouth manager Andoni Iraola's attacking options following the departure of Antoine Semenyo to Manchester City this month.
"We believe he has enormous potential, and Bournemouth is the right environment for him to continue his development and make an impact," Bournemouth's president of football operations, Tiago Pinto, said in a statement.
Bournemouth, who last week claimed a shock 3-2 win over champions Liverpool, are 13th in the Premier League standings on 30 points. They travel to bottom club Wolverhampton Wanderers on Saturday.