Kumar defeated Richard Veenstra 3-1 in sets to become the first Indian player to win a match at the World Championship.
The Dutchman threatened to derail the moment with a series of heavy finishes, including 144 and 126, forcing the contest into a deciding set.
Kumar responded by holding his nerve in the closing stages, breaking at a key moment and sealing victory with assured finishing.
"I don't know what to say. I'm overwhelmed. If you dream it, anything is possible," Kumar told Sky Sports afterwards.
"There are so many good players from India, they just need to see someone who can do it, and I'm willing to be the puppet!"
Tata stuns Edhouse
Jonny Tata claimed a 3-0 victory over former European champion Ritchie Edhouse in a match shaped by missed opportunities.
Edhouse failed to convert multiple set darts in the opener, allowing Tata to escape and take control.
The New Zealander grew increasingly dominant, breaking regularly and closing out the match with a strong third set that included a maximum and a clean finish on double 8.
Dom Taylor continued his Alexandra Palace debut with a 3-0 win over Oskar Lukasiak, relying on consistent finishing to progress.
Taylor recorded three ton-plus checkouts of 114, 110 and 100 and punished Lukasiak's missed chances throughout.
Although the Swede avoided a complete whitewash by taking the third set, Taylor quickly closed out the contest to reach the second round.
Convincing Cullen
The session ended with Joe Cullen producing the most convincing performance of the afternoon, sweeping aside Bradley Brooks in straight sets.
Cullen averaged just under 100, landed four 180s in the second set and finished the match with a 116 checkout.
"My son told me Bradley was the favourite for this match and I was like 'really'?" Cullen said on stage after the victory.
"I think that's what I had to do today. I was up for today, and I thank my son for that."
The PDC Darts World Championship continues on Sunday evening from 19:00 GMT, with Lukas Wenig facing Wesley Plaisier, Dimitri Van den Bergh against Darren Beveridge, Stephen Bunting taking on Sebastian Bialecki and James Hurrell playing Stowe Buntz.