Wilfried Nancy was sacked just eight games into his reign at the Glasgow giants on Monday after overseeing six defeats.
Before the Frenchman's arrival, O'Neill had masterminded seven wins in eight games.
The 73-year-old ditched Nancy's failed 3-4-3 formation and was rewarded with a dominant display.
Yang Hyun-jun, Arne Engels, Benjamin Nygren and Daizen Maeda were on target as Celtic climbed back above fierce rivals Rangers into second place on goal difference and cut the gap on league leaders Hearts to three points.
"It was delightful. We played really well in the game and in the end we won quite convincingly," said O'Neill.
"Today was great, there was a nice atmosphere. Now, being in a dog fight for the championship, we need the crowd behind us.
"It was all about winning the game but the performance was terrific, I don't think I should overlook that. We played really well, got confidence back again and it was just lovely to win."
Both Hearts and Rangers are in action on Sunday, away to Dundee and Aberdeen respectively.