The 68-year-old joined the Yorkshire side at the start of May, replacing Javi Gracia.
The Leeds board hoped Allardyce's reputation for keeping teams up would help them in a last-ditch attempt to avoid relegation from the Premier League, but with just four games to turn things around, 'Big Sam' lost three and drew one of his matches in charge.
In a joint statement, Allardyce described his time at Elland Road as "an honour" and praised the fans for trying to play their part in the club's survival bid.
"It has been an honour to manage Leeds United, a great club with an incredible fan base, who deserve to be in the Premier League," Allardyce said.
"I have really enjoyed working with the staff and players at the club and I would like to thank Angus Kinnear for the opportunity. I'd also like to highlight the outstanding work of Karl Robinson and Robbie Keane who have worked tirelessly over the past few weeks.
"At this stage in my career I am not sure taking on this challenge, which is potentially a long-term project, is something I could commit to, but I wish the club every success for the future and hope the club returns to the Premier League, where they belong."
Leeds United CEO Angus Kinnear thanked Allardyce for his efforts during the short tenure and revealed a new head coach would be announced in the coming weeks.
"We thank Sam for being brave enough to step in and do all he could to save us," Kinnear said in his press release.
"Sadly, it was not to be, but Sam, Karl and Robbie did everything they could to lift the mood at Thorp Arch and Elland Road and for that we are all extremely grateful."
Leeds had spent three consecutive seasons in the Premier League after a nearly two-decade hiatus from England's top flight, but a turbulent 18 months saw the Yorkshire outfit go through four managers in the past two campaigns with Marcelo Bielsa, Jesse Marsch, Javi Gracia all preceding Allardyce.
They were resoundingly beaten 4-1 by Tottenham at home on the final day of the season, finishing five points away from safety.