Goalkeeper Fernando Muslera's mistake condemned Bielsa's side to an early exit in a 1-0 defeat to Spain on Friday after disappointing draws against Cape Verde and Saudi Arabia.
"We deserved to get seven points and got two," said Bielsa.
"We failed to achieve an acceptable ratio between taking the chances we created and the relationship between the goals we conceded and the role our opponents played in scoring them, was also poor. They were avoidable goals.
"Journalists and fans of Uruguayan football legitimately want to pin the blame on me. I'm the person responsible and I have to accept it."
Muslera had also been at fault for both Cape Verde's goals in a 2-2 draw and Bielsa said the decision to replace the 40-year-old at half-time had been taken by the player.
Bielsa also controversially substituted Real Madrid's Federico Valverde on the hour mark.
"Muslera decided to come off at half-time, and I replaced Valverde with Federico Vinas because I wanted to add a more physical presence to the attack."
Bielsa conceded the decision to keep faith with Muslera had backfired but explained his thinking in sticking by one of Uruguay's heroes in a run to the semi-finals of the 2010 World Cup.
"I cannot deny or dispute the assessment of Muslera's performance but, at the same time, I believe he is a goalkeeper who was coming off a magnificent year and is a player with tremendous personality and character," added the Argentine coach.
"I certainly share the assessment you are making, but the decision I took was not to undermine Muslera's confidence, but rather to maintain it."
World Cup 2026
The 2026 World Cup will be held from June 11th to July 19th in the United States, Canada and Mexico. The tournament will feature 48 national teams and will be played in 16 modern stadiums.
Match schedule and times | Group tables | England at the World Cup | Scotland at the World Cup | Team rosters at the World Cup | How to watch the World Cup | Predictions and odds | Which teams could be considered dark horses?