The decision was an important defeat for Netanyahu as he seeks to continue his tenure as Israel's longest-serving prime minister.
Jerusalem: Prime Minister Benjamin Netanyahu informed Israel's president Monday he could not form a new government following deadlocked September elections, making way for his opponent Benny Gantz to try.
The decision was an important defeat for Netanyahu as he seeks to continue his tenure as Israel's longest-serving prime minister, but it does not mean the end of those efforts since he still has various cards to play.
Gantz will also face long odds in forming a government, with many analysts predicting he too will fail to negotiate a unity coalition.
In that case, President Reuven Rivlin could ask a majority of parliament members to agree on a candidate.
Rivlin has vowed to do all he can to prevent yet another election -- a third since April -- but the possibility remains that the stalemate could eventually trigger it.
Netanyahu has been battling the threat to his political survival on two fronts, also facing the possibility of corruption charges in the weeks ahead.
"A short time ago I informed the president that I was handing back my mandate to try to form a government," Netanyahu said in a video posted on his official Facebook page.
The prime minister blamed Gantz for refusing to negotiate on Netanyahu's preferred terms.
Rivlin said shortly afterwards that he intended to ask ex-military chief Gantz, head of the centrist Blue and White alliance, to try to form a government.
Like Netanyahu, he will have 28 days to attempt to do so.
Rivlin can take up to three days to hear out the parties elected to parliament before officially tasking Gantz, who had no political experience before mounting his challenge to the premier he once worked with as military chief of staff.
Blue and White said in a statement: "The time of spin is over, and it is now time for action."
"Blue and White is determined to form the liberal unity government, led by Benny Gantz, that the people of Israel voted for a month ago," it said.
By "liberal", it signalled it would seek to limit the influence of religious parties in forming a coalition.
Netanyahu argued that he had "made every effort to bring Benny Gantz to the negotiating table, every effort to form a broad national government, every effort to prevent further elections".
"Unfortunately, time and time again he simply refused."
But Gantz has repeatedly said he cannot negotiate based on the terms Netanyahu wants.
Netanyahu's right-wing Likud has been seeking to negotiate based on a compromise set out by Rivlin that takes into account the possibility the premier will be indicted for corruption in the upcoming weeks.
It could see him remain prime minister for now, but step aside at some point later as he combats the charges.
Gantz would take over as acting premier under such a scenario.
Who goes first?
The Blue and White leader, however, says he should be prime minister first under any rotation arrangement since his party won the most seats in September 17 elections, finishing with 33 compared to Likud's 32.
Gantz also says Blue and White cannot serve in a government with a prime minister facing serious indictment.
Netanyahu has added a further complication to the coalition talks.
He has pledged not to abandon the smaller right-wing and religious parties that support him in parliament, saying he represents the entire bloc in coalition negotiations.
That condition is also unacceptable to Gantz, who says it would mean Blue and White joining a Netanyahu government as a junior member.
Netanyahu received the endorsement of 55 members of parliament for the post of prime minister after the election, while Gantz received 54.
Ten of the parliament members endorsing Gantz, however, are from Arab parties and have said they will not serve in a government with the ex-military leader.
While the 70-year-old Netanyahu was yielding to his opponent at least temporarily on Monday, he has shown no sign of willingly giving up the post he has held for a total of more than 13 years.
A prime minister does not have to step down if indicted -- only if convicted with all appeals exhausted -- while other ministers can be forced to do so when charged.