Lieutenant-General Dan Halutz's departure on Wednesday added to the pressure on Olmert after earlier news that a probe into corruption allegations is to be launched.
Olmert is facing an investigation by the justice ministry over his alleged role in the privatisation of Israel's second-largest bank when he was acting finance minister.
A poll broadcast on national television reinforced the feeling that Halutz could be just the first senior figure to go.
Güncelleme Tarihi: 20 Eylül 2018, 18:16Amir Peretz, the defence minister, also defied calls for his resignation. The TV poll showed 85 per cent of respondents hoped he would quit.
The results of the survey shown on Wednesday evening on Channel 10 showed that 69 per cent of people wanted Olmert to resign.
The Smith Research poll, conducted by telephone, questioned 425 Israelis and quoted a margin of error of 4.8 per cent.
Olmert is believed to have tried to dissuade Halutz from quitting but he is seen as having paid the price for the army's failure to stop Hezbollah fighters in Lebanon launching rockets into Israel during the conflict between the two countries last summer.
Halutz himself said in his letter of resignation that he had "accomplished the objective that was set for me at the end of the Lebanon war, which was to study and learn the lessons from what transpired".
Tzipi Livni, the Israeli foreign minister, said on Thursday: "Israel is strong enough to face these challenges. The prime minister was elected in a democratic election," she said on a visit to Tokyo.
She said she had "high respect" for Halutz and stressed that Israel had a separation of powers.
Livni said: "I think in his resignation he expressed his values as an officer in the Israeli army. And this is part of the values that we are trying also to teach to our young soldiers."
Hussein Haj Hassan, a Hezbollah member in Lebanon, called Halutz's resignation "the result of the defeat of the Israeli enemy in Lebanon".
The fallout in Israel was described as an "earthquake" by one national newspaper as the country's political commentators turned their attention to who could be next to pay the price for the mid-summer war.
That offensive failed to achieve its goals of freeing two captured Israeli soldiers and stopping Hezbollah rocket fire.
Olmert's approval rating is currently 14 per cent and half of Israelis think he should resign while 85 per cent think the nation's leadership is corrupt.
Israeli army radio said Halutz's resignation would, in a "domino effect", lead to the resignations of Peretz and Olmert himself, sentiments echoed by opposition politicians.
Roni Zweigenboim, a leader of military reservists protesting over the conduct of the war, told AFP: "We will continue our struggle until Olmert and Peretz step down, because we are worried for the country.
As long as they are prime minister and defence minister, our condition is very dangerous."
| Peretz, the defence minister, is even more|
unpopular than Olmert, polls say [AFP]
Pressure on Olmert and Peretz could increase further ahead of the release of a wide-ranging government probe into the war.
The investigative panel, focusing on the performance of military and political leaders, is expected to announce its conclusions in the coming weeks.
Meanwhile, the ratings of the Likud party have surged.
Polls show it would nearly triple its current number of 11 seats in the 120-member parliament at the expense of Olmert's Kadima party and its main coalition partner, Labour.