In a speech to members of his centrist Kadima party, he outlined his plans to fulfil an election pledge to establish permanent borders for Israel by 2010.
He said he was prepared to hold peace talks with the Palestinians but would act unilaterally if he had to.
With nearly all votes counted, Kadima has 28 of the 120 seats in parliament.
But this margin of victory was much less decisive than the party hoped, says the BBC's Jon Leyne in Jerusalem.
It means Ehud Olmert could have trouble forming and maintaining a stable coalition, he says.
RESULTS - 99% COUNTED
1. Kadima: 28 seats, centrist
2. Labour: 20 seats, centre-left
3. Shas: 13 seats, ultra-orthodox
4. Yisrael Beitenu: 12 seats, Russian emigres, far-right
5. Likud: 11 seats, right-wing
6. Arab parties: 10
7. Others: 26 seats
"In the coming period we will move to set the final borders of the state of Israel, a Jewish state with a Jewish majority," Mr Olmert told Kadima party members in what was effectively his victory speech.
"We will try to achieve this in an agreement with the Palestinians. This is our hope and prayer."
He told the Palestinian leadership: "We are ready to compromise, to give up parts of the beloved Land of Israel... and evacuate, under great pain, Jews living there, in order to create the conditions that will enable you to fulfil your dream and live alongside us."
But he said it was time for the Palestinians to "relate to the existence of the state of Israel, to accept only part of their dream, to stop terror, to accept democracy and accept compromise and peace with us".
Electoral officials said voter turnout was 62.3%, the lowest in Israel's history and 5.7% points lower than in the 2003 election.
With 99% of ballots counted, Kadima has won 21.8% or 28 seats, with the centre-left Labour party coming second with 20 seats, a 15.1% share.
Under Israel's complex proportional representation, the exact number of seats may change as the final votes are redistributed.
The right-wing former ruling party, Likud, is trailing with just 11 seats - behind the ultra-Orthodox Shas, with 13, and the far-right Yisrael Beitenu party, which proposes forcibly transferring Arab populations inside Israel to Palestinian territory, with 12.
Likud leader Benjamin Netanyahu admitted they had "suffered a tough blow" but vowed to rebuild the party.
Since the creation of Israel in 1948, the country has been governed either by the Labour or Likud parties, so a Kadima victory is historic.
Kadima, which means "forward" in Hebrew, was founded by Mr Sharon last year after he left Likud amid bitter rows over his withdrawal of settlers and troops from the Gaza Strip.
Israel occupied Gaza and the West Bank, including east Jerusalem in the 1967 war. Its settlements are illegal under international law, although Israel rejects that.