"Tonight, friends, our great country has voted for change," Mr Harper said in his victory speech, pledging to lower taxes and root out corruption. Mr Martin said he would step down as Liberal leader.
In a speech at his party headquarters in Calgary, Mr Harper hailed the election as a sign of change.
"And Canadians have asked our party to take the lead in delivering that change. We will honour your words and deliver on our commitments," he said.
"We differ on many things, but we all share the belief of the potential and the promise of Canada and the desire of our country to succeed," he said.
He admitted defeat in a telephone call to Mr Harper.
The Harvard professor and author Michael Ignatieff, who has been tipped as a possible future Liberal leader, won his seat in a Toronto suburb.
The Conservatives won 124 seats, the Liberals 103, the Bloc Quebecois 51 and the New Democratic Party 29, official results showed after counts at Canada's 66,000 polling stations.
Results indicated that the Conservatives made significant gains in Ontario, Canada's most populous province, and in the Francophone region of Quebec.
The Conservatives had 36% of the vote and the Liberals 30%.
The New Democrats gained 11 seats, a change hailed by their leader as an opportunity to "balance" the domination of the Conservative party.
The Conservatives had enjoyed a comfortable lead in opinion polls throughout the campaign.
But the party fell well short of the 155 seats needed for an outright majority in the 308-seat parliament.
Deputy leader Peter MacKay said: "Minority means we have to be constructive, and we have to be working together and finding common ground."
The weather across most of the country was mild for the 22.7 million registered voters, and officials in the east said turnout was brisk.
Paul Martin lost a confidence vote late last year
The confidence vote that forced the election was triggered by a public inquiry that found Liberal politicians in Quebec had taken kickbacks in return for government contracts.
The Liberals used the election campaign to focus on economic successes, pointing to eight consecutive budget surpluses.
But the BBC's Lee Carter in Toronto says the corruption scandals that have beset the Liberals in recent years seemed to stick this time.
*Increased to 15 seats in 2002 after two by-elections and the admission of a New Democrat MP
**Reduced to 98 after defection of an MP
Total seats in Canadian Parliament: 308
Last Mod: 20 Eylül 2018, 18:16