Mr Bush said that both he and Mr Rumsfeld had agreed the time was right for new leadership at the Pentagon.
Former CIA Director Robert Gates has been nominated to replace Mr Rumsfeld.
The Democrats won control of the House of Representatives in the polls, and the Senate balance of power hangs on a tight race in just one state, Virginia.
Mr Bush said that his administration's Iraq policy was "not working well enough, fast enough", and that Mr Rumsfeld agreed that a "fresh perspective" was needed on the issue.
The US president described as "thumping" the Republicans' set-back in the elections, in which the Democrats gained control of the House of Representatives for the first time in 12 years
But he said that with victory, there had to be responsibility, and that was why he would be working with his Democratic opponents to get legislation passed.
Mr Bush described Mr Rumsfeld as a "patriot who served this country with honour and distinction", as well as "a trusted adviser and a friend".
With popular anger over the war in Iraq a major factor during the election campaign, there had been growing calls for Mr Rumsfeld, the longest serving defence secretary in American history, to quit.
Correspondents say he was a key architect of the war in Iraq and had been looking increasingly beleaguered because of its apparent failures.
Neither Mr Rumsfeld or Vice-President Dick Cheney were present at the news conference where Mr Bush spoke, which correspondents said was unusual at such events.
Asked if Mr Cheney would be with him for the rest of his term, Mr Bush said "yes he will".
Mr Bush spoke minutes after the news came that the Democrats had won the Senate race in Montana, one of the two seats needed to wrest control of the upper chamber of Congress from the Republicans.
Mr Bush said he was appointing Robert Gates, 63, to take over as defence secretary.
Mr Gates served as CIA director for just over a year in the early 1990s, during the presidency of Mr Bush's father, former President George Bush.
He is a member of the bipartisan Iraq Study Group, which is tasked with recommending ways of tackling the problems the US faces in Iraq.
Correspondents say Mr Gates is widely respected by both Democrats and Republicans, which will help gain Senate confirmation even if the Republicans lose their majority.
The BBC's John Simpson in Baghdad says the news of Mr Rumsfeld's resignation will be met with delight on the streets of the Iraqi capital.
Resentment has grown against the US-led troop presence there as violence has spiralled.
Despite a bitter, fiercely fought election campaign, Mr Bush said he was looking forward to working with Democrats.
"If you hold grudges in this line of work, you never get anything done," he said.
He said he would seek to find "common ground" with House Democratic leader Nancy Pelosi, who is set to become the first female speaker in the lower chamber.
Earlier she pledged that the Democrats would work with "civility" and "partnership, not partisanship" in their newly empowered position.
The resignation of the US Defence Secretary Donald Rumsfeld shows how much the Bush administration is in disarray about Iraq.
The president made it quite clear at a news conference after the election that he had decided beforehand that a "fresh perspective" was needed at the Pentagon.
This means that, win or lose the election, Mr Bush had decided that things were going badly enough to remove one of the architects of the war.
In fact, when Mr Bush told reporters last week that Mr Rumsfeld would be staying on, he had already spoken to Mr Rumsfeld about leaving. He said to the news conference that "win or lose, Bob Gates was going to become the nominee."
Whether Robert Gates, an ex CIA director, is the kind of man to provide much of a fresh perspective remains to be seen. Until now he has always been an establishment figure. But he seems to be about to be one of the pegs on which new hopes will be hung.
The departure of Donald Rumsfeld is a major moment in the history of the Bush administration and the war in Iraq.
His resignation is a sign and an admission that the policy in Iraq has not worked, so far.
Apart from Vice-President Dick Cheney and President Bush himself, there was nobody who symbolised the administration's determination to wage the war on terror and to get rid of Saddam Hussein.
"We know they have weapons of mass destruction," he announced of the Iraqis at one stage. "We don't need any debate about it." His confidence and brusque dismissal of dissent was typical. For some, it amounted to arrogance.
Rumsfeld brought to the Pentagon years of ambition to stir up a department he had run as a much younger man under President Ford.
The recent book about the administration at war by Washington Post reporter Bob Woodward, State of Denial, tells of the blizzard of handwritten memos known as "snowflakes" with which he bombarded his officials.
He was determined to break what he saw as the old guard and to get control of policy himself, which he felt was too much in the hands of the generals and admirals.
He wanted a slimmer, more mobile military, one more capable of waging war on international terrorists and governments that supported them and less concentrated on the massive weapons systems that were being developed as if the Cold War had not ended.
Donald Rumsfeld felt himself to be the right man, in the right place, at the right time.
His direct, irascible, sometimes even folksy style appealed to many when things were going well. His famous dictum about there being "known unknowns" and "unknown unknowns", made pre-Iraq, was seen as quirky and "Rummy" at his most idiosyncratic.
However the very confidence that allowed him to make his mark on the Pentagon also led to his downfall.
It became overconfidence.
He ignored warnings that his reliance on hard-hitting, relatively small units would win the ground war in Iraq but would not win a guerrilla war.
Like most US policymakers, he simply did not believe that Iraqis would not welcome the invaders and take care of events for themselves from then on.
He was not a man of patience and did not in the end have the necessary patience for a long drawn out counter insurgency war. Nor did he show the flexibility of tactics needed to demonstrate to his commander-in-chief that he was going to deliver the victory the president believes is so necessary.
He had to go, whatever the results of the elections.
Source: BBCGüncelleme Tarihi: 20 Eylül 2018, 18:16