They blew them up with landmines then doused them with petrol before setting them alight, finally using bayonets to finish off anyone still left alive.
This was just one small incident in what has become known as the Nanjing massacre, a six-week orgy of violence in which tens of thousands of Chinese people died.
A day before the massacre's 70th anniversary, a small group of survivors, dignitaries and visiting Japanese citizens gathered at Taiping Gate to remember the dead.
It is just one of a number of commemoration events being held in Nanjing this week to mark a bloody episode that still reverberates in East Asia today.
We only know about the Taiping Gate killings because of the tenacity of Japanese teacher Tamaki Matsuoka, who wanted to know more about the massacre.
With a number of others, she set up telephone hotlines in several Japanese cities in 1997, inviting former soldiers who had served in Nanjing to call up.
The group interviewed more than 200 old soldiers and, from the information it gathered, was able to identify the exact army unit that had carried out the Taiping killings.
Güncelleme Tarihi: 13 Aralık 2007, 16:00
Ms Matsuoka told her story at the Taiping memorial ceremony, which took place on a cold, wet Nanjing morning.
Before helping to unveil a small monument, she said she hoped Chinese young people would not forget what had happened in Nanjing.
There is little chance of that. The atrocities carried out by the Japanese in Nanjing, and elsewhere in China during World War II, are drilled into Chinese schoolchildren.
Some of them - from Nanjing 's 34th Middle School - were even on hand at the Taiping ceremony to lend their support. They promised to look after the monument.
In a speech, 17-year-old Wang Lin, head of the school's student association, said historical events should not be allowed to simply fade from memory.
Using a well-known Chinese phrase, she said past experience, if not forgotten, is a guide to the future.
She added: "Chinese people love peace, but you will be bullied if you fall behind. This is a serious lesson passed down by our ancestors."
But not everyone is as keen about remembering the past as this Chinese schoolgirl.
Although the Japanese government - and most Japanese people - acknowledge what went on in Nanjing, a small group of people claim the massacre has been fabricated.
Others say the 300,000 death toll often cited by the Chinese government is too high.
It was an issue touched upon at the Taiping ceremony by Lin Boyao, who is ethnically Chinese but has lived in Japan since 1978.
"Even today, surprisingly, there are some people who deny the Nanjing massacre is a historical fact," he said.
"How can they be so shameless? This shows hatred for the dead and brings shame on the living."
The debate about what exactly happened after the Japanese army entered Nanjing on 13 December is not just about how to remember the past.
Even though the two sides are increasingly close trading partners, Japan's attitude towards World War II colours Chinese government policy to this day.
Former Japanese Prime Minister Junichiro Koizumi's frequent trips to the Yasukuni shrine, which honours convicted war criminals as well as other war dead, soured Sino-Japanese relations for many years.
They only improved in 2006 when Koizumi handed over power to Shinzo Abe, who did not visit the shrine while he was in office - he sent a pot plant instead.
Inevitably, this debate also fuels the anger of some Chinese people, a sentiment Beijing has occasionally encouraged.
In 2005, there were rare street demonstrations in several major Chinese cities over Japanese history textbooks which critics claimed whitewashed the country's World War II record.
Despite the possible anger aroused by the anniversary, the owner of a Japanese restaurant in Nanjing said he was unconcerned.
"I've been here 15 years and am not worried at all. No-one talks to me about the massacre," he told the BBC as he waved goodbye to the final customers of the evening.
But the memory of the Nanjing massacre will not simply go away, at least as long as there are survivors such as Xiang Yuansong.
Now 80, Mr Xiang was just 10 when Japanese soldiers entered what was then the capital of China.
His 25-year-old brother was killed by the invaders. He remembers looking for him among the piles of dead bodies.
"We can never forget history," was his angry comment as he made his way home from the newly unveiled Taiping Gate monument.