As the leader of a coalition government that led the nation for six years until its defeat in 2004, he ordered tests of nuclear warheads in 1998, prompting Pakistan to do the same and eliciting economic sanctions from abroad.
"I will not participate in any electoral politics," Mr. Vajpayee, 81, told a party gathering in Mumbai, also known as Bombay, late Thursday. His retirement was not a surprise, because he has been in poor health.
Mr. Vajpayee's exit coincides with the expected departure this weekend of Lal Krishna Advani as the party president. Mr. Advani, a minister in Mr. Vajpayee's government, prompted an outcry within the party in June by offering praise for the man Hindu nationalists have always demonized: Mohammed Ali Jinnah, the founder of Pakistan.
Mr. Advani will stay on as leader of the opposition in Parliament and will most likely continue to play an important role in the party, but the impending vacuum at the party's top and tussle for power below have the potential to upset the balance of Indian politics.
"It's going to be a tense and difficult year," said Mahesh Rangarajan, a political analyst who writes for several Indian newspapers. "It's a period of enormous confusion, division in the ranks at the very top and lack of clarity. Where do they go?"
Bharatiya Janata has been knocked around from within in recent weeks. Earlier this month, Uma Bharati, a colorful populist politician and a Hindu nun, was thrown out of the party. Last week, six of its members of Parliament were expelled by their fellow legislators over a corruption scandal.
In recent days the party's general secretary, Sanjay Joshi, resigned after a video was circulated purporting to show him in an intimate relationship with a woman. As a member of a radical Hindu group, the Rashtriya Swayamsevak, Mr. Joshi had taken a vow of celibacy.
The leadership change now under way presents a set of formidable challenges to the party in the new year: how to quiet the public bickering among its midlevel leaders and how to reconcile the demands of its hard-line Hindu radical backers while maintaining enough support from the general public to have a chance at regaining power nationally.
Swapan Dasgupta, a newspaper columnist sympathetic to Bharatiya Janata, minimized the impact of Mr. Vajpayee's retirement. "You have a generation that is moving away and a generation that hasn't necessarily moved in," he said.
In his speech to the party faithful on Thursday, Mr. Vajpayee, famous for his economy with words, appeared to give the nod to fresh leadership when he declared that Mr. Advani and a younger party official, Pramod Mahajan, would together "lead the party."