The government moved the leaders after claiming it had information of a possible escape plot from the former holiday camp where they had previously been held.
As well as the alleged escape plot,
The breakdown of talks comes amid scrutiny of close ties between some of Uribe's top political allies and the paramilitary groups, which were originally created by landowners and drug-traffickers to take control of the countryside from leftist rebels.
Earlier this week, the supreme court interviewed members of congress on their alleged ties with the paramilitaries.
Three members of congress and a former congresswoman are already in jail on charges of creating and bankrolling paramilitary groups.
Restrepo said the leaders would now appear before special tribunals where they will be judged for their actions in the course of the conflict.
According to terms of the peace pact the leaders will receive no more than eight years in prison each.
Under the pact, the paramilitary bosses must confess to their crimes, pay reparations to their victims and return land they took from people who fled areas affected by violence.
While Uribe's government has engaged the paramilitaries in peace talks, it has not negotiated with the Revolutionary Armed Forces of Colombia (FARC), the country's main left-wing guerrilla group.
Government peace talks with another group, the 4,500-strong National Liberation Army (ELN), have also made little progress.