Tripoli, Libya (CNN) -- Fighters pushed Sunday to the outskirts of one of Moammar Gadhafi's last bastions of support, setting the stage for possible clashes ahead of this week's deadline for loyalists of the ousted leader to surrender.

Libya's new leaders gave Gadhafi loyalists in Sirte, Bani Walid and a handful of towns until Saturday to surrender or face military force.

"We're giving them a chance to come forward and negotiate surrender," Col. Ahmed Bani, spokesman for the National Transitional Council's defense ministry, told reporters Sunday.

But there were indications that fighters were planning to enter Bani Walid, where a powerful tribe is sympathetic to Gadhafi, before the deadline.

One of Gadhafi's sons, Saadi Gadhafi, told CNN's Nic Robertson in a phone conversation that negotiations aimed at a cease-fire had been going well, but that an "aggressive" speech broadcast by his brother, Saif al-Islam Gadhafi, a few days ago caused talks to break down. He said he saw no further point in negotiating and believes the NTC is ready to go into Bani Walid.

NTC negotiator Abdallah Kinshel said the planned talks fell apart after local leaders refused to come to the designated meeting spot -- a mosque outside of town -- and demanded the NTC fighters come into the town unarmed. The anti-Gadhafi forces refused.

"As far as we are concerned, negotiations are over from our side," Kinshel said. "Military commanders must determine what the next steps will be."

Asked his location, Saadi Gadhafi said he is "a little bit outside" of Bani Walid but had been moving around. "It seems that his options are running out," Robertson said.

Saadi Gadhafi said he had not seen his father or brother for two months. He said he is "neutral" and not on his father's side or that of the rebels, but remains "ready to help negotiate a cease-fire."

NTC fighters positioned outside Bani Walid said they remain unwilling to shed blood or fire shots.

"Some tribal leaders and many of the residents have surrendered their weapons, but there are still many loyalists who are protecting Moammar Gadhafi and his sons," National Transitional Council media coordinator Adel Zintani said.

The deadline was extended by a week to curb further bloodshed and prevent further destruction, military commanders have said.

"This extension does not mean we are unaware of what Gadhafi's accomplices are up to," NTC chairman Mustafa Abdul Jalil said at a news conference, countering earlier criticism that a grace period might give Gadhafi's forces to regroup.

But Gadhafi's spokesman, who has not been seen since Tripoli fell, said key tribal leaders in Bani Walid remain loyal to the ousted leader.

Musa Ibrahim, in a telephone call from an undisclosed location, told Reuters that the transitional council's messages "are not being heeded here in Bani Walid."

Ibrahim also told Reuters that Gadhafi remained in Libya, though he said he did not know where. He did say Gadhafi was well defended.

Anti-Gadhafi forces were positioned around the former leader's hometown of Sirte as well as Bani Walid, Jalil said Saturday.

Asked to comment Sunday on a report that the NTC knows Gadhafi's whereabouts, Col. Bani refused to do so.

Ali Tarhouni, interim deputy prime minister and oil minister of the NTC, said Bani Walid was close to falling.

"It's possible, although we are not sure, that the Bani Walid (tribe) has joined the revolution, and now it's under control of the revolutionaries," he said.

Anti-Gadhafi fighters from the east pushed toward Bani Walid Saturday with virtually no resistance. They were able to reach El Mardum, which sits on the border of Bani Walid province and is home to Khamis Gadhafi's 32nd Brigade base.

The anti-Gadhafi forces entered the base and arrested three men in civilian clothes they claim were loyalists. They also took seven armored personnel carriers.

Tarhouni said Libya's new leadership will move their headquarters from Benghazi to Tripoli this week to begin implementing political plans to shape a new future.

But for the time being, guns trump government on the streets of the capital.

Tripoli has become a city of checkpoints, weapons and no real authority as the threat of Gadhafi's loyalists lingers.

Jittery and suspicious anti-Gadhafi fighters blocked a road Saturday where a drive-by shooting occurred earlier. They collected weapons and registered them at police stations. Those who called themselves rebels just a week ago were now working with Tripoli's law enforcement authorities.

With Gadhafi's armories emptied, guns -- always in large supply in Libya -- have proliferated on the streets.

Those who want to carry weapons now must be issued identification cards, but the selection process is not centralized -- neighborhood councils are making that decision.

A group called the Tripoli Revolutionary Council is trying to exert control over the city, creating the potential for further conflict with the established National Transitional Council in a volatile situation.

Tarhouni announced Saturday the formation of the Supreme Security Committee, which held its first meeting Friday. Among the priorities for the committee were the protection of public institutions and weapons in Tripoli.

The transitional council, meanwhile, is trying to get Libya's oil-dependent economy going again.

It expects to restart oil production at the Misla and Sarir oil fields in less than two weeks, Tarhouni said.

In related news, Cuba withdrew its ambassador from Libya, saying it does not recognize the NTC.

CNN's Arwa Damon, Ben Wedeman, Mohammed Tawfeeq and Frederik Pleitgen contributed to this report.