Paris (CNN) -- Francois Hollande defeated French President Nicolas Sarkozy in a presidential runoff Sunday, signaling a shift to the left as the country and Europe fight to dig out of a weak economy.

In a victory speech to supporters in Tulle, Hollande declared his win "a great date for our country, and a new start for Europe." The Socialist challenger promised to govern for all of France.

"Many people have been waiting for this moment for many long years. Others, younger, have never known such a time. ... I am proud to be capable to bring about hope again," he told the cheering crowd.

"Tonight, there are not two Frances. ... There is only one France, only one nation that is united with the same destiny," Hollande said.

The president-elect, who will be the nation's first left-wing president since Francois Mitterrand left office in 1995, suggested there was a sense of relief in many European countries because of his win.

"Austerity can no longer be something that is inevitable," he said.

Later, Hollande went to Paris, where he addressed a large crowd at the Bastille. Supporters waved flags and shouted his name.

"I know what many people feel -- years and years of wounds, of ruptures, and we have to repair, recover, unite. That is what we're going to do together," he vowed.

Hollande congratulated Sarkozy, who earlier conceded to Hollande as results from exit polls and official tallies in the runoff election came in.

François Hollande thanks supporters

Nicolas Sarkozy concedes election

Sarkozy loses presidency to Hollande

Why some feel discrimination in France

"I carry the entire responsibility for this defeat, and I'm going to say why. I fought for the values of responsibility, and I'm not a man who does not accept his responsibilities," Sarkozy said from his Paris campaign headquarters, as members of the crowd shouted, "No!"

"I'm ready to become a French person amongst French people, and more than ever I have the love for my country deeply ingrained in my heart," Sarkozy said.

With almost all of the votes counted, Hollande was leading with 51.6% to Sarkozy's 48.4%, the nation's Interior Ministry said. Voter turnout was reported at more than 80%.

Exit polls said Hollande won 51.9% of the vote, France 2 television reported.

Crowds roared at the center-left candidate's campaign headquarters as the exit poll results came out Sunday evening. Celebratory car horns blared along the Champs-Elysees in Paris.

"It's a great night, full of joy for so many young people all across the country," said Thierry Marchal-Beck, president of the Movement of Young Socialists.

U.S. President Barack Obama called Hollande shortly after results were in to congratulate the newly elected president and invite him to the White House.

Sarkozy had fought to keep his job amid a wave of discontent over his inability to rein in unemployment. He defended his economic record despite low growth and unemployment at about 10%, saying the impact of Europe's debt crisis could have been far worse.

France is a key player in plans to lead the eurozone out of its debt crisis, making the election vital to the region.

Sarkozy's defeat marks the latest -- and most significant -- of at least half a dozen European leaders swept from office during the eurozone economic crisis, including the Greek and Italian prime ministers.

France's vote came the same day as the Greek parliamentary election. Exit polls in that country indicated Greece's ruling coalition appeared headed for steep losses, while parties on the far left and far right were poised to make significant gains amid anger over austerity measures.

Hollande and Sarkozy traded insults last week in the only televised head-to-head debate of the campaign. Sarkozy labeled Hollande a liar and a "little slanderer," while Hollande accused the president of shirking his responsibilities, cronyism and favoring the privileged over France's poor.

Both candidates reached out to France's undecided voters since the first-round vote on April 22 left them the only two standing.

Centrist Francois Bayrou, who took 9% of the first round vote, delivered a boost to Hollande's campaign Thursday when he said he would vote for the Socialist and urged his supporters to vote according to their conscience.

Sarkozy has been president since 2007.