Francois Hollande came top with 28.6% and Mr Sarkozy got 27.1% - the first time a sitting president has lost in the first round.
Third-place Marine Le Pen took the largest share of the vote her far-right National Front has ever won, with 18%.
Referring to her voters, Mr Sarkozy said: "I have heard you."
"There was this crisis vote that doubled from one election to another - an answer must be given to this crisis vote," he said.
Pollsters say Mr Hollande is the clear favourite to win the second round on 6 May, a duel between him and Mr Sarkozy, who leads the centre-right UMP.
If Mr Hollande wins he will become the first Socialist president in France in 17 years.
Watch from Reuters: