French international Youri Djorkaeff gave them a flying start with a second minute goal, but Chris Bart-Williams' first-half penalty and Jason Euell's 67th minute close-range goal earned Charlton their first win.
Wanderers' bright start owed much to Paul Warhurst's shooting power and Djorkaeff's quick reactions.
Warhurst's stinging right-foot shot was brilliantly tipped away by Dean Kiely but as the ball trickled towards goal Djorkaeff reacted before the Charlton defence could save the situation, helping the ball over the line.
Wanderers should have built on that but spurned a couple of good chances.
Henrik Pedersen could only turn and fire wide after Djorkaeff and Warhurst had split the Charlton defence then Ricardo Gardner found himself clear at the far post but could not direct Pedersen's cross on target.
That wastefulness cost the Wanderers when Charlton suddenly found themselves back on level terms after 26 minutes.
Bruno N'Gotty's challenge on Richard Rufus as they went up to meet Claus Jensen's corner seemed innocuous but York referee, Mr Messias immediately pointed to the spot.
Bart-Williams tucked the ball just inside the left-hand post.
The second half reflected much of what had happened in the first. Wanderers creating and wasting chances, while Charlton made sure of theirs.
Michael Ricketts, Anthony Barness and Kevin Nolan all threatened the Charlton goal but in the end paid the price when Euell was left unmarked on the six-yard box to tap in Graham Stuart's cross.
Charlton manager Alan Cubishley admitted he feared the worst after Wanderers had gone in front: "We were still looking for our first win, we were a goal down and we lost Gary Rowett with an injury only after five minutes.
"I thought we were going to get a battering but the lads have shown tremendous spirit and pulled themselves together to come away with a good win. I am delighted. After we went in front we gave them room to play but fortunately Dean Kiely has been superb."
Sam Allardyce could only reflect on missed chances: "We created more today than we did in the first three games of last season all of which we won.
"Unfortunately we are not taking responsibility in defence and that has let us down.
"We just weren't resilient enough at the back while we have to start taking out chances up front."