The diminutive winger latched onto a long punt forward by goalkeeper Jussi Jaaskelainen to catch the Derby defence sleeping.
He flicked the ball over goalkeeper Lewis Price, before heading into the empty net to send a frustrated home crowd into ecstasy.
The rock-bottom Rams had been on the back foot for most of the clash but went close twice in a first-half minute through a Craig Fagan header and a Michael Johnson strike that rattled the upright.
Nicolas Anelka did exactly the same midway through the second period when his fizzing 25-yard drive was tipped onto the bar.
The Frenchman admitted before the game that he fancies a move to Chelsea, but he was desperate to give the Wanderers a perfect leaving gift.
Kevin Nolan went close on numerous occasions, failing to control the ball in the box twice in the first half.
But it was his second-half sitter that caused him to be booed off by the fans when he was subbed just moments later.
He shanked an Anelka cross over from just three yards when it seemed easier to score than miss.
El Hadji Diouf had a goal ruled out for offside, while Swedish winger Christian Wilhelmsson was the best player on the park even though he only came on at half-time for Kevin Davies.
But then it was left for Stelios to steel the show with his late, late winner.
Derby have only claimed one point from 24 under Paul Jewell, while Bolton ended a run of two successive defeats.