Edmonton had 14 shots in the first while Colorado only managed three on Dubnyk.
The Oilers took a three-goal lead midway through the second period on yet another man-advantage opportunity as Hall's shot hit Horcoff's skate in front and squeaked past Varlamov.
Colorado had its best chance to that point of the game a minute later as Jamie McGinn had Dubnyk beat but rang a shot off the post.
Persistent pressure by the Avalanche in the third finally paid off 8 minutes in as Parenteau fired a shot under Dubnyk's pads at the side of the net to make it 3-1.
Yakupov scored his third of the season on yet another power play with 30 seconds left and Varlamov on the Colorado bench for an extra attacker.
Varlamov finished with 23 saves.
NOTES: It was the first of five meetings between the Northwest Division rivals. The teams split their six season meetings last season. ... The Avalanche were without captain Gabriel Landeskog, variously listed as out with leg and/or a head injury. Landeskog was levelled late in the first period by San Jose's Brad Stuart in Colorado's last game, but did return to finish the contest. ... The Avs remain without last year's leading scorer, Ryan O'Reilly, who is holding out due to a contract dispute. ... Edmonton made one lineup change, pulling out newly-acquired defenseman Mark Fistric for Corey Potter. ... The Oilers were without Ben Eager (concussion), Nikolai Khabibulin (hip) and Theo Peckham (hip), who are all out day-to-day. Ryan Jones (eye) and Andy Sutton (knee) are out indefinitely. ... The Oilers came into the game having been short-handed 13 times in the first period in four games so far this season, the most in the NHL. They took two first-period penalties against Colorado and only three in the game.