Hardy hit a two-out solo shot in the second, his 15th, to cut it to 2-1, but the Blue Jays got a run in the fifth on Kawasaki's two-out RBI single.
Nate McLouth singled to open the sixth and Nick Markakis drew a one-out walk. McLouth scored on Adam Jones' base hit and Davis followed with his third homer in the past two games.
Davis had two homers and five RBIs in Wednesday's 13-3 win over Detroit.
Encarnacion answered with a leadoff drive in the bottom half, his 20th, cutting it to 5-4, but Flaherty's leadoff drive in the seventh restored Baltimore's two-run lead.
Tommy Hunter replaced Hammel to begin the seventh and gave up a leadoff single to Izturis. Two outs later, Kawasaki tied it at 6 by hitting a 1-1 pitch into the right field bullpen.
Called out of the dugout for a curtain call, Kawasaki bowed several times before tipping his cap.
All three Baltimore homers came off Dickey, who matched a season-worst by allowing three long balls.
Dickey allowed six runs and seven hits in six-plus innings. The right-hander, who walked two and struck out five, has a 6.02 ERA in eight home starts this season.
Showalter was ejected in the second for arguing with plate umpire Angel Hernandez. He was replaced by bench coach John Russell.
Davis struck out swinging to begin the inning, but the pitch got away from catcher Thole and Hernandez ruled it a foul ball as Thole threw to first base. Gibbons came out and the umpires huddled, then called Davis out.
That brought out Showalter for an animated discussion with Hernandez. After being tossed, Showalter pointed at all four umpires, then mimicked throwing them out of the game.
"I don't know if the umpires considered the fact that two wrongs don't make a right," said Showalter, who was ejected for the first time this season.
NOTES: Baltimore 3B Manny Machado's single in the first was his ML-leading 104th hit. ... Blue Jays SS Jose Reyes (left ankle) went 1 for 4 and scored the game-winning run for Buffalo in the first Triple-A game of his rehab assignment. Reyes could be activated Monday. ... Baltimore activated RHP Miguel Gonzalez off the paternity list and put OF Steve Pearce (left wrist) on the 15-day DL, retroactive to Thursday. Gonzalez will start against Toronto on Saturday. ... Baltimore 2B Brian Roberts (right hamstring) is scheduled to begin a minor league rehab assignment with Triple-A Norfolk on Tuesday.