Ellington's two free throws made it 104-101, and Williams was short with a 3-pointer in the final second.
It was a crushing loss for the Jazz, who are trying to hold off the Los Angeles Lakers for the eighth playoff spot in the Western Conference.
Paul Millsap scored 16 points and Alec Burks 14 for the Jazz, which lost in overtime at Milwaukee on Monday. Utah played its third straight game without starting forward Al Jefferson, who has a sprained left ankle.
"These are tough losses," Utah coach Tyrone Corbin said. "You don't want to minimize that. They are games we felt we should have won. Closing games down, especially without Al out there, we're so used to going to him to close the games for us and he's not there.
"We've got to find a way to get better."
Corbin benched starting forward Marvin Williams for DeMarre Carroll, hoping it would spark Utah's first unit. But Carroll scored just four points and the Jazz reserves outscored the starters 56-45. Williams did not score in 14 minutes.
Randy Foye scored 11 points in the third quarter, when the Jazz went on a 16-3 run to open a 71-58 lead.
The Cavaliers were only missing two players with the flu, but they all appeared under the weather.
Cleveland lacked energy, but a dunk by Alonzo Gee in the final second of the quarter seemed to ignite the Cavs. But they fell behind 88-76, when Irving returned with 7:26 left. Shortly after that, he was bringing them back to an unexpected win.
"Absolutely we did (steal this win)," Irving said. "But the one thing I can say is we did get stops at the end. Our defense led to a couple of turnovers and baskets. I've got to be a leader out there, and it starts on the defensive end. Now that I'm preaching it, I've got to do it."
The Cavs were without starting rookie guard Dion Waiters, who missed his second straight game with a flu bug that has flattened a few teammates. Daniel Gibson also sat out with nasty stomach virus, which sidelined Tyler Zeller for one game.
NOTES: Hayward scored 15 points in the second quarter, helping the Jazz open a 51-49 halftime lead. ... The Cavs had a season-high 12 blocks. ... Corbin said there was no timetable on Jefferson, who rolled his ankle last week in a loss to Atlanta. ... Irving is the ninth player in NBA history to surpass 2,000 career points before turning 21. Irving's birthday is March 23.