Petraeus aides said Kelley took it to another level, winning the title of "honorary ambassador" for her extensive entertaining. Petraeus even honored Kelley and her husband with an award in a special ceremony at the Pentagon.
The Kelleys invited then-Florida Gov. Charlie Crist to their home, but he never accepted. Instead, he had dinner with the Kelleys at a steakhouse in Tampa with Petraeus.
"Lovely people. I remember I had a nice dinner with she and her husband and the general and his wife," Crist said.
Ken Walters, a neighbor, said he went to a party the couple had to celebrate their first son's baptism. The Kelleys have two other children.
"Natalie and her sister, they're certainly not shrinking violets," Walters said. He recalled that when the sisters first entered the South Tampa social scene, they "rubbed people the wrong way. I think they probably stepped on a couple of toes."
Petraeus' affair with Broadwell was discovered after Kelley told an FBI agent friend she had received email warning her to stay away from Petraeus. The email turned out to be from Broadwell, who apparently regarded Kelley as a rival for Petraeus' affections. Kelley's family and Petraeus aides have said Petraeus and Kelley were just friends.
In another strange footnote to the scandal, long before the case involving Petraeus got under way, the FBI agent sent Kelley shirtless photos of himself, according to a federal law enforcement official.
Kelley's brother, David Khawam, told WPVI-TV in Philadelphia on Monday night that the allegations have spilled out quickly.
"It's a shock. We're trying to figure out where the pieces are falling right now," he said in an interview at his law office in Westmont, N.J.
He said his family left Lebanon for suburban Philadelphia's Huntingdon Valley in the 1970s to escape the turmoil in their homeland. His parents opened a Middle Eastern restaurant in Voorhees, N.J., called Sahara.
"My family is very patriotic; we came from Lebanon at a young age," he said.
Kelley later married and moved to Florida with her husband, who works at a cancer clinic.
Jill and Scott Kelley have been involved in at least nine legal actions since arriving in Tampa, according to court records. Most involve real estate transactions, including one foreclosure and an $11,000 judgment against the couple in a Pennsylvania case.
In another twist in the scandal, court records indicate that Petraeus and Allen intervened two months ago in a messy custody dispute on behalf of Jill Kelley's sister. Both four-star generals wrote letters supporting Natalie Khawam.
The judge in the case awarded Khawam's ex-husband custody last year of their son. He also called Khawam, a lawyer, dishonest and lacking in integrity.
Follow Tamara Lush on Twitter at http://twitter.com/tamaralush .
Associated Press writers Kimberly Dozier and Pauline Jelinek in Washington and Brendan Farrington in Tallahassee, Fla., contributed to this report.