Children services workers met with both sides of the family within the past week, most recently on Saturday, said Dean Sparks, executive director of Lucas County Children Services.
"We only know that there were a lot of allegations back and forth," he said, adding that the grandmother was worried about placing the children back in the home with their 9-year-old brother, who had been disruptive in the past.
But the agency had no authority to decide who should keep the children, he said, and the parents had every right to bring them back into their home.
While the children were living with their grandparents, Hayes and her husband saw them often and went on outings to parks and the zoo, Turner said.
Turner said she never saw any indication of a strained relationship between Hayes and her mother, and they never went to court over the issue of custody.
Family members declined to comment.
Doug Hall, a neighbor who lives across the street, said he saw the Fords' son and the children raking leaves last week. He said the only unusual thing he's noticed was a police car at the house last Thursday. He said he didn't know why it was there.
Another neighbor said he saw the kids helping with the yard work and playing in the leaves just a few days ago.
"One minute they're doing the leaves, and then the next there are cop cars all over," Eric Pieper said.
Associated Press writer Kantele Franko in Columbus, Ohio, contributed to this report.