The letter said the database may only be queried for intelligence purposes where there is a reasonable articulable suspicion "based on specific facts" that an identifier, such as a phone number, is associated with a specific foreign terrorist organization previously identified and approved by the court.
"Consequently, only a very small fraction of the records acquired under this program is ever reviewed by intelligence analysts," the letter said.
It said that in 2012, fewer than "300 unique identifiers were authorized for query" under the standard.
According to the letter, congressional intelligence committees are regularly briefed on the program.
"Thus, the program has been approved and is rigorously overseen by all three branches of the government," the letter said.
The lawyers wrote that the ACLU is trying to stop a program "that plays an important role in the government's overall strategy to protect the American people from terrorist threats."
It said storing a large volume of information is necessary to ensure that a much smaller subset of terrorist-related phone records are preserved, since phone companies store data for a limited period. The letter added that aggregating the data is necessary as well to identify records that involve different telecommunications networks.
ACLU attorney Alex Abdo called the government's view of Americans' right to privacy "shockingly narrow."
"If the government is correct, then the NSA may warehouse all of our domestic communications now so long as it only searches for a subset of them later on," Abdo said in a statement. "The Constitution does not permit the government to cast a dragnet over every Americans' communications on the theory that the NSA might one day need them.
"There is also a tension between the government's public insistence that its collection of every Americans' phone records is lawful and its vigorous attempt to prevent that very question from being addressed by the court."