In the ensuing debate, intelligence officials could not point to any specific attack the program had thwarted. But they defended it as a useful tool when new terrorism-linked phone numbers were identified, and suggested that had it been in place before Sept. 11, it might have helped uncover Al Qaeda’s plot. Critics rejected the Sept. 11 argument as exaggerated and portrayed the program as ripe for abuse and as a legally dubious invasion of privacy.
Eventually, the Obama administration and Congress agreed on a reform law that would end the N.S.A.’s bulk collection of domestic calling data, but preserve its ability to swiftly gain access to records held by telecoms when a judge agreed that a specific number had terrorism links. The idea was to reduce the risk of abuse while preserving the analytical capability.
That law — the USA Freedom Act of 2015 — permitted the N.S.A. to build a system linking up with the telecoms under which the agency could retrieve logs of phone calls and texts for a specific suspect, as well as the logs of communications by everyone who had ever been in contact with that suspect — even when they were customers of different phone companies.
Under the old program, the N.S.A. had been vacuuming up billions of logs about Americans’ communications every day. Under the replacement Freedom Act system, that number dropped significantly, although its scale remained large in absolute terms. In 2016, the agency obtained orders to target 42 suspects and collected 151 million records. In 2017, it obtained orders to target 40 suspects and collected 534 million records.
But public signs of trouble with the Freedom Act system began to emerge in June 2018, when the N.S.A. announced that it had discovered “technical irregularities” that caused it to collect more phone records than it had legal authority to gather.
The agency has been coy about the details, saying they are classified, but in broad strokes has said that for various reasons telecoms were returning both accurate and inaccurate numbers in the list of accounts a suspect had been in contact with. When the N.S.A. fed those numbers back into the system to get the “second hop” of calling data from the suspect’s contacts, it compounded the problem.
Unable to separate the good data from the bad, the agency deleted its entire collection of Americans’ phone records — hundreds of millions of communications logs — and started over. But in October 2018, it discovered that the problem was happening again, and, once again, had to purge the data, according to a recently declassified inspector general report.