Whorlton Hall hospital had at least 100 visits by official agencies in the year before abuse of inpatients there was discovered, the BBC has learned.
Inspectors, council officials and NHS staff all visited the County Durham unit – sometimes in teams of two or three over the course of several days.
But the scale of mistreatment of people there with learning disabilities and autism was not spotted.
Campaigners said the authorities had failed in their jobs.
Undercover filming by the BBC’s Panorama programme – aired on Wednesday – showed patients at the 17-bed unit being mocked, taunted, intimidated and repeatedly restrained.
The footage also included shocking scenes where some staff can be heard using offensive language to describe patients, while another calls the hospital a “house of mongs”.
A police investigation has been launched and 16 staff suspended.
The Care Quality Commission (CQC), which regulates the sector, went in three times – in March, April and July of last year. One of the visits lasted two days and involved a team of three after concerns were raised by a whistle-blower.
The inspection found breaches of regulations in relation to staffing and good governance, but the hospital kept its good rating.
CQC deputy chief inspector Dr Paul Lelliott said it was “now clear we missed what was going on”.
He said the regulator was sorry. He said inspectors spoke to staff and patients as well as independent people familiar with the hospital. But no concerns were raised.
“This illustrates how difficult it is to get under the skin of this type of ‘closed culture’,” he added.
On top of the CQC visit, there were multiple visits by 10 different councils and local NHS bodies.
Durham Council visited the hospital 33 times over the past year – 12 because of safeguarding concerns – with the rest largely related to the placement of new patients at the 17-bed unit.
The council’s corporate director of adult and health service, Jane Robinson, said: “We found no evidence suggesting issues of the nature shown in the programme.”
Richard Kramer, chief executive of disability charity Sense, criticised the approach taken by authorities.
He said agencies were maybe too likely to take the word of staff at “face value”, rather than insist on observations and on seeing the person.
“This appears to be a case of professionals not investing time and resources to fully review the care and support,” he added.
Earlier on Thursday, Care Minister Caroline Dinenage told the House of Commons she was “deeply sorry that this has happened”.
Ms Dinenage said that after the government and the Care Quality Commission were told of the allegations of abuse at Whorlton Hall, “immediate steps” were taken to ensure the safety of the patients there.
And she questions needed to be asked over whether the activity at Whorlton Hall was criminal, if the regulatory and inspection framework is working and also over the commissioning of care services.
The unit has now been closed and all the patients moved to other services.
Watch BBC Panorama: Undercover Abuse Scandal on BBC iPlayer.