The Church of England is examining the personnel files of thousands of clergymen and women dating back to the 1950s as part of a wide-ranging investigation into historical allegations of child abuse, the archbishop of Canterbury has confirmed.

The Most Rev Justin Welby, the principal leader of the Church of England, said that the move is part of a renewed effort to ensure that there are no allegations of abuse which have been ignored, overlooked or covered up.

Following recent disclosures of evidence of child abuse within the church, Welby conceded that the investigations would show that “there is more that has not been revealed”.

He also spoke for the first time about plans by the church to launch a study which will examine the confidentiality of the confessional – a move which could overturn 1,800 years of tradition.

Last week, the archbishop of York, Dr John Sentamu, said he was deeply ashamed of the church’s failure to protect vulnerable children after an independent inquiry found “systematic failures” in its attempts to stop the Very Rev Robert Waddington, the former dean of Manchester, who died seven years ago.

At a press gallery lunch in Westminster, Welby said that he had regularly listened to abuse survivors.

“It is beyond description – terrible. I had a meeting with some survivors a few weeks ago and was giving a talk later that afternoon, somewhere else and on a completely different subject, but someone asked, it was a theological colleague, about issues of safeguarding, and to my intense surprise – and I don’t normally do this sort of thing – I broke down completely.

“It was the shredding effect of hearing what we did – what we did – to those people and the sense of total failure and betrayal,” he said.

Detailing the process of trawling the clergy’s personnel folders, known as blue files, he said the inquiry is being conducted by external investigators as well as church staff to ensure there is no question of a cover-up.

“We will systematically bring those transparently and openly first of all working with the survivors where they are still alive and then seeing what they want.

“The rule is survivors come first, not our own interests, and however important the person was, however distinguished, however well-known, survivors come first,” he said.

Waddington, who died in 2007, was head of education for the Church of England, the dean of Manchester Cathedral, and governor of a music school where he was responsible for mass abuse against children.

Welby also said that the General Synod is launching a study of the confidentiality of the confessional – which is “actually an incredibly radical move which challenges more than 1,800 years of church tradition”.

The current Church of England understanding is that what is mentioned in confession may not be revealed by the priest, whether or not the penitent gives his or her permission.

Most confessors would counsel a penitent to reveal a crime to the police as a condition of absolution, but would not “break the seal of the confessional” themselves. This is the change that is being discussed.

Such a step would be “incredibly radical” in breaching 1,800 years of tradition, Welby said.

The failure of churches – not just Welby’s own – was of far greater importance than that of other institutions, including the media, because they set themselves higher standards, he said.

Speaking at the Commons, Welby made a candid admission of past failures and a future commitment not to countenance cover-ups.

He revealed that the blue files on every Anglican clergyman since 1950 are being audited, diocese by diocese, for any evidence – even a cryptic remark – which might point to an abuse case.

In nine out of 10 cases, the follow-up showed that such remarks had been misunderstood or that all the parties were now found to be dead.

Welby said that bishops had heard a lot of evidence from victims of clerical abuse. He spoke after it was revealed that a woman whose three sons had all been subjected to clerical abuse had been sent a nearly identical brush-off letter, no more than two paragraphs, from officials in Lambeth Palace under both himself and his predecessor.