The 2007-09 crisis had been "hugely trust-busting", dragging banking and public institutions into disrepute, Haldane told a London audience at the Royal Society for the encouragement of Arts, Manufactures and Commerce (RSA).
"Even as the scars of the crisis heal, this trust deficit might not repair itself naturally ... (and) may be not cyclical, not temporary, but permanent," he said.
"If that's true, it strikes me as plausible, then those of us in financial services - including central banks - will really have to go some to repair that deficit," he added.
A report by public relations firm Edelman this year showed Britain ranked near the bottom for public trust in institutions.
A recent Bank of England survey of the public conducted by TNS showed it had a net satisfaction rating of 24 percent for its handling of interest rates - the lowest in almost four years and a far cry from its pre-recession average of 43 percent.
(Reporting by Andy Bruce and William Schomberg; editing by Andrew Roche)