Hong Kong's Duplicate Image Problem: What Comes Next and the Key Decisions Ahead

Hong Kong's public and commercial institutions are sitting on digital libraries riddled with duplicate images — identical or near-identical files stored multiple times across servers — and the bill for inaction is growing. The question now is not whether to clean house, but who decides how, and on what timeline.

The issue has sharpened in 2026 because several major digitisation pushes launched around 2021 and 2022, partly to support remote working under pandemic restrictions, are now reaching their three- to five-year review points. That means storage contracts are coming up for renegotiation and institutions must decide whether to renew capacity, or finally deduplicate and reduce their footprint.

Where the Problem Is Concentrated

The Hong Kong Public Libraries network, which operates 72 branch locations including the flagship Hong Kong Central Library on Causeway Bay's Moreton Terrace, has been digitising historical photograph collections since at least 2019. Sources familiar with large-scale archival projects say duplicated scan files — created when batches are processed more than once without adequate checksums — can represent anywhere from 15 to 30 percent of raw storage in an active digitisation workflow. The Hong Kong Film Archive in Sai Wan Ho faces a structurally similar challenge, managing a catalogue that spans decades of physical and born-digital material ingested from multiple donor streams.

Private-sector pressure is equally real. The city's financial institutions, clustered in Central and in the newer towers of Kowloon's East commercial district around Kwun Tong, maintain marketing and compliance image libraries subject to Securities and Futures Commission recordkeeping rules. When the same branded asset is stored in multiple departments under slightly different filenames, retrieval failures during audits become a compliance exposure, not just a storage inefficiency.

Hong Kong's commercial data centre market, anchored by facilities in Tseung Kwan O's industrial belt and in Tsuen Wan, charges enterprise clients roughly HK$2,000 to HK$4,500 per rack unit per month depending on tier and redundancy level. At that price, even modest deduplication — cutting storage by 20 percent across a medium-sized institution — translates into material annual savings. For organisations reviewing budgets in the second half of 2026, that arithmetic is difficult to ignore.

The Decisions That Will Shape the Next 12 Months

Three choices will define how this unfolds. The first is technical: whether institutions adopt perceptual hashing tools — software that identifies visually identical images even when file sizes or metadata differ — or rely on simpler filename and checksum matching. Perceptual hashing catches more duplicates but demands more processing overhead and trained staff to review edge cases.

The second decision is governance. The Innovation, Technology and Industry Bureau, which has been driving Hong Kong's smart city agenda under its Smart City Blueprint framework, has not yet issued sector-specific guidance on image deduplication standards. Without a common benchmark, each institution develops its own policy, creating interoperability problems when collections are eventually linked across, for example, the Hong Kong Heritage Museum in Sha Tin and university special collections at HKU Libraries in Pok Fu Lam.

The third is timing. Institutions that wait until their next full storage contract renewal — often a three-year cycle — risk accumulating another year or two of unchecked duplication. Those that act now face the upfront cost of a deduplication audit but gain cleaner data and lower ongoing storage spend.

The practical path forward for most organisations involves commissioning an inventory audit in the third quarter of 2026, before year-end budget submissions lock in 2027 storage allocations. Institutions should document their deduplication methodology clearly, since regulators and auditors increasingly treat data governance records as part of broader compliance frameworks under Hong Kong's evolving personal data and cybersecurity requirements. Doing nothing is itself a decision — and an increasingly expensive one.