New York's War on Duplicate Images in Public Records: How the City Stacks Up Against London, Tokyo, and São Paulo

New York City's Department of City Records is sitting on a problem that costs real money and wastes real time: tens of thousands of duplicate images embedded in public-facing property databases, permit filings, and land-use archives that have accumulated over decades of piecemeal digital migration. The city has known about the redundancy issue since at least a 2023 audit flagged it internally, and now, in the middle of a FIFA World Cup summer that has drawn unprecedented international scrutiny to municipal infrastructure, the pressure to clean house has sharpened.

The timing matters. With roughly 1.5 million visitors projected through New York's five boroughs during the tournament window, government portals like the Department of Buildings' BIS system and the Department of Finance's ACRIS property database are being accessed at volumes that stress-test backend storage and retrieval speeds. Duplicate image files — sometimes three or four copies of the same scanned deed or construction photo occupying separate database entries — slow query response times and inflate cloud storage costs that ultimately land on the city budget.

What New York Is Actually Doing

The Office of Technology and Innovation, housed at 255 Greenwich Street in Lower Manhattan, confirmed earlier this year that it had launched a deduplication initiative across several agency data systems as part of its broader NYC Digital Services modernization program. The effort targets legacy records that were digitized between 2005 and 2018, a period when scanning protocols varied wildly from agency to agency and quality control was inconsistent. The Department of Buildings and the Landmarks Preservation Commission, which maintains archives for more than 37,000 individually designated properties citywide, are both listed as priority agencies in the initiative's first phase.

The Landmarks Preservation Commission's archives are particularly tangled. Records for historically significant districts — including the nearly 2,500 buildings in the Brooklyn Heights Historic District and documentation files tied to landmarked structures along West 118th Street in Harlem — contain image files that were scanned multiple times under different archivists using different naming conventions. The result is a sprawl of near-identical JPEGs and TIFFs that consume storage without adding informational value.

How Peer Cities Are Handling the Same Problem

London's approach offers an instructive contrast. The Greater London Authority completed a systematic deduplication sweep of its Planning London Datahub in late 2024, using hash-matching algorithms to identify identical files regardless of filename or upload date. The GLA publicly reported that the process reduced its active image repository by roughly 22 percent, freeing meaningful server capacity ahead of migrating the system to a consolidated cloud environment. That migration was completed by March 2025.

Tokyo's ward offices — particularly Shinjuku Ward and Minato Ward, which manage dense commercial real estate documentation — have operated under a unified digital asset management standard since 2022, meaning duplicate uploads are rejected at the point of entry rather than caught after the fact. São Paulo's municipal government, through its Secretaria Municipal de Gestão, rolled out a similar front-end validation layer in 2023 as part of a World Bank-supported e-government initiative, catching redundant files before they enter the archive rather than requiring retroactive cleanup.

New York, by contrast, is still largely working backward through existing records rather than preventing new duplication at the source. The Office of Technology and Innovation has indicated that front-end validation tools are on the roadmap, but no public timeline has been attached to that commitment.

For residents, the practical fallout is most visible when pulling property records through ACRIS at the Manhattan recording office on 66 John Street, or when contractors try to retrieve permit photographs through the BIS portal. Redundant records can return multiple search results for the same document, requiring users to manually determine which version is authoritative — an irritating, time-consuming process that hits small property owners and independent contractors hardest.

The city's first-phase deduplication work is expected to reach completion by the end of calendar year 2026. Anyone dealing with duplicate record returns in the meantime can flag discrepancies directly to the Department of City Records via its online constituent portal, which launched a dedicated data-quality reporting form in January 2026.