How New York's Public Records Ended Up Full of Duplicate Images — and Why It Took a Decade to Fix

New York City's open-data infrastructure has a problem hiding in plain sight. Across multiple municipal agencies, digitized public records — property filings, building permits, environmental assessments — contain duplicate images embedded in the same document packages, a technical failure that inflated storage costs, slowed retrieval systems, and in some cases made records harder, not easier, to navigate. The city's Department of Information Technology and Telecommunications, known as DoITT, and the Department of City Records have been working since late 2024 to audit and replace those redundant files, a process that officials have described in budget documents as ongoing through the 2026 fiscal year.

The timing matters. New York is hosting FIFA World Cup matches this summer, drawing global scrutiny to city operations. At the same time, the Adams administration has made streamlining city services a recurring talking point in its budget presentations. Duplicate image files are, on their own, a mundane technical issue — but they represent a broader pattern in how the city's digitization push of the early 2010s was executed without consistent quality controls, and how the downstream costs of that era are still being absorbed now.

The Digitization Rush That Left Loose Ends

The roots of the problem go back to roughly 2011 and 2012, when the Bloomberg administration accelerated a push to move paper records into digital systems. The Municipal Archives on Chambers Street in lower Manhattan processed millions of pages during that period. So did the Department of Buildings, whose online portal — the BIS system, later succeeded by DOB NOW — became the primary public interface for construction permit records in neighborhoods from Mott Haven in the Bronx to Sunset Park in Brooklyn.

The digitization was done quickly and, in many cases, through third-party scanning contractors whose quality-assurance standards varied. When the same document was scanned more than once — because a paper copy had been refiled, or because different agency units held overlapping records — both versions often ended up in the same digital package. The DOB NOW system, which the Department of Buildings launched in phases beginning around 2016, inherited many of those legacy files without a systematic deduplication pass.

The result is measurable. City technology budget documents from fiscal year 2025 cited data storage for legacy agency records as one of the top five cost drivers in the municipal cloud migration program, though those documents did not break out duplicate-image storage as a separate line item. Independent technology auditors have estimated, in filings reviewed by open-government advocates, that redundant files in large municipal digitization projects can account for anywhere between 12 and 20 percent of total stored data — a significant figure when the city's aggregate record storage runs into the petabyte range.

What the Fix Actually Looks Like

DoITT's current approach involves automated hash-matching — a standard technique that assigns each image file a unique digital fingerprint and flags identical copies for review. Flagged duplicates are not deleted outright; instead, they are moved to a segregated archive and a replacement index image is inserted, preserving the document chain while removing the redundant file from active retrieval queues. The Municipal Archives on Chambers Street is serving as the pilot site for the replacement program before the process scales to other agency document repositories.

For ordinary New Yorkers trying to look up a property record or a zoning filing, the practical effect has been intermittent: some documents pulled through the city's ACRIS property records system or through the DOB NOW portal have displayed placeholder images while the underlying files are being reindexed. Staff at the Brooklyn Public Library's Central Branch on Grand Army Plaza, which maintains a public-access terminal for city records, have fielded questions from patrons about incomplete document displays since early 2025.

The deduplication project is expected to complete its first full pass across Department of Buildings records by the end of calendar year 2026. Anyone who needs a complete copy of a filed document in the meantime can request a certified paper copy through the Department of City Records office at 31 Chambers Street, where walk-in service is available weekdays. Processing time for certified copies currently runs five to seven business days.