NYC Agencies Remove Millions of Duplicate Files From Digital Records System

New York City's effort to purge duplicate images from its sprawling municipal database systems notched a meaningful step forward this week, with the Department of Buildings and the Department of City Records both confirming progress on a joint audit that began in January 2026. The cleanup targets tens of thousands of redundant scanned files — permit documents, inspection photos, deed records — that have accumulated across city servers for more than a decade, slowing processing times at a moment when the Adams administration is under pressure to speed up housing approvals citywide.

The timing matters. With congestion pricing now fully operational and the 2026 FIFA World Cup drawing global attention to New York infrastructure, city agencies face heightened scrutiny over how efficiently they handle permitting, licensing and public records requests. Duplicate image files — the same scanned document stored multiple times under different file names or case numbers — have been flagged internally as a drag on system performance, inflating storage costs and, in some cases, confusing inspectors pulling records in the field.

Where the Backlog Is Worst

The problem is most acute in the Department of Buildings' BIS portal, the Buildings Information System that contractors, architects and property owners use daily to track permit status across the five boroughs. DOB staff identified the duplication issue as particularly concentrated in records tied to properties in Bushwick, the South Bronx and parts of Lower Manhattan, where high volumes of renovation permits were filed rapidly during the 2021-2023 building boom. The Municipal Archives on Chambers Street, which holds physical and digital records going back generations, has also been working with the city's Office of Technology and Innovation since March to flag redundant image files uploaded during a 2019 mass-scanning initiative.

The Office of Technology and Innovation — based at 1 Liberty Plaza — has deployed automated deduplication software across several agency servers since April. The tool cross-references file hashes and metadata to identify matches before flagging them for human review, rather than deleting files outright. That human-review step was added after an early pilot in 2024 mistakenly tagged legitimate amended permits as duplicates.

Why It Hits Homeowners and Developers Now

For ordinary New Yorkers, the practical effect shows up in wait times. The city's own performance dashboard, updated quarterly, has shown DOB permit-processing times averaging over 30 days for certain residential job types in 2025 — well above the 15-day target the agency set in its fiscal year 2024 plan. City officials have attributed part of that lag to system slowdowns, not just staffing. Deduplication is one piece of a broader IT modernization effort the Adams administration has said is central to its housing agenda, which includes a goal of permitting 100,000 new units over the next five years.

Storage costs are also a real line item. City agencies collectively pay for cloud and on-premises storage under contracts managed through the Mayor's Office of Contract Services. While specific dollar figures for storage expenditure are not publicly broken out by agency in the current fiscal year budget, technology officials have publicly noted that redundant data inflation is a measurable cost driver.

The World Cup connection is not incidental. Venue-adjacent facilities at Metlife Stadium in East Rutherford rely on coordinated permitting and event-licensing records that flow through city and state systems. Getting those digital pipelines clean before the tournament's New York-area matches arrive later this month has added urgency to what might otherwise be a low-visibility IT housekeeping task.

For property owners and contractors watching this closely, the immediate practical advice is straightforward: if you have outstanding permit applications filed before July 2025, log into the DOB BIS portal and verify your documents are displaying correctly. The Office of Technology and Innovation has set up a dedicated help line — reachable through NYC311 — for users who believe a deduplication flag has incorrectly touched their active filings. The full audit is expected to wrap by September 30, ahead of the next fiscal year IT review cycle.