New York's Agencies Are Drowning in Duplicate Images. Here's What Officials and Experts Say Needs to Change.

New York City's sprawling network of municipal agencies is sitting on a quiet, expensive mess: thousands of duplicate image files clogging servers, slowing down public-facing portals, and inflating storage costs that ultimately land on taxpayers. The city's Department of Information Technology and Telecommunications, known as DoITT, has flagged the problem in successive infrastructure reviews, and digital records managers say the scale of redundancy has grown substantially since the pandemic-era push to digitize everything from housing inspection photos to transit surveillance stills.

The timing matters. With the 2026 FIFA World Cup already drawing millions of additional visitors through New York — MetLife Stadium in East Rutherford is hosting eight matches, with fan zones anchored at Hudson Yards and along the West Side — city agencies are under pressure to keep public-facing digital infrastructure running cleanly. Sluggish permit portals and photo-heavy databases are not abstract inconveniences right now. They affect how quickly vendors get licensed, how fast journalists can pull public records, and how reliably tourist-facing city apps load under heavy traffic.

Officials at the Mayor's Office of Data Analytics have described the duplicate-image problem as a downstream consequence of agencies operating in silos. The MTA's communications department, for instance, maintains separate photo repositories for its New York City Transit, Long Island Rail Road, and Metro-North divisions, with no unified deduplication protocol applied across all three. The Department of Buildings, which processes hundreds of permit applications daily through its NYC Development Hub portal on Broadway, has similarly allowed redundant site-inspection photographs to accumulate across its eFiling system for years.

What the Experts Are Recommending

Digital preservation specialists at the New York Public Library's Digital Imaging Unit have long advocated for hash-based deduplication — a technical process that assigns each image a unique fingerprint, allowing systems to automatically identify and remove exact copies. The approach is standard practice at institutions including the Library of Congress and the Smithsonian. Applied at the city level, proponents argue it could cut storage overhead by a meaningful fraction without deleting any unique records.

Columbia University's School of Professional Studies runs a graduate program in Information and Knowledge Strategy whose faculty have consulted with city agencies on exactly this kind of records hygiene. Instructors there have pointed to the city's Open Data portal — hosted at data.cityofnewyork.us and updated daily — as a model of what structured, deduplicated public data can look like when agencies commit to standards. The argument is that images need the same discipline that tabular datasets have received over the past decade.

Storage is not cheap at municipal scale. Commercial cloud storage for government entities typically runs between $0.02 and $0.05 per gigabyte per month depending on the vendor contract tier. City agencies collectively hold petabytes of image data, and without deduplication, a meaningful share of that capacity is redundant. The city's fiscal year 2026 technology budget, approved by the City Council in June, allocated funds for cloud migration across multiple agencies — but critics on the Council's Technology Committee have questioned whether migration without deduplication simply moves the problem to a more expensive address.

The Path Forward

The Department of Records and Information Services, headquartered at 31 Chambers Street in lower Manhattan, is the logical home for a city-wide deduplication standard, according to several records management professionals familiar with the agency's mandate. The department already oversees retention schedules and digitization guidelines. Extending its authority to include image-file standards would require either a mayoral executive order or a directive through the city's chief technology officer — neither of which has been issued as of this writing.

Advocates are urging the Adams administration to act before the next budget cycle locks in cloud-storage contracts that assume current, bloated file counts. For ordinary New Yorkers, the practical stakes show up in concrete ways: a slower NYC.gov, longer load times on the 311 app in Washington Heights or the Bronx, and permit delays that ripple outward to small contractors and homeowners. Agencies that get their image archives in order before the next contract renewal stand to save real money. Those that don't will keep paying for the same picture, over and over again.