U.S. climate data preservation crisis
Pangaea races to archive NOAA climate datasets as U.S. budget cuts trigger purge Plain Schwarz (Berlin Buzzwords, Haystack)TL;DW
- NOAA began decommissioning 100+ years of climate data after Trump administration eliminated personnel and funding for climate science, making datasets inaccessible and halting new data collection.
- NSF grant funding dropped 20-30% in 2025; executive orders banned words like 'diversity' and 'inclusion' from grant applications, forcing scientists to self-censor research proposals.
- Pangia data center rescued U.S. climate datasets by transforming them into FAIR format (findable, accessible, interoperable, reusable) with proper metadata and persistent identifiers.
- U.S. data centers lack basic infrastructure like HTTP content negotiation and fair data access; European scientists must manually reconstruct missing metadata and convert arbitrary proprietary formats.
- International meeting in Berlin established strategy: multiformat metadata, integrated data storage across borders, portable packaging, and global coordination to preserve datasets outside U.S. infrastructure risk.
- Scientists historically withheld raw data to prevent competitors from extracting value; now NSF and other agencies mandate open, accessible data publication as condition of funding.
- Preserving historical climate data requires semantic layers and terminology annotations beyond raw measurements to track how biodiversity and measurement methodologies evolved over time.
- Data rescue prioritization critical: some datasets exist only on tape archives requiring weeks or months to retrieve, so preservation must focus on scientifically important datasets first.
TL;DW
- NOAA began decommissioning 100+ years of climate data after Trump administration eliminated personnel and funding for climate science, making datasets inaccessible and halting new data collection.
- NSF grant funding dropped 20-30% in 2025; executive orders banned words like 'diversity' and 'inclusion' from grant applications, forcing scientists to self-censor research proposals.
- Pangia data center rescued U.S. climate datasets by transforming them into FAIR format (findable, accessible, interoperable, reusable) with proper metadata and persistent identifiers.
- U.S. data centers lack basic infrastructure like HTTP content negotiation and fair data access; European scientists must manually reconstruct missing metadata and convert arbitrary proprietary formats.
- International meeting in Berlin established strategy: multiformat metadata, integrated data storage across borders, portable packaging, and global coordination to preserve datasets outside U.S. infrastructure risk.
- Scientists historically withheld raw data to prevent competitors from extracting value; now NSF and other agencies mandate open, accessible data publication as condition of funding.
- Preserving historical climate data requires semantic layers and terminology annotations beyond raw measurements to track how biodiversity and measurement methodologies evolved over time.
- Data rescue prioritization critical: some datasets exist only on tape archives requiring weeks or months to retrieve, so preservation must focus on scientifically important datasets first.
Uwe Schindler (Pangaea) details how NSF cuts of 20–30% and executive orders restricting terminology have pushed NOAA to decommission century-scale atmospheric and oceanographic databases. European centers are ingesting rescued datasets, but recovery requires rebuilding sparse metadata, converting tape archives, and enforcing FAIR compliance across incompatible formats.