The IEM attempts to archive a lot of data each day. The goal being to make accessible long term archives available for both funded research projects that support the IEM and also for anybody else to freely use. This archiving activity takes on a number of different forms with a "flat file" directory tree being the subject of today's Daily Feature. This directory tree is organized by UTC date and subsequent sub-folders. The predictable and stable URLs allow for automated downloads and scraping. The featured chart presents the daily total of archived file size, the accumulated storage, and some annotated dates explaining why the archive sizes increased at that time. There are some practical constraints to archival capacity, so the scope of archival has not changed much over the past five years or so. There is an associated ISU project called MTArchive with a similar directory structure, but archives more meteorological specific datasets. MTArchive also has a different funding model, so thus why the service is separate.

Voting:
Good = 22
Bad = 0

Tags:   iem   archive