News and Notes

For various convoluted reasons, the IEM maintains a downloadable archive of many National Weather Service Watch, Warning, and Advisories. This archive is very unique to the known public Internet with no known freely available alternatives, which I suppose is a prideful statement, but also a scary one as there is no known "answer key" to compare IEM processing with. Anyway, this archive intends to be a complete archive of NWS products implementing VTEC (Valid Time Event Code), which was another in the line of convoluted things attempting to make parsing NWS ASCII text products sane. The VTEC implementation started in 2005, but many moons ago the IEM did some special processing attempting to create VTEC events prior to 2005 for a limited set of warning types.

Enough backstory, this news item denotes an addition of fake-VTEC warning types covering certain marine products in Alaska. Again with the convoluted term, but for such reasons Alaska forecast offices do not implement VTEC for things like Gale Warnings and Small Craft Advisories. Instead, they have a zone based forecast product that implements these events as headlines within the forecast. An example product is CWFNSB with an ASCII text like so:

PKZ811-210330-
Cape Thompson to Cape Beaufort out to 15 NM-
402 AM AKST Wed Nov 20 2024

...STORM WARNING IN EFFECT THROUGH LATE THIS MORNING...
...HEAVY FREEZING SPRAY WARNING IN EFFECT FROM LATE THIS
AFTERNOON THROUGH EARLY THURSDAY MORNING...

So in this case, we have a Storm Warning (SR.W) and a Heavy Freezing Spray Warning (UP.W) for Marine zone PKZ811.

The IEM now has processing that attempts to create these "fake-VTEC" events in realtime and has back-filled this processing to the start of 2005. FAQ Item 7 for the VTEC dataset has a bit more color on this. The biggest caveat is that the temporal handling of these events is very crude. Absent of any NWS documentation on how these events are to be timed, the least painful coding life choices were made with the pyWWA ingestor written.

As always, I am happy to field any questions about this and hope the community finds this addition useful.

The IEM maintains an archive of parsed National Weather Service Watch, Warning, and Advisory (WWA) data. This parsing is based on the NWS Valid Time Event Code (VTEC) encoding that is found within their raw text products. This encoding looks like so:

WFUS53 KDMX 282344
TORDMX
IAC125-181-290015-
/O.NEW.KDMX.TO.W.0026.170628T2344Z-170629T0015Z/

BULLETIN - EAS ACTIVATION REQUESTED
Tornado Warning
National Weather Service Des Moines IA
644 PM CDT WED JUN 28 2017

This encoding allows computer programs to better understand and process text products that contain lots of free-form content. This encoding is also used to drive various IEM web applications that take URL parameters to describe a warning product, for example.
https://mesonet.agron.iastate.edu/vtec/#2017-O-NEW-KDMX-TO-W-0026

A part of the VTEC encoding is the "Event ID", which is a sequential number used throughout a given year to order events as they are issued. In general, these Event IDs should be unique for a given year and most all of them start with the number 1. Sadly, a database issue at the NWS is now causing NWS Forecast Offices to reuse "Event IDs". This reuse means that the "Event IDs" are no longer unique for 2017. The impact is that URLs like the one above are now ambiguous and the presented data may contain the first event earlier this year, the last event, or some corrupted combination of the two.

After discussing the matter with the NWS management, they indicated they would not be fixing this situation before the end of the year. Of course, when 1 January 2018 comes, all the "Event IDs" reset back to 1 anyway. Sadly, this situation could again happen next year :(

I shutter to think of the amount of code necessary to better account for this situation and keep the website / web services straight when this problem happens. So I likely won't be attempting to make this work :(

Updated 22 Jan 2018: I decided to update the database and attempt to fix the duplicated Event IDs used. This change means that the eventid metadata will mismatch against the raw information found in the text, but there are no winning solutions in this situation. The update changed the reused Event ID, so if an office issued number 2 again and it should have been 10, the database now contains 10.