Some TV stations delete public files from the FCC system following the minimum retention period (~2yrs).

The deleted file ids can be fetched from the “File History” endpoint on the FCC API by looking for files with the status of “delete” (https://publicfiles.fcc.gov/api/manager/file/history.json?startDate=2017-01-01&status=delete)

These files, although deleted, can be fetched by using the file_manager_id value from FCC’s File Manager: https://files.fcc.gov/download/{file_manager_id}.pdf

A pipeline to fetch these files and signal for deletes would be helpful for reporting purposes. Additionally, we need to identify a place to store these files for long term (e.g. Archive.org)

TV Stations often produce successive versions of the same invoice/receipt when the candidate updates their Ad-Buy.

A good number of these can be identified by looking for keywords such as “REV” in the file name through the search API. A more thorough approach for the same is to look for the Contract or Order ID number inside the PDF to view when successive filings have the same ID suggesting that it could be a revision

Based on this detection, we could offer notifications to alert when certain candidates/races have revised orders

Per Derek Willis' suggestion.

Develop minimal wrapper to work with FCC's API

Schema available here: https://publicfiles.fcc.gov/developer/

By and large there seem to be a few (~4-5) types of layouts that stations upload their document in - if we can classify these documents into groups based on the layout - it might make it easier to implement layout specific parsers.

The file path from the public files in the FCC API are largely structured in the following format:

{StationCallsign}/political-files/{Year}/{OfficeLevel}/{OfficeName}/{Candidate/PAC Name}

For instance, the following is the file path for Bernie Sanders filings for WBTV:

wbtv/political-files/2016/federal/president/bernie-sanders/

However, the names aren't always consistent - some entities use PAC Names (e.g. Hillary For America), some use last names (Clinton) and some use full names (Bernie Sanders) - this format is largely station specific. While its easy to write rule based lookups for federal candidates by matching names with data from GovTrack or Propublica - this problem gets more challenging when handling local candidates.

An approach to canonicalize these candidate names across TV Stations and States would greatly help streamline how data is aggregated later.

WIP