The last name initial used to build the bibcode does not use the simple normalized code implemented here:

Dump a sample of ADS events in JSON format

Implement unit tests for forwarded messages and consider expanding other tests too.

Software source/concept doi records do not have metadata independent of the most recent version, even though they have a unique doi. As the metadata updates, We keep the bibcode year in these records frozen. This ultimately results in the bibcode year and publication date being out of sync, sometimes by several years. To address this:

1. Modify maintenance_metadata to retain the publication year for concept dois
2. Write a new maintenance task that extracts the publication year from the citation_target_version table and updates the current metadata accordingly.

We need to implement a function to guess first and last names with different possible formats (e.g., "Firstname Lastname", "Lastname, Firstname") and multiple last names or middle names. Then this should be applied to address this:

Unit tests do not currently cover the file import portion of CitationCapture. It would be good to add some tests for both DeltaComputation and ReaderImport when it is merged into the mainline branch so that way we can confirm the behavior of the two imports are consistent.

ADS Classic can output a list of citation coming from the same citing bibcode but different source. Sometimes, for one source the resolver did not work so it provides and empty cited bibcode. We must ensure that when we discard duplicates, we keep those that are resolved.

Store results in a database as citation changes are processed.

In order to remove duplicates and invalid citing bibcodes from a list of citations, two options are available:

1. Use metadata from ADS classic at the time of citation ingest / processing
2. Use metadata from our API when software records are serialized and sent to the pipeline

If we go with option 1, we would use the mapping between all known bibcodes and canonical bibcodes, which is found in file /proj/ads/abstracts/config/bibcodes.list.all2can
The first column contains all the known bibcodes in ADS, and the second column contains their canonical version. Given a set of citing bibcodes, one would look each one up in this list. If an entry exist, then replace the bibcode with its canonical version. If an entry does not exist, the bibcode is invalid and should be ignored. At the end of the process, the resulting list of bibcodes is uniqued.

If we go with option 2, we would use an API call to bigquery and submit our list of citing bibcodes at once. The search engine will resolve each bibcode in its canonical form and return a list of unique records. The bibcodes of these records are the ones to use as our citation list.

Since bibcodes can be remapped at any point, one would periodically need to perform this operation and update the relevant records on a regular basis (at least weekly). ADS classic does this via updates to its nonbib pipeline, so a similar model could be used by the citation capture pipeline.

Remove ORCID data from affiliations and store it in a useful place.

Direct inges to master pipeline without going through ADS Classic.

Use a random TOKEN for the asclepias webhook prototype and update eb-deploy PR accordingly.

Pubdate/year both are filled using the same source:

For instance, 2016zndo.....55143A in solr has year":"2016" and "pubdate":"2016-06-21". But date is filled using the current timestamp:

This makes that a query like year:2016 NOT pubdate:[2016-01 TO 2016-12] returns results, because solr is using date and not pubdate fields when the user queries using pubdate:.