Index "tombstone" meta_data column in Elasticsearch about cccatalog-api HOT 1 CLOSED

cc-archive commented on July 28, 2024

Index "tombstone" meta_data column in Elasticsearch

from cccatalog-api.

Comments (1)

aldenstpage commented on July 28, 2024

@pa-w I set up the new Elasticsearch replicator to migrate this column over.

curl -i 'localhost:9200/image/_search?q=pg_id:9619372&pretty=true'

          ...
          ...
          "foreign_landing_url": "http:\/\/www.metmuseum.org\/art\/collection\/search\/71211",
          "removed_from_source": false,
          "meta_data": {
            "date": "18th century",
            "image": "http:\/\/images.metmuseum.org\/CRDImages\/as\/web-large\/DP15698.jpg",
            "title": "Piece",
            "medium": "Silk \/ Compound weave",
            "period": "Edo period (1615\\u20131868)",
            "source": "http:\/\/www.metmuseum.org\/art\/collection\/search\/71211",
            "culture": "Japan",
            "license": "\/publicdomain\/zero\/1.0\/",
            "date_era": "A.D. 1600\\u20131800",
            "dimensions": "6 1\/2 \\xd7 1 1\/8 in. (16.5 \\xd7 2.9 cm)",
            "credit_line": "Gift of Kihei Hattori, in memory of his father, Kihei Hattori, 1920",
            "description": "",
            "classification": "Textiles-Woven",
            "accession_number": "20.148.222",
            "geographic_location": "Asia,Japan",
            "object_type_material": "Silk,Textiles,Woven"
           }
         ...
         ...

This should let you query documents by century like you wanted, with the caveat that there is no standardization on the date format (e.g. this example has "date": "18th century" but others instead have ca.1715 or late 13th\\u201314th century). The best way to address that is probably to clean it up and standardize it somewhere in the ETL pipeline before it ends up in Postgres.

I still need to stand up the replicator in our staging environment, so you won't see these changes quite yet.

from cccatalog-api.

Recommend Projects