Data from NLP API missing in the unofficial data snapshot taken at 29/03/2020, 16:30 about api-covid19-in HOT 9 CLOSED

Found the issue. Was a bug on my side

from api-covid19-in.

Hey,
Just checked the logs, they seem to be working from 16:30 to 17:30. Logs from 16:30 to 17:30

The expected input data for the NLP API looks something like this

{
    "patients": [
        {
            "patientId": "1",
            "notes": "Indian Student Travelled from Italy, Family Member of P13 Friend of P12"
        }
    ]
}

Looking at Unofficial Patient Tracing API

{
    "success": true,
    "data": {
        "source": "covid19india.org",
        "lastRefreshed": "2020-03-30T00:54:06.269Z",
        "summary": {
            "total": 1139
        },
        "rawPatientData": [
            {
                "patientId": 1,
                "reportedOn": "30/01/2020",
                "onsetEstimate": "",
                "ageEstimate": "20",
                "gender": "female",
                "city": "Thrissur",
                "district": "Thrissur",
                "state": "Kerala",
                "status": "Recovered",
                "notes": "Travelled from Wuhan",
                "contractedFrom": "",
                "sources": [
                    "https://twitter.com/vijayanpinarayi/status/1222819465143832577"
                ]
            }
        ]
    }
}

the data is in a format where the key rawPatientData is used instead of patients, so I am guessing the API is having some trouble there. To confirm this, the API must be responding with when the key is not found.

{
    "error": "Not the correct request format!"
}

from api-covid19-in.

Likely. @NirantK, can you comment here?

from api-covid19-in.

A small input:

I wrote a small Python snipped that is using data from the Unofficial Patient Tracing API , filtering it for patient ID and notes only and running it through the NLP Data API. I am able to get responses that has valid travel data. If it helps, I am also sending my data in batches of 500.

``

from api-covid19-in.

cc @meghanabhange please check logs for the shared timestamp and confirm if we had an outage at the time?

from api-covid19-in.

@meghanabhange rawPatientData is the response from the rootnet API, but the query to the NLP API is still sent as patients.

from api-covid19-in.

@meghanabhange you can confirm my assertion here: https://github.com/amodm/api-covid19-in/blob/master/data/unofficial/index.js#L60

from api-covid19-in.

Thanks for the confirmation @amodm.

Confirming via logs, that we are receiving requests with the patients key. We are also returning a response as shared:

{
    "patients": [
        {
            "1": {
                "nationality": [],
                "place_attributes": [
                    {
                        "is_foreign": true,
                        "place": "Nepal"
                    }
                ],
                "relationship": [],
                "travel": [
                    "Nepal"
                ]
            }
        }
    ]
}

How can I repro this?

Would it help if we could look at the response/error you're getting from the NLP endpoint via your logs? Something a little more descriptive than throw e.g. 500 or a parsing error (possible here)

from api-covid19-in.

I've enabled the error logging now (was disabled earlier). Will revert.

from api-covid19-in.