Describe the bug
I have been working with text data and using Amazon comprehend NLP plugin for various purposes in daily work at client environment. As a prerequisite step, I often deal with Language detection plugin recipe to understand which language the text data falls in, followed by language translation piece. One of our workflows seeks incremental records every week and passes on to subsequent workflows. It is imperative that there is a concrete dependency among workflows which runs one after another with no error at prior steps.
However, it has transpired that we hardly get any incremental nowadays in text data whereas other workflows are significant to run every week. Thus, it is expected to have no records in the text dataset and to let other workflows run. Being input dataset empty to language detection plugin recipe, the recipe throws an error (precisely Python error).
My request is to add an empty dataset as output with columns that come up when the input dataset has records. Instead of Python error, it is better to have an empty one, I think. In this way, we can prevent our entire data pipeline from failing. Kindly look into this when you have a moment.
To Reproduce
Steps to reproduce the behavior:
- Please use Amazon Comprehend NLP's language detection recipe with an empty dataset.
- Kindly check error.
Expected behavior
An empty dataset with input dataset's schema plus columns returned by API.
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.