googleapis / nodejs-document-ai Goto Github PK

View Code? Open in Web Editor NEW

28.0 45.0 19.0 7.38 MB

This repository is deprecated. All of its content and history has been moved to googleapis/google-cloud-node.

License: Apache License 2.0

nodejs-document-ai's Introduction

THIS REPOSITORY IS DEPRECATED. ALL OF ITS CONTENT AND HISTORY HAS BEEN MOVED TO GOOGLE-CLOUD-NODE

Document AI: Node.js Client

Document AI client for Node.js

A comprehensive list of changes in each version may be found in the CHANGELOG.

Read more about the client libraries for Cloud APIs, including the older Google APIs Client Libraries, in Client Libraries Explained.

Table of contents:

Quickstart
Samples
Versioning
Contributing
License

Quickstart

Before you begin

Select or create a Cloud Platform project.
Enable billing for your project.
Enable the Document AI API.
Set up authentication with a service account so you can access the API from your local workstation.

Installing the client library

npm install @google-cloud/documentai

Using the client library

/**
 * TODO(developer): Uncomment these variables before running the sample.
 */
// const projectId = 'YOUR_PROJECT_ID';
// const location = 'YOUR_PROJECT_LOCATION'; // Format is 'us' or 'eu'
// const processorId = 'YOUR_PROCESSOR_ID'; // Create processor in Cloud Console
// const filePath = '/path/to/local/pdf';

const {DocumentProcessorServiceClient} =
  require('@google-cloud/documentai').v1;

// Instantiates a client
// apiEndpoint regions available: eu-documentai.googleapis.com, us-documentai.googleapis.com (Required if using eu based processor)
// const client = new DocumentProcessorServiceClient({apiEndpoint: 'eu-documentai.googleapis.com'});
const client = new DocumentProcessorServiceClient();

async function quickstart() {
  // The full resource name of the processor, e.g.:
  // projects/project-id/locations/location/processor/processor-id
  // You must create new processors in the Cloud Console first
  const name = `projects/${projectId}/locations/${location}/processors/${processorId}`;

  // Read the file into memory.
  const fs = require('fs').promises;
  const imageFile = await fs.readFile(filePath);

  // Convert the image data to a Buffer and base64 encode it.
  const encodedImage = Buffer.from(imageFile).toString('base64');

  const request = {
    name,
    rawDocument: {
      content: encodedImage,
      mimeType: 'application/pdf',
    },
  };

  // Recognizes text entities in the PDF document
  const [result] = await client.processDocument(request);
  const {document} = result;

  // Get all of the document text as one big string
  const {text} = document;

  // Extract shards from the text field
  const getText = textAnchor => {
    if (!textAnchor.textSegments || textAnchor.textSegments.length === 0) {
      return '';
    }

    // First shard in document doesn't have startIndex property
    const startIndex = textAnchor.textSegments[0].startIndex || 0;
    const endIndex = textAnchor.textSegments[0].endIndex;

    return text.substring(startIndex, endIndex);
  };

  // Read the text recognition output from the processor
  console.log('The document contains the following paragraphs:');
  const [page1] = document.pages;
  const {paragraphs} = page1;

  for (const paragraph of paragraphs) {
    const paragraphText = getText(paragraph.layout.textAnchor);
    console.log(`Paragraph text:
${paragraphText}`);
  }
}

Samples

Samples are in the samples/ directory. Each sample's README.md has instructions for running its sample.

Sample	Source Code	Try it
Batch-process-document	source code
Process-document-form	source code
Process-document-ocr	source code
Process-document-quality	source code
Process-document-specialized	source code
Process-document-splitter	source code
Process-document	source code
Quickstart	source code

The Document AI Node.js Client API Reference documentation also contains samples.

Supported Node.js Versions

Our client libraries follow the Node.js release schedule. Libraries are compatible with all current active and maintenance versions of Node.js. If you are using an end-of-life version of Node.js, we recommend that you update as soon as possible to an actively supported LTS version.

Google's client libraries support legacy versions of Node.js runtimes on a best-efforts basis with the following warnings:

Legacy versions are not tested in continuous integration.
Some security patches and features cannot be backported.
Dependencies cannot be kept up-to-date.

Client libraries targeting some end-of-life versions of Node.js are available, and can be installed through npm dist-tags. The dist-tags follow the naming convention legacy-(version). For example, npm install @google-cloud/documentai@legacy-8 installs client libraries for versions compatible with Node.js 8.

Versioning

This library follows Semantic Versioning.

This library is considered to be stable. The code surface will not change in backwards-incompatible ways unless absolutely necessary (e.g. because of critical security issues) or with an extensive deprecation period. Issues and requests against stable libraries are addressed with the highest priority.

More Information: Google Cloud Platform Launch Stages

Contributing

Contributions welcome! See the Contributing Guide.

Please note that this README.md, the samples/README.md, and a variety of configuration files in this repository (including .nycrc and tsconfig.json) are generated from a central template. To edit one of these files, make an edit to its templates in directory.

License

Apache Version 2.0

See LICENSE

nodejs-document-ai's People

Contributors

Stargazers

Watchers

Forkers

renovate-bot chingor13 memodrnn bigbri64 tindoantrong asrivas fdescamps mtannerfors tryweirder igor-mardari-ft galz10 seeergio16 isabella232 wafabenyettou allymao85 ssumit-cs andreistelmach alokkumar8 kilicay

nodejs-document-ai's Issues

Process form document: should run document (process form) (v1) failed

This test failed!

To configure my behavior, see the Flaky Bot documentation.

If I'm commenting on this issue too often, add the flakybot: quiet label and
I will stop commenting.

commit: b465fd2
buildURL: Build Status, Sponge
status: failed

Test output

Command failed: node ./process-document-form.js long-door-651 us 8f1123c1b125e0b7 /workspace/samples/resources/invoice.pdf Error: 16 UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project. at Object.callErrorFromStatus (/workspace/node_modules/@grpc/grpc-js/build/src/call.js:31:19) at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:190:52) at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:365:141) at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:328:181) at /workspace/node_modules/@grpc/grpc-js/build/src/call-stream.js:188:78 at processTicksAndRejections (internal/process/task_queues.js:79:11) for call at at ServiceClientImpl.makeUnaryRequest (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:160:30) at ServiceClientImpl. (/workspace/node_modules/@grpc/grpc-js/build/src/make-client.js:105:19) at /workspace/build/src/v1beta3/document_processor_service_client.js:296:29 at /workspace/node_modules/google-gax/build/src/normalCalls/timeout.js:44:16 at repeat (/workspace/node_modules/google-gax/build/src/normalCalls/retries.js:80:25) at /workspace/node_modules/google-gax/build/src/normalCalls/retries.js:118:13 at OngoingCallPromise.call (/workspace/node_modules/google-gax/build/src/call.js:67:27) at NormalApiCaller.call (/workspace/node_modules/google-gax/build/src/normalCalls/normalApiCaller.js:34:19) at /workspace/node_modules/google-gax/build/src/createApiCall.js:84:30 { code: 16, details: 'Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.', metadata: Metadata { internalRepr: Map { 'grpc-server-stats-bin' => [Array] }, options: {} }, note: 'Exception occurred in retry method that was not classified as transient' }

Error: Command failed: node ./process-document-form.js long-door-651 us 8f1123c1b125e0b7 /workspace/samples/resources/invoice.pdf
Error: 16 UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
at Object.callErrorFromStatus (/workspace/node_modules/@grpc/grpc-js/build/src/call.js:31:19)
at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:190:52)
at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:365:141)
at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:328:181)
at /workspace/node_modules/@grpc/grpc-js/build/src/call-stream.js:188:78
at processTicksAndRejections (internal/process/task_queues.js:79:11)
for call at
at ServiceClientImpl.makeUnaryRequest (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:160:30)
at ServiceClientImpl. (/workspace/node_modules/@grpc/grpc-js/build/src/make-client.js:105:19)
at /workspace/build/src/v1beta3/document_processor_service_client.js:296:29
at /workspace/node_modules/google-gax/build/src/normalCalls/timeout.js:44:16
at repeat (/workspace/node_modules/google-gax/build/src/normalCalls/retries.js:80:25)
at /workspace/node_modules/google-gax/build/src/normalCalls/retries.js:118:13
at OngoingCallPromise.call (/workspace/node_modules/google-gax/build/src/call.js:67:27)
at NormalApiCaller.call (/workspace/node_modules/google-gax/build/src/normalCalls/normalApiCaller.js:34:19)
at /workspace/node_modules/google-gax/build/src/createApiCall.js:84:30 {
code: 16,
details: 'Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.',
metadata: Metadata {
internalRepr: Map { 'grpc-server-stats-bin' => [Array] },
options: {}
},
note: 'Exception occurred in retry method that was not classified as transient'
}

at checkExecSyncError (child_process.js:635:11)
at Object.execSync (child_process.js:671:15)
at execSync (test/process-document-form.test.js:28:28)
at Context.<anonymous> (test/process-document-form.test.js:43:20)
at processImmediate (internal/timers.js:461:21)</pre></details>

Process splitter document: should run document (splitter) (v1beta3) failed

This test failed!

To configure my behavior, see the Flaky Bot documentation.

If I'm commenting on this issue too often, add the flakybot: quiet label and
I will stop commenting.

commit: b465fd2
buildURL: Build Status, Sponge
status: failed

Test output

Command failed: node ./process-document-splitter.js long-door-651 us 8f447646e4ec6fa2 /workspace/samples/resources/multi_document.pdf Error: 16 UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project. at Object.callErrorFromStatus (/workspace/node_modules/@grpc/grpc-js/build/src/call.js:31:19) at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:190:52) at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:365:141) at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:328:181) at /workspace/node_modules/@grpc/grpc-js/build/src/call-stream.js:188:78 at processTicksAndRejections (internal/process/task_queues.js:79:11) for call at at ServiceClientImpl.makeUnaryRequest (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:160:30) at ServiceClientImpl. (/workspace/node_modules/@grpc/grpc-js/build/src/make-client.js:105:19) at /workspace/build/src/v1beta3/document_processor_service_client.js:296:29 at /workspace/node_modules/google-gax/build/src/normalCalls/timeout.js:44:16 at repeat (/workspace/node_modules/google-gax/build/src/normalCalls/retries.js:80:25) at /workspace/node_modules/google-gax/build/src/normalCalls/retries.js:118:13 at OngoingCallPromise.call (/workspace/node_modules/google-gax/build/src/call.js:67:27) at NormalApiCaller.call (/workspace/node_modules/google-gax/build/src/normalCalls/normalApiCaller.js:34:19) at /workspace/node_modules/google-gax/build/src/createApiCall.js:84:30 { code: 16, details: 'Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.', metadata: Metadata { internalRepr: Map { 'grpc-server-stats-bin' => [Array] }, options: {} }, note: 'Exception occurred in retry method that was not classified as transient' }

Error: Command failed: node ./process-document-splitter.js long-door-651 us 8f447646e4ec6fa2 /workspace/samples/resources/multi_document.pdf
Error: 16 UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
at Object.callErrorFromStatus (/workspace/node_modules/@grpc/grpc-js/build/src/call.js:31:19)
at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:190:52)
at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:365:141)
at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:328:181)
at /workspace/node_modules/@grpc/grpc-js/build/src/call-stream.js:188:78
at processTicksAndRejections (internal/process/task_queues.js:79:11)
for call at
at ServiceClientImpl.makeUnaryRequest (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:160:30)
at ServiceClientImpl. (/workspace/node_modules/@grpc/grpc-js/build/src/make-client.js:105:19)
at /workspace/build/src/v1beta3/document_processor_service_client.js:296:29
at /workspace/node_modules/google-gax/build/src/normalCalls/timeout.js:44:16
at repeat (/workspace/node_modules/google-gax/build/src/normalCalls/retries.js:80:25)
at /workspace/node_modules/google-gax/build/src/normalCalls/retries.js:118:13
at OngoingCallPromise.call (/workspace/node_modules/google-gax/build/src/call.js:67:27)
at NormalApiCaller.call (/workspace/node_modules/google-gax/build/src/normalCalls/normalApiCaller.js:34:19)
at /workspace/node_modules/google-gax/build/src/createApiCall.js:84:30 {
code: 16,
details: 'Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.',
metadata: Metadata {
internalRepr: Map { 'grpc-server-stats-bin' => [Array] },
options: {}
},
note: 'Exception occurred in retry method that was not classified as transient'
}

at checkExecSyncError (child_process.js:635:11)
at Object.execSync (child_process.js:671:15)
at execSync (test/process-document-splitter.test.js:28:28)
at Context.<anonymous> (test/process-document-splitter.test.js:43:20)
at processImmediate (internal/timers.js:461:21)</pre></details>

📦 pack-n-play test: TypeScript code failed

This test failed!

To configure my behavior, see the Flaky Bot documentation.

If I'm commenting on this issue too often, add the flakybot: quiet label and
I will stop commenting.

commit: fa59273
buildURL: Build Status, Sponge
status: failed

Test output

Process 201 exited with code 2.
Error: Process 201 exited with code 2.
    at ChildProcess. (node_modules/pack-n-play/build/src/utils.js:48:22)
        -> /workspace/node_modules/pack-n-play/src/utils.ts:62:11
    at maybeClose (internal/child_process.js:1022:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:287:5)

(node:6877) UnhandledPromiseRejectionWarning: ReferenceError: processorId is not defined

Thanks for stopping by to let us know something could be better!

PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.

Is this a client library issue or a product issue?
Sample code
Did someone already solve this?
Not sure
Do you have a support contract?
Please create an issue in the support console to ensure a timely response.

If the support paths suggested above still do not result in a resolution, please provide the following details.

Environment details

OS: OS X
Node.js version: 12.19
npm version: 6.14.8
@google-cloud/documentai version: v1

Steps to reproduce

Run the quickstart or processDocument code

Lines:
https://github.com/googleapis/nodejs-document-ai/blob/master/samples/quickstart.js#L25
and
https://github.com/googleapis/nodejs-document-ai/blob/master/samples/process-document.v1beta3.js#L25

have
const processor = 'YOUR_PROCESSOR_ID'; // Create processor in Cloud Console

but the name variable further down expects 'processorId'
https://github.com/googleapis/nodejs-document-ai/blob/master/samples/process-document.v1beta3.js#L39
const name = projects/${projectId}/locations/${location}/processors/${processorId};

remove exceptions from linkinator.config.json

Document AI batch parse form: "after all" hook for "should parse the GCS invoice example as a form" failed

Note: #269 was also for this test, but it was closed more than 10 days ago. So, I didn't mark it flaky.

commit: b465fd2
buildURL: Build Status, Sponge
status: failed

Test output

invalid_grant: Invalid JWT Signature.
Error: invalid_grant: Invalid JWT Signature.
    at Gaxios._request (node_modules/gaxios/build/src/gaxios.js:130:23)
        -> /workspace/samples/node_modules/gaxios/src/gaxios.ts:158:15
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async GoogleToken.requestToken (node_modules/gtoken/build/src/index.js:238:23)
    at async GoogleToken.getTokenAsync (node_modules/gtoken/build/src/index.js:145:20)
    at async JWT.refreshTokenNoCache (node_modules/google-auth-library/build/src/auth/jwtclient.js:172:23)
    at async JWT.getRequestMetadataAsync (node_modules/google-auth-library/build/src/auth/oauth2client.js:284:17)
    at async JWT.getRequestHeaders (node_modules/google-auth-library/build/src/auth/oauth2client.js:247:26)
    at async GoogleAuth.authorizeRequest (node_modules/google-auth-library/build/src/auth/googleauth.js:677:25)
    at async Promise.all (index 1)
    at async prepareRequest (node_modules/@google-cloud/storage/build/src/nodejs-common/util.js:447:61)

Synthesis failed for nodejs-document-ai

Hello! Autosynth couldn't regenerate nodejs-document-ai. 💔

Here's the output from running synth.py:

b'ing dependencies.\n2020-06-05 04:26:55,883 synthtool [DEBUG] > Cloning googleapis.\nDEBUG:synthtool:Cloning googleapis.\n2020-06-05 04:26:55,884 synthtool [DEBUG] > Using precloned repo /home/kbuilder/.cache/synthtool/googleapis\nDEBUG:synthtool:Using precloned repo /home/kbuilder/.cache/synthtool/googleapis\n2020-06-05 04:26:55,888 synthtool [DEBUG] > Pulling Docker image: gapic-generator-typescript:latest\nDEBUG:synthtool:Pulling Docker image: gapic-generator-typescript:latest\nlatest: Pulling from gapic-images/gapic-generator-typescript\nDigest: sha256:c9bc12024eddcfb94501627ff5b3ea302370995e9a2c9cde6b3317375d7e7b66\nStatus: Image is up to date for gcr.io/gapic-images/gapic-generator-typescript:latest\n2020-06-05 04:26:56,792 synthtool [DEBUG] > Generating code for: google/cloud/documentai/v1beta2.\nDEBUG:synthtool:Generating code for: google/cloud/documentai/v1beta2.\n2020-06-05 04:26:57,713 synthtool [DEBUG] > Wrote metadata to synth.metadata.\nDEBUG:synthtool:Wrote metadata to synth.metadata.\nTraceback (most recent call last):\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main\n    "__main__", mod_spec)\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code\n    exec(code, run_globals)\n  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 102, in <module>\n    main()\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 829, in __call__\n    return self.main(*args, **kwargs)\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 782, in main\n    rv = self.invoke(ctx)\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke\n    return ctx.invoke(self.callback, **ctx.params)\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 610, in invoke\n    return callback(*args, **kwargs)\n  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 94, in main\n    spec.loader.exec_module(synth_module)  # type: ignore\n  File "<frozen importlib._bootstrap_external>", line 678, in exec_module\n  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed\n  File "/home/kbuilder/.cache/synthtool/nodejs-document-ai/synth.py", line 38, in <module>\n    version=version)\n  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_microgenerator.py", line 66, in typescript_library\n    return self._generate_code(service, version, "typescript", **kwargs)\n  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_microgenerator.py", line 195, in _generate_code\n    f"Code generation seemed to succeed, but {output_dir} is empty."\nRuntimeError: Code generation seemed to succeed, but /tmpfs/tmp/tmp0jcg2377 is empty.\n2020-06-05 04:26:57,752 autosynth [ERROR] > Synthesis failed\n2020-06-05 04:26:57,753 autosynth [DEBUG] > Running: git reset --hard HEAD\nHEAD is now at 0862794 build: update protos.js (#57)\n2020-06-05 04:26:57,758 autosynth [DEBUG] > Running: git checkout autosynth-self\nSwitched to branch \'autosynth-self\'\n2020-06-05 04:26:57,762 autosynth [ERROR] > Command \'[\'/tmpfs/src/github/synthtool/env/bin/python3\', \'-m\', \'synthtool\', \'--metadata\', \'synth.metadata\', \'synth.py\', \'--\']\' returned non-zero exit status 1.\n2020-06-05 04:26:57,939 autosynth [DEBUG] > Running: git checkout 0862794737fe7f2cafd8760f566fd255618c12ae\nNote: checking out \'0862794737fe7f2cafd8760f566fd255618c12ae\'.\n\nYou are in \'detached HEAD\' state. You can look around, make experimental\nchanges and commit them, and you can discard any commits you make in this\nstate without impacting any branches by performing another checkout.\n\nIf you want to create a new branch to retain commits you create, you may\ndo so (now or later) by using -b with the checkout command again. Example:\n\n  git checkout -b <new-branch-name>\n\nHEAD is now at 0862794 build: update protos.js (#57)\n2020-06-05 04:26:57,944 autosynth [DEBUG] > Running: git checkout d53a5b45c46920932dbe7d0a95e10d8b58933dae\nPrevious HEAD position was be74d3e build: do not fail builds on codecov errors (#528)\nHEAD is now at d53a5b4 docs: improve README (#600)\n2020-06-05 04:26:57,960 autosynth [DEBUG] > Running: git checkout cd804bab06e46dd1a4f16c32155fd3cddb931b52\nPrevious HEAD position was 2fc2caaa chore: enable gapic v2 and proto annotation for bigquery/storage/v1beta2 API.\nHEAD is now at cd804bab docs: cleaned docs for the Agents service and resource.\n2020-06-05 04:26:58,030 autosynth [DEBUG] > Running: git branch -f autosynth-123\n2020-06-05 04:26:58,034 autosynth [DEBUG] > Running: git checkout autosynth-123\nSwitched to branch \'autosynth-123\'\n2020-06-05 04:26:58,039 autosynth [INFO] > Running synthtool\n2020-06-05 04:26:58,039 autosynth [INFO] > [\'/tmpfs/src/github/synthtool/env/bin/python3\', \'-m\', \'synthtool\', \'--metadata\', \'synth.metadata\', \'synth.py\', \'--\']\n2020-06-05 04:26:58,041 autosynth [DEBUG] > Running: /tmpfs/src/github/synthtool/env/bin/python3 -m synthtool --metadata synth.metadata synth.py --\n2020-06-05 04:26:58,245 synthtool [DEBUG] > Executing /home/kbuilder/.cache/synthtool/nodejs-document-ai/synth.py.\nOn branch autosynth-123\nnothing to commit, working tree clean\n2020-06-05 04:26:58,375 synthtool [DEBUG] > Ensuring dependencies.\nDEBUG:synthtool:Ensuring dependencies.\n2020-06-05 04:26:58,380 synthtool [DEBUG] > Cloning googleapis.\nDEBUG:synthtool:Cloning googleapis.\n2020-06-05 04:26:58,381 synthtool [DEBUG] > Using precloned repo /home/kbuilder/.cache/synthtool/googleapis\nDEBUG:synthtool:Using precloned repo /home/kbuilder/.cache/synthtool/googleapis\n2020-06-05 04:26:58,385 synthtool [DEBUG] > Pulling Docker image: gapic-generator-typescript:latest\nDEBUG:synthtool:Pulling Docker image: gapic-generator-typescript:latest\nlatest: Pulling from gapic-images/gapic-generator-typescript\nDigest: sha256:c9bc12024eddcfb94501627ff5b3ea302370995e9a2c9cde6b3317375d7e7b66\nStatus: Image is up to date for gcr.io/gapic-images/gapic-generator-typescript:latest\n2020-06-05 04:26:59,250 synthtool [DEBUG] > Generating code for: google/cloud/documentai/v1beta2.\nDEBUG:synthtool:Generating code for: google/cloud/documentai/v1beta2.\n2020-06-05 04:27:00,066 synthtool [DEBUG] > Wrote metadata to synth.metadata.\nDEBUG:synthtool:Wrote metadata to synth.metadata.\nTraceback (most recent call last):\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main\n    "__main__", mod_spec)\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code\n    exec(code, run_globals)\n  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 102, in <module>\n    main()\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 829, in __call__\n    return self.main(*args, **kwargs)\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 782, in main\n    rv = self.invoke(ctx)\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke\n    return ctx.invoke(self.callback, **ctx.params)\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 610, in invoke\n    return callback(*args, **kwargs)\n  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 94, in main\n    spec.loader.exec_module(synth_module)  # type: ignore\n  File "<frozen importlib._bootstrap_external>", line 678, in exec_module\n  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed\n  File "/home/kbuilder/.cache/synthtool/nodejs-document-ai/synth.py", line 38, in <module>\n    version=version)\n  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_microgenerator.py", line 66, in typescript_library\n    return self._generate_code(service, version, "typescript", **kwargs)\n  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_microgenerator.py", line 195, in _generate_code\n    f"Code generation seemed to succeed, but {output_dir} is empty."\nRuntimeError: Code generation seemed to succeed, but /tmpfs/tmp/tmpd4yv7sk4 is empty.\n2020-06-05 04:27:00,107 autosynth [ERROR] > Synthesis failed\n2020-06-05 04:27:00,107 autosynth [DEBUG] > Running: git reset --hard HEAD\nHEAD is now at 0862794 build: update protos.js (#57)\n2020-06-05 04:27:00,113 autosynth [DEBUG] > Running: git checkout autosynth\nSwitched to branch \'autosynth\'\n2020-06-05 04:27:00,119 autosynth [DEBUG] > Running: git clean -fdx\nRemoving __pycache__/\nTraceback (most recent call last):\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main\n    "__main__", mod_spec)\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code\n    exec(code, run_globals)\n  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 615, in <module>\n    main()\n  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 476, in main\n    return _inner_main(temp_dir)\n  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 595, in _inner_main\n    commit_count = synthesize_loop(x, multiple_prs, change_pusher, synthesizer)\n  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 371, in synthesize_loop\n    synthesize_inner_loop(toolbox, synthesizer)\n  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 381, in synthesize_inner_loop\n    synthesizer, len(toolbox.versions) - 1\n  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 266, in synthesize_version_in_new_branch\n    synthesizer.synthesize(synth_log_path, self.environ)\n  File "/tmpfs/src/github/synthtool/autosynth/synthesizer.py", line 119, in synthesize\n    synth_proc.check_returncode()  # Raise an exception.\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 389, in check_returncode\n    self.stderr)\nsubprocess.CalledProcessError: Command \'[\'/tmpfs/src/github/synthtool/env/bin/python3\', \'-m\', \'synthtool\', \'--metadata\', \'synth.metadata\', \'synth.py\', \'--\']\' returned non-zero exit status 1.\n'

Google internal developers can see the full log here.

Process specialized document: should run document (specialized) (v1beta3) failed

This test failed!

To configure my behavior, see the Flaky Bot documentation.

If I'm commenting on this issue too often, add the flakybot: quiet label and
I will stop commenting.

commit: 34df222
buildURL: Build Status, Sponge
status: failed

Test output

Command failed: node ./process-document-specialized.js long-door-651 us bb340a5e47a6c1e /workspace/samples/resources/us_driver_license.pdf
TypeError: Cannot read property 'content' of null
    at processDocument (/workspace/samples/process-document-specialized.js:77:43)
    at async main (/workspace/samples/process-document-specialized.js:88:3)
Error: Command failed: node ./process-document-specialized.js long-door-651 us bb340a5e47a6c1e /workspace/samples/resources/us_driver_license.pdf

TypeError: Cannot read property 'content' of null

at processDocument (process-document-specialized.js:77:43)

at async main (process-document-specialized.js:88:3)
at checkExecSyncError (child_process.js:635:11)
at Object.execSync (child_process.js:671:15)
at execSync (test/process-document-specialized.test.js:28:28)
at Context.<anonymous> (test/process-document-specialized.test.js:43:20)
at processImmediate (internal/timers.js:461:21)</pre></details>

Document AI batch parse table: should parse the GCS invoice example as as table failed

This test failed!

To configure my behavior, see the Build Cop Bot documentation.

If I'm commenting on this issue too often, add the buildcop: quiet label and
I will stop commenting.

commit: 483df36
buildURL: Build Status, Sponge
status: failed

Test output

expected 'Document processing complete.\nFetching results ...\nFetched file #1:\nResults from first table processed:\n' to match /First detected language:/
AssertionError: expected 'Document processing complete.\nFetching results ...\nFetched file #1:\nResults from first table processed:\n' to match /First detected language:/
    at Context. (test/batch_parse_table.test.js:50:12)
    at processImmediate (internal/timers.js:456:21)

not getting the same key/value pairs as those in the browser test

I'm having issues with getting the same key/value pairs for the Expense Parser Processor. The uploaded test in the Document AI browser has keys like total_amount, currency, supplier_address... but it's not showing up when I ran it using the provided code in your example.

        console.log("\nThe following form key/value pairs were detected:");

        const { formFields } = page1;
        for (const field of formFields) {
          const fieldName = getText(field.fieldName.textAnchor);
          const fieldValue = getText(field.fieldValue.textAnchor);

          console.log("Extracted key value pair:");
          console.log(`\t(${fieldName}, ${fieldValue})`);
        }

Its keys are like the actual text name of the items.

Error in samples/process-document.v1beta3.js

With node samples/process-document.v1beta3.js I got this error:

{ Error: 3 INVALID_ARGUMENT: Request contains an invalid argument.
    at Object.callErrorFromStatus (/home/info/cloudshell_open/nodejs-document-ai/samples/node_modules/@grpc/grpc-js/build/src/call.js:31:26)
    at Object.onReceiveStatus (/home/info/cloudshell_open/nodejs-document-ai/samples/node_modules/@grpc/grpc-js/build/src/client.js:176:52)
    at Object.onReceiveStatus (/home/info/cloudshell_open/nodejs-document-ai/samples/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:342:141)
    at Object.onReceiveStatus (/home/info/cloudshell_open/nodejs-document-ai/samples/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:305:181)
    at process.nextTick (/home/info/cloudshell_open/nodejs-document-ai/samples/node_modules/@grpc/grpc-js/build/src/call-stream.js:124:78)
    at process._tickCallback (internal/process/next_tick.js:61:11)
  code: 3,
  details: 'Request contains an invalid argument.',
  metadata:
   Metadata {
     internalRepr: Map { 'grpc-server-stats-bin' => [Array] },
     options: {} },
  note:
   'Exception occurred in retry method that was not classified as transient' }

I had the same error in localhost and CloudShell.

Im using a service account in json format: export GOOGLE_APPLICATION_CREDENTIALS="[PATH]"

library should have @google-cloud prefix in gapic configuration

Process quality document: should run document (process quality) (v1beta3) failed

This test failed!

To configure my behavior, see the Flaky Bot documentation.

If I'm commenting on this issue too often, add the flakybot: quiet label and
I will stop commenting.

commit: b465fd2
buildURL: Build Status, Sponge
status: failed

Test output

Command failed: node ./process-document-quality.js long-door-651 us face29cdcce978c0 /workspace/samples/resources/document_quality_poor.pdf Error: 16 UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project. at Object.callErrorFromStatus (/workspace/node_modules/@grpc/grpc-js/build/src/call.js:31:19) at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:190:52) at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:365:141) at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:328:181) at /workspace/node_modules/@grpc/grpc-js/build/src/call-stream.js:188:78 at processTicksAndRejections (internal/process/task_queues.js:79:11) for call at at ServiceClientImpl.makeUnaryRequest (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:160:30) at ServiceClientImpl. (/workspace/node_modules/@grpc/grpc-js/build/src/make-client.js:105:19) at /workspace/build/src/v1beta3/document_processor_service_client.js:296:29 at /workspace/node_modules/google-gax/build/src/normalCalls/timeout.js:44:16 at repeat (/workspace/node_modules/google-gax/build/src/normalCalls/retries.js:80:25) at /workspace/node_modules/google-gax/build/src/normalCalls/retries.js:118:13 at OngoingCallPromise.call (/workspace/node_modules/google-gax/build/src/call.js:67:27) at NormalApiCaller.call (/workspace/node_modules/google-gax/build/src/normalCalls/normalApiCaller.js:34:19) at /workspace/node_modules/google-gax/build/src/createApiCall.js:84:30 { code: 16, details: 'Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.', metadata: Metadata { internalRepr: Map { 'grpc-server-stats-bin' => [Array] }, options: {} }, note: 'Exception occurred in retry method that was not classified as transient' }

Error: Command failed: node ./process-document-quality.js long-door-651 us face29cdcce978c0 /workspace/samples/resources/document_quality_poor.pdf
Error: 16 UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
at Object.callErrorFromStatus (/workspace/node_modules/@grpc/grpc-js/build/src/call.js:31:19)
at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:190:52)
at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:365:141)
at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:328:181)
at /workspace/node_modules/@grpc/grpc-js/build/src/call-stream.js:188:78
at processTicksAndRejections (internal/process/task_queues.js:79:11)
for call at
at ServiceClientImpl.makeUnaryRequest (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:160:30)
at ServiceClientImpl. (/workspace/node_modules/@grpc/grpc-js/build/src/make-client.js:105:19)
at /workspace/build/src/v1beta3/document_processor_service_client.js:296:29
at /workspace/node_modules/google-gax/build/src/normalCalls/timeout.js:44:16
at repeat (/workspace/node_modules/google-gax/build/src/normalCalls/retries.js:80:25)
at /workspace/node_modules/google-gax/build/src/normalCalls/retries.js:118:13
at OngoingCallPromise.call (/workspace/node_modules/google-gax/build/src/call.js:67:27)
at NormalApiCaller.call (/workspace/node_modules/google-gax/build/src/normalCalls/normalApiCaller.js:34:19)
at /workspace/node_modules/google-gax/build/src/createApiCall.js:84:30 {
code: 16,
details: 'Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.',
metadata: Metadata {
internalRepr: Map { 'grpc-server-stats-bin' => [Array] },
options: {}
},
note: 'Exception occurred in retry method that was not classified as transient'
}

at checkExecSyncError (child_process.js:635:11)
at Object.execSync (child_process.js:671:15)
at execSync (test/process-document-quality.test.js:28:28)
at Context.<anonymous> (test/process-document-quality.test.js:43:20)
at processImmediate (internal/timers.js:461:21)</pre></details>

Warning: a recent release failed

The release PR #61 is still in a pending state after several hours

Broken link breaking doc tests

Doc tests are not passing due to a broken link in protos:

https://cloud.google.com/document-understanding/docs/

This makes us unable to land work on the library.

@telpirion are you the right person to talk to about this?

Synthesis failed for nodejs-document-ai

Hello! Autosynth couldn't regenerate nodejs-document-ai. 💔

Here's the output from running synth.py:

ythonhosted.org/packages/30/9e/f663a2aa66a09d838042ae1a2c5659828bb9b41ea3a6efa20a20fd92b121/Jinja2-2.11.2-py2.py3-none-any.whl
  Saved ./Jinja2-2.11.2-py2.py3-none-any.whl
Collecting MarkupSafe==1.1.1 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 5))
  Using cached https://files.pythonhosted.org/packages/b2/5f/23e0023be6bb885d00ffbefad2942bc51a620328ee910f64abe5a8d18dd1/MarkupSafe-1.1.1-cp36-cp36m-manylinux1_x86_64.whl
  Saved ./MarkupSafe-1.1.1-cp36-cp36m-manylinux1_x86_64.whl
Collecting protobuf==3.13.0 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 6))
  Using cached https://files.pythonhosted.org/packages/30/79/510974552cebff2ba04038544799450defe75e96ea5f1675dbf72cc8744f/protobuf-3.13.0-cp36-cp36m-manylinux1_x86_64.whl
  Saved ./protobuf-3.13.0-cp36-cp36m-manylinux1_x86_64.whl
Collecting pypandoc==1.5 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 7))
  Using cached https://files.pythonhosted.org/packages/d6/b7/5050dc1769c8a93d3ec7c4bd55be161991c94b8b235f88bf7c764449e708/pypandoc-1.5.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmpfs/tmp/tmpbtx14b04/setuptools-tmp/setuptools/__init__.py", line 6, in <module>
        import distutils.core
      File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/_distutils_hack/__init__.py", line 82, in create_module
        return importlib.import_module('._distutils', 'setuptools')
      File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/importlib/__init__.py", line 126, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
    ModuleNotFoundError: No module named 'setuptools._distutils'
    
    ----------------------------------------
 (Command "python setup.py egg_info" failed with error code 1 in /tmpfs/tmp/pip-build-07l_ypne/pypandoc/
)
Loading: 0 packages loaded
ERROR: no such package '@gapic_generator_python_pip_deps//': pip_import failed: Collecting click==7.1.2 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 1))
  Using cached https://files.pythonhosted.org/packages/d2/3d/fa76db83bf75c4f8d338c2fd15c8d33fdd7ad23a9b5e57eb6c5de26b430e/click-7.1.2-py2.py3-none-any.whl
  Saved ./click-7.1.2-py2.py3-none-any.whl
Collecting google-api-core==1.22.1 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 2))
  Using cached https://files.pythonhosted.org/packages/e0/2d/7c6c75013105e1d2b6eaa1bf18a56995be1dbc673c38885aea31136e9918/google_api_core-1.22.1-py2.py3-none-any.whl
  Saved ./google_api_core-1.22.1-py2.py3-none-any.whl
Collecting googleapis-common-protos==1.52.0 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 3))
  Using cached https://files.pythonhosted.org/packages/03/74/3956721ea1eb4bcf7502a311fdaa60b85bd751de4e57d1943afe9b334141/googleapis_common_protos-1.52.0-py2.py3-none-any.whl
  Saved ./googleapis_common_protos-1.52.0-py2.py3-none-any.whl
Collecting jinja2==2.11.2 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 4))
  Using cached https://files.pythonhosted.org/packages/30/9e/f663a2aa66a09d838042ae1a2c5659828bb9b41ea3a6efa20a20fd92b121/Jinja2-2.11.2-py2.py3-none-any.whl
  Saved ./Jinja2-2.11.2-py2.py3-none-any.whl
Collecting MarkupSafe==1.1.1 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 5))
  Using cached https://files.pythonhosted.org/packages/b2/5f/23e0023be6bb885d00ffbefad2942bc51a620328ee910f64abe5a8d18dd1/MarkupSafe-1.1.1-cp36-cp36m-manylinux1_x86_64.whl
  Saved ./MarkupSafe-1.1.1-cp36-cp36m-manylinux1_x86_64.whl
Collecting protobuf==3.13.0 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 6))
  Using cached https://files.pythonhosted.org/packages/30/79/510974552cebff2ba04038544799450defe75e96ea5f1675dbf72cc8744f/protobuf-3.13.0-cp36-cp36m-manylinux1_x86_64.whl
  Saved ./protobuf-3.13.0-cp36-cp36m-manylinux1_x86_64.whl
Collecting pypandoc==1.5 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 7))
  Using cached https://files.pythonhosted.org/packages/d6/b7/5050dc1769c8a93d3ec7c4bd55be161991c94b8b235f88bf7c764449e708/pypandoc-1.5.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmpfs/tmp/tmpbtx14b04/setuptools-tmp/setuptools/__init__.py", line 6, in <module>
        import distutils.core
      File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/_distutils_hack/__init__.py", line 82, in create_module
        return importlib.import_module('._distutils', 'setuptools')
      File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/importlib/__init__.py", line 126, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
    ModuleNotFoundError: No module named 'setuptools._distutils'
    
    ----------------------------------------
 (Command "python setup.py egg_info" failed with error code 1 in /tmpfs/tmp/pip-build-07l_ypne/pypandoc/
)
INFO: Elapsed time: 2.104s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
FAILED: Build did NOT complete successfully (0 packages loaded)

Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 102, in <module>
    main()
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 94, in main
    spec.loader.exec_module(synth_module)  # type: ignore
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/kbuilder/.cache/synthtool/nodejs-document-ai/synth.py", line 31, in <module>
    library = gapic.node_library(name, version)
  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_bazel.py", line 52, in node_library
    return self._generate_code(service, version, "nodejs", **kwargs)
  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_bazel.py", line 183, in _generate_code
    shell.run(bazel_run_args)
  File "/tmpfs/src/github/synthtool/synthtool/shell.py", line 39, in run
    raise exc
  File "/tmpfs/src/github/synthtool/synthtool/shell.py", line 33, in run
    encoding="utf-8",
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['bazel', '--max_idle_secs=240', 'build', '//google/cloud/documentai/v1beta2:documentai-v1beta2-nodejs']' returned non-zero exit status 1.
2020-08-31 02:29:58,573 autosynth [ERROR] > Synthesis failed
2020-08-31 02:29:58,573 autosynth [DEBUG] > Running: git reset --hard HEAD
HEAD is now at 483df36 build: track flaky tests for "nightly", add new secrets for tagging (#96)
2020-08-31 02:29:58,579 autosynth [DEBUG] > Running: git checkout autosynth
Switched to branch 'autosynth'
2020-08-31 02:29:58,583 autosynth [DEBUG] > Running: git clean -fdx
Removing __pycache__/
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 690, in <module>
    main()
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 539, in main
    return _inner_main(temp_dir)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 670, in _inner_main
    commit_count = synthesize_loop(x, multiple_prs, change_pusher, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 375, in synthesize_loop
    has_changes = toolbox.synthesize_version_in_new_branch(synthesizer, youngest)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 273, in synthesize_version_in_new_branch
    synthesizer.synthesize(synth_log_path, self.environ)
  File "/tmpfs/src/github/synthtool/autosynth/synthesizer.py", line 120, in synthesize
    synth_proc.check_returncode()  # Raise an exception.
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 389, in check_returncode
    self.stderr)
subprocess.CalledProcessError: Command '['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']' returned non-zero exit status 1.

Google internal developers can see the full log here.

library should have quickstart sample added

Warning: a recent release failed

The following release PRs may have failed:

#344 - The release job failed -- check the build log.

Quickstart: should run quickstart failed

This test failed!

To configure my behavior, see the Flaky Bot documentation.

If I'm commenting on this issue too often, add the flakybot: quiet label and
I will stop commenting.

commit: b465fd2
buildURL: Build Status, Sponge
status: failed

Test output

Command failed: node ./quickstart.js long-door-651 us 8f1123c1b125e0b7 /workspace/samples/resources/invoice.pdf Error: 16 UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project. at Object.callErrorFromStatus (/workspace/node_modules/@grpc/grpc-js/build/src/call.js:31:19) at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:190:52) at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:365:141) at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:328:181) at /workspace/node_modules/@grpc/grpc-js/build/src/call-stream.js:188:78 at processTicksAndRejections (internal/process/task_queues.js:79:11) for call at at ServiceClientImpl.makeUnaryRequest (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:160:30) at ServiceClientImpl. (/workspace/node_modules/@grpc/grpc-js/build/src/make-client.js:105:19) at /workspace/build/src/v1/document_processor_service_client.js:297:29 at /workspace/node_modules/google-gax/build/src/normalCalls/timeout.js:44:16 at repeat (/workspace/node_modules/google-gax/build/src/normalCalls/retries.js:80:25) at /workspace/node_modules/google-gax/build/src/normalCalls/retries.js:118:13 at OngoingCallPromise.call (/workspace/node_modules/google-gax/build/src/call.js:67:27) at NormalApiCaller.call (/workspace/node_modules/google-gax/build/src/normalCalls/normalApiCaller.js:34:19) at /workspace/node_modules/google-gax/build/src/createApiCall.js:84:30 { code: 16, details: 'Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.', metadata: Metadata { internalRepr: Map { 'grpc-server-stats-bin' => [Array] }, options: {} }, note: 'Exception occurred in retry method that was not classified as transient' }

Error: Command failed: node ./quickstart.js long-door-651 us 8f1123c1b125e0b7 /workspace/samples/resources/invoice.pdf
Error: 16 UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
at Object.callErrorFromStatus (/workspace/node_modules/@grpc/grpc-js/build/src/call.js:31:19)
at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:190:52)
at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:365:141)
at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:328:181)
at /workspace/node_modules/@grpc/grpc-js/build/src/call-stream.js:188:78
at processTicksAndRejections (internal/process/task_queues.js:79:11)
for call at
at ServiceClientImpl.makeUnaryRequest (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:160:30)
at ServiceClientImpl. (/workspace/node_modules/@grpc/grpc-js/build/src/make-client.js:105:19)
at /workspace/build/src/v1/document_processor_service_client.js:297:29
at /workspace/node_modules/google-gax/build/src/normalCalls/timeout.js:44:16
at repeat (/workspace/node_modules/google-gax/build/src/normalCalls/retries.js:80:25)
at /workspace/node_modules/google-gax/build/src/normalCalls/retries.js:118:13
at OngoingCallPromise.call (/workspace/node_modules/google-gax/build/src/call.js:67:27)
at NormalApiCaller.call (/workspace/node_modules/google-gax/build/src/normalCalls/normalApiCaller.js:34:19)
at /workspace/node_modules/google-gax/build/src/createApiCall.js:84:30 {
code: 16,
details: 'Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.',
metadata: Metadata {
internalRepr: Map { 'grpc-server-stats-bin' => [Array] },
options: {}
},
note: 'Exception occurred in retry method that was not classified as transient'
}

at checkExecSyncError (child_process.js:635:11)
at Object.execSync (child_process.js:671:15)
at execSync (test/quickstart.test.js:28:28)
at Context.<anonymous> (test/quickstart.test.js:44:20)
at processImmediate (internal/timers.js:461:21)</pre></details>

library's import is called DocumentUnderstandingServiceClient

is this import name DocumentUnderstandingServiceClient correct, given our conversation about not calling the library Document Understanding?

Document AI Node.js Client API Reference documentation on version 2.1.1 as "latest version"

Environment details

OS: N/A
Node.js version: N/A
npm version: N/A
@google-cloud/documentai version: 2.2.1

Steps to reproduce

Visit https://googleapis.dev/nodejs/documentai/latest/index.html

The info box in bottom right corner has Version: 2.1.1 (latest)
but latest version is 2.3.0 atm. (as seen in the npm badge at top of page)

[Policy Bot] found one or more issues with this repository.

Policy Bot found one or more issues with this repository.

Document AI batch parse table (v1beta2): "after all" hook for "should parse the GCS invoice example as as table" failed

This test failed!

To configure my behavior, see the Flaky Bot documentation.

If I'm commenting on this issue too often, add the flakybot: quiet label and
I will stop commenting.

commit: 1c4ab7d
buildURL: Build Status, Sponge
status: failed

Test output

The specified bucket does not exist.
Error: The specified bucket does not exist.
    at new ApiError (node_modules/@google-cloud/common/build/src/util.js:73:15)
    at Util.parseHttpRespBody (node_modules/@google-cloud/common/build/src/util.js:208:38)
    at Util.handleResp (node_modules/@google-cloud/common/build/src/util.js:149:117)
    at /workspace/samples/node_modules/@google-cloud/common/build/src/util.js:477:22
    at onResponse (node_modules/retry-request/index.js:228:7)
    at /workspace/samples/node_modules/teeny-request/build/src/index.js:226:13
        -> /workspace/samples/node_modules/teeny-request/src/index.ts:333:11
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: undefined. Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.

Mocha Tests: should process a PDF in another region failed

This test failed!

To configure my behavior, see the Build Cop Bot documentation.

If I'm commenting on this issue too often, add the buildcop: quiet label and
I will stop commenting.

commit: ab12c1b
buildURL: Build Status, Sponge
status: failed

Synthesis failed for nodejs-document-ai

Hello! Autosynth couldn't regenerate nodejs-document-ai. 💔

Here's the output from running synth.py:

est.sh
.kokoro/test.bat
.kokoro/test.sh
.kokoro/trampoline.sh
.kokoro/trampoline_v2.sh
.mocharc.js
.nycrc
.prettierignore
.prettierrc.js
.trampolinerc
CODE_OF_CONDUCT.md
CONTRIBUTING.md
LICENSE
README.md
api-extractor.json
renovate.json
samples/README.md
2020-11-17 02:41:05,589 synthtool [DEBUG] > Post-processing GAPIC library...
DEBUG:synthtool:Post-processing GAPIC library...
2020-11-17 02:41:05,590 synthtool [DEBUG] > Installing dependencies...
DEBUG:synthtool:Installing dependencies...

> [email protected] postinstall /home/kbuilder/.cache/synthtool/nodejs-document-ai/node_modules/protobufjs
> node scripts/postinstall


> @google-cloud/[email protected] prepare /home/kbuilder/.cache/synthtool/nodejs-document-ai
> npm run compile-protos && npm run compile


> @google-cloud/[email protected] compile-protos /home/kbuilder/.cache/synthtool/nodejs-document-ai
> compileProtos src

installing semver@^7.1.2
installing tmp@^0.2.0
installing uglify-js@^3.7.7
installing espree@^7.0.0
installing escodegen@^2.0.0
installing estraverse@^5.1.0

> @google-cloud/[email protected] precompile /home/kbuilder/.cache/synthtool/nodejs-document-ai
> gts clean

version: 14
Removing build ...

> @google-cloud/[email protected] compile /home/kbuilder/.cache/synthtool/nodejs-document-ai
> tsc -p . && cp -r protos build/

npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@~2.1.2 (node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

added 673 packages from 777 contributors and audited 675 packages in 31.696s

57 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

2020-11-17 02:41:37,927 synthtool [DEBUG] > Running prelint...
DEBUG:synthtool:Running prelint...

> @google-cloud/[email protected] prelint /home/kbuilder/.cache/synthtool/nodejs-document-ai
> cd samples; npm link ../; npm install


> @google-cloud/[email protected] prepare /home/kbuilder/.cache/synthtool/nodejs-document-ai
> npm run compile-protos && npm run compile


> @google-cloud/[email protected] compile-protos /home/kbuilder/.cache/synthtool/nodejs-document-ai
> compileProtos src


> @google-cloud/[email protected] precompile /home/kbuilder/.cache/synthtool/nodejs-document-ai
> gts clean

version: 14
Removing build ...

> @google-cloud/[email protected] compile /home/kbuilder/.cache/synthtool/nodejs-document-ai
> tsc -p . && cp -r protos build/

npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

audited 675 packages in 16.857s

57 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

/home/kbuilder/.nvm/versions/node/v14.15.1/lib/node_modules/@google-cloud/documentai -> /home/kbuilder/.cache/synthtool/nodejs-document-ai
/home/kbuilder/.cache/synthtool/nodejs-document-ai/samples/node_modules/@google-cloud/documentai -> /home/kbuilder/.nvm/versions/node/v14.15.1/lib/node_modules/@google-cloud/documentai -> /home/kbuilder/.cache/synthtool/nodejs-document-ai
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@~2.1.2 (node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

added 183 packages from 107 contributors and audited 272 packages in 5.728s

58 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

2020-11-17 02:42:02,098 synthtool [DEBUG] > Running fix...
DEBUG:synthtool:Running fix...

> @google-cloud/[email protected] fix /home/kbuilder/.cache/synthtool/nodejs-document-ai
> gts fix

version: 14

/home/kbuilder/.cache/synthtool/nodejs-document-ai/src/v1beta2/document_understanding_service_client.ts
  103:48  error  'window' is not defined  no-undef

/home/kbuilder/.cache/synthtool/nodejs-document-ai/src/v1beta3/document_processor_service_client.ts
  105:48  error  'window' is not defined  no-undef

✖ 2 problems (2 errors, 0 warnings)

Error: Command failed with exit code 1: node ./node_modules/eslint/bin/eslint --fix **/*.ts **/*.js **/*.tsx **/*.jsx --no-error-on-unmatched-pattern
    at makeError (/home/kbuilder/.cache/synthtool/nodejs-document-ai/node_modules/execa/lib/error.js:59:11)
    at handlePromise (/home/kbuilder/.cache/synthtool/nodejs-document-ai/node_modules/execa/index.js:114:26)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async run (/home/kbuilder/.cache/synthtool/nodejs-document-ai/node_modules/gts/build/src/cli.js:120:17) {
  shortMessage: 'Command failed with exit code 1: node ./node_modules/eslint/bin/eslint --fix **/*.ts **/*.js **/*.tsx **/*.jsx --no-error-on-unmatched-pattern',
  command: 'node ./node_modules/eslint/bin/eslint --fix **/*.ts **/*.js **/*.tsx **/*.jsx --no-error-on-unmatched-pattern',
  exitCode: 1,
  signal: undefined,
  signalDescription: undefined,
  stdout: undefined,
  stderr: undefined,
  failed: true,
  timedOut: false,
  isCanceled: false,
  killed: false
}
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! @google-cloud/[email protected] fix: `gts fix`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the @google-cloud/[email protected] fix script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/kbuilder/.npm/_logs/2020-11-17T10_42_08_260Z-debug.log
2020-11-17 02:42:08,269 synthtool [ERROR] > Failed executing npm run fix:

None
ERROR:synthtool:Failed executing npm run fix:

None
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 102, in <module>
    main()
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 94, in main
    spec.loader.exec_module(synth_module)  # type: ignore
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/kbuilder/.cache/synthtool/nodejs-document-ai/synth.py", line 54, in <module>
    node.postprocess_gapic_library()
  File "/tmpfs/src/github/synthtool/synthtool/languages/node.py", line 194, in postprocess_gapic_library
    fix(hide_output=hide_output)
  File "/tmpfs/src/github/synthtool/synthtool/languages/node.py", line 179, in fix
    shell.run(["npm", "run", "fix"], hide_output=hide_output)
  File "/tmpfs/src/github/synthtool/synthtool/shell.py", line 39, in run
    raise exc
  File "/tmpfs/src/github/synthtool/synthtool/shell.py", line 33, in run
    encoding="utf-8",
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['npm', 'run', 'fix']' returned non-zero exit status 1.
2020-11-17 02:42:08,342 autosynth [ERROR] > Synthesis failed
2020-11-17 02:42:08,343 autosynth [DEBUG] > Running: git reset --hard HEAD
HEAD is now at dccd43e chore: release 2.2.1 (#128)
2020-11-17 02:42:08,371 autosynth [DEBUG] > Running: git checkout autosynth
Switched to branch 'autosynth'
2020-11-17 02:42:08,399 autosynth [DEBUG] > Running: git clean -fdx
Removing __pycache__/
Removing build/
Removing node_modules/
Removing package-lock.json
Removing samples/node_modules/
Removing samples/package-lock.json
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 354, in <module>
    main()
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 189, in main
    return _inner_main(temp_dir)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 334, in _inner_main
    commit_count = synthesize_loop(x, multiple_prs, change_pusher, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 65, in synthesize_loop
    has_changes = toolbox.synthesize_version_in_new_branch(synthesizer, youngest)
  File "/tmpfs/src/github/synthtool/autosynth/synth_toolbox.py", line 259, in synthesize_version_in_new_branch
    synthesizer.synthesize(synth_log_path, self.environ)
  File "/tmpfs/src/github/synthtool/autosynth/synthesizer.py", line 120, in synthesize
    synth_proc.check_returncode()  # Raise an exception.
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 389, in check_returncode
    self.stderr)
subprocess.CalledProcessError: Command '['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']' returned non-zero exit status 1.

Google internal developers can see the full log here.

Document AI batch parse form: "before all" hook for "should parse the GCS invoice example as a form" failed

Note: #268 was also for this test, but it was closed more than 10 days ago. So, I didn't mark it flaky.

commit: b465fd2
buildURL: Build Status, Sponge
status: failed

Test output

invalid_grant: Invalid JWT Signature.
Error: invalid_grant: Invalid JWT Signature.
    at Gaxios._request (node_modules/gaxios/build/src/gaxios.js:130:23)
        -> /workspace/samples/node_modules/gaxios/src/gaxios.ts:158:15
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async GoogleToken.requestToken (node_modules/gtoken/build/src/index.js:238:23)
    at async GoogleToken.getTokenAsync (node_modules/gtoken/build/src/index.js:145:20)
    at async JWT.refreshTokenNoCache (node_modules/google-auth-library/build/src/auth/jwtclient.js:172:23)
    at async JWT.getRequestMetadataAsync (node_modules/google-auth-library/build/src/auth/oauth2client.js:284:17)
    at async JWT.getRequestHeaders (node_modules/google-auth-library/build/src/auth/oauth2client.js:247:26)
    at async GoogleAuth.authorizeRequest (node_modules/google-auth-library/build/src/auth/googleauth.js:677:25)
    at async Promise.all (index 1)
    at async prepareRequest (node_modules/@google-cloud/storage/build/src/nodejs-common/util.js:447:61)

Set v1beta3 as default version of library

Once releases, the v1beta3 version of Document AI will be the default version that we want customers using.

We should update accordingly in these locations (there may be others):

Process document: should run document (process invoice) (v1) failed

This test failed!

To configure my behavior, see the Flaky Bot documentation.

If I'm commenting on this issue too often, add the flakybot: quiet label and
I will stop commenting.

commit: b465fd2
buildURL: Build Status, Sponge
status: failed

Test output

Command failed: node ./process-document.js long-door-651 us 8f1123c1b125e0b7 /workspace/samples/resources/invoice.pdf Error: 16 UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project. at Object.callErrorFromStatus (/workspace/node_modules/@grpc/grpc-js/build/src/call.js:31:19) at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:190:52) at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:365:141) at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:328:181) at /workspace/node_modules/@grpc/grpc-js/build/src/call-stream.js:188:78 at processTicksAndRejections (internal/process/task_queues.js:79:11) for call at at ServiceClientImpl.makeUnaryRequest (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:160:30) at ServiceClientImpl. (/workspace/node_modules/@grpc/grpc-js/build/src/make-client.js:105:19) at /workspace/build/src/v1/document_processor_service_client.js:297:29 at /workspace/node_modules/google-gax/build/src/normalCalls/timeout.js:44:16 at repeat (/workspace/node_modules/google-gax/build/src/normalCalls/retries.js:80:25) at /workspace/node_modules/google-gax/build/src/normalCalls/retries.js:118:13 at OngoingCallPromise.call (/workspace/node_modules/google-gax/build/src/call.js:67:27) at NormalApiCaller.call (/workspace/node_modules/google-gax/build/src/normalCalls/normalApiCaller.js:34:19) at /workspace/node_modules/google-gax/build/src/createApiCall.js:84:30 { code: 16, details: 'Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.', metadata: Metadata { internalRepr: Map { 'grpc-server-stats-bin' => [Array] }, options: {} }, note: 'Exception occurred in retry method that was not classified as transient' }

Error: Command failed: node ./process-document.js long-door-651 us 8f1123c1b125e0b7 /workspace/samples/resources/invoice.pdf
Error: 16 UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
at Object.callErrorFromStatus (/workspace/node_modules/@grpc/grpc-js/build/src/call.js:31:19)
at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:190:52)
at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:365:141)
at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:328:181)
at /workspace/node_modules/@grpc/grpc-js/build/src/call-stream.js:188:78
at processTicksAndRejections (internal/process/task_queues.js:79:11)
for call at
at ServiceClientImpl.makeUnaryRequest (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:160:30)
at ServiceClientImpl. (/workspace/node_modules/@grpc/grpc-js/build/src/make-client.js:105:19)
at /workspace/build/src/v1/document_processor_service_client.js:297:29
at /workspace/node_modules/google-gax/build/src/normalCalls/timeout.js:44:16
at repeat (/workspace/node_modules/google-gax/build/src/normalCalls/retries.js:80:25)
at /workspace/node_modules/google-gax/build/src/normalCalls/retries.js:118:13
at OngoingCallPromise.call (/workspace/node_modules/google-gax/build/src/call.js:67:27)
at NormalApiCaller.call (/workspace/node_modules/google-gax/build/src/normalCalls/normalApiCaller.js:34:19)
at /workspace/node_modules/google-gax/build/src/createApiCall.js:84:30 {
code: 16,
details: 'Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.',
metadata: Metadata {
internalRepr: Map { 'grpc-server-stats-bin' => [Array] },
options: {}
},
note: 'Exception occurred in retry method that was not classified as transient'
}

at checkExecSyncError (child_process.js:635:11)
at Object.execSync (child_process.js:671:15)
at execSync (test/process-document.test.js:28:28)
at Context.<anonymous> (test/process-document.test.js:43:20)
at processImmediate (internal/timers.js:461:21)</pre></details>

Add support for content being a Buffer

As I understand the story today, I can supply a base64 encoded string for the content property of a Document object that I wish to process where the content will be either from a PDF, a TIFF or a GIF. This request asks for consideration of other equivalent data structures such as a NodeJS Buffer. I am reading GCS data and getting Buffer objects and would like to pass in Buffers without explicitly converting to base64 (in user code) first.

Process OCR document: should run document (process invoice) (v1) failed

This test failed!

To configure my behavior, see the Flaky Bot documentation.

If I'm commenting on this issue too often, add the flakybot: quiet label and
I will stop commenting.

commit: b465fd2
buildURL: Build Status, Sponge
status: failed

Test output

Command failed: node ./process-document-ocr.js long-door-651 us d9f262d374c21325 /workspace/samples/resources/handwritten_form.pdf Error: 16 UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project. at Object.callErrorFromStatus (/workspace/node_modules/@grpc/grpc-js/build/src/call.js:31:19) at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:190:52) at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:365:141) at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:328:181) at /workspace/node_modules/@grpc/grpc-js/build/src/call-stream.js:188:78 at processTicksAndRejections (internal/process/task_queues.js:79:11) for call at at ServiceClientImpl.makeUnaryRequest (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:160:30) at ServiceClientImpl. (/workspace/node_modules/@grpc/grpc-js/build/src/make-client.js:105:19) at /workspace/build/src/v1beta3/document_processor_service_client.js:296:29 at /workspace/node_modules/google-gax/build/src/normalCalls/timeout.js:44:16 at repeat (/workspace/node_modules/google-gax/build/src/normalCalls/retries.js:80:25) at /workspace/node_modules/google-gax/build/src/normalCalls/retries.js:118:13 at OngoingCallPromise.call (/workspace/node_modules/google-gax/build/src/call.js:67:27) at NormalApiCaller.call (/workspace/node_modules/google-gax/build/src/normalCalls/normalApiCaller.js:34:19) at /workspace/node_modules/google-gax/build/src/createApiCall.js:84:30 { code: 16, details: 'Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.', metadata: Metadata { internalRepr: Map { 'grpc-server-stats-bin' => [Array] }, options: {} }, note: 'Exception occurred in retry method that was not classified as transient' }

Error: Command failed: node ./process-document-ocr.js long-door-651 us d9f262d374c21325 /workspace/samples/resources/handwritten_form.pdf
Error: 16 UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
at Object.callErrorFromStatus (/workspace/node_modules/@grpc/grpc-js/build/src/call.js:31:19)
at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:190:52)
at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:365:141)
at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:328:181)
at /workspace/node_modules/@grpc/grpc-js/build/src/call-stream.js:188:78
at processTicksAndRejections (internal/process/task_queues.js:79:11)
for call at
at ServiceClientImpl.makeUnaryRequest (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:160:30)
at ServiceClientImpl. (/workspace/node_modules/@grpc/grpc-js/build/src/make-client.js:105:19)
at /workspace/build/src/v1beta3/document_processor_service_client.js:296:29
at /workspace/node_modules/google-gax/build/src/normalCalls/timeout.js:44:16
at repeat (/workspace/node_modules/google-gax/build/src/normalCalls/retries.js:80:25)
at /workspace/node_modules/google-gax/build/src/normalCalls/retries.js:118:13
at OngoingCallPromise.call (/workspace/node_modules/google-gax/build/src/call.js:67:27)
at NormalApiCaller.call (/workspace/node_modules/google-gax/build/src/normalCalls/normalApiCaller.js:34:19)
at /workspace/node_modules/google-gax/build/src/createApiCall.js:84:30 {
code: 16,
details: 'Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.',
metadata: Metadata {
internalRepr: Map { 'grpc-server-stats-bin' => [Array] },
options: {}
},
note: 'Exception occurred in retry method that was not classified as transient'
}

at checkExecSyncError (child_process.js:635:11)
at Object.execSync (child_process.js:671:15)
at execSync (test/process-document-ocr.test.js:28:28)
at Context.<anonymous> (test/process-document-ocr.test.js:43:20)
at processImmediate (internal/timers.js:461:21)</pre></details>

Synthesis failed for nodejs-document-ai

Hello! Autosynth couldn't regenerate nodejs-document-ai. 💔

Here's the output from running synth.py:

_tools/tools/build_defs/repo/http.bzl:296:16):
 - <builtin>
 - /home/kbuilder/.cache/synthtool/googleapis/WORKSPACE:42:1
DEBUG: Rule 'gapic_generator_python' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "8d11f06b408ac5f1c01da3ca17f3a75dc008831509c5c1a4f24f9bde37792a57"
DEBUG: Call stack for the definition of repository 'gapic_generator_python' which is a http_archive (rule definition at /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/bazel_tools/tools/build_defs/repo/http.bzl:296:16):
 - <builtin>
 - /home/kbuilder/.cache/synthtool/googleapis/WORKSPACE:224:1
DEBUG: Rule 'com_googleapis_gapic_generator_go' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "12bfed7873f085093cd60615bd113178ecf36396af0c2ca25e6cd4d4bebdd198"
DEBUG: Call stack for the definition of repository 'com_googleapis_gapic_generator_go' which is a http_archive (rule definition at /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/bazel_tools/tools/build_defs/repo/http.bzl:296:16):
 - <builtin>
 - /home/kbuilder/.cache/synthtool/googleapis/WORKSPACE:261:1
DEBUG: Rule 'gapic_generator_typescript' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "ca322b5e7b0d03b3cc44a90444e3a7f944c9ba3345f0505ee48c8e715d19dd95"
DEBUG: Call stack for the definition of repository 'gapic_generator_typescript' which is a http_archive (rule definition at /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/bazel_tools/tools/build_defs/repo/http.bzl:296:16):
 - <builtin>
 - /home/kbuilder/.cache/synthtool/googleapis/WORKSPACE:280:1
DEBUG: Rule 'gapic_generator_csharp' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "40ddae63d2729ef5ccbd8b60123327ea200ce9400d0629238193ff530dcaea18"
DEBUG: Call stack for the definition of repository 'gapic_generator_csharp' which is a http_archive (rule definition at /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/bazel_tools/tools/build_defs/repo/http.bzl:296:16):
 - <builtin>
 - /home/kbuilder/.cache/synthtool/googleapis/WORKSPACE:318:1
DEBUG: Rule 'bazel_skylib' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "1dde365491125a3db70731e25658dfdd3bc5dbdfd11b840b3e987ecf043c7ca0"
DEBUG: Call stack for the definition of repository 'bazel_skylib' which is a http_archive (rule definition at /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/bazel_tools/tools/build_defs/repo/http.bzl:296:16):
 - <builtin>
 - /home/kbuilder/.cache/synthtool/googleapis/WORKSPACE:35:1
Analyzing: target //google/cloud/documentai/v1beta2:documentai-v1beta2-nodejs (1 packages loaded, 0 targets configured)
INFO: Call stack for the definition of repository 'npm' which is a yarn_install (rule definition at /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/build_bazel_rules_nodejs/internal/npm_install/npm_install.bzl:411:16):
 - <builtin>
 - /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/build_bazel_rules_nodejs/index.bzl:87:5
 - /home/kbuilder/.cache/synthtool/googleapis/WORKSPACE:293:1
ERROR: An error occurred during the fetch of repository 'npm':
   yarn_install failed: yarn install v1.19.1
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
info If you think this is a bug, please open a bug report with the information provided in "/home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_typescript/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
 (error An unexpected error occurred: "https://registry.yarnpkg.com/@grpc/grpc-js/-/grpc-js-1.1.2.tgz: Request failed \"404 Not Found\"".
)
ERROR: /home/kbuilder/.cache/synthtool/googleapis/google/cloud/documentai/v1beta2/BUILD.bazel:228:1: //google/cloud/documentai/v1beta2:documentai_nodejs_gapic depends on @gapic_generator_typescript//:protoc_plugin in repository @gapic_generator_typescript which failed to fetch. no such package '@npm//@bazel/typescript': yarn_install failed: yarn install v1.19.1
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
info If you think this is a bug, please open a bug report with the information provided in "/home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_typescript/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
 (error An unexpected error occurred: "https://registry.yarnpkg.com/@grpc/grpc-js/-/grpc-js-1.1.2.tgz: Request failed \"404 Not Found\"".
)
ERROR: Analysis of target '//google/cloud/documentai/v1beta2:documentai-v1beta2-nodejs' failed; build aborted: no such package '@npm//@bazel/typescript': yarn_install failed: yarn install v1.19.1
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
info If you think this is a bug, please open a bug report with the information provided in "/home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_typescript/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
 (error An unexpected error occurred: "https://registry.yarnpkg.com/@grpc/grpc-js/-/grpc-js-1.1.2.tgz: Request failed \"404 Not Found\"".
)
INFO: Elapsed time: 1.185s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (1 packages loaded, 6 targets configured)
FAILED: Build did NOT complete successfully (1 packages loaded, 6 targets configured)

2020-08-06 04:15:02,953 synthtool [DEBUG] > Wrote metadata to synth.metadata.
DEBUG:synthtool:Wrote metadata to synth.metadata.
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 102, in <module>
    main()
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 94, in main
    spec.loader.exec_module(synth_module)  # type: ignore
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/kbuilder/.cache/synthtool/nodejs-document-ai/synth.py", line 31, in <module>
    library = gapic.node_library(name, version)
  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_bazel.py", line 52, in node_library
    return self._generate_code(service, version, "nodejs", **kwargs)
  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_bazel.py", line 183, in _generate_code
    shell.run(bazel_run_args)
  File "/tmpfs/src/github/synthtool/synthtool/shell.py", line 39, in run
    raise exc
  File "/tmpfs/src/github/synthtool/synthtool/shell.py", line 33, in run
    encoding="utf-8",
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['bazel', '--max_idle_secs=240', 'build', '//google/cloud/documentai/v1beta2:documentai-v1beta2-nodejs']' returned non-zero exit status 1.
2020-08-06 04:15:03,006 autosynth [ERROR] > Synthesis failed
2020-08-06 04:15:03,006 autosynth [DEBUG] > Running: git reset --hard HEAD
HEAD is now at 3b743a6 docs: add links to the CHANGELOG from the README.md for Java and Node (#88)
2020-08-06 04:15:03,012 autosynth [DEBUG] > Running: git checkout autosynth
Switched to branch 'autosynth'
2020-08-06 04:15:03,017 autosynth [DEBUG] > Running: git clean -fdx
Removing __pycache__/
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 690, in <module>
    main()
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 539, in main
    return _inner_main(temp_dir)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 670, in _inner_main
    commit_count = synthesize_loop(x, multiple_prs, change_pusher, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 375, in synthesize_loop
    has_changes = toolbox.synthesize_version_in_new_branch(synthesizer, youngest)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 273, in synthesize_version_in_new_branch
    synthesizer.synthesize(synth_log_path, self.environ)
  File "/tmpfs/src/github/synthtool/autosynth/synthesizer.py", line 120, in synthesize
    synth_proc.check_returncode()  # Raise an exception.
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 389, in check_returncode
    self.stderr)
subprocess.CalledProcessError: Command '['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']' returned non-zero exit status 1.

Google internal developers can see the full log here.

Process specialized document: should run document (specialized) (v1beta3) failed

Note: #285 was also for this test, but it was closed more than 10 days ago. So, I didn't mark it flaky.

commit: b465fd2
buildURL: Build Status, Sponge
status: failed

Test output

Command failed: node ./process-document-specialized.js long-door-651 us bb340a5e47a6c1e /workspace/samples/resources/us_driver_license.pdf Error: 16 UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project. at Object.callErrorFromStatus (/workspace/node_modules/@grpc/grpc-js/build/src/call.js:31:19) at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:190:52) at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:365:141) at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:328:181) at /workspace/node_modules/@grpc/grpc-js/build/src/call-stream.js:188:78 at processTicksAndRejections (internal/process/task_queues.js:79:11) for call at at ServiceClientImpl.makeUnaryRequest (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:160:30) at ServiceClientImpl. (/workspace/node_modules/@grpc/grpc-js/build/src/make-client.js:105:19) at /workspace/build/src/v1beta3/document_processor_service_client.js:296:29 at /workspace/node_modules/google-gax/build/src/normalCalls/timeout.js:44:16 at repeat (/workspace/node_modules/google-gax/build/src/normalCalls/retries.js:80:25) at /workspace/node_modules/google-gax/build/src/normalCalls/retries.js:118:13 at OngoingCallPromise.call (/workspace/node_modules/google-gax/build/src/call.js:67:27) at NormalApiCaller.call (/workspace/node_modules/google-gax/build/src/normalCalls/normalApiCaller.js:34:19) at /workspace/node_modules/google-gax/build/src/createApiCall.js:84:30 { code: 16, details: 'Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.', metadata: Metadata { internalRepr: Map { 'grpc-server-stats-bin' => [Array] }, options: {} }, note: 'Exception occurred in retry method that was not classified as transient' }

Error: Command failed: node ./process-document-specialized.js long-door-651 us bb340a5e47a6c1e /workspace/samples/resources/us_driver_license.pdf
Error: 16 UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
at Object.callErrorFromStatus (/workspace/node_modules/@grpc/grpc-js/build/src/call.js:31:19)
at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:190:52)
at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:365:141)
at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:328:181)
at /workspace/node_modules/@grpc/grpc-js/build/src/call-stream.js:188:78
at processTicksAndRejections (internal/process/task_queues.js:79:11)
for call at
at ServiceClientImpl.makeUnaryRequest (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:160:30)
at ServiceClientImpl. (/workspace/node_modules/@grpc/grpc-js/build/src/make-client.js:105:19)
at /workspace/build/src/v1beta3/document_processor_service_client.js:296:29
at /workspace/node_modules/google-gax/build/src/normalCalls/timeout.js:44:16
at repeat (/workspace/node_modules/google-gax/build/src/normalCalls/retries.js:80:25)
at /workspace/node_modules/google-gax/build/src/normalCalls/retries.js:118:13
at OngoingCallPromise.call (/workspace/node_modules/google-gax/build/src/call.js:67:27)
at NormalApiCaller.call (/workspace/node_modules/google-gax/build/src/normalCalls/normalApiCaller.js:34:19)
at /workspace/node_modules/google-gax/build/src/createApiCall.js:84:30 {
code: 16,
details: 'Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.',
metadata: Metadata {
internalRepr: Map { 'grpc-server-stats-bin' => [Array] },
options: {}
},
note: 'Exception occurred in retry method that was not classified as transient'
}

at checkExecSyncError (child_process.js:635:11)
at Object.execSync (child_process.js:671:15)
at execSync (test/process-document-specialized.test.js:28:28)
at Context.<anonymous> (test/process-document-specialized.test.js:43:20)
at processImmediate (internal/timers.js:461:21)</pre></details>

Files stored in Cloud Storage buckets

With Document AI request options, can have something similar to Vision API where we can just enter the Google Bucket URI like below for the file?

const gcsSourceUri = `gs://${bucketName}/${fileName}`;
const name = `projects/${projectId}/locations/${location}/processors/${processorId}`;

const request = {
  name,
  gcsSource: {
    uri: gcsSourceUri,
  },
};

// Recognizes text entities in the PDF document
      const [result] = await client.processDocument(request);

Vision API

Europe endpoint is not supported

Environment details

OS: MacOS
Node.js version: 10
npm version:
@google-cloud/documentai version: 2.2.2

Steps to reproduce

Prepare the sample code
Make any request with location='eu'

The library does not support eu endpoint. Looks like the us endpoint is hardcoded here
https://github.com/googleapis/nodejs-document-ai/blob/master/src/v1beta3/document_processor_service_client.ts
on servicePath and apiEndpoint stati methods.

If one try to set

const location = 'eu';
request.name = "projects/${projectId}/locations/${location}/processors/${processorId}"

Any call to the api fails with the error:

3 INVALID_ARGUMENT: Request contains an invalid argument

promote library to GA

Package name: @google-cloud/documentai
Current release: beta
Proposed release: GA

Instructions

Check the lists below, adding tests / documentation as required. Once all the "required" boxes are ticked, please create a release and close this issue.

Required

28 days elapsed since last beta release with new API surface
Server API is GA
Package API is stable, and we can commit to backward compatibility
All dependencies are GA

Optional

Most common / important scenarios have descriptive samples
Public manual methods have at least one usage sample each (excluding overloads)
Per-API README includes a full description of the API
Per-API README contains at least one “getting started” sample using the most common API scenario
Manual code has been reviewed by API producer
Manual code has been reviewed by a DPE responsible for samples
'Client Libraries' page is added to the product documentation in 'APIs & Reference' section of the product's documentation on Cloud Site

Bug - error on GCS uri from external project

Description

I've got two projects, one with Document AI API enabled and another one to store PDFs on Cloud Storage. I use document ai with a service account which has access to the external Cloud Storage Bucket (where the PDFs are stored). When I call document ai api sending the GCS uri of the pdf it returns "PERMISSION_DENIED". However, the service account has access to the bucket (with uniform permissions configured on the bucket).

I know that the service account has permission to the bucket and the PDFs.

Workaround

I can download the PDFs via GCS node client (that's why I know that the service account has access to the object). I download it as base64 and send to te api request

Desired behavior

I expected that sending the GCS object uri in the Document AI request would work.
Downloading as base64 is more memory consuming.

Environment details

OS: Windows 10
Node.js version: 14.15.1
npm version: 6.14.18
@google-cloud/documentai version: 2.10.0

Steps to reproduce

Create an object in a bucket of Google Cloud Storage from an external project (for example with uniform permissions)
Create a service account on the Document AI project
Add that service account to the external project bucket
Make a document ai request to extract parameters from that external bucket

Additional info

The Cloud Storage and Document AI node clients are initialized with the service account.
The service account is created in the project which has Document AI enabled.

add kokoro tests

Document AI batch parse table (v1beta2): "before all" hook for "should parse the GCS invoice example as as table" failed

This test failed!

To configure my behavior, see the Flaky Bot documentation.

If I'm commenting on this issue too often, add the flakybot: quiet label and
I will stop commenting.

commit: 1c4ab7d
buildURL: Build Status, Sponge
status: failed

Test output

Required parameter: project
Error: Required parameter: project
    at new ApiError (node_modules/@google-cloud/common/build/src/util.js:73:15)
    at Util.parseHttpRespBody (node_modules/@google-cloud/common/build/src/util.js:208:38)
    at Util.handleResp (node_modules/@google-cloud/common/build/src/util.js:149:117)
    at /workspace/samples/node_modules/@google-cloud/common/build/src/util.js:477:22
    at onResponse (node_modules/retry-request/index.js:228:7)
    at /workspace/samples/node_modules/teeny-request/build/src/index.js:226:13
        -> /workspace/samples/node_modules/teeny-request/src/index.ts:333:11
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

Document AI batch parse form: "after all" hook for "should parse the GCS invoice example as a form" failed

This test failed!

To configure my behavior, see the Flaky Bot documentation.

If I'm commenting on this issue too often, add the flakybot: quiet label and
I will stop commenting.

commit: 1c4ab7d
buildURL: Build Status, Sponge
status: failed

Test output

The specified bucket does not exist.
Error: The specified bucket does not exist.
    at new ApiError (node_modules/@google-cloud/common/build/src/util.js:73:15)
    at Util.parseHttpRespBody (node_modules/@google-cloud/common/build/src/util.js:208:38)
    at Util.handleResp (node_modules/@google-cloud/common/build/src/util.js:149:117)
    at /workspace/samples/node_modules/@google-cloud/common/build/src/util.js:477:22
    at onResponse (node_modules/retry-request/index.js:228:7)
    at /workspace/samples/node_modules/teeny-request/build/src/index.js:226:13
        -> /workspace/samples/node_modules/teeny-request/src/index.ts:333:11
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

Synthesis failed for nodejs-document-ai

Hello! Autosynth couldn't regenerate nodejs-document-ai. 💔

Here's the output from running synth.py:

2020-10-08 03:10:16,319 autosynth [INFO] > logs will be written to: /tmpfs/src/logs/nodejs-document-ai
2020-10-08 03:10:17,029 autosynth [DEBUG] > Running: git config --global core.excludesfile /home/kbuilder/.autosynth-gitignore
2020-10-08 03:10:17,032 autosynth [DEBUG] > Running: git config user.name yoshi-automation
2020-10-08 03:10:17,036 autosynth [DEBUG] > Running: git config user.email [email protected]
2020-10-08 03:10:17,039 autosynth [DEBUG] > Running: git config push.default simple
2020-10-08 03:10:17,042 autosynth [DEBUG] > Running: git branch -f autosynth
2020-10-08 03:10:17,045 autosynth [DEBUG] > Running: git checkout autosynth
Switched to branch 'autosynth'
2020-10-08 03:10:17,051 autosynth [DEBUG] > Running: git clean -fdx
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 706, in <module>
    main()
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 541, in main
    return _inner_main(temp_dir)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 627, in _inner_main
    metadata = load_metadata(metadata_path)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 55, in load_metadata
    return json.loads(path.read_text())
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 45 column 3 (char 1036)

Google internal developers can see the full log here.

Document AI batch parse form: "before all" hook for "should parse the GCS invoice example as a form" failed

This test failed!

To configure my behavior, see the Flaky Bot documentation.

If I'm commenting on this issue too often, add the flakybot: quiet label and
I will stop commenting.

commit: 1c4ab7d
buildURL: Build Status, Sponge
status: failed

Test output

Required parameter: project
Error: Required parameter: project
    at new ApiError (node_modules/@google-cloud/common/build/src/util.js:73:15)
    at Util.parseHttpRespBody (node_modules/@google-cloud/common/build/src/util.js:208:38)
    at Util.handleResp (node_modules/@google-cloud/common/build/src/util.js:149:117)
    at /workspace/samples/node_modules/@google-cloud/common/build/src/util.js:477:22
    at onResponse (node_modules/retry-request/index.js:228:7)
    at /workspace/samples/node_modules/teeny-request/build/src/index.js:226:13
        -> /workspace/samples/node_modules/teeny-request/src/index.ts:333:11
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

Document AI batch parse form (v1beta2): "before all" hook for "should parse the GCS invoice example as a form" failed

This test failed!

To configure my behavior, see the Flaky Bot documentation.

If I'm commenting on this issue too often, add the flakybot: quiet label and
I will stop commenting.

commit: 1c4ab7d
buildURL: Build Status, Sponge
status: failed

Test output

Required parameter: project
Error: Required parameter: project
    at new ApiError (node_modules/@google-cloud/common/build/src/util.js:73:15)
    at Util.parseHttpRespBody (node_modules/@google-cloud/common/build/src/util.js:208:38)
    at Util.handleResp (node_modules/@google-cloud/common/build/src/util.js:149:117)
    at /workspace/samples/node_modules/@google-cloud/common/build/src/util.js:477:22
    at onResponse (node_modules/retry-request/index.js:228:7)
    at /workspace/samples/node_modules/teeny-request/build/src/index.js:226:13
        -> /workspace/samples/node_modules/teeny-request/src/index.ts:333:11
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

batch Processor with local files

Hi,

I am doing POC on document AI document processors.
Currently for batch processing sample is provided with use of google cloud storage.

I need to know if similar approach can be followed for batch processing local files or not.

Tried using similar request object as used in simple form processor with batch processor but getting "Request contains invalid argument ".
Using following with batch processor :
const request = {
name,
rawDocument: {
content: encodedImage,
mimeType: 'application/pdf',
},
};

Let me know if using similar request in batch processor is possible or we need to use google cloud storage for accessing files for batch processing with document Ai.

Regards

Document AI batch parse form (v1beta2): "after all" hook for "should parse the GCS invoice example as a form" failed

This test failed!

To configure my behavior, see the Flaky Bot documentation.

If I'm commenting on this issue too often, add the flakybot: quiet label and
I will stop commenting.

commit: 1c4ab7d
buildURL: Build Status, Sponge
status: failed

Test output

The specified bucket does not exist.
Error: The specified bucket does not exist.
    at new ApiError (node_modules/@google-cloud/common/build/src/util.js:73:15)
    at Util.parseHttpRespBody (node_modules/@google-cloud/common/build/src/util.js:208:38)
    at Util.handleResp (node_modules/@google-cloud/common/build/src/util.js:149:117)
    at /workspace/samples/node_modules/@google-cloud/common/build/src/util.js:477:22
    at onResponse (node_modules/retry-request/index.js:228:7)
    at /workspace/samples/node_modules/teeny-request/build/src/index.js:226:13
        -> /workspace/samples/node_modules/teeny-request/src/index.ts:333:11
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

googleapis / nodejs-document-ai Goto Github PK

nodejs-document-ai's Introduction

Quickstart

Before you begin

Installing the client library

Using the client library

Samples

Supported Node.js Versions

Versioning

Contributing

License

nodejs-document-ai's People

Contributors

Stargazers

Watchers

Forkers

nodejs-document-ai's Issues

Environment details

Steps to reproduce

Environment details

Steps to reproduce

Environment details

Steps to reproduce

Instructions

Required

Optional

Description

Workaround

Desired behavior

Environment details

Steps to reproduce

Additional info

Recommend Projects

Recommend Topics

Recommend Org