Giter Club home page Giter Club logo

microsoft-onedrive-component's Introduction

Microsoft OneDrive Component

Table of Contents

Description

Microsoft OneDrive Component is designed to manage files and folders in OneDrive service using Microsoft Graph API

Environment variables

Name Mandatory Description Values
API_RETRIES_COUNT false Set how many time system try to make request to API on server errors (3 by default) integer above 0 and below 5
API_REQUEST_TIMEOUT false HTTP requests timeout in milliseconds (15000 by default) integer above 500 and below 20000

API version

Current release of component tested on Graph API v1 (https://graph.microsoft.com/v1.0/)

Credentials

Microsoft OneDrive uses the OAuth 2.0. How to register an application look here. Redirect URI for platform is https://{your-tenant-address}/callback/oauth2

  • Type (dropdown, required) - OAuth2
  • Choose Auth Client (dropdown, required) - select one of created before or Add New Auth Client:
    • Name (string, required) - provide any name you want
    • Client ID (string, required) - put here Application (client) ID
    • Client Secret (string, required) - put here Client credentials
    • Authorization Endpoint (string, required) - OneDrive authorization endpoint https://login.microsoftonline.com/common/oauth2/v2.0/authorize
    • Token Endpoint (string, required) - OneDrive refresh token endpoint https://login.microsoftonline.com/common/oauth2/v2.0/token
  • Name Your Credential (string, required) - provide any name you want
  • Scopes (Space-separated list) (string, required) - Put here scopes to get access to your OneDrive - offline_access Files.ReadWrite.All

Actions

Create Folder

Create new folder in provided path. If path not exist component will fail.

Configuration Fields

  • Drive Identity - (dropdown, required): OneDrive instance to work with
  • Conflict Behavior - (dropdown, optional, Fail by default) - Select one of options to handle case when folder already exists:
    • Fail - Fails if folder with same name already exists under provided path
    • Generate new name - If folder with same name already exists under provided path then will be created new folder with different name. Examples: folder_name (already exist) -> folder_name 1 (will be created), folder_name 1 (already exist) -> folder_name 1 1 (will be created)
    • Upsert - If folder already exist, you will get information about this folder

Input Metadata

  • Path - (string, required): Full path to folder where you need to create new folder, ex: Monthly reports/November
  • Name - (string, required): Name of new folder

Output Metadata

Metadata of created folder

Delete File

Action to delete item from OneDrive by provided path in selected disc.

Configuration Fields

  • Drive Identity - (dropdown, required): OneDrive instance to work with
  • Emit strategy when file not found - (dropdown, optional, Emit nothing by default) - select one of options to handle case when file not exist:
    • Emit nothing - component will not produce any messages
    • Emit an empty object - result will be empty object: {}
    • Throw an error - error will be thrown

Input Metadata

  • Path - (string, required): Full path to item, ex: Monthly reports/November/Cars sales.pdf

Output Metadata

  • Path - (string, required): Full path to item, ex: Monthly reports/November/Cars sales.pdf

Get File

Lookup a single file by its path.

Configuration Fields

  • Drive Identity - (dropdown, required): OneDrive instance to work with
  • Enable File Attachments - (checkbox, optional): If checked, file will be uploaded to local storage and link provided in response

Input Metadata

  • Path - (string, required): Full path to item, ex: Monthly reports/November/Cars sales.pdf

Output Metadata

File information as JSON object, if Enable File Attachments checked, there also will be additional field attachmentUrl with link to file on platform

Upsert File

Updates (if record exist) or creates a new file

Configuration Fields

  • Drive Identity - (dropdown, required): OneDrive instance to work with
  • Upload single file - (checkbox, optional): Use this option if you want to upload a single file

Input Metadata

If Upload single file checked, there will be 2 fields:

  • URL - (string, required): link to file on Internet or platform
  • Path - (string, required): Full path to item on OneDrive, ex: Monthly reports/November/Cars sales.pdf

If Upload single file unchecked:

  • Files - (array, required): Collection of files to upload, each record contains object with two keys:
    • URL - (string, required): link to file on Internet or platform
    • Path - (string, required): Full path to item on OneDrive, ex: Monthly reports/November/Cars sales.pdf

Output Metadata

Result object from upsert.

Triggers

Get New and Updated Objects Polling

Retrieve all the updated or created objects within a given time range.

Configuration Fields

  • Drive Identity - (dropdown, required): OneDrive instance to work with
  • Folders path - (multiselect dropdown, required): Select folders to follow
  • Include subfolders - (checkbox, optional): If checked, trigger will follow to each subfolder of selected folder.
    ❗Note: this will increase the number of API calls
  • Enable File Attachments - (checkbox, optional): If checked, file will be uploaded to local storage and link provided in response
  • Time stamp field to poll on - (dropdown, optional, default Last Modified): Select which date will be used to track files - Last Modified or Created
  • Emit Behavior - (dropdown, optional, default Emit individually): Defines the way result objects will be emitted, one of Emit page or Emit individually.
  • Page Size - (number, optional, defaults to 999, max 999): Indicates the size of pages to be fetched per request
  • Start Time - (string, optional): The timestamp to start polling from (inclusive) - using ISO 8601 Date time utc format - YYYY-MM-DDThh:mm:ssZ. Default value is the beginning of time (January 1, 1970 at 00:00).
  • End Time - (string, optional): The timestamp to stop polling (exclusive) - using ISO 8601 Date time utc format - YYYY-MM-DDThh:mm:ssZ. Default value is flow execution time.

Input/Output Metadata

None.

Output Metadata

Depends on Enable File Attachments and Emit behavior fields.

  • If Emit behavior field is equal to Emit page - object with property results that contains array of files
  • If Emit behavior field is equal to Emit individually, file information will fulfill whole message
  • If Enable File Attachments checked, for each file there will be additional field - attachmentUrl

Limitations

  • OneDrive API doesn't support filtering - as result we collect information about all files from selected folders and filter them locally (inside component) for each trigger execution
  • From point above, option Emit page not always emit records according to Page Size
  • Include subfolders increase amount of API calls - additional call for each subfolder and inner folders

microsoft-onedrive-component's People

Contributors

denyshld avatar emptyinfinity avatar if0s avatar jhorbulyk avatar olegosh avatar shkarupanick avatar stas-fomenko avatar zubairov avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Forkers

mitulshahcg

microsoft-onedrive-component's Issues

Add error handling for `Start Time` and `End Time` in `Get New...` trigger

Description:

  • 1. Trigger suppose to support following min and max values (see readme):
**Start Time** - Start datetime of polling. Default min date: `-271821-04-20T00:00:00.000Z`.
* **End Time** - End datetime of polling. Default max date: `+275760-09-13T00:00:00.000Z`.

So in case when expected values are violated by actulal values - error is expected

  • 2. Error is also expected when End Time < Start Time

STR for case 1:

  1. Input
    image
  2. Retrieve sample

AR: all files specified by path are retrieved

ER: error regarding invalid input values thrown


STR for case 2.

  1. Input
    image

  2. Retrieve sample

AR: all files specified by path are retrieved

ER: error thrown that End Time must not be less then Start Time

Refresh token lifetime investigation

Description:
Need to find proper application settings of token expiration on microsoft graph and set them in order to make token lifetime longer as much as it possible

STR:

  1. Create a draft with any microsoft-onedrive-component trigger/action
  2. Refresh valid creds
  3. Leave draft as is for some time (3600sec +) and do not publish it.
  4. Continue draft configuration

AR: Credentials expired: 401
ER: Credentials are not expired for a long time (time need to be discussed), as the current component behavior makes flow developing very painful

Deployment is failing - Timeout of 2000ms exceeded.

Component is not possible to deploy into more restricted systems. It fails the tests:

       1) Upsert File
       Returns response for uploaded file:
       Error: Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/tmp/build/spec/actions/upsertFile.spec.js)
       at listOnTimeout (internal/timers.js:549:17)
       at processTimers (internal/timers.js:492:7)



remote: npm ERR! Test failed.  See above for more details.

-----> Build failed

       We're sorry this build is failing! If you can't find the issue in component code,
       please submit a ticket so we can help: https://support.elastic.io
       You can also try upgrading Sailor to the most recent version. Check out
       https://www.npmjs.com/package/elasticio-sailor-nodejs

       Your elastic.io team
remote:
remote:
remote: Removing build container and slug
remote: Saving build
remote: Detaching from log stream
remote: Build failed
remote: Error: Appbuilder failed with code 1
remote: Error: Appbuilder failed with code 1
remote:     at ChildProcess.<anonymous> (/home/node/gitreceiver/src/lib/builder.js:74:31)
remote:     at ChildProcess.emit (events.js:310:20)
remote:     at Process.ChildProcess._handle.onexit (internal/child_process.js:275:12)
remote: {"name":"gitreceiver","hostname":"gitreceiver-7b85ccd445-xv2kk","pid":"4576","level":"50","err":{"message":"Appbuilder failed with code 1","name":"Error","stack":"Error: Appbuilder failed with code 1\n    at ChildProcess.<anonymous> (/home/node/gitreceiver/src/lib/builder.js:74:31)\n    at ChildProcess.emit (events.js:310:20)\n    at Process.ChildProcess._handle.onexit (internal/child_process.js:275:12)"},"msg":"Appbuilder failed with code 1","time":"2020-06-08T15:02:22.946Z","v":"0"}
To app-msp-test:ms-onedrive.git
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to '****@****:ms-onedrive.git'

Upsert File action fails with 500 when attachment for upsert ~4MB

Description: Attachment (3711661 bytes) may be successfully upserted, but attacmpt to upsert a bit bigger attachment (4194304 bytes) causes an error.

STR:

  1. Use ANY trigger action that is able to pass attachment to the next step (Onedrive Get FIle)
  2. Pass an attachment ~3711661 bytes
  3. Add next step with Upsert File action (select available drive and set pass like: { "path": "UPSERTED file.csv"}) and retrieve sample.
  4. Get a sample and find a file by specified drive and path.
  5. Repeate steps 1 -4 for a bigger file ~4194304 bytes.
  6. Get an error thrown:
    Error! Error in making request to https://graph.microsoft.com/v1.0/drives/7161fc17af0d3ce4/root:/test_big_files/UPSERTED 4_194_304 _bytes.csv:/content Status code: 500, Body: {"error":{"code":"InternalServerError","message":"Stream read failure","innerError":{"date":"2021-03-23T14:13:42","request-id":"d30f46bc-81d1-4009-99eb-f9ae7a5cc183","client-request-id":"d30f46bc-81d1-4009-99eb-f9ae7a5cc183"}}}

AR: error thrown when file size is ~4 MB
ER: no error, attachment saved to Onedrive

`path` should not be required field in `Create folder` action

Description:
In designer mode user cannot create folder in root (My files)
image

But, in dev mode it is possible

{
  "name": "create_Folder",
  "path": ""
}

So, it seems that path should not be required.

However, if path completely missed in input:

{
  "name": "create_Folder"
}

error is thrown: Error! Error: Path: "undefined" is not exist in drive: 7161fc17af0d3ce4. This case is also should be handled: when no path in input - folder should be created in root (My files)

Bug: Modified/Order By Error

Bug occurs when making a standard request to get files from a folder.

Status code: 501, Body: {"error":{"code":"notSupported","message":"The property Modified is not supported for OrderBy.","innerError":{"date":"2020-07-20T13:27:24","request-id":"935c055d-2809-4dda-ae4f-77b866150e48"}}} at OneDriveRestClient.handleRestResponse (/home/apprunner/lib/utils/OneDriveRestClient.js:50:17) at OneDriveRestClient.makeRequest (/home/apprunner/lib/utils/OneDriveRestClient.js:42:21) at processTicksAndRejections (internal/process/task_queues.js:97:5) { status: 501, data: { error: { code: 'notSupported', message: 'The property Modified is not supported for OrderBy.', innerError: [Object] } } } Error in making request to https://graph.microsoft.com/v1.0/drives/b!XXX/items/root/children?$orderby=lastModifiedDateTime Status code: 501, Body: {"error":{"code":"notSupported","message":"The property Modified is not supported for OrderBy.","innerError":{"date":"2020-07-20T13:27:24","request-id":"935c055d-2809-4dda-ae4f-77b866150e48"}}}

https://app.elastic.io/c/5c9a2c0b938cbf0011eb6916#/w/5d6d3876cb5a28b8062c5c18/designer/5eb16b44f7a49069d64e5dd5/draft

Component interacts with attachments regardless of the value of the `ATTACHMENT_MAX_SIZE` variable

Description: ATTACHMENT_MAX_SIZE may be set as 1,100, 1500, however user is still able to Get file 2+ MB with trigger and action (with help of Enable File Attachments option), Upsert File (also with help of Enable File Attachments option), so it seems that this environment variable is broken.

Not sure if this variable would be needed in case if Maester will be used here,if so - this bug should be fixed, if no - variable should be removed from code and readme

UPD: seems that this was added into readme by mistake as there is no such variable in code. So going to remove this from readme. Such variable may be implemented if necessary

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.