radumg / dynaweb Goto Github PK

DynaWeb is a Dynamo package providing support for interaction with the interwebz in general and with REST APIs in particular.

Home Page: https://radumg.github.io/DynaWeb/

License: GNU Affero General Public License v3.0

C# 100.00%

dynamods rest-api csharp

dynaweb's Introduction

DynaWeb

DynaWeb is a Dynamo package providing support for interaction with the interwebz in general and with REST APIs in particular.

What does it do ?

It helps you

retrieve (GET) stuff from the web
send (POST) information to the web
everything in between (PUT, DELETE, PATCH, etc).
the package also includes some rather nifty JSON de/serialisation nodes, so you can use the information you get from the web directly in Dynamo graphs, as native types.

How it came about

DynaWeb was designed as a package to make other packages, so it provides building blocks enabling you to build Dynamo integrations with just about any web service out there. After making DynaSlack & DynAsana, it became clear that writing a ZeroTouch-based package for every web service I or the community would want to integrate with was simply not scalable or sustainable, no matter how much code was re-used. DynAsana is an abstracted DynaSlack and DynaWeb is an even more abstracted & modularised DynAsana.

Getting Started

Package manager

DynaWeb is now available on the Dynamo package manager, search for DynaWeb and install it from there. See Alternative installation methods at the end of this document for alternative install methods.

Using DynaWeb

This repository has quite a few sample files provided to help you get started with DynaWeb.

I highly recommed starting with the samples as they contain detailed notes and instructions on how to use each of the nodes. Feel free to open an issue or submit a PR if you'd like to see further some documentation added here.

Samples

There are 8 sample Dynamo graphs included with the package, provided in both Dynamo 1.3 XML file format and Dynamo 2.0's new JSON format.

You can find the samples in this repository's samples folder folder, as well as and in the extra folder of the package you download using the Dynamo Package Manager, typically found here : %appdata%\Dynamo\Dynamo Revit\1.3\packages\DynaWeb (note version and Revit flavour of Dynamo, your location may vary).

The samples start from super-simple and progressively increase in complexity :

Sample 1 - A first request

3 nodes, similar to out-of-the-box (OOTB) Dynamo experience today.

Sample 2 - A simple request

Introduces the 3 stages of performing web requests and explains quite a few things. Also show how to achieve same thing with the OOTB node.

Sample 3 - Requst + benchmarking

Same as sample 2 but with added nodes that provide more information about the request (timing, etc) and output the results to text files.

Sample 4 - REST API example

This introduces the use of the WebClient class and some of the basic priciples of interacting with REST services. Uses a REST API that is freely accessible and returns JSON reponses. Contrasts using a WebClient and a WebRequest to achieve same thing and also introduces Deserialisation.

Sample 5 - REST API advanced

Introduces POST-ing to a REST API service and handling JSON payloads. Once the request is submitted, the response is deserialised too.

Sample 6 - Complex POST request

Further expands on the above example, building a complex WebRequest with 6 steps before its execution.

Sample 7 - Autodesk Forge - Upload file

This example builds a WebRequest and attaches a file to it, to upload directly to the Autodesk Forge service. See the issue that sparked this sample here.

Sample 8 - Autodesk Forge - Request auth token

This example builds a POST WebRequest, used to request an authorisation token from the Autodesk Forge service. See the issue that sparked this sample here.

Structure

There's 5 main components in DynaWeb :

WebRequest : the web request that gets executed
WebClient : the context in which a request is executed
WebResponse : this contains the response from the server, as well as additional metadata about the response & server itself
Execution : this provides nodes that simply execute requests, making it easier & clearer to use standard http verbs such as GET, POST, etc.
Helpers : a few helper nodes, with a particular focus on Deserialisation.

Simply put, use WebRequest nodes for one-off requests and start using a WebClient when you are interacting with REST APIs and/or have multiple request to similar endpoints/URLs. When using a WebClient, the WebRequest is still what gets executed, but it allows you more control over how that occurs (custom timeouts, etc)

Fun facts

when executing a WebRequest on its own, the DynaWeb package constructs an empty WebClient in the background anyway as it's needed for execution
the strucuture of the source code shows up directly in Dynamo

Alternative installation methods

Manual install

If you prefer to install one of the more experimental/work-in-progress builds, you can still follow the instructions below.

Download the latest release from the Releases page
unzip the downloaded file
once unzipped, copy the DynaWeb folder to the location of your Dynamo packages :
- %appdata%\Dynamo\Dynamo Core\1.3\packages for Dynamo Sandbox, replacing 1.3 with your version of Dynamo
- %appdata%\Dynamo\Dynamo Revit\1.3\packages for Dynamo for Revit, replacing 1.3 with your version of Dynamo
start Dynamo, the package should now be listed as DynWWW in the library.

Still can't see the package in Dynamo ?

This issue should be fixed now the package is distributed through the package manager, I definitely recommending getting it that way. However, in case you still have issues, see instructions below :

As reported by users, Windows sometimes blocks .dll files for security reasons. To resolve this, you'll have to go through the steps below for each assembly (.dll file) in the package :

Right-click on .dll file and select properties
Tick the Unblock checkbox at the bottom, in the Security section.
Launch Dynamo again, the package should now load.

Updating from alpha-0.5 build ?

The changes in 1.0 are breaking, meaning graphs using the previous version will not work. However, instead of re-creating them, you can simply open the .dyn files using Notepad (though i recommend SublimeText) and perform the following text find/replaces :

replace DSCore.Web. with DynaWeb.
replace DynWWW.dll with DynaWeb.dll
replace WebClient.WebClient with WebClient.ByUrl

Prerequisites

This project requires the following applications or libraries be installed :

Dynamo : version 1.3 or later

.Net : version 4.5 or later

Please note the project has no dependency to Revit and its APIs, so it will happily run in Dynamo Sandbox or Dynamo Studio.

Built with

The DynaWeb project relies on a few community-published NuGet packages as listed below :

Newtonsoft - handles serializing and deserializing to JSON
RestSharp - enables easier interaction with REST API endpoints
DynamoServices - an official Dynamo package providing support for better mapping of C# code to Dynamo nodes

Contributing

Please read CONTRIBUTING.md for details on how to contribute to this package. Please also read the CODE OF CONDUCT.md.

Authors

Radu Gidei : Github profile, Twitter profile

License

This project is licensed under the GNU AGPL 3.0 License - see the LICENSE FILE for details.

dynaweb's People

Contributors

Stargazers

Watchers

Forkers

22mn alexandrpopov maurosabiu threepointsquare shknudsen albertotono ianmcgaw tuulay mengelmn chuongmep benrobinson20 daltongoo kumarravi007 thebimsider

dynaweb's Issues

add simple Request DYN sample

Expected Behavior

sample to showcase a simple request, fully annotated.

Actual Behavior

got nothing yet.

Merge old DynWWW repo with all source code

Expected Behavior

This repo should contain the full source code for DynaWeb

Actual Behavior

This repo has no source code.
The actual C# source code for the DynaWeb package was in a private repository called DynWWW

Steps

deduplicate DynWWW repo docs & files
Merge DynWWW repo in here & archive it
Rename the solution without needing version bump
Update CI (Travis) & CodeClimate settings

Web services you'd like to use with Dynamo

Please add your comments below with what web services you'd like to see support for in Dynamo, so :

we can test if this package supports that out-of-the-box
see if there's any improvements that can be made to add support if not

It would be great if aside from the name of the service, you included

a link 🌐
maybe a 1 sentence description too

There's no right/wrong answer, but I'm more interested in services that are used frequently & actively in AEC, so please bear in mind testing of those will be prioritised.

Switch to static for overloaded methods

Expected Behavior

If a WebRequest is modified through 3 daisy-chained nodes that Add/Set some of its properties, the result after each stage should be different.
Same goes for WebClient class.

Actual Behavior

What happens currently is the above will be true on the first run of the graph, but in subsequent runs each node will return the latest version of the object. This is because the nodes are essentially returning pointers to their shared/common instance instead of the objects themselves.

Add Trello sample

As reported by @VVlcev here : #1 (comment)

add REST API DYN sample

Expected Behavior

provide a DYN sample that showcases interaction with a REST API and includes :

use of WebClient
an endpoint that returns an array JSON-encoded objects
Deserialisation of array and of individual records

Actual Behavior

have a sample that is not documented consistently and is not commited to repo.

Wish: Support for deserializing JSON containing multiple items

Expected Behavior

Hey Radu, my webservice returns multiple items for some requests, e.g.:

{
   "property1":"ABC",
   "property2":"DEF"
},
{
   "property1":"GHI",
   "property2":"JKL"
}

What I would like to see the Helpers.Deserialize nodes return is something like this:

Actual Behavior

Currently, the Helpers.Deserialize nodes cannot deal with that and throw this warning:

Additional text encountered after finished reading JSON content: ,. Path '', line 4, position 1.

Steps to Reproduce the Problem

Copy sample JSON from above
Paste into String Input node
Connect to Helpers.DeserializeAsDictionary
Run

Specifications

- Dynamo : 1.3.3
- Operating system : Windows 10
- .NET :  4.7.2

Deserialise method names inconsistent

Expected Behavior

Class & method names to be consistent.

Actual Behavior

Some classes use deserialisation whilst others use deserialization

Content-Type header ignored (UnsupportedMediaType returned)

Expected Behavior

I'm trying to use Dynaweb to talk to a RESTheart docker instance, in order to read/write to a mongoDB.

Following this (little outdated) tutorial, I'm trying to create a collection via
PUT http://localhost:8080/restheart/movies with JSON body {"desc": "my favorite movies"}.

The equivalent (working) curl call is:

curl -i -H "Content-Type: application/json" -X PUT http://localhost/restheart/movies -d '{"desc": "my favorite movies"}'

I'm expecting that using the example 5 and changing the url, endpoint, json string and method would work without issues.

Actual Behavior

The WebResponse returns a 415 UnsupportedMediaType status code, with content

{
  "http status code": 415,
  "http status description": "Unsupported Media Type",
  "message": "Content-Type must be either: application/hal+json or application/json"
}

It seems that Content-Type is not correctly passed as header.

Steps to Reproduce the Problem

open the example 5 in dynamo studio (I also changed the execution type to manual)
change the url to http://localhost:8080/restheart (the url pointing to the restheart instance+database name)
change the endpoint to movies
change the json string to {"desc": "my favorite movies"}
change the metod to PUT
run the project

Specifications

- DynaWeb: 1.0.3
- Dynamo : Dynamo Studio 1.3.3.4111
- Operating system :  Windows 10 1809
- .NET :  4.6.2

Auth Request: POST-data:write

Expected Behavior

Auth Request: Data Write

Actual Behavior

Hey! I would like to generate my data-Write Token using a Post Request but I'm encountering an error "ERR-002: The Requested resource does not exist".

Steps to Reproduce the Problem

In Postman I usually do something like the screenshot.

Following your "Post example" I'm replicating it.. but without success:

Any ideas on how I could make it work?
Thanks a lot!!

Specifications

- Dynamo :  1.3.1

add complex POST request DYN sample

Expected Behavior

a more complex DYN sample that builds and executes a request :

POST method
with Headers
with Cookies
with JSON body

Actual Behavior

got nothing.

add documentation to all DYN samples

Expected Behavior

all sample DYN files should be documented consistently and rather comprehensively

Actual Behavior

only 1 file is documented (REST API sample)

Add unit tests

From @radumg on August 8, 2017 23:15

Expected Behavior

Unit tests that follow the Dynamo repository guidelines

Actual Behavior

No tests

Copied from original issue: radumg/DynWWW#11

Add support for web sockets

Real-time services require a more efficient communication protocol than restful web requests.
Look into google protocol buffers as well

Verbosity of Nodes

From @radumg on December 24, 2017 15:35

Expected Behavior

Users should :

be able to build & execute web requests in as few nodes as possible
not need extensive background knowledge on how web requests work (shallow learning curve)

Actual Behavior

The bad bits

The current implementation of DynaWeb :

uses a builder pattern (albeit incomplete to be simpler)
is quite verbose in that it exposes many methods to progressively construct a web request (AddHeader, AddBody, AddParameter, etc)
as such, complex requests can require many nodes to build (see sample 8 for a Forge request)
assumes the complexity of a request will be hidden in a custom node (package is designed to be used to make specific integration packages).

The good bits

The current implementation of DynaWeb :

matches closely the standard terminology used with web requests in public documentation (wiki, mozzila dev docs, etc) or the terminology & steps presented in middleman apps for interacting with the web such as Postman
the C# code generator in Postman translates 1-to-1 to steps required to build a request with DynaWeb
highly flexible library, allowing you to build very specific request structures
the structure of the codebase maps nicely to a Dynamo package
once the terminology of web requests is known, usage of the package is quite easy & clear

Discuss

This issue is meant to be a place to discuss how the package verbosity should be tackled in the context of integrating it into the Dynamo Core Library. Screenshots, ideas, options, every suggestion is welcomed :)

Copied from original issue: radumg/DynWWW#40

WebRequest.AddParameter not parsing through list of values

Expected Behavior

add a parameter for each item of the list

Actual Behavior

repeats that first item of the list any number of time instead of going through the list

Specifications

- Dynamo : 2.2.1.5175
- Operating system : Win 10 Pro

Caching & repeated execution

From @radumg on December 24, 2017 17:23

Expected Behavior

Executing a graph with DynaWeb nodes a second (or Nth) time should consistently produce the same result.

Actual Behavior

Repeated execution of the same graph can have unintended consequences :

may send malformed requests (doubled up headers for example)
fail to send an identical request a second time.

Main reason for this is Dynamo's caching of node results.
Note that I've successfully tested a workaround : using the ProRUBIM node that forces execution of children nodes. This suggests that these shortcomings can be tackled by using NodeModels so a discussion around this is needed.

Copied from original issue: radumg/DynWWW#41

I am struggling to pass a body with my POST request

Expected Behavior

What I submit a POST to my login API using the WebRequest.AddParameter node and supplying a string node with my JSON body, I would expect that my parameters email_address and password are passed in the body of my request and that my response will be as I see in postman or in my application.

Actual Behavior

What I am seeing instead is that my parameters email_address and password are not passed and my request return an error stating that my email_address parameter's value is undefined.

Steps to Reproduce the Problem

I've attached the .dyn file that I'm using here, and copied the contents of the string node that I'm using with for the contents of the Body for my request.

Get Data from API_post.zip

I would imagine that I simply have a syntax issue, potentially with my quotes or apostrophes.

As a note, I am using this login process to receive a JWT so that I can pass it with my other requests. If I log in via Postman and copy the contents of my JWT into my GET request (not included in this file), I am able to return a 200 response with my data as-expected. The bottleneck I seem to be having here is restricted to how I properly pass a JSON body with my POST request.

Specifications

- Dynamo : 2.3
- Operating system : Windows 10 (1809)
- .NET :  4.8

Add HTML parser

Implement missing setters for WebClient

Expected Behavior

All WebClient properties that are not intended to be read-only (such as timeout, etc) should have a node that allows setting their value.

Actual Behavior

The WebClient has a properties that are exposed as part of query ? nodes but there's no way to set them using action nodes.

update samples to new build

Expected Behavior

sample DYN files should be compatible with the latest build

Actual Behavior

samples still reference deprecated WebRequest constructor nodes, etc.

Add unit testing infrastructure

From @radumg on August 8, 2017 23:16

Expected Behavior

Reproducing the set-up for the Dynamo repo, using NUnit

Actual Behavior

No unit testing infrastructure currently.

Copied from original issue: radumg/DynWWW#12

Dictionary not correctly deserialized

Expected Behavior

The SerializeToJSON (and the WebRequest.AddJsonBody) node should convert Dynamo dictionary to JSON objects.

Actual Behavior

Specifications

- Dynamo : 2.3.0
- Operating system : Windows 10

Nodes used in samples reference assembly from missing location

Expected Behavior

Expected Node to be resolved (available)

Actual Behavior

Node's path points to a non existing location (C:\Users*username*\AppData\Roaming\Dynamo\Dynamo Core\Desktop\testing\DynWWW.dll)

Steps to Reproduce the Problem

Open DynaWeb.dyn sample

Specifications

- Dynamo : (ex : 1.3.1)
- Operating system : Windows 10
- .NET : 4.6

Best practice for storing (secret) passwords?

I have a few questions related to handling personal passwords.
What is the best way to save personal passwords?
Many web APIs require a user to log in with their personal user and code. How do I ensure that the code is handled confidentially in Dynamo?
For some API we might have a company password that I wnat to hide from the end user of my Dynamo nodes/ graphs. What is the best way to do this?

Add Jira sample

As reported by @josols here : #1 (comment)

Package does not show in Dynamo due to Windows security

Expected Behavior

Following the instructions, users should see the package loaded in Dynamo

Actual Behavior

Dynamo does not load the package because the DLL is blocked by Windows.

Reported steps to fix the problem

For each assembly (.dll file) in the package :

Right-click on assembly and select properties
Tick the Unblock checkbox at the bottom, in the Security section.
Launch Dynamo again, the package should load.

Specifications

- Dynamo : presumed 1.3
- Operating system : Windows 10 Pro

Report

As originally reported by @3sk0r

I only got it to work after unblocking all files that came with the package. (right click > Properties)
.
Maybe an issue with Windows 10's Windows Defender? Might want to mention this in the README.md

Upload .rvt file ( PUT method)

Expected Behavior

I would want to upload a .rvt file on a bucket hosted online.

Actual Behavior

I'm using PUT method. I can add Headers and Parameters to the URL but I'm not sure how upload the Revit file.

Steps to Reproduce the Problem

The WebRequest.AddBody doesn't let me specify the type ( "binary") of the model: any suggestion?

Specifications

- Dynamo : 1.3