Giter Club home page Giter Club logo

resty's Introduction

Resty

Simple HTTP and REST client library for Go (inspired by Ruby rest-client)

Features section describes in detail about Resty capabilities

Build Status Code Coverage Go Report Card Release Version GoDoc License Mentioned in Awesome Go

Resty Communication Channels

Chat on Gitter - Resty Community Twitter @go_resty

News

  • v2.13.1 released and tagged on May 10, 2024.
  • v2.0.0 released and tagged on Jul 16, 2019.
  • v1.12.0 released and tagged on Feb 27, 2019.
  • v1.0 released and tagged on Sep 25, 2017. - Resty's first version was released on Sep 15, 2015 then it grew gradually as a very handy and helpful library. Its been a two years since first release. I'm very thankful to Resty users and its contributors.

Features

  • GET, POST, PUT, DELETE, HEAD, PATCH, OPTIONS, etc.
  • Simple and chainable methods for settings and request
  • Request Body can be string, []byte, struct, map, slice and io.Reader too
    • Auto detects Content-Type
    • Buffer less processing for io.Reader
    • Native *http.Request instance may be accessed during middleware and request execution via Request.RawRequest
    • Request Body can be read multiple times via Request.RawRequest.GetBody()
  • Response object gives you more possibility
    • Access as []byte array - response.Body() OR Access as string - response.String()
    • Know your response.Time() and when we response.ReceivedAt()
  • Automatic marshal and unmarshal for JSON and XML content type
  • Easy to upload one or more file(s) via multipart/form-data
    • Auto detects file content type
  • Request URL Path Params (aka URI Params)
  • Backoff Retry Mechanism with retry condition function reference
  • Resty client HTTP & REST Request and Response middlewares
  • Request.SetContext supported
  • Authorization option of BasicAuth and Bearer token
  • Set request ContentLength value for all request or particular request
  • Custom Root Certificates and Client Certificates
  • Download/Save HTTP response directly into File, like curl -o flag. See SetOutputDirectory & SetOutput.
  • Cookies for your request and CookieJar support
  • SRV Record based request instead of Host URL
  • Client settings like Timeout, RedirectPolicy, Proxy, TLSClientConfig, Transport, etc.
  • Optionally allows GET request with payload, see SetAllowGetMethodPayload
  • Supports registering external JSON library into resty, see how to use
  • Exposes Response reader without reading response (no auto-unmarshaling) if need be, see how to use
  • Option to specify expected Content-Type when response Content-Type header missing. Refer to #92
  • Resty design
    • Have client level settings & options and also override at Request level if you want to
    • Request and Response middleware
    • Create Multiple clients if you want to resty.New()
    • Supports http.RoundTripper implementation, see SetTransport
    • goroutine concurrent safe
    • Resty Client trace, see Client.EnableTrace and Request.EnableTrace
      • Since v2.4.0, trace info contains a RequestAttempt value, and the Request object contains an Attempt attribute
    • Debug mode - clean and informative logging presentation
    • Gzip - Go does it automatically also resty has fallback handling too
    • Works fine with HTTP/2 and HTTP/1.1
  • Bazel support
  • Easily mock Resty for testing, for e.g.
  • Well tested client library

Included Batteries

  • Redirect Policies - see how to use
    • NoRedirectPolicy
    • FlexibleRedirectPolicy
    • DomainCheckRedirectPolicy
    • etc. more info
  • Retry Mechanism how to use
    • Backoff Retry
    • Conditional Retry
    • Since v2.6.0, Retry Hooks - Client, Request
  • SRV Record based request instead of Host URL how to use
  • etc (upcoming - throw your idea's here).

Supported Go Versions

Recommended to use go1.16 and above.

Initially Resty started supporting go modules since v1.10.0 release.

Starting Resty v2 and higher versions, it fully embraces go modules package release. It requires a Go version capable of understanding /vN suffixed imports:

  • 1.9.7+
  • 1.10.3+
  • 1.11+

It might be beneficial for your project 😄

Resty author also published following projects for Go Community.

  • aah framework - A secure, flexible, rapid Go web framework.
  • THUMBAI - Go Mod Repository, Go Vanity Service and Simple Proxy Server.
  • go-model - Robust & Easy to use model mapper and utility methods for Go struct.

Installation

# Go Modules
require github.com/go-resty/resty/v2 v2.11.0

Usage

The following samples will assist you to become as comfortable as possible with resty library.

// Import resty into your code and refer it as `resty`.
import "github.com/go-resty/resty/v2"

Simple GET

// Create a Resty Client
client := resty.New()

resp, err := client.R().
    EnableTrace().
    Get("https://httpbin.org/get")
curlCmdExecuted := resp.Request.GenerateCurlCommand()


// Explore curl command
fmt.Println("Curl Command:\n  ", curlCmdExecuted+"\n")

// Explore response object
fmt.Println("Response Info:")
fmt.Println("  Error      :", err)
fmt.Println("  Status Code:", resp.StatusCode())
fmt.Println("  Status     :", resp.Status())
fmt.Println("  Proto      :", resp.Proto())
fmt.Println("  Time       :", resp.Time())
fmt.Println("  Received At:", resp.ReceivedAt())
fmt.Println("  Body       :\n", resp)
fmt.Println()

// Explore trace info
fmt.Println("Request Trace Info:")
ti := resp.Request.TraceInfo()
fmt.Println("  DNSLookup     :", ti.DNSLookup)
fmt.Println("  ConnTime      :", ti.ConnTime)
fmt.Println("  TCPConnTime   :", ti.TCPConnTime)
fmt.Println("  TLSHandshake  :", ti.TLSHandshake)
fmt.Println("  ServerTime    :", ti.ServerTime)
fmt.Println("  ResponseTime  :", ti.ResponseTime)
fmt.Println("  TotalTime     :", ti.TotalTime)
fmt.Println("  IsConnReused  :", ti.IsConnReused)
fmt.Println("  IsConnWasIdle :", ti.IsConnWasIdle)
fmt.Println("  ConnIdleTime  :", ti.ConnIdleTime)
fmt.Println("  RequestAttempt:", ti.RequestAttempt)
fmt.Println("  RemoteAddr    :", ti.RemoteAddr.String())

/* Output
Curl Command:
  curl -X GET -H 'User-Agent: go-resty/2.12.0 (https://github.com/go-resty/resty)'  https://httpbin.org/get

Response Info:
  Error      : <nil>
  Status Code: 200
  Status     : 200 OK
  Proto      : HTTP/2.0
  Time       : 457.034718ms
  Received At: 2020-09-14 15:35:29.784681 -0700 PDT m=+0.458137045
  Body       :
  {
    "args": {},
    "headers": {
      "Accept-Encoding": "gzip",
      "Host": "httpbin.org",
      "User-Agent": "go-resty/2.4.0 (https://github.com/go-resty/resty)",
      "X-Amzn-Trace-Id": "Root=1-5f5ff031-000ff6292204aa6898e4de49"
    },
    "origin": "0.0.0.0",
    "url": "https://httpbin.org/get"
  }

Request Trace Info:
  DNSLookup     : 4.074657ms
  ConnTime      : 381.709936ms
  TCPConnTime   : 77.428048ms
  TLSHandshake  : 299.623597ms
  ServerTime    : 75.414703ms
  ResponseTime  : 79.337µs
  TotalTime     : 457.034718ms
  IsConnReused  : false
  IsConnWasIdle : false
  ConnIdleTime  : 0s
  RequestAttempt: 1
  RemoteAddr    : 3.221.81.55:443
*/

Enhanced GET

// Create a Resty Client
client := resty.New()

resp, err := client.R().
      SetQueryParams(map[string]string{
          "page_no": "1",
          "limit": "20",
          "sort":"name",
          "order": "asc",
          "random":strconv.FormatInt(time.Now().Unix(), 10),
      }).
      SetHeader("Accept", "application/json").
      SetAuthToken("BC594900518B4F7EAC75BD37F019E08FBC594900518B4F7EAC75BD37F019E08F").
      Get("/search_result")


// Sample of using Request.SetQueryString method
resp, err := client.R().
      SetQueryString("productId=232&template=fresh-sample&cat=resty&source=google&kw=buy a lot more").
      SetHeader("Accept", "application/json").
      SetAuthToken("BC594900518B4F7EAC75BD37F019E08FBC594900518B4F7EAC75BD37F019E08F").
      Get("/show_product")


// If necessary, you can force response content type to tell Resty to parse a JSON response into your struct
resp, err := client.R().
      SetResult(result).
      ForceContentType("application/json").
      Get("v2/alpine/manifests/latest")

Various POST method combinations

// Create a Resty Client
client := resty.New()

// POST JSON string
// No need to set content type, if you have client level setting
resp, err := client.R().
      SetHeader("Content-Type", "application/json").
      SetBody(`{"username":"testuser", "password":"testpass"}`).
      SetResult(&AuthSuccess{}).    // or SetResult(AuthSuccess{}).
      Post("https://myapp.com/login")

// POST []byte array
// No need to set content type, if you have client level setting
resp, err := client.R().
      SetHeader("Content-Type", "application/json").
      SetBody([]byte(`{"username":"testuser", "password":"testpass"}`)).
      SetResult(&AuthSuccess{}).    // or SetResult(AuthSuccess{}).
      Post("https://myapp.com/login")

// POST Struct, default is JSON content type. No need to set one
resp, err := client.R().
      SetBody(User{Username: "testuser", Password: "testpass"}).
      SetResult(&AuthSuccess{}).    // or SetResult(AuthSuccess{}).
      SetError(&AuthError{}).       // or SetError(AuthError{}).
      Post("https://myapp.com/login")

// POST Map, default is JSON content type. No need to set one
resp, err := client.R().
      SetBody(map[string]interface{}{"username": "testuser", "password": "testpass"}).
      SetResult(&AuthSuccess{}).    // or SetResult(AuthSuccess{}).
      SetError(&AuthError{}).       // or SetError(AuthError{}).
      Post("https://myapp.com/login")

// POST of raw bytes for file upload. For example: upload file to Dropbox
fileBytes, _ := os.ReadFile("/Users/jeeva/mydocument.pdf")

// See we are not setting content-type header, since go-resty automatically detects Content-Type for you
resp, err := client.R().
      SetBody(fileBytes).
      SetContentLength(true).          // Dropbox expects this value
      SetAuthToken("<your-auth-token>").
      SetError(&DropboxError{}).       // or SetError(DropboxError{}).
      Post("https://content.dropboxapi.com/1/files_put/auto/resty/mydocument.pdf") // for upload Dropbox supports PUT too

// Note: resty detects Content-Type for request body/payload if content type header is not set.
//   * For struct and map data type defaults to 'application/json'
//   * Fallback is plain text content type

Sample PUT

You can use various combinations of PUT method call like demonstrated for POST.

// Note: This is one sample of PUT method usage, refer POST for more combination

// Create a Resty Client
client := resty.New()

// Request goes as JSON content type
// No need to set auth token, error, if you have client level settings
resp, err := client.R().
      SetBody(Article{
        Title: "go-resty",
        Content: "This is my article content, oh ya!",
        Author: "Jeevanandam M",
        Tags: []string{"article", "sample", "resty"},
      }).
      SetAuthToken("C6A79608-782F-4ED0-A11D-BD82FAD829CD").
      SetError(&Error{}).       // or SetError(Error{}).
      Put("https://myapp.com/article/1234")

Sample PATCH

You can use various combinations of PATCH method call like demonstrated for POST.

// Note: This is one sample of PUT method usage, refer POST for more combination

// Create a Resty Client
client := resty.New()

// Request goes as JSON content type
// No need to set auth token, error, if you have client level settings
resp, err := client.R().
      SetBody(Article{
        Tags: []string{"new tag1", "new tag2"},
      }).
      SetAuthToken("C6A79608-782F-4ED0-A11D-BD82FAD829CD").
      SetError(&Error{}).       // or SetError(Error{}).
      Patch("https://myapp.com/articles/1234")

Sample DELETE, HEAD, OPTIONS

// Create a Resty Client
client := resty.New()

// DELETE a article
// No need to set auth token, error, if you have client level settings
resp, err := client.R().
      SetAuthToken("C6A79608-782F-4ED0-A11D-BD82FAD829CD").
      SetError(&Error{}).       // or SetError(Error{}).
      Delete("https://myapp.com/articles/1234")

// DELETE a articles with payload/body as a JSON string
// No need to set auth token, error, if you have client level settings
resp, err := client.R().
      SetAuthToken("C6A79608-782F-4ED0-A11D-BD82FAD829CD").
      SetError(&Error{}).       // or SetError(Error{}).
      SetHeader("Content-Type", "application/json").
      SetBody(`{article_ids: [1002, 1006, 1007, 87683, 45432] }`).
      Delete("https://myapp.com/articles")

// HEAD of resource
// No need to set auth token, if you have client level settings
resp, err := client.R().
      SetAuthToken("C6A79608-782F-4ED0-A11D-BD82FAD829CD").
      Head("https://myapp.com/videos/hi-res-video")

// OPTIONS of resource
// No need to set auth token, if you have client level settings
resp, err := client.R().
      SetAuthToken("C6A79608-782F-4ED0-A11D-BD82FAD829CD").
      Options("https://myapp.com/servers/nyc-dc-01")

Override JSON & XML Marshal/Unmarshal

User could register choice of JSON/XML library into resty or write your own. By default resty registers standard encoding/json and encoding/xml respectively.

// Example of registering json-iterator
import jsoniter "github.com/json-iterator/go"

json := jsoniter.ConfigCompatibleWithStandardLibrary

client := resty.New().
    SetJSONMarshaler(json.Marshal).
    SetJSONUnmarshaler(json.Unmarshal)

// similarly user could do for XML too with -
client.SetXMLMarshaler(xml.Marshal).
    SetXMLUnmarshaler(xml.Unmarshal)

Multipart File(s) upload

Using io.Reader

profileImgBytes, _ := os.ReadFile("/Users/jeeva/test-img.png")
notesBytes, _ := os.ReadFile("/Users/jeeva/text-file.txt")

// Create a Resty Client
client := resty.New()

resp, err := client.R().
      SetFileReader("profile_img", "test-img.png", bytes.NewReader(profileImgBytes)).
      SetFileReader("notes", "text-file.txt", bytes.NewReader(notesBytes)).
      SetFormData(map[string]string{
          "first_name": "Jeevanandam",
          "last_name": "M",
      }).
      Post("http://myapp.com/upload")

Using File directly from Path

// Create a Resty Client
client := resty.New()

// Single file scenario
resp, err := client.R().
      SetFile("profile_img", "/Users/jeeva/test-img.png").
      Post("http://myapp.com/upload")

// Multiple files scenario
resp, err := client.R().
      SetFiles(map[string]string{
        "profile_img": "/Users/jeeva/test-img.png",
        "notes": "/Users/jeeva/text-file.txt",
      }).
      Post("http://myapp.com/upload")

// Multipart of form fields and files
resp, err := client.R().
      SetFiles(map[string]string{
        "profile_img": "/Users/jeeva/test-img.png",
        "notes": "/Users/jeeva/text-file.txt",
      }).
      SetFormData(map[string]string{
        "first_name": "Jeevanandam",
        "last_name": "M",
        "zip_code": "00001",
        "city": "my city",
        "access_token": "C6A79608-782F-4ED0-A11D-BD82FAD829CD",
      }).
      Post("http://myapp.com/profile")

Sample Form submission

// Create a Resty Client
client := resty.New()

// just mentioning about POST as an example with simple flow
// User Login
resp, err := client.R().
      SetFormData(map[string]string{
        "username": "jeeva",
        "password": "mypass",
      }).
      Post("http://myapp.com/login")

// Followed by profile update
resp, err := client.R().
      SetFormData(map[string]string{
        "first_name": "Jeevanandam",
        "last_name": "M",
        "zip_code": "00001",
        "city": "new city update",
      }).
      Post("http://myapp.com/profile")

// Multi value form data
criteria := url.Values{
  "search_criteria": []string{"book", "glass", "pencil"},
}
resp, err := client.R().
      SetFormDataFromValues(criteria).
      Post("http://myapp.com/search")

Save HTTP Response into File

// Create a Resty Client
client := resty.New()

// Setting output directory path, If directory not exists then resty creates one!
// This is optional one, if you're planning using absolute path in
// `Request.SetOutput` and can used together.
client.SetOutputDirectory("/Users/jeeva/Downloads")

// HTTP response gets saved into file, similar to curl -o flag
_, err := client.R().
          SetOutput("plugin/ReplyWithHeader-v5.1-beta.zip").
          Get("http://bit.ly/1LouEKr")

// OR using absolute path
// Note: output directory path is not used for absolute path
_, err := client.R().
          SetOutput("/MyDownloads/plugin/ReplyWithHeader-v5.1-beta.zip").
          Get("http://bit.ly/1LouEKr")

Request URL Path Params

Resty provides easy to use dynamic request URL path params. Params can be set at client and request level. Client level params value can be overridden at request level.

// Create a Resty Client
client := resty.New()

client.R().SetPathParams(map[string]string{
   "userId": "[email protected]",
   "subAccountId": "100002",
}).
Get("/v1/users/{userId}/{subAccountId}/details")

// Result:
//   Composed URL - /v1/users/[email protected]/100002/details

Request and Response Middleware

Resty provides middleware ability to manipulate for Request and Response. It is more flexible than callback approach.

// Create a Resty Client
client := resty.New()

// Registering Request Middleware
client.OnBeforeRequest(func(c *resty.Client, req *resty.Request) error {
    // Now you have access to Client and current Request object
    // manipulate it as per your need

    return nil  // if its success otherwise return error
  })

// Registering Response Middleware
client.OnAfterResponse(func(c *resty.Client, resp *resty.Response) error {
    // Now you have access to Client and current Response object
    // manipulate it as per your need

    return nil  // if its success otherwise return error
  })

OnError Hooks

Resty provides OnError hooks that may be called because:

  • The client failed to send the request due to connection timeout, TLS handshake failure, etc...
  • The request was retried the maximum amount of times, and still failed.

If there was a response from the server, the original error will be wrapped in *resty.ResponseError which contains the last response received.

// Create a Resty Client
client := resty.New()

client.OnError(func(req *resty.Request, err error) {
  if v, ok := err.(*resty.ResponseError); ok {
    // v.Response contains the last response from the server
    // v.Err contains the original error
  }
  // Log the error, increment a metric, etc...
})

Redirect Policy

Resty provides few ready to use redirect policy(s) also it supports multiple policies together.

// Create a Resty Client
client := resty.New()

// Assign Client Redirect Policy. Create one as per you need
client.SetRedirectPolicy(resty.FlexibleRedirectPolicy(15))

// Wanna multiple policies such as redirect count, domain name check, etc
client.SetRedirectPolicy(resty.FlexibleRedirectPolicy(20),
                        resty.DomainCheckRedirectPolicy("host1.com", "host2.org", "host3.net"))
Custom Redirect Policy

Implement RedirectPolicy interface and register it with resty client. Have a look redirect.go for more information.

// Create a Resty Client
client := resty.New()

// Using raw func into resty.SetRedirectPolicy
client.SetRedirectPolicy(resty.RedirectPolicyFunc(func(req *http.Request, via []*http.Request) error {
  // Implement your logic here

  // return nil for continue redirect otherwise return error to stop/prevent redirect
  return nil
}))

//---------------------------------------------------

// Using struct create more flexible redirect policy
type CustomRedirectPolicy struct {
  // variables goes here
}

func (c *CustomRedirectPolicy) Apply(req *http.Request, via []*http.Request) error {
  // Implement your logic here

  // return nil for continue redirect otherwise return error to stop/prevent redirect
  return nil
}

// Registering in resty
client.SetRedirectPolicy(CustomRedirectPolicy{/* initialize variables */})

Custom Root Certificates and Client Certificates

// Create a Resty Client
client := resty.New()

// Custom Root certificates, just supply .pem file.
// you can add one or more root certificates, its get appended
client.SetRootCertificate("/path/to/root/pemFile1.pem")
client.SetRootCertificate("/path/to/root/pemFile2.pem")
// ... and so on!

// Adding Client Certificates, you add one or more certificates
// Sample for creating certificate object
// Parsing public/private key pair from a pair of files. The files must contain PEM encoded data.
cert1, err := tls.LoadX509KeyPair("certs/client.pem", "certs/client.key")
if err != nil {
  log.Fatalf("ERROR client certificate: %s", err)
}
// ...

// You add one or more certificates
client.SetCertificates(cert1, cert2, cert3)

Custom Root Certificates and Client Certificates from string

// Custom Root certificates from string
// You can pass you certificates through env variables as strings
// you can add one or more root certificates, its get appended
client.SetRootCertificateFromString("-----BEGIN CERTIFICATE-----content-----END CERTIFICATE-----")
client.SetRootCertificateFromString("-----BEGIN CERTIFICATE-----content-----END CERTIFICATE-----")
// ... and so on!

// Adding Client Certificates, you add one or more certificates
// Sample for creating certificate object
// Parsing public/private key pair from a pair of files. The files must contain PEM encoded data.
cert1, err := tls.X509KeyPair([]byte("-----BEGIN CERTIFICATE-----content-----END CERTIFICATE-----"), []byte("-----BEGIN CERTIFICATE-----content-----END CERTIFICATE-----"))
if err != nil {
  log.Fatalf("ERROR client certificate: %s", err)
}
// ...

// You add one or more certificates
client.SetCertificates(cert1, cert2, cert3)

Proxy Settings

Default Go supports Proxy via environment variable HTTP_PROXY. Resty provides support via SetProxy & RemoveProxy. Choose as per your need.

Client Level Proxy settings applied to all the request

// Create a Resty Client
client := resty.New()

// Setting a Proxy URL and Port
client.SetProxy("http://proxyserver:8888")

// Want to remove proxy setting
client.RemoveProxy()

Retries

Resty uses backoff to increase retry intervals after each attempt.

Usage example:

// Create a Resty Client
client := resty.New()

// Retries are configured per client
client.
    // Set retry count to non zero to enable retries
    SetRetryCount(3).
    // You can override initial retry wait time.
    // Default is 100 milliseconds.
    SetRetryWaitTime(5 * time.Second).
    // MaxWaitTime can be overridden as well.
    // Default is 2 seconds.
    SetRetryMaxWaitTime(20 * time.Second).
    // SetRetryAfter sets callback to calculate wait time between retries.
    // Default (nil) implies exponential backoff with jitter
    SetRetryAfter(func(client *resty.Client, resp *resty.Response) (time.Duration, error) {
        return 0, errors.New("quota exceeded")
    })

By default, resty will retry requests that return a non-nil error during execution. Therefore, the above setup will result in resty retrying requests with non-nil errors up to 3 times, with the delay increasing after each attempt.

You can optionally provide client with custom retry conditions:

// Create a Resty Client
client := resty.New()

client.AddRetryCondition(
    // RetryConditionFunc type is for retry condition function
    // input: non-nil Response OR request execution error
    func(r *resty.Response, err error) bool {
        return r.StatusCode() == http.StatusTooManyRequests
    },
)

The above example will make resty retry requests that end with a 429 Too Many Requests status code. It's important to note that when you specify conditions using AddRetryCondition, it will override the default retry behavior, which retries on errors encountered during the request. If you want to retry on errors encountered during the request, similar to the default behavior, you'll need to configure it as follows:

// Create a Resty Client
client := resty.New()

client.AddRetryCondition(
    func(r *resty.Response, err error) bool {
        // Including "err != nil" emulates the default retry behavior for errors encountered during the request.
        return err != nil || r.StatusCode() == http.StatusTooManyRequests
    },
)

Multiple retry conditions can be added. Note that if multiple conditions are specified, a retry will occur if any of the conditions are met.

It is also possible to use resty.Backoff(...) to get arbitrary retry scenarios implemented. Reference.

Allow GET request with Payload

// Create a Resty Client
client := resty.New()

// Allow GET request with Payload. This is disabled by default.
client.SetAllowGetMethodPayload(true)

Wanna Multiple Clients

// Here you go!
// Client 1
client1 := resty.New()
client1.R().Get("http://httpbin.org")
// ...

// Client 2
client2 := resty.New()
client2.R().Head("http://httpbin.org")
// ...

// Bend it as per your need!!!

Remaining Client Settings & its Options

// Create a Resty Client
client := resty.New()

// Unique settings at Client level
//--------------------------------
// Enable debug mode
client.SetDebug(true)

// Assign Client TLSClientConfig
// One can set custom root-certificate. Refer: http://golang.org/pkg/crypto/tls/#example_Dial
client.SetTLSClientConfig(&tls.Config{ RootCAs: roots })

// or One can disable security check (https)
client.SetTLSClientConfig(&tls.Config{ InsecureSkipVerify: true })

// Set client timeout as per your need
client.SetTimeout(1 * time.Minute)


// You can override all below settings and options at request level if you want to
//--------------------------------------------------------------------------------
// Host URL for all request. So you can use relative URL in the request
client.SetBaseURL("http://httpbin.org")

// Headers for all request
client.SetHeader("Accept", "application/json")
client.SetHeaders(map[string]string{
        "Content-Type": "application/json",
        "User-Agent": "My custom User Agent String",
      })

// Cookies for all request
client.SetCookie(&http.Cookie{
      Name:"go-resty",
      Value:"This is cookie value",
      Path: "/",
      Domain: "sample.com",
      MaxAge: 36000,
      HttpOnly: true,
      Secure: false,
    })
client.SetCookies(cookies)

// URL query parameters for all request
client.SetQueryParam("user_id", "00001")
client.SetQueryParams(map[string]string{ // sample of those who use this manner
      "api_key": "api-key-here",
      "api_secret": "api-secret",
    })
client.R().SetQueryString("productId=232&template=fresh-sample&cat=resty&source=google&kw=buy a lot more")

// Form data for all request. Typically used with POST and PUT
client.SetFormData(map[string]string{
    "access_token": "BC594900-518B-4F7E-AC75-BD37F019E08F",
  })

// Basic Auth for all request
client.SetBasicAuth("myuser", "mypass")

// Bearer Auth Token for all request
client.SetAuthToken("BC594900518B4F7EAC75BD37F019E08FBC594900518B4F7EAC75BD37F019E08F")

// Enabling Content length value for all request
client.SetContentLength(true)

// Registering global Error object structure for JSON/XML request
client.SetError(&Error{})    // or resty.SetError(Error{})

Unix Socket

unixSocket := "/var/run/my_socket.sock"

// Create a Go's http.Transport so we can set it in resty.
transport := http.Transport{
	Dial: func(_, _ string) (net.Conn, error) {
		return net.Dial("unix", unixSocket)
	},
}

// Create a Resty Client
client := resty.New()

// Set the previous transport that we created, set the scheme of the communication to the
// socket and set the unixSocket as the HostURL.
client.SetTransport(&transport).SetScheme("http").SetBaseURL(unixSocket)

// No need to write the host's URL on the request, just the path.
client.R().Get("http://localhost/index.html")

Bazel Support

Resty can be built, tested and depended upon via Bazel. For example, to run all tests:

bazel test :resty_test

Mocking http requests using httpmock library

In order to mock the http requests when testing your application you could use the httpmock library.

When using the default resty client, you should pass the client to the library as follow:

// Create a Resty Client
client := resty.New()

// Get the underlying HTTP Client and set it to Mock
httpmock.ActivateNonDefault(client.GetClient())

More detailed example of mocking resty http requests using ginko could be found here.

Versioning

Resty releases versions according to Semantic Versioning

  • Resty v2 does not use gopkg.in service for library versioning.
  • Resty fully adapted to go mod capabilities since v1.10.0 release.
  • Resty v1 series was using gopkg.in to provide versioning. gopkg.in/resty.vX points to appropriate tagged versions; X denotes version series number and it's a stable release for production use. For e.g. gopkg.in/resty.v0.
  • Development takes place at the master branch. Although the code in master should always compile and test successfully, it might break API's. I aim to maintain backwards compatibility, but sometimes API's and behavior might be changed to fix a bug.

Contribution

I would welcome your contribution! If you find any improvement or issue you want to fix, feel free to send a pull request, I like pull requests that include test cases for fix/enhancement. I have done my best to bring pretty good code coverage. Feel free to write tests.

BTW, I'd like to know what you think about Resty. Kindly open an issue or send me an email; it'd mean a lot to me.

Creator

Jeevanandam M. ([email protected])

Core Team

Have a look on Members page.

Contributors

Have a look on Contributors page.

License

Resty released under MIT license, refer LICENSE file.

resty's People

Contributors

aanm avatar ahuigo avatar arcticsnowman avatar bak1an avatar creekorful avatar ei-grad avatar gitter-badger avatar icepie avatar jeevatkm avatar kishaningithub avatar lavoiesl avatar lggomez avatar lrita avatar moorereason avatar muir avatar neganovalexey avatar paradoxengine avatar robbilie avatar sabandi avatar sakateka avatar sandyydk avatar segevda avatar sherzberg avatar sudo-suhas avatar svilgelm avatar tanyabouman avatar testwill avatar tkrop avatar victoraugustolls avatar vivekv96 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

resty's Issues

Support trailing backslashes in URLs

Many REST APIs still include different behavior for addresses with a trailing slash vs without. For example:

GET /api/v1/dog

might possibly be different from

GET /api/v1/dog/

Current behavior is to trim the trailing backslash. I propose to include trailing backslashes and leave it to the user to specify the verbose path.

Save http response into file - similar to curl -o flag

Bring ability to save HTTP response into file.

Client Level feature:

  • Setting Directory path for saving response

Request Level feature:

  • Setting output filename for request. Absolute and Relative path option.
    • For relative path option, if output directory path is not set at client level then default 'go-resty' directory will be created at current working directory

Add custom timeout per client basis + FastHTTP support

So I'm looking for something like

myresty := resty.New()
req, err := myresty.R().
            SetHeader("User-Agent", UserAgent).
            SetProxy(proxyUrl).
            SetTimeout(duration)
            Get(link)

Also wanted to ask if it's possible to use fasthttp instead of net/http. From what I read it is significantly more performant:

Refer: https://medium.com/@valyala/net-http-client-has-the-following-additional-limitations-318ac870ce9d#.502tduhmn

Refer: https://medium.com/@nate510/don-t-use-go-s-default-http-client-4804cb19f779#.pla7aw7gz

Thanks!

Add support for WaitTime and MaxWaitTime for retries

Currently client does not expose a way to configure wait times for retry, only conditions and retry count are configurable -

}, Retries(r.client.RetryCount), RetryConditions(r.client.RetryConditions))

We need to make wait times configurable as well, default 100ms wait time is too little in my case.

I can provide a pull request for this if the issue is valid

stop redirect

can resty stop redirect without getting error ?! i tried custom redirectpolicy but still nil continue the redirect and error panic and not returning with response

Add an option for user-defined middleware after the built-in middlware, to modify the RawRequest object

I wanted to use resty for requests to an AWS endpoint that requires requests to be signed with AWS credentials. A library already exists for this (https://github.com/smartystreets/go-aws-auth), however using that one requires access to the http.Request object directly. Currently, user defined middleware is run before the built-in middleware, which makes sense when modifying the resty.Request object. However for resty to be compatible with go-aws-auth, I need middleware that is processed after the built-in middleware, so that the RawRequest object is available.

Unable to set custom `http.RoundTripper`

At the moment it is not possible to set custom http.RoundTripper as a transport in resty since its transport field wants exactly http.Transport struct. On the contrary, golang's stdlib client allows us to use any RoundTripper - https://golang.org/src/net/http/client.go?s=1933:4055#L46

Providing custom RoundTripper can be usable in testing, i.e. one could hijack transport to be provided by some mocking library (https://github.com/dnaeon/go-vcr) or when there is a need to use custom transport for whatever reasons like connecting to weird places with weird protocols or using transport with more features than golang's one provides (https://github.com/pkulak/simpletransport).

Resty is currently relying on http.Transport to provide a way for controlling proxies, tls and timeouts.

We could use type assertion in those places to check if RoundTripper we have is an http.Transport and do nothing in case it is not (or returning error). We could also mention it in the documentation that those things are controlled by underlying transport so SetProxy, SetTimeout and SetTLSClientConfig won't function in case custom non http.Transport transport used.

I can provide a pull request if you find the idea to be valid.

Make Request.execute public

There are cases where we have to programmatically decide which http method to use when invoking an API. While we can create a map of handler functions to call the various put/post/get/delete/options functions on a Request, it would be simpler if we could just pass the desired method to Execute directly, i.e:

resp, err := resty.R().Execute(resty.POST, url)

vs what I have to do right now:

type handler func(*resty.Request, string) (*resty.Response, error)

func get(r *resty.Request, url string) (*resty.Response, error) {
  return r.Get(url)
}

func post(r *resty.Request, url string) (*resty.Response, error) {
  return r.Post(url)
}

handlers := make(map[string]handler)
handlers[resty.GET] = get
handlers[resty.POST] = post
// etc

resp, err := handlers[resty.POST](resty.R(), url)

httpClient.Do is protected by a mutex?

Because of the locking, there can only be one outgoing call, which is really bad to the performance. If I understand it correctly,

  • if req1 has proxyURL set but req2 doesn't, then req2 will be sent with req1's proxy. Is this expected?
  • if req1 and req2 both have proxyURL set, then req1 will be sent with req1's proxy and req2 will be sent with req2's proxy. This works. But the overhead of this is to not have more than 1 outgoing calls.
  • if Unlock() is moved up before httpClient.Do. For the above case, there is no guarantee on which proxy will be used, which is not good.

I hope this per request proxy is an optional feature. Providing it is fine but without sacrificing the normal use case.

Code ref:
https://github.com/go-resty/resty/blob/master/client.go#L634-L647

	c.mutex.Lock()

	if req.proxyURL != nil {
		c.transport.Proxy = http.ProxyURL(req.proxyURL)
	} else if c.proxyURL != nil {
		c.transport.Proxy = http.ProxyURL(c.proxyURL)
	}

	req.Time = time.Now()
	c.httpClient.Transport = c.transport

	resp, err := c.httpClient.Do(req.RawRequest)

        c.mutex.Unlock()

Resty not binding for a non 200 response

Hi,

Thanks for the awesome library!

I have a struct that I'm filling with a JSON response from an API endpoint using SetResult. When the endpoint returns a 200 (success status), this struct is filled in just fine. However, when it returns say a 4XX or 5XX error message this struct is empty even though the response body itself has proper JSON.

Convenient Query string (SetQueryString) support for Request

Currently go-resty supports query string via SetQueryParam and SetQueryParams at Client and Request. However I believe in-addition to those methods. It would be better to have more comfortable experience with SetQueryString in the Request level.

resp1, err1 := resty.R().
            SetQueryString("Param1=Param1Vaule&Param2=Param2 Value&Param3=Param3 Value").
            Get("http://httpbin.org/get")

Support socks5 proxies

Seeing resty allows proxies to be set by URL which involves parsing and identifying its scheme, most people would expect that it supports other proxy schemes such as socks5. However, when I try to set socks5 proxy, It connects to the socks5 proxy host by using a http proxy model.

The reason why I need this is typically Tor listens only for socks5 connections. I know of no project using golang that currently supports socks5 proxy, but is there any way this could be built into the resty?

SetBody() does not support array of models

Hi
I am a team member of Swagger CodeGen, we use the resty API, we recently ran into issue when setting body with an array of models, is it a known issue that resty does not allow passing array of objects in the body? below is the test exception:

--- FAIL: TestCreateUsersWithArrayInput (0.00s)
panic: interface conversion: interface is *[]petstore.User, not []uint8 [recovered]
    panic: interface conversion: interface is *[]petstore.User, not []uint8

goroutine 18 [running]:
testing.tRunner.func1(0xc8200f6090)
    /usr/local/go/src/testing/testing.go:450 +0x171
github.com/go-resty/resty.DetectContentType(0x2e9320, 0xc8200fe140, 0x0, 0x0)
    /Users/xx/go/src/github.com/go-resty/resty/client.go:780 +0xd6
github.com/go-resty/resty.handleRequestBody(0xc8200bc000, 0xc82010a000, 0x0, 0x0)
    /Users/xx/go/src/github.com/go-resty/resty/middleware.go:295 +0xc4
github.com/go-resty/resty.parseRequestBody(0xc8200bc000, 0xc82010a000, 0x0, 0x0)
    /Users/xx/go/src/github.com/go-resty/resty/middleware.go:107 +0x245
github.com/go-resty/resty.(*Client).execute(0xc8200bc000, 0xc82010a000, 0x0, 0x0, 0x0)
    /Users/xx/go/src/github.com/go-resty/resty/client.go:579 +0xc7
github.com/go-resty/resty.(*Request).Execute(0xc82010a000, 0x40ab28, 0x4, 0xc8200f2240, 0x32, 0x4, 0x0, 0x0)
    /Users/xx/go/src/github.com/go-resty/resty/request.go:394 +0x105
github.com/go-resty/resty.(*Request).Post(0xc82010a000, 0xc8200f2240, 0x32, 0x4

FYI, with the same code, single object works perfectly.

Lost repsonse and no error return

Someone api will cost N ms.

If we SetTimeout M ms (N< M < 2N) to Client and we call this api twice, the server side will receive three request and send three response.

In client side, user only see 2 api call but 3 api call be sent actually.

This issue will be duplicated in go 1.7.3, here is the code.

I found a bad implement of SetTimeout and fix it.

OnBeforeRequest does not change Request.RawRequest attributes

Trying to manipulate *resty.Request fields inside an OnBeforeRequest function does not affect the actual request, which is finally executed:

https://github.com/go-resty/resty/blob/master/client.go#L627-L645

for _, f := range c.beforeRequest {
		err = f(c, req)
		if err != nil {
			return nil, err
		}
	}

	c.mutex.Lock()

	if req.proxyURL != nil {
		c.transport.Proxy = http.ProxyURL(req.proxyURL)
	} else if c.proxyURL != nil {
		c.transport.Proxy = http.ProxyURL(c.proxyURL)
	}

	req.Time = time.Now()
	c.httpClient.Transport = c.transport

	resp, err := c.httpClient.Do(req.RawRequest)

All OnBeforeRequest functions will be executed but they have no effect when they call methods directly on *resty.Request because the RawRequest stays untouched until real request execution.

For example setting an auth token header inside an OnBeforeRequest function won't work with

	httpClient.OnBeforeRequest(func(c *resty.Client, r *resty.Request) error {
		r.SetAuthToken("bla")
                return nil
        })

As a workaround one has to manipulate directly the r.RawRequest struct.

SetQueryParams does not support "Multi" inputs

Does Resty support Multiple params? I am passing any array of statuses for query param, i am expecting the querystring is
?status=pending&status=approved, but what i got from the debugger is only appending the first value,(status=pending)
here is my code

    var status = []string {"pending","approved"}
    for _, value := range status {
        resty.R().SetQueryParam("status",value)
    }

fyi, i also tried SetQueryParams, but since it is a dictionary, you can't have multiple values, so it won't work either. I know the work around is converting status array to a comma separated strings, but I prefer to set multiple parameters.

are you going to add this feature in the near future?

cc @wing328

No DNS failover possible with resty

Hi!

I have a daemon, which polls some API endpoints. If the API endpoint switch traffic with DNS (changing the IP), then resty will not do a DNS lookup to get the new IP.
I use the default client use the following code:

urlString := fmt.Sprintf("https://api.endpoint.foo.com/events/%v", ID) // DNS name changed
resp, err := resty.R().
    SetHeader("Content-Type", "application/json").
    SetBody(body).
    SetAuthToken(cli.AccessToken).
    Put(urlString)

Maybe I can get resty to lookup DNS somehow, but I did not find it, yet.

Thank you

for such wonderfully commented code. Feel free to close this :)

Nil Pointer access on timeout using retry

When a request interrupts due to a client timeout and when using a retry condition function then the underlying resty.Response.RawResponse is nil. Which is pretty obvious because the remote haven't sent any. But one has to make a nil check on the RawResponse in RetryConditionFuncs because functions like resty.Response.StatusCode() access directly the underlying RawResponses fields.

Add support for automatic json unmarshalling using map on response

Can you please bring automatic JSON unmarshalling using map on response like struct?

I'm trying to do something like the following:

resp, err := resty.R().SetResult(map[string]interface{}{}).Get(url)
result := resp.Result().(*map[string]interface{})
fmt.Printf("%+v", result["message"])

API streamlining (Minor breaking changes)

For API consistency and progressing toward v1.0 goal, following changes taking place in v0.5-

  • Response.Body turns to Response.Body() method
  • Response.ReceivedAt turns to Response.ReceivedAt() method

Don't set the transport's proxy settings to nil by default

In client.go, at this location, there's this code:

    if req.proxyURL != nil {
        c.transport.Proxy = http.ProxyURL(req.proxyURL)
    } else if c.proxyURL != nil {
        c.transport.Proxy = http.ProxyURL(c.proxyURL)
    } else {
        c.transport.Proxy = nil
    }

The final else clause should be removed. If the caller provides a transport, it may already have a proxy configured, there's no reason to set the transport's proxy to nil.

Disable warnings possible?

I get this continously:

RESTY 2016/04/13 16:28:18 WARNING - Using Basic Auth in HTTP mode is not secure.

It would be nice if it was (perhaps it already is!) possible to disable warnings.

DATA RACE in resty.(*Client).execute()

Hello,
I am use resty in multi-goroutine and case the DATA RACE

test.go
`package main

import (
"github.com/go-resty/resty"
"sync"
)

func main() {
w := sync.WaitGroup{}
for i := 0; i < 50; i++ {
w.Add(1)
go func() {
defer w.Done()
resty.R().Get("http://httpbin.org/get")
}()
}
w.Wait()
} go run -race test.go`

Support pointer and non-pointer for SetResult and SetError methods

SetResult:

// Note: Result object can be pointer or non-pointer.
resty.R().SetResult(&AuthToken{})
// OR
resty.R().SetResult(AuthToken{})

// Accessing a result value
response.Result().(*AuthToken)

SetError:

// Note: Error object can be pointer or non-pointer.
resty.R().SetError(&AuthError{})
// OR
resty.R().SetError(AuthError{})

// Accessing a error value
response.Error().(*AuthError)

Confusion between `set` and `add` methods

Hi,

I noticed an unexpected behavior for most of the set methods. This is from request.go:

func (r *Request) SetFormData(data map[string]string) *Request {
	for k, v := range data {
		r.FormData.Add(k, v)
	}
	return r
}

Most of the set methods are actually add which is misleading and prevents from overriding existing keys.
I don't know whether it is a design choice or not.

My suggestion would be to have both set and add methods (I have the changes locally). It would make things cleaner imo.

However, I am also aware that for functions such as SetQueryString it might be possible to have multiple times the same key.
It would be possible for the set functions to make them smarter and in case of multiple occurrences of one key, to first call set (which would erase the previously stored key) then call add.
Though I am not sure it would really be better.

Also, changing the current set methods to internally use a set call instead of an add call will break compatibility with code using the current behavior.

What is your opinion on the topic ?

Bulk Request feature

Bring bulk request feature as a batteries for go-resty. For example: very useful for Upload/Download, batch processing, etc.

Add an option to support meta redirection

This is not a standard yet but it'd be great if Resty support redirection using meta refresh such as:

<meta http-equiv="refresh" content="0; url=/index.jsp">
resty.FollowMetaRedirects(true)

Most websites follow the HTTP standard, which contains declarations for standard redirection method, but some don't. However, this is not so important because it's often not the case and can cause performance issues.

SetMultiValueFormData

Hello,

It would be nice to have SetMultiValueFormData which should work in the same way as SetMultiValueQueryParams but for FormData:

q, err := query.Values(struct {}{})

for p, v := range q {
    for _, pv := range v {
        client.FormData.Add(p, pv)
    }
}
q, err := query.Values(struct {}{})

client.SetMultiValueFormData(q)

Implement extensible and multiple redirect policies capability

  • RedirectPolicy and RedirectPolicyFunc approach for extensible redirect policy definition. User can define N number of policies as they need
  • Design and structure SetRedirectPolicy method, so it can accept multiple redirect policies as variadic arguments
  • Bring new policy called 'Host check' redirect policy

Allow file uploads with io.Reader and a name

Hey, first of all: thanks for this wonderful library! I've been using it for quite some time now, and it works like a charm.

There's one thing I'm missing, though: being able to upload files (as multipart) using an io.Reader and a name for the file. The scenario is, for example, something like this:

I have a database of many small image files which are stored as BLOBs along with their original file name (or file type). I want to upload some of them to some endpoints, I know the server will check the extension of the file name to determine if I really send images. It'd be great if I could directly "pipe" the image from the DB along with its name (or a made up name) into resty. It's a pain to create temporary files for every upload I need (imagine the DB running on another machine)...

Other scenarios might include receiving files over the network - I don't want to store a copy locally, I just want to pipe them through.

Would it be possible to get that? I'm thinking of something like

func (r *Request) SetFileReader(param, fileName string, reader io.Reader) *Request

What do you think?

Enable proper retrying of POST requests

It looks like retries on POST requests do not resend the request body. Some would suggest that POSTs should not be retried, but in my case the request is idempotent, so it's fine. It should be noted that the log indicates the retried attempt contains the post body, but it's not actually there.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.