Giter Club home page Giter Club logo

gowebdav's Introduction

GoWebDAV

Unit Tests Status Build Artifacts Status GoDoc Go Report Card

A pure Golang WebDAV client library that comes with a reference implementation.

Features at a glance

Our gowebdav library allows to perform following actions on the remote WebDAV server:

It also provides an authentication API that makes it easy to encapsulate and control complex authentication challenges. The default implementation negotiates the algorithm based on the user's preferences and the methods offered by the remote server.

Out-of-box authentication support for:

Usage

First of all you should create Client instance using NewClient() function:

root := "https://webdav.mydomain.me"
user := "user"
password := "password"

c := gowebdav.NewClient(root, user, password)
c.Connect()
// kick of your work!

After you can use this Client to perform actions, described below.

NOTICE: We will not check for errors in the examples, to focus you on the gowebdav library's code, but you should do it in your code!

Create path on a WebDAV server

err := c.Mkdir("folder", 0644)

In case you want to create several folders you can use c.MkdirAll():

err := c.MkdirAll("folder/subfolder/subfolder2", 0644)

Get files list

files, _ := c.ReadDir("folder/subfolder")
for _, file := range files {
    //notice that [file] has os.FileInfo type
    fmt.Println(file.Name())
}

Download file to byte array

webdavFilePath := "folder/subfolder/file.txt"
localFilePath := "/tmp/webdav/file.txt"

bytes, _ := c.Read(webdavFilePath)
os.WriteFile(localFilePath, bytes, 0644)

Download file via reader

Also you can use c.ReadStream() method:

webdavFilePath := "folder/subfolder/file.txt"
localFilePath := "/tmp/webdav/file.txt"

reader, _ := c.ReadStream(webdavFilePath)

file, _ := os.Create(localFilePath)
defer file.Close()

io.Copy(file, reader)

Upload file from byte array

webdavFilePath := "folder/subfolder/file.txt"
localFilePath := "/tmp/webdav/file.txt"

bytes, _ := os.ReadFile(localFilePath)

c.Write(webdavFilePath, bytes, 0644)

Upload file via writer

webdavFilePath := "folder/subfolder/file.txt"
localFilePath := "/tmp/webdav/file.txt"

file, _ := os.Open(localFilePath)
defer file.Close()

c.WriteStream(webdavFilePath, file, 0644)

Get information about specified file/folder

webdavFilePath := "folder/subfolder/file.txt"

info := c.Stat(webdavFilePath)
//notice that [info] has os.FileInfo type
fmt.Println(info)

Move file to another location

oldPath := "folder/subfolder/file.txt"
newPath := "folder/subfolder/moved.txt"
isOverwrite := true

c.Rename(oldPath, newPath, isOverwrite)

Copy file to another location

oldPath := "folder/subfolder/file.txt"
newPath := "folder/subfolder/file-copy.txt"
isOverwrite := true

c.Copy(oldPath, newPath, isOverwrite)

Delete file

webdavFilePath := "folder/subfolder/file.txt"

c.Remove(webdavFilePath)

Links

More details about WebDAV server you can read from following resources:

NOTICE: RFC 2518 is obsoleted by RFC 4918 in June 2007

Contributing

All contributing are welcome. If you have any suggestions or find some bug - please create an Issue to let us make this project better. We appreciate your help!

License

This library is distributed under the BSD 3-Clause license found in the LICENSE file.

API

import "github.com/studio-b12/gowebdav"

Package gowebdav is a WebDAV client library with a command line tool included.

auth.go basicAuth.go client.go digestAuth.go doc.go errors.go file.go netrc.go passportAuth.go requests.go utils.go

const XInhibitRedirect = "X-Gowebdav-Inhibit-Redirect"
var ErrAuthChanged = errors.New("authentication failed, change algorithm")

ErrAuthChanged must be returned from the Verify method as an error to trigger a re-authentication / negotiation with a new authenticator.

var ErrTooManyRedirects = errors.New("stopped after 10 redirects")

ErrTooManyRedirects will be used as return error if a request exceeds 10 redirects.

func FixSlash(s string) string

FixSlash appends a trailing / to our string

func FixSlashes(s string) string

FixSlashes appends and prepends a / if they are missing

func IsErrCode(err error, code int) bool

IsErrCode returns true if the given error is an os.PathError wrapping a StatusError with the given status code.

func IsErrNotFound(err error) bool

IsErrNotFound is shorthand for IsErrCode for status 404.

func Join(path0 string, path1 string) string

Join joins two paths

func NewPathError(op string, path string, statusCode int) error
func NewPathErrorErr(op string, path string, err error) error
func PathEscape(path string) string

PathEscape escapes all segments of a given path

func ReadConfig(uri, netrc string) (string, string)

ReadConfig reads login and password configuration from ~/.netrc machine foo.com login username password 123456

func String(r io.Reader) string

String pulls a string out of our io.Reader

type AuthFactory func(c *http.Client, rs *http.Response, path string) (auth Authenticator, err error)

AuthFactory prototype function to create a new Authenticator

type Authenticator interface {
    // Authorizes a request. Usually by adding some authorization headers.
    Authorize(c *http.Client, rq *http.Request, path string) error
    // Verifies the response if the authorization was successful.
    // May trigger some round trips to pass the authentication.
    // May also trigger a new Authenticator negotiation by returning `ErrAuthChenged`
    Verify(c *http.Client, rs *http.Response, path string) (redo bool, err error)
    // Creates a copy of the underlying Authenticator.
    Clone() Authenticator
    io.Closer
}

A Authenticator implements a specific way to authorize requests. Each request is bound to a separate Authenticator instance.

The authentication flow itself is broken down into Authorize and Verify steps. The former method runs before, and the latter runs after the Request is submitted. This makes it easy to encapsulate and control complex authentication challenges.

Some authentication flows causing authentication round trips, which can be archived by returning the redo of the Verify method. True restarts the authentication process for the current action: A new Request is spawned, which must be authorized, sent, and re-verified again, until the action is successfully submitted. The preferred way is to handle the authentication ping-pong within Verify, and then redo with fresh credentials.

The result of the Verify method can also trigger an Authenticator change by returning the ErrAuthChanged as an error. Depending on the Authorizer this may trigger an Authenticator negotiation.

Set the XInhibitRedirect header to '1' in the Authorize method to get control over request redirection. Attention! You must handle the incoming request yourself.

To store a shared session state the Clone method must return a new instance, initialized with the shared state.

func NewDigestAuth(login, secret string, rs *http.Response) (Authenticator, error)

NewDigestAuth creates a new instance of our Digest Authenticator

func NewPassportAuth(c *http.Client, user, pw, partnerURL string, header *http.Header) (Authenticator, error)

constructor for PassportAuth creates a new PassportAuth object and automatically authenticates against the given partnerURL

type Authorizer interface {
    // Creates a new Authenticator Shim per request.
    // It may track request related states and perform payload buffering
    // for authentication round trips.
    // The underlying Authenticator will perform the real authentication.
    NewAuthenticator(body io.Reader) (Authenticator, io.Reader)
    // Registers a new Authenticator factory to a key.
    AddAuthenticator(key string, fn AuthFactory)
}

Authorizer our Authenticator factory which creates an Authenticator per action/request.

func NewAutoAuth(login string, secret string) Authorizer

NewAutoAuth creates an auto Authenticator factory. It negotiates the default authentication method based on the order of the registered Authenticators and the remotely offered authentication methods. First In, First Out.

func NewEmptyAuth() Authorizer

NewEmptyAuth creates an empty Authenticator factory The order of adding the Authenticator matters. First In, First Out. It offers the NewAutoAuth features.

func NewPreemptiveAuth(auth Authenticator) Authorizer

NewPreemptiveAuth creates a preemptive Authenticator The preemptive authorizer uses the provided Authenticator for every request regardless of any Www-Authenticate header.

It may only have one authentication method, so calling AddAuthenticator will panic!

Look out!! This offers the skinniest and slickest implementation without any synchronisation!! Still applicable with BasicAuth within go routines.

type BasicAuth struct {
    // contains filtered or unexported fields
}

BasicAuth structure holds our credentials

func (*BasicAuth) Authorize

func (b *BasicAuth) Authorize(c *http.Client, rq *http.Request, path string) error

Authorize the current request

func (*BasicAuth) Clone

func (b *BasicAuth) Clone() Authenticator

Clone creates a Copy of itself

func (*BasicAuth) Close

func (b *BasicAuth) Close() error

Close cleans up all resources

func (*BasicAuth) String

func (b *BasicAuth) String() string

String toString

func (*BasicAuth) Verify

func (b *BasicAuth) Verify(c *http.Client, rs *http.Response, path string) (redo bool, err error)

Verify verifies if the authentication

type Client struct {
    // contains filtered or unexported fields
}

Client defines our structure

func NewAuthClient(uri string, auth Authorizer) *Client

NewAuthClient creates a new client instance with a custom Authorizer

func NewClient(uri, user, pw string) *Client

NewClient creates a new instance of client

func (*Client) Connect

func (c *Client) Connect() error

Connect connects to our dav server

func (*Client) Copy

func (c *Client) Copy(oldpath, newpath string, overwrite bool) error

Copy copies a file from A to B

func (*Client) Mkdir

func (c *Client) Mkdir(path string, _ os.FileMode) (err error)

Mkdir makes a directory

func (*Client) MkdirAll

func (c *Client) MkdirAll(path string, _ os.FileMode) (err error)

MkdirAll like mkdir -p, but for webdav

func (*Client) Read

func (c *Client) Read(path string) ([]byte, error)

Read reads the contents of a remote file

func (*Client) ReadDir

func (c *Client) ReadDir(path string) ([]os.FileInfo, error)

ReadDir reads the contents of a remote directory

func (*Client) ReadStream

func (c *Client) ReadStream(path string) (io.ReadCloser, error)

ReadStream reads the stream for a given path

func (c *Client) ReadStreamRange(path string, offset, length int64) (io.ReadCloser, error)

ReadStreamRange reads the stream representing a subset of bytes for a given path, utilizing HTTP Range Requests if the server supports it. The range is expressed as offset from the start of the file and length, for example offset=10, length=10 will return bytes 10 through 19.

If the server does not support partial content requests and returns full content instead, this function will emulate the behavior by skipping offset bytes and limiting the result to length.

func (*Client) Remove

func (c *Client) Remove(path string) error

Remove removes a remote file

func (*Client) RemoveAll

func (c *Client) RemoveAll(path string) error

RemoveAll removes remote files

func (*Client) Rename

func (c *Client) Rename(oldpath, newpath string, overwrite bool) error

Rename moves a file from A to B

func (*Client) SetHeader

func (c *Client) SetHeader(key, value string)

SetHeader lets us set arbitrary headers for a given client

func (c *Client) SetInterceptor(interceptor func(method string, rq *http.Request))

SetInterceptor lets us set an arbitrary interceptor for a given client

func (*Client) SetJar

func (c *Client) SetJar(jar http.CookieJar)

SetJar exposes the ability to set a cookie jar to the client.

func (*Client) SetTimeout

func (c *Client) SetTimeout(timeout time.Duration)

SetTimeout exposes the ability to set a time limit for requests

func (*Client) SetTransport

func (c *Client) SetTransport(transport http.RoundTripper)

SetTransport exposes the ability to define custom transports

func (*Client) Stat

func (c *Client) Stat(path string) (os.FileInfo, error)

Stat returns the file stats for a specified path

func (*Client) Write

func (c *Client) Write(path string, data []byte, _ os.FileMode) (err error)

Write writes data to a given path

func (*Client) WriteStream

func (c *Client) WriteStream(path string, stream io.Reader, _ os.FileMode) (err error)

WriteStream writes a stream

type DigestAuth struct {
    // contains filtered or unexported fields
}

DigestAuth structure holds our credentials

func (*DigestAuth) Authorize

func (d *DigestAuth) Authorize(c *http.Client, rq *http.Request, path string) error

Authorize the current request

func (*DigestAuth) Clone

func (d *DigestAuth) Clone() Authenticator

Clone creates a copy of itself

func (*DigestAuth) Close

func (d *DigestAuth) Close() error

Close cleans up all resources

func (*DigestAuth) String

func (d *DigestAuth) String() string

String toString

func (*DigestAuth) Verify

func (d *DigestAuth) Verify(c *http.Client, rs *http.Response, path string) (redo bool, err error)

Verify checks for authentication issues and may trigger a re-authentication

type File struct {
    // contains filtered or unexported fields
}

File is our structure for a given file

func (f File) ContentType() string

ContentType returns the content type of a file

func (File) ETag

func (f File) ETag() string

ETag returns the ETag of a file

func (File) IsDir

func (f File) IsDir() bool

IsDir let us see if a given file is a directory or not

func (File) ModTime

func (f File) ModTime() time.Time

ModTime returns the modified time of a file

func (File) Mode

func (f File) Mode() os.FileMode

Mode will return the mode of a given file

func (File) Name

func (f File) Name() string

Name returns the name of a file

func (File) Path

func (f File) Path() string

Path returns the full path of a file

func (File) Size

func (f File) Size() int64

Size returns the size of a file

func (File) String

func (f File) String() string

String lets us see file information

func (File) Sys

func (f File) Sys() interface{}

Sys ????

type PassportAuth struct {
    // contains filtered or unexported fields
}

PassportAuth structure holds our credentials

func (*PassportAuth) Authorize

func (p *PassportAuth) Authorize(c *http.Client, rq *http.Request, path string) error

Authorize the current request

func (*PassportAuth) Clone

func (p *PassportAuth) Clone() Authenticator

Clone creates a Copy of itself

func (*PassportAuth) Close

func (p *PassportAuth) Close() error

Close cleans up all resources

func (*PassportAuth) String

func (p *PassportAuth) String() string

String toString

func (*PassportAuth) Verify

func (p *PassportAuth) Verify(c *http.Client, rs *http.Response, path string) (redo bool, err error)

Verify verifies if the authentication is good

type StatusError struct {
    Status int
}

StatusError implements error and wraps an erroneous status code.

func (StatusError) Error

func (se StatusError) Error() string

Generated by godoc2md

gowebdav's People

Contributors

chripo avatar chuckwagoncomputing avatar ferada avatar fmartingr avatar gamer92000 avatar jkowalski avatar kballbrabblerag avatar kitech avatar lalinsky avatar mania25 avatar marcelblijleven avatar mattn avatar miquella avatar misha-plus avatar mrvine avatar needsalongholiday avatar pojntfx avatar programyazar avatar qbit avatar racoon-devel avatar stazer avatar ueffel avatar yyeltsyn avatar zekrotja avatar zhijian-pro avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

gowebdav's Issues

404 isn't treated as expected

In Read/ReadStream no HTTP response code handling is done. The expected behaviour IMO is to promote 404 and similar errors to actual err responses.

Authentication fails when deployed to Windows machine without Go installed

I wrote an app on Windows using basic authentication, which appears as Go-http-client/2.0 on the server, if run from my developer machine with Go installed, and works well.

On a different machine without Go installed (or on my machine after uninstalling Go) the same client appears as Microsoft-WebDAV-MiniRedir/10.0.16299 on the server, and authentication fails.

I read quite a bit about dynamic linking if using the net package, but right now I am stuck.

Error on Folders with '+'

Hello Collaborators,

Describe the bug

When we want to Propfind a folder which contains folders with "+", the call fails. Due to the QueryUnescape at
https://github.com/studio-b12/gowebdav/blob/master/client.go#L141
the "+" results in a wrong foldername. in this case "lost+found" becomes "lost found".

Not sure what the purpose of QueryUnescape is, but once we remove it, it works just fine.

Software

  • OS: linux
  • Golang: 1.15
  • Version: latest master

To Reproduce

  1. Create a folder 'lost+found' in a folder 'a'
  2. Propfind folder 'a'
  3. Propfind returns 'lost found' as the folder name

Expected
Expected the folder name "lost+found" to stay that way

Www-Authenticate header's token should be treated as case-insensitive

Hi,

I've run into trouble using this library with a server that returns a Www-Authenticate header of "BASIC" instead of "Basic". This is allowed as per https://tools.ietf.org/html/rfc2617, but the check on Lines 46 and 48 of requests.go does a case-sensitive check against 'Digest' or 'Basic'.

Can this be changed to a case-insensitive match like
strings.Index(strings.ToLower(rs.Header.Get("Www-Authenticate")), "digest")?

Is there any workaround? I can't see any way to manually specify the authentication method.

Regards,

Wade

panic due to concurrent map writes

I'm sometimes getting this panic when running multiple requests in parallel using a single webdav client:

	/usr/local/Cellar/go/1.14.6/libexec/src/runtime/panic.go:1116 +0x72 fp=0xc0001d9448 sp=0xc0001d9418 pc=0x10351d2
runtime.mapassign_faststr(0x1c15400, 0xc00001f0b0, 0x1ce5cd4, 0xd, 0x64)
	/usr/local/Cellar/go/1.14.6/libexec/src/runtime/map_faststr.go:291 +0x3de fp=0xc0001d94b0 sp=0xc0001d9448 pc=0x101529e
net/textproto.MIMEHeader.Set(...)
	/usr/local/Cellar/go/1.14.6/libexec/src/net/textproto/header.go:22
net/http.Header.Set(...)
	/usr/local/Cellar/go/1.14.6/libexec/src/net/http/header.go:37
github.com/studio-b12/gowebdav.(*BasicAuth).Authorize(0xc0005585c0, 0xc00001f0e0, 0x1ce1500, 0x8, 0x1cdc5ce, 0x1)
	/Users/jarek/go/pkg/mod/github.com/studio-b12/[email protected]/basicAuth.go:32 +0x1d0 fp=0xc0001d9578 sp=0xc0001d94b0 pc=0x18d1c80
github.com/studio-b12/gowebdav.(*Client).req(0xc00001f0e0, 0x1ce1500, 0x8, 0x1cdc5ce, 0x1, 0x1e96e40, 0xc0002807a0, 0xc0001d97a0, 0x0, 0x0, ...)
	/Users/jarek/go/pkg/mod/github.com/studio-b12/[email protected]/requests.go:28 +0x283 fp=0xc0001d9740 sp=0xc0001d9578 pc=0x18d6353
github.com/studio-b12/gowebdav.(*Client).propfind(0xc00001f0e0, 0x1cdc5ce, 0x1, 0x26cdc00, 0x1d1eebf, 0xcb, 0x1acf280, 0xc00028d2f0, 0xc0001d98b0, 0x0, ...)
	/Users/jarek/go/pkg/mod/github.com/studio-b12/[email protected]/requests.go:93 +0x114 fp=0xc0001d9808 sp=0xc0001d9740 pc=0x18d7544
github.com/studio-b12/gowebdav.(*Client).ReadDir(0xc00001f0e0, 0x1cdc5ce, 0x1, 0x1cdc5ce, 0x1, 0x1cdc5ce, 0x1, 0xc0002a8520)
	/Users/jarek/go/pkg/mod/github.com/studio-b12/[email protected]/client.go:164 +0x1a6 fp=0xc0001d98e8 sp=0xc0001d9808 pc=0x18d2476

Should parallel calls using single webdav client be supported?

codereview

hi contributors,
hi folks,

we improved the project in a nice way.
thank you for your work and interest.
most of your commits were picked without a pull requests, this was fine.
please see my modifications as a kind of talking to and with you by source code.
so feel free to review and to improve the source and project.

let's start to review these files

  • requests.go
  • client.go

improve concurrency

Here are my points which I have to think more about.

  1. API changes in Authorize. I'd love to avoid it.
  2. Losing the context (Client) in Authorize.
  3. The Mutext, which is just needed for concurrency.

Originally posted by @chripo in #36 (comment)

Improve README

I propose to:

  1. create separate README file for gowebdav library and its cmd client
  2. remove godoc text from README file

Apache HTTPS Server does not return `Dav` or `DAV` header on `OPTIONS` request

Here is the current version of Connect function:

// Connect connects to our dav server
func (c *Client) Connect() error {
	rs, err := c.options("/")
	if err != nil {
		return err
	}

	err = rs.Body.Close()
	if err != nil {
		return err
	}

	if rs.StatusCode != 200 || (rs.Header.Get("Dav") == "" && rs.Header.Get("DAV") == "") {
		return newPathError("Connect", c.root, rs.StatusCode)
	}

	return nil
}

but Apache HTTPS Server does not return Dav or DAV header, so application returns following message on any request:

Failed to connect due to: Connect https://mywebdav.me/: 200

Is it bug in library, or wrong settings of Apache HTTPS Server?

401 Unauthorized when trying to login to Apache HTTPS Server

Hello, studio-b12!

I use following software:
OS: Windows 10 x64
Golang: 1.10.1

I trying to use your app to connect to my WebDav sever (Apache HTTPS Server), via following commands in cmd:

set ROOT=mywebdav.lan
set USER=user
set PASSWORD=password

gowebdav -X LS /user

and expect to see my files and folders in /user folder:

folder-1
folder-2
folder-3
file.txt

but instead I got following:

ReadDir /data/: 401 Unauthorized - PROPFIND /data/

My server works good and I have no problem to connect to it via Google Chrome.

How can I fix this?
Thanks in advance.

improve authentication

fix #14 in commit b45378c changed the behaviour of the req() method. the auth checks were moved from Connect() to req().

how to improve the code to keep the modification inside the client, because it's a client improvement and to keep the request as small as possible.

suggestions and discussion are welcome!

[BUG/FEATURE] OneDrive Support

Hello Collaborators,

Describe the bug
When using OneDrive as the WebDav server things get weird. In short: The authentication fails because of Microsoft.

Software

  • OS: Linux (Docker)
  • Golang: 1.19.4
  • Version: 60ec5ad

To Reproduce

  1. Create a client using OneDrive credentials
  2. Try to use any (authenticated) request (e.g. ReadDir())
  3. Response is empty / unexpected (e.g. no files)

Expected
At least an authentication error should be returned. Even better would be support for MS's need to be proprietary.

Additional context
Microsoft (as expected) does not follow industry standards and instead of returning 401 on unauthenticated requests they return 302 to redirect you to the login page. The login page then return a 200 status so the webdav client thinks it is a valid result for the original request. On its own this would not be too bad as you could catch the redirect and handle it like a 401. But since Microsoft use their own authentication method (MS-PASS with Passport1.4) you also need a new authenticator.

nil pointer exception in req() method

Software

OS: Windows 10 x64
GO: go1.9.2 windows/amd64
gowebdav: latest version from master

go env

set GOARCH=386
set GOBIN=
set GOEXE=.exe
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=D:\Go\
set GORACE=
set GOROOT=D:\Programs\Go
set GOTOOLDIR=D:\Programs\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set GO386=sse2
set CC=gcc
set GOGCCFLAGS=-m32 -fmessage-length=0 -fdebug-prefix-map=C:\Users\PC\AppData\Local\Temp\go-build886287271=/tmp/go-build -gno-record-gcc-switches
set CXX=g++
set CGO_ENABLED=0
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config

Issue description

We use http.Do() method inside c.req(), and do not check for errors returned. So, it is possible to receive nil response, and got nil pointer dereference panic in line 42.
http.Do() method will return non-nil error in following cases:

  1. Request is nil
  2. Response missing Location header
  3. Client failed to parse Location header
  4. Method "request.GetBody()" returns error
  5. Http.Client.Send() returns error
  6. Client timeout was exceeded
    In all cases except of (5) this method returns nil as a poiner to http.Response, which will force app to panic later.

Conclusion

Taking into account the above mentioned, we need to perform error-checking for http.Do() method results.

File remove problem

Some webdav servers (like yandex) send 204 status for successful delete operation. On client.go file (line: 195) you can add 204 status. Note: Thanks for this good client.

Can't upload file with content

Hello Collaborators,
during testing current version of gowebdav I notice that client's method WriteStream does not set request body even if body should be set for current request (for example during uploading non-empty file via gowebdav.exe -X PUT /my_folder/file.txt file.txt). Also, @chuckwagoncomputing face with this problem too, but he used propfind request instead.

Software

  • OS: Windows 10 x64
  • Golang: 1.10
  • Version: latest commit in master is 008b27e

To Reproduce

  1. build gowebdav/cmd/gowebdav with go build to get gowebdav.exe
  2. set following environment's variables:
    1. ROOT -- WebDav Server's URL
    2. USER -- WebDav Server's username
    3. PASSWORD -- WebDav Server's password
  3. suppose your WebDav Server's folder is my_folder, and your local file to upload located near gowebdav.exe, named file.txt and has some text (so actually file's size is not zero).
  4. run following command: gowebdav.exe -X PUT /my_folder/file.txt file.txt
  5. go to your WebDav Server via any other interface (putty, web browser etc), and you will see that file is exist but file's size is zero

Expected
Uploaded file contains body

Actually
Uploaded file does not contains body

memleak during multiple file upload when authentication is required

Hello Collaborators,

Describe the bug
A short description of what you think the bug is.

Software

  • OS: Linux Mint 19 Tara
  • Golang: go version go1.10.1 linux/amd64
  • Version: latest studio-b12/gowebdav (loaded via go get)

To Reproduce
I've started to build a small tool which would allow file syncing via WebDAV. It's not fully ready but at least it allows to scan a folder and create a list of files which should be uploaded. The flow of the program now is the following:

  1. Create list of files for uploading
  2. Create N goroutines, each has its own gowebdav.Client
  3. Filenames for uploading are sent to these goroutines via channels (fan-out), then each file is uploaded independently via Client.WriteStream. The file is closed via defer.
  4. I used WebDAV API for Yandex.Disk, but I believe the same problem could be reproduced with any server with configured Basic authentication.

What I observe is enormous RAM usage by my tool (e.g. even if a folder contains files for 1G, it can easily take up to several gigabytes of RAM). I've tried to narrow the scope of the problem and here's what I've got:

  • GODEBUG=gctrace=1 shows that upon each gc cycle it could collect 0 space, and the space constantly grows
  • pprof.WriteHeapProfile at the end gives me a profile which shows that there's some inuse_space related to TeeReader from requests, though acquired by persistConn inside of http.Transport. It's not several gigabytes, but up to several hundred megabytes.
  • I tried to use Client.SetTransport() providing my http.Transport with reduced IdleConnTimeout, MaxIdleConns, DisableKeepAlive and other similar things like explicit call . Nothing changed.
  • Finally, I reverted gowebdav to revision prior to the fix for issue #14 - no additional memory has been taken (memory footprint was just about several megabytes according to gctrace)

I believe there's something wrong with current usage of TeeReader introduced in #b45378c. It looks like persistConn inside of a Transport somehow doesn't allow to collect the unused objects (i.e even if a new file gets uploaded, the data for the old one doesn't get cleared). Unfortunately I'm pretty new to Go and couldn't resolve the issue by myself.

Expected
Well, no memleak

Additional context
Just in case you're curious how do I use gowebdav exactly, here is the tool I'm using.

Please let me know if you need any additional info.

support Walk func

Webdav does not require the server to support recursive traversal, but recursive traversal is commonly used in some scenarios, so we should support the walk function for users. just like filepath.Walk()

[WIP] Supports upload files via WriteCloser

Supports write file via io.WriteCloser i.e., create a function that returns io.WriteCloser.

Could be as follows:
#63 func (c *Client)CreateWdFile(path string,r io.Reader) (writer io.WriteCloser, err error) {}

Support range bytes request

I tried to call ReadStream with header SetHeader("Range", "bytes=10-100"), but it failed because ReadStream only accepts 200 http response code. The response code of range bytes request is 206 (Partial Content).

if rs.StatusCode == 200 {

I think the above code needs to be revised as follows:
if rs.StatusCode == 200 || rs.StatusCode == 206 {

No support for Bearer Token authentication

Hello Collaborators,

Describe the bug
401 Authorization error on trying to connect to a WebDAV server with Bearer type of authentication

Software

  • OS: Ubuntu 20.04 LTS
  • Golang: go version go1.19.2 linux/amd64
  • Version: latest studio-b12/gowebdav (loaded via go get)

To Reproduce

  1. Connect to any WebDAV server which requires Bearer type of authentication. I used OwnCloud
  2. Try to read the list of directories using c.ReadDir("/") (Any other operation also fails)
  3. Error: 401 Authorization Error

Expected
No authorization error, the client should be able to connect to the server and execute any operation

Additional context
This seems to be due the fact that only two types of auth: Basic and Digest are supported.
I would be willing to work on a PR to add Bearer authentication support

Wrong content of uploaded file

Hello Collaborators,
During testing current version of gowebdav/cmd I notice that uploaded file has incorrect content if function getStream() can not find required file to open.

Software

  • OS: Windows
  • Golang: go version go1.11.4 windows/amd64
  • Version: current version of gowebdav/cmd

To Reproduce

  • call gowebdav.exe with args: -X PUT /webdav_folder/file.txt

Expected
File not found error returned

Actual
Application uploads file with it's path as content

Set GetBody on http.Request

A user of Kopia reported this error in kopia/kopia#1509:

Propfind "https://myhost.mydomain.com/kopia/p5f/7be/": http2: Transport: cannot retry err [http2: Transport received Serv
er's graceful shutdown GOAWAY] after Request.Body was written; define Request.GetBody to avoid this error

There appears to be an issue with some proxy servers that force connection closure after receiving the request and Go cannot automatically retry the request unless http.Request.GetBody is set.

Could this be added in gowebdav ?

400 when passing whitespace in path args to c.Copy()

Hello Collaborators,

Describe the bug

c.Copy() doesn't handle whitespace characters in file/folder names.

Software

  • OS: ubuntu
  • Golang: 1.15.7
  • Version: latest

To Reproduce

  1. Create a file or folder with whitespace in the name on the remote webdav server.
  2. Make a request with client.Copy with oldpath and newpath being strings with spaces in the name.
  3. 400 Error is returned

Expected
Spaces should be handled.

Additional context
Add any other context about the problem here.

ReadDir omits returning the first result if it is a directory

Hello Collaborators,

I am reading a directory using ReadDir API. The contents of the directory are two directories. ReadDir only returns the second directory.

I debugged the code, and the culprit is the following block of code in client.go file

	if skipSelf {
		skipSelf = false
		if p := getProps(r, "200"); p != nil && p.Type.Local == "collection" {
			r.Props = nil
			return nil
		}
		return newPathError("ReadDir", path, 405)
	}

If I comment out this code, it works as expected. I do not know what this skipSelf thingy is. It does not seem to have it there be necessary when I used it

Software

  • OS: Windows 10
  • Golang:
  • Version: 1.15

To Reproduce

  1. ReadDir on a folder that has two folders (no files)
  2. You will the second folder in the result

Expected
ReadDir should return both dirs.

Additional context
My test file: main.go - you need to add your WebDav Url and username/password.

package main

import (
"log"
"path/filepath"

"github.com/studio-b12/gowebdav"

)

func main() {
root := "xxxx"
user := "yyyy"
password := "zzzz"

c := gowebdav.NewClient(root, user, password)
err := c.Connect()
if err != nil {
	log.Fatalf("could not connect %v\n", err)
}

readDir(c, "/")

}

func readDir(c *gowebdav.Client, dpath string) {
fi, err := c.ReadDir(dpath)
if err != nil {
log.Fatalf("could not read dir %v\n", err)
}
for _, file := range fi {
if file.IsDir() {
d := filepath.ToSlash(filepath.Join(dpath, file.Name()))
log.Printf("Directory: %v\n", d)
readDir(c, d)
continue
}
log.Printf("File: %v\n", file.Name())
}
}

[err:WriteStream] file write problem: err:WriteStream

Hello Collaborators,

Describe the bug
A short description of what you think the bug is.

Software

  • OS: win 11
  • Golang: 1.18.1
  • Version: 8

To Reproduce

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected
A short description of what you expected to happen.

err:WriteStream xx: 500

Additional context
Add any other context about the problem here.

server is on WSL2 of docker, images is derkades/webdav.
I try to upload a image file

dav_error

gowebdav library should not hush up errors

If we carefully see on following methods:

  • client.mkcol(), client.Mkdir(), client.MkdirAll()
  • client.copymove(), client.doCopyMove()
  • client.Write() and client.WriteStream()
  • it suppress real errors and returns only status code in the best case.

I propose to change this behaviour to allow user of our library to get original error message.

Can't authorize at server

Hello.
I found a bug about authorization.

At requests.go:req(...) authorization processed after copying headers from WebDAV client to HTTP client, not before. So authorization is impossible.

I created pull request which solves this issue -- #13

Stat() for directory does not return last modified time

Hello!

When I use Stat() for retrieve directory last modification time, I got zero-time.

In source code (client.go):

  if p.Type.Local == "collection" {
  if !strings.HasSuffix(f.path, "/") {
	  f.path += "/"
  }
  f.size = 0
  f.modified = time.Unix(0, 0) // Why ??
  f.isdir = true
  } else {
  f.size = parseInt64(&p.Size)
  f.modified = parseModified(&p.Modified)
  f.isdir = false
}

File upload uploads files with 0 bytes - No error given

Describe the bug
I discovered a bug in the file upload that has some relation to #20.
When uploading a file to Nextcloud, the command succeeds and exits without any error, but the uploaded file has a size of 0 bytes.

Software

  • OS: Windows (also tested on Linux)
  • Golang: go1.14.6 windows/amd64
  • Version: current master

To Reproduce

  1. checkout this project
  2. Setup your environment with the ROOT, USER and PASSWORD
  3. Pick a file you want to upload
  4. Upload the file using the command
gowebdav -X PUT /webdav_test.png .\Chart_KAD.PNG
Put: .\Chart_KAD.PNG -> /webdav_test.png`
  1. Look at the Nextcloud instance to see that the actual uploaded file-size is 0 bytes.

image
image

Apache access log (nextcloud)

80.187.108.153 - - [12/Aug/2020:12:32:26 +0200] "PUT /remote.php/dav/files/RAYs3T/webdav_test.png HTTP/1.1" 401 557
80.187.108.153 - - [12/Aug/2020:12:32:27 +0200] "PUT /remote.php/dav/files/RAYs3T/webdav_test.png HTTP/1.1" 201 -

Expected
The file should upload completly or fail with an error

Additional context
I initially discovered this issue via PhotoPrism (which uses this client for the uploads).
At least one other user is facing the same issue (photoprism/photoprism#443).

This is a library usage question,about ContentType.

I saw that the Type File Struct has a field of ContentType that can be used, but this is just a struct and cannot get the ContentType of the file. Maybe I'm not very familiar with golang and would like to ask how to get it?

type File struct {
	path        string
	name        string
	contentType string
	size        int64
	modified    time.Time
	etag        string
	isdir       bool
}

func (f File) ContentType() string {
	return f.contentType
}

Support for cookies

Hello Collaborators,

can we add the ability to set the CookieJar of the internal http.Client of the gowebdav.Client?
I got proprietary webdav server, which does authentication via kerberos. It is way faster to use the cookie after the initial authentication than to reauthenticate at every request.

I see 3 possibilities to implement this:

  1. provide a SetJar method for the gowebdav.Client like this

    func (c *Client) SetJar(jar http.CookieJar) {
    	c.c.Jar = jar
    }
  2. expose the internal http.Client by making the c field public or add an option to gowebdav.NewClient so a configured http.Client for the webdav client to use can be provided.

  3. use the http.DefaultClient in gowebdav.NewClient function to initialize the c field.

I like number 2. with an option to provide a http.Client best, because it adds the most flexibility. Maybe there is a good reason why the library uses its own clean http.Client that I don't see right now. In that case I would like number 1 implemented.

Thoughts?

(I can provide the pull request if we reach a decision)

Digest auth usage

Hello Collaborators,

From the docs I see there is a support to use DigestAuth, but I couldn't find how exactly am I supposed to pass the credentials for using Digest Auth, I see by default gowebdav uses basic auth. How can I explicitly ask it to use digest auth? Could you provide an example on how to use DigestAuth with gowebdav client?

Software

  • OS: Mac
  • Golang:
go version
go version go1.17.1 darwin/amd64
  • Version:
github.com/studio-b12/gowebdav v0.0.0-20230203202212-3282f94193f2

Additional context
I'm trying to upload my HLS content to AWS Elemental MediaPackager

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.