Giter Club home page Giter Club logo

multipart-kit's Introduction

MultipartKit

Documentation Team Chat MIT License Continuous Integration Swift 5.7+

🏞 Multipart parser and serializer with Codable support for Multipart Form Data.

Installation

The table below shows a list of MultipartKit major releases alongside their compatible NIO and Swift versions.

Version NIO Swift SPM
4.0 2.2 5.4+ from: "4.0.0"
3.0 1.0 4.0+ from: "3.0.0"
2.0 N/A 3.1+ from: "2.0.0"
1.0 N/A 3.1+ from: "1.0.0"

Use the SPM string to easily include the dependency in your Package.swift file.

Add MultiPartKit to your package dependencies:

dependencies: [
    // ...
    .package(url: "https://github.com/vapor/multipart-kit.git", from: "4.0.0"),
]

Add MultiPartKit to your target's dependencies:

targets: [
    .target(name: "MyAppTarget", dependencies: [
        // ...
        .product(name: "MultipartKit", package: "multipart-kit"),
    ])
]

Supported Platforms

MultipartKit supports the following platforms:

  • All Linux distributions supported by Swift 5.7+
  • macOS 10.15+

Overview

MultipartKit is a multipart parsing and serializing library. It provides Codable support for the special case of the multipart/form-data media type through a FormDataEncoder and FormDataDecoder. The parser delivers its output as it is parsed through callbacks suitable for streaming.

Multipart Form Data

Let's define a Codable type and a choose a boundary used to separate the multipart parts.

struct User: Codable {
    let name: String
    let email: String
}
let user = User(name: "Ed", email: "[email protected]")
let boundary = "abc123"

We can encode this instance of a our type using a FormDataEncoder.

let encoded = try FormDataEncoder().encode(foo, boundary: boundary)

The output looks then looks like this.

--abc123
Content-Disposition: form-data; name="name"

Ed
--abc123
Content-Disposition: form-data; name="email"

[email protected]
--abc123--

In order to decode this message we feed this output and the same boundary to a FormDataDecoder and we get back an identical instance to the one we started with.

let decoded = try FormDataDecoder().decode(User.self, from: encoded, boundary: boundary)

A note on null

As there is no standard defined for how to represent null in Multipart (unlike, for instance, JSON), FormDataEncoder and FormDataDecoder do not support encoding or decoding null respectively.

Nesting and Collections

Nested structures can be represented by naming the parts such that they describe a path using square brackets to denote contained properties or elements in a collection. The following example shows what that looks like in practice.

struct Nested: Encodable {
    let tag: String
    let flag: Bool
    let nested: [Nested]
}
let boundary = "abc123"
let nested = Nested(tag: "a", flag: true, nested: [Nested(tag: "b", flag: false, nested: [])])
let encoded = try FormDataEncoder().encode(nested, boundary: boundary)

This results in the content below.

--abc123
Content-Disposition: form-data; name="tag"

a
--abc123
Content-Disposition: form-data; name="flag"

true
--abc123
Content-Disposition: form-data; name="nested[0][tag]"

b
--abc123
Content-Disposition: form-data; name="nested[0][flag]"

false
--abc123--

Note that the array elements always include the index (as opposed to just []) in order to support complex nesting.

multipart-kit's People

Contributors

0xtim avatar adam-fowler avatar ahmdyasser avatar bennydebock avatar bre7 avatar brettrtoomey avatar calebkleveter avatar ckd avatar dkolas avatar grundoon avatar gwynne avatar jaapwijnen avatar loganwright avatar mihaelisaev avatar ole avatar patrick-kladek avatar siemensikkema avatar t-ae avatar tanner0101 avatar ulrikdamm avatar vi4m avatar vzsg avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

multipart-kit's Issues

Crash on multipart/form-data decoding with empty mime type.

Describe the bug

Server crashes on a multipart/form-data request with an empty mimeType.

To Reproduce

Steps to reproduce the behavior:

  1. Send a multipart/form-data request from a client with an empty mime type.
  2. Try decoding the request in a Content conform struct on the server.
  3. Server crashes with a fatalError, no error messages provided.

Expected behavior

The request throw a decoding error avoiding server crashes and providing a readable error message.

Environment

  • Vapor Framework version: 4.38.0
  • OS version: Ubuntu 20.04, macOS 10.15.7

Additional context

This is how the AlamoFire request is sent (See MultipartFormData):

multipart.append(file, withName: "file", fileName: "my_file.zip", mimeType: "")

and this is how I decode it :


func upload(_ req: Request) throws {
      let file = try req.content.decode(Upload.self)
      // doing some other stuff
}
struct Upload: Content {
     let file : File 
}

Using Backtrace I found the line where the fatalError is raised :

Fatal error: file Vapor/MultipartParser.swift, line 186

The error is raised on the handleHeadersComplete function.

Me and the iOS developer spent some hours digging into the MultipartParser decoding strategy and we found the headers decoding is made using three different functions:

  • handleHeaderField (Vapor/MultipartParser.swift, line 158)
  • handleHeaderValue (Vapor/MultipartParser.swift, line 170)
  • handleHeadersComplete (Vapor/MultipartParser.swift, line 186) where we got the fatal error.

Simplifying a lot the handleHeaderField function read the header name, handleHeaderValue the value for that header and handleHeadersComplete declares the header is complete.

Passing an empty mime type, the parser fail to decode the "Content-Type" header raising a fatalError, stopping the execution and causing a server crash.

Data or File Data Member of Decodable Parsed Incorrectly on Linux

Short backstory, I pushed a new version of my backend a few days ago after testing it on my Mac extensively first, including file/photo upload. After pushing it and making sure things were still working, I discovered that file upload was broken everywhere. I boiled the issue down to an update in multipart to 3.0.4 from 3.0.3 that was the cause of the issue.

This issue at least exists on Linux running Swift 4.2 and later.

Pictures and files were being uploaded but the files saved were not what was uploaded. The files being saved included part of the actual file data but also included the parts of the multipart requests including the boundaries. The file/picture data portion is always cut off and never complete.

For example, I have a request that includes an integer and then a file for the other part. The file from the request once it has been decoded would look like:

--__X_PAW_BOUNDARY__
Content-Disposition: form-data; name="id"

10
--__X_PAW_BOUNDARY__
Content-Disposition: form-data; name="image"; filename="image.png"
Content-Type: image/png

âPNG
�
���
IHDR���Y���>�����@˙∑M���ziCCPICC Profile��(ë}êM+DQ�«�3à�çba°‹º≠Ü�%6 L�S≥ò�ÂmsÁö�53nwÆêçÖ≤Uîÿx[	ÿX(k•�)Ÿ¯�ƒF∫ûchºîߌ9øÛúÁ˘˜ú?∏}∫ifJ;!õ≥≠X(†ççOhÂ�∏RC#•∫ë7�¢—��_Áœxπñjâ´v•ı˜˝fl®öN‰
pU�˜�¶e��	7œ€¶b•Wg…P¬ÀäS�fiP�/—GÕH,(|*¨�i}Z¯Nÿg§≠,∏ï~K¸[MÍ�g3s∆Á<Í'ûDntXŒ&Y
‰â�"ÄFòAÇÙ–EüÏ=¥„ßCnÿâ�[5�gÕEk&ï∂µ�q"°ÖsFáOÛwvıÅÚı∑_≈‹Ï.Ù>C…Z1�flÑìU®ø-ÊZv¿ª�«Á¶nÈ�©�YÓd���°z�j/°r2üψ�~‰	@ŸΩ„<µB˘:º≠9ŒÎû„ºÌK≥xtñ+xÙ©≈¡
å,A‰�∂∂°M¥ΩSÔfi¥g�R‰¿����	pHYs��.#��.#�x•?v����iTXtXML:com.adobe.xmp�����<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 5.4.0">
   <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
      <rdf:Description rdf:about=""
            xmlns:xmp="http://ns.adobe.com/xap/1.0/"
            xmlns:tiff="http://ns.adobe.com/tiff/1.0/"
            xmlns:xmpMM="http://ns.adobe.com/xap/1.0/mm/"
            xmlns:stRef="http://ns.adobe.com/xap/1.0/sType/ResourceRef#"
            xmlns:stEvt="http://ns.adobe.com/xap/1.0/sType/ResourceEvent#"
            xmlns:exif="http://ns.adobe.com/exif/1.0/"
            xmlns:dc="http://purl.org/dc/elements/1.1/"
            xmlns:photoshop="http://ns.adobe.com/photoshop/1.0/">
         <xmp:ModifyDate>2018-12-24T10:05:48-06:00</xmp:ModifyDate>
         <xmp:CreateDate>2012-01-16T10:46:07-06:00</xmp:CreateDate>
... (there's more just I'm not including it here)

I'll also include a file that is exactly what is saved when decoded from multipart 3.0.4 that was running on my server. If you open the file, you can see that the image portion of the request is cut off and incomplete.

Here's the struct I was using:

struct BackflowPayload: Decodable {
     var image: File
     var id: Int
 }

I also tried this struct to see if there was a difference:

    struct BackflowPayload: Decodable {
        var image: Data
        var id: Int
    }

I did do some testing to see if I could find the exact cause. The image size in bytes is reported correctly on Linux when compared against the backend running on my Mac. I think the data start offset that is copied into the respective data member is incorrect for parts of type Data or File on Linux. My guess is that the number of bytes taken up by the other parts of the multipart requests is why the data/file portion being saved is cut off.

test.txt

Uploading multiple files result in zero files uploaded

multipart 3.0.1, vapor 3.0.0-rc.2.7, leaf 3.0.0-rc.2

I have a handler of the following form, but when I upload one or multiple files, the result is always an empty array.
Tested in Safari and Chrome on a Mac.

  router.post("multiple") { req -> Future<Response> in
       struct ImageFile : Content {
           var upload: [File]
       }
       return try req.content.decode(ImageFile.self).map(to: Response.self, { forms in
           let ploep = forms.upload            // Xcode lldb cannot decode forms
                                               // for some reason, ploep.count is always 0
           print("files uploaded: \(ploep)")
           guard forms.upload.count > 0 else {
               return req.redirect(to: "/", type: RedirectType.normal)
           }
           return req.redirect(to: "/", type: RedirectType.normal)          //.redirect(to: "/", [:])
      })
   }

I created a sample Vapor 3 project at https://github.com/axello/VaporMultipart.
When upload is defined as a single File, it works.
When uploading multiple files, the result is an empty array.

Unable to decode nested `multipart` array.

Describe the bug

struct Payload: Content {
    struct NestedPayload: Content {
        let name: String
        let age: Int
    }
    let nestedPayload: [NestedPayload]
}

let encoder = FormDataEncoder()
let encoded = try encoder.encode(Payload(nestedPayload: [.init(name: "abc", age: 10),
                                                         .init(name: "def", age: 20)]),
                                 boundary: "-")
let decoder = FormDataDecoder()
let decoded = try decoder.decode(Payload.self, from: encoded, boundary: "-")

got output [ WARNING ] Value required for key 'nestedPayload.0.age'.

Environment

  • Vapor Framework version: 4.27.0
  • Multipart-Kit Framework version: 4.2.1
  • Vapor Toolbox version: 18.3.3
  • OS version: Mac OS 11.3

Add null-termination to strncpy call

It seems there is a bug of this nature https://stackoverflow.com/questions/1176737/strncpy-and-using-sizeof-to-copy-maximum-characters

Located here https://github.com/francoiscolas/multipart-parser/blob/b0bc117dba58a03bb99f46bd8d03a720bb802e7b/multipartparser.c#L96-L97

It seems it is fixable by adding - 1 e.g. strncpy(parser->boundary, boundary, sizeof(parser->boundary) - 1);, because of the preceding memset(parser, 0, sizeof(*parser)); call.

Apologies in advanced if this is a false-positive.

Need support `Bool` values

I tried to decode multipart request with Bool property in my codable model, but I caught an error

[ ERROR ] MultipartError.convertible: `Bool` is not `MultipartPartConvertible`.

It looks like a bug 🙂
I use the latest version of Multipart (v3.0.1) from April, 20.

Decoding String enum

Describe the bug

Server show the error on a multipart/form-data request with a String enum - "Could not convert value at "license" to type License from multipart part. for key license". Earlier versions of multipart-kit work as expected. At least 4.0.0.

enum License: String, Codable, CaseIterable {
    case dme1
}

To Reproduce

Steps to reproduce the behavior:

  1. Send a multipart/form-data request from a client with a String enum.
  2. Try decoding the request in a Content conform struct on the server.
  3. Server shows the error.

Expected behavior

Correct decoding String enum.

Environment

  • Vapor Framework version: 4.47.0
  • OS version: Ubuntu 20.04, macOS 11.4
  • multipart-kit: 4.2.0

Performance issue

Describe the bug

OS - Ubuntu 20.04
When i try to upload the file more than 65 kb , server hangs sometimes . I got the error from nginx
request timed out. i have set proxy_read_time set to 10s .

I have also tried this in local setup (mac mini M1). Its takes 6-7 seconds to upload the file. the issue seems to exist in 4.0.0 + version

When i tried to upload file with cmultiparser (4.0.0) . Server never hangs and response time is also fast. In local setup it takes milliseconds

To Reproduce

Steps to reproduce the behavior:

  1. Upload the file larger than 65kb you will see the lag . You can try in local setup too

I think issue exist in this function

private func execute() throws {

Expected behavior

It should you work like cmultiparser

Environment

  • Vapor Framework version: 4.67.0

  • Vapor Toolbox version:

  • OS version: Ubuntu 20.04

  • Vapor Framework version: 4.67.0

  • Vapor Toolbox version:

  • OS version: Mac mini M1 ( Ventura 13.0 (22A380))

Missing quotes from values

Missing quotes in values other than encoded file content. Interesting discussion in akka/akka-http#386 (includes spec requirements as well as browser compatibility comments)

E.g:

--BOUNDARY
Content-Disposition: form-data; filename=test.pdf; name=data
Content-Type: application/pdf

%PDF-1.3
...

Un-vendor the `swift-collections` implementation

multipart-kit currently contains a somewhat outdated version of OrderedSet and OrderedDictionary taken from the then-current pre-release version of swift-collections; this vendored implementation should be replaced with a dependency on swift-collections 1.x.

cc @siemensikkema

Add support for decoding multipart/form-data without knowing the boundary

Is your feature request related to a problem? Please describe.
I need to parse multipart/form-data that is sent from an iOS app using Alamofire without knowing what the boundary is before decoding. I've looked around — a little bit — and it appears that this is not something supported by this library. Alamofire does not have a convenience method that permits you to specify a custom value for the boundary. By default, the boundary appears to be a randomly generated value. Alamofire does seem to support creating multipart/form-data requests with a custom boundary value, but it's a bit more involved than a single method call. It's likely the route that I'm going to take, but I thought I'd add this feature request here in case others ended up experiencing the same inconvenience.

Describe the solution you'd like
It looks to me that the boundary is defined within the body of multipart/form-data. So what I'd like is for a solution to exist that grabs this information from the data, and then uses it instead of requiring the consumer to provide it themselves.

Describe alternatives you've considered

  1. Researching how to set a custom boundary using Alamofire.
  2. Implementing a solution in Swift that suits my needs.
  3. Not uploading data in the multipart/form-data format.
  4. Using a Python script to parse the request body that is called from Swift code.

Additional context
None

Support fom multiple files from a single form field

I cannot get multiple files from a single field in a form.

With HTML like this:

<input class="form-control" type="file" id="images" name="images" multiple>

The form gets submitted, but only one of the files is passed by Vapor and available in the controller.

Decoding optional type cause error

Describe the bug

This one works:

let foo = try FormDataDecoder().decode(Foo.self, from: data, boundary: "12345")

this one doesn't

let foo = try FormDataDecoder().decode(Foo?.self, from: data, boundary: "12345")

Error:

caught error: "typeMismatch(MultipartKit.OrderedDictionary<Swift.String, MultipartKit.MultipartFormData>, Swift.DecodingError.Context(codingPath: [], debugDescription: "expected single value but encountered dictionary", underlyingError: nil))"

To Reproduce

You can adjust func testFormDataDecoderW3() to see this behaviour

Expected behavior

Content should be decoded no matter if type is Foo.self or Foo?.self

Environment

Tested od mian branch

Bug in FormDataEncoder with Optional<CustomValue>

Describe the bug

I don't know english...
file FormDataEncoder.swift

private final class FormDataEncoderContext {
    var parts: [MultipartPart]
    init() {
        self.parts = []
    }

    func encode<E>(_ encodable: E, at codingPath: [CodingKey]) throws where E: Encodable {
        guard let convertible = encodable as? MultipartPartConvertible else {
            throw MultipartError.convertibleType(E.self)
        }

        guard var part = convertible.multipart else {
            throw MultipartError.convertibleType(E.self)
        }
        
        switch codingPath.count {
        case 1: part.name = codingPath[0].stringValue
        case 2:
            guard codingPath[1].intValue != nil else {
                throw MultipartError.nesting
            }
            part.name = codingPath[0].stringValue + "[]"
        default:
            throw MultipartError.nesting
        }
        self.parts.append(part)
    }
}


private struct _FormDataEncoder: Encoder {
    let codingPath: [CodingKey]
    let multipart: FormDataEncoderContext
    var userInfo: [CodingUserInfoKey: Any] {
        return [:]
    }

    init(multipart: FormDataEncoderContext, codingPath: [CodingKey]) {
        self.multipart = multipart
        self.codingPath = codingPath
    }

    func container<Key>(keyedBy type: Key.Type) -> KeyedEncodingContainer<Key> where Key : CodingKey {
        return KeyedEncodingContainer(_FormDataKeyedEncoder(multipart: multipart, codingPath: codingPath))
    }

    func unkeyedContainer() -> UnkeyedEncodingContainer {
        return _FormDataUnkeyedEncoder(multipart: multipart, codingPath: codingPath)
    }

    func singleValueContainer() -> SingleValueEncodingContainer {
        return _FormDataSingleValueEncoder(multipart: multipart, codingPath: codingPath)
    }
}
struct A: Encodable {
    var number: Int 
}

struct B: Encodable {
    var a: A? /// this Optional<A>
}

and such optional objects follow the call path func singleValueContainer() -> SingleValueEncodingContainer

after the method is called

func encode(_ encodable: E, at codingPath: [CodingKey]) throws where E: Encodable {

and does not pass this test

encodable as? MultipartPartConvertible

because the struct A not conform MultipartPartConvertible

end

Serialization seems to contain wrong data

Hi,

I've been trying using this library (version 4.0.0-beta.2) together with AsyncHTTPClient to execute a multipart POST request, where I need to upload a file.

The expected request should look like the following

> POST /rest/api/content/595596971/child/attachment/609652887/data HTTP/1.1
> Host: myHost.com
> Authorization: Basic myBasic
> User-Agent: insomnia/7.1.1
> X-Atlassian-Token: nocheck
> Content-Type: multipart/form-data; boundary=X-INSOMNIA-BOUNDARY
> Accept: */*
> Content-Length: 656151

| --X-INSOMNIA-BOUNDARY
| Content-Disposition: form-data; name="file"; filename="picture.jpg"
| Content-Type: image/jpeg
It continues. I cut it because irrelevant.

This request works well through Insomnia and I tried to convert it to Swift using MultipartKit as below

import MultipartKit
import Foundation

let boundary = "MY-BOUNDARY"
let fileUrl = URL(fileURLWithPath: "/Users/myUser/Desktop/picture.jpg")
let fileData = try! Data(contentsOf: fileUrl)
var multipart = MultipartPart(body: fileData)
multipart.headers.replaceOrAdd(
    name: "Content-Disposition",
    value: "form-data; name=\"file\"; filename=\"picture.jpg\""
)
multipart.headers.replaceOrAdd(
    name: "Content-Type",
    value: "image/jpeg"
)

let body = try MultipartSerializer().serialize(
    parts: [multipart],
    boundary: boundary
)

print(Data(body.utf8).count) // 1130517

Unfortunately the data count (1130517) is quite bigger than expected (see Content-Length from Insomnia output).

I've tried my own implementation of this simple case using Foundation and I actually am able to create the correct payload

let multipartData = try MultipartHelper().multipart(
    from: fileUrl.path,
    partName: "file",
    boundary: boundary
)

print(multipartData.count) // 656135

And MultipartHelper.swift

import Foundation

struct MultipartHelper {
    enum Error: Swift.Error {
        case fileDoesNotExist
        case unsupportedFileType(String)
    }

    func multipart(from filePath: String, partName: String, boundary: String) throws -> Data {
        guard FileManager.default.fileExists(atPath: filePath) else {
            throw Error.fileDoesNotExist
        }

        let fileUrl = URL(fileURLWithPath: filePath)
        let fileName = fileUrl.lastPathComponent
        let fileExtension = fileUrl.pathExtension
        guard !fileName.isEmpty, !fileExtension.isEmpty else {
            throw Error.fileDoesNotExist
        }

        guard let contentType = ContentType(fileExt: fileExtension) else {
            throw Error.unsupportedFileType(fileName)
        }

        let fileData: Data
        do {
            fileData = try Data(contentsOf: fileUrl)
        } catch {
            throw Error.fileDoesNotExist
        }

        let headersPart = Data("""
        --\(boundary)\r\n\
        Content-Disposition: form-data; name="\(partName)"; filename="\(fileName)"\r\n\
        Content-Type: \(contentType.header)\r\n\
        \r\n
        """.utf8)
        let footerPart = Data("""
        \r\n--\(boundary)--\r\n
        """.utf8)

        var data = Data()
        data.append(headersPart)
        data.append(fileData)
        data.append(footerPart)

        return data
    }
}

enum ContentType {
    case png, jpeg, gif

    init?(fileExt: String) {
        switch fileExt.lowercased() {
        case "png":
            self = .png

        case "jpeg", "jpg":
            self = .jpeg

        case "gif":
            self = .gif

        default:
            return nil
        }
    }

    var header: String {
        switch self {
        case .gif:
            return "image/gif"

        case .jpeg:
            return "image/jpeg"

        case .png:
            return "image/png"
        }
    }
}

Am I somehow using the library incorrectly?

How to upload file on iOS?

I was trying to encode struct like that:

struct MyStruct {
    let imageFile: Data
}
let encoder = FormDataEncoder()
let body = try encoder.encode(options, boundary: boundary)

But imageFile is not recognized as file part. Also I could not find any example.

Also sorry for bug label - don't know how to remove it 😅

Replace [File] encoding with a real, generic solution

I pushed forward #28 to scratch an immediate itch, but the current implementation is more of a hack as it's handling [File] as a concrete type.

A generic extension on Array (where Element: MultipartPartConvertible) would probably be a cleaner solution.

Unnecessary conversions between `[UInt8]` and `ByteBuffer`

Describe the bug

When decoding a multipart buffer using FormDataDecoder you generally start with a ByteBuffer. FormDataDecoder forces you to convert this to [UInt8] which then it converts back to a ByteBuffer internally inside MultipartParser.execute. This is causing unnecessary data copying.

Expected behavior

Add

FormDataDecoder .decode<D: Decodable>(_ decodable: D.Type, from data: ByteBuffer, boundary: String) throws -> D

which uses ByteBuffer version of MultipartParser.execute.

Set other versions of FormDataDecoder.decode to use this new version

Encoding corrupts data

Describe the bug

When using FormDataEncoder to encode data objects, specifically JPEG image data, the data gets corrupted.

To Reproduce

This can easily be reproduced by feeding JPEG image data into FormDataEncoder's encode function, then feeding it's output into FormDataDecoder's decode function and comparing the input data with the output data.

let folder = #filePath.split(separator: "/").dropLast().joined(separator: "/")
let path = "/" + folder + "/Utilities/image.jpeg"
let originalData = try Data(contentsOf: URL(fileURLWithPath: path))

struct ObjectToEncode: Codable {
    let data: Data
}

let object = ObjectToEncode(data: originalData)
let boundary = UUID().uuidString

let encoded = try FormDataEncoder().encode(object, boundary: boundary)
let decoded = try FormDataDecoder().decode(ObjectToEncode.self, from: encoded, boundary: boundary)

XCTAssertEqual(originalData, decoded.data)

The comparison fails indicating that the original data is not equal to the decoded data.

When inspecting the bytes it shows a ~90% increase in size. @Joannis suggested one can expect such increase with base64 encoding while @vzsg was more specific pointing out that base64's "expanded size" is 4:3 not 3:2.

@vzsg also expressed concern that the encode method returns a String instead of Data as binary files may not be valid UTF-8.

Expected behavior

The decoded data is equal to the original data.

Environment

multipart-kit version: 4.6.0
swift-tools-version: 5.7
macOS version: 14.2.1

POSTing file with filename as single part

Is there a way to encode a file into a single part so it can be sent via POST ?
Use case: User sends "file.pdf" to Vapor *** Vapor posts the file to a remote API

Code

public struct APIFile: Content, MultipartPartConvertible {
    public static let defaultMediaType: MediaType = .formData

    public let file: Core.File

    public func convertToMultipartPart() throws -> MultipartPart {
        print(#function + " \(file.contentType!)")
        var part = try file.convertToMultipartPart()
        part.contentType = file.contentType!
        return part
    }

    public static func convertFromMultipartPart(_ part: MultipartPart) throws -> APIFile {
        let file = try File.convertFromMultipartPart(part)
        return APIFile(file: file)
    }
}

// Called by a route
public func _uploadFile(on req: Request, userFile: APIFile) throws -> Future<Response> {
    return try userFile.encode(for: req)
//    let client = try req.make(Client.self)
//    return client.post(url, content: userFile)
}

Expected:

--BOUNDARY
Content-Disposition: form-data; name="data"; filename="file.pdf"
Content-Type: application/pdf

%PDF-1.3
...

Actual result:

--BOUNDARY
Content-Disposition: form-data; name=filename

file.pdf
--BOUNDARY
Content-Disposition: form-data; name=data

%PDF-1.3
...

A Core.File can be encoded using convertToMultipartPart() (MultipartPartConvertible) but the result doesn't conform to Content, hence it can't be post'ed

streaming support

Support for parsing streaming bodies could greatly reduce memory usage for decoding multipart forms. This API would need to be asynchronous, so it could not be used with Codable. However, it could allow you to access multipart parts as they become available, and even stream large parts (like files) by chunk. All while using very little memory if done correctly.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.