Comments (20)
I'm working on this now, is taking a bit of time as it's a relatively large feature. A proposal with the new API/generated code is coming hopefully next week.
In the meantime, please drop here your use cases of multipart, such as:
- request body or response body?
- many parts of the same type (e.g. file), or heterogeneous, i.e. a raw log, a JSON object, and an image all in our request/response
- what are you representing with the parts? (files, images, logs, JSON blobs, ...)
Thanks! 🙏
from swift-openapi-generator.
Thanks for the suggestions @czechboy0! I found a workaround that is okay for the time being. I changed the upload parameters to accept an application/octet-stream
instead and simply pass data unadorned.
requestBody:
content:
application/octet-stream:
schema:
type: string
format: binary
This only works because in my particular case I don't have other metadata to upload in the body, so I will keep watching progress on this issue.
from swift-openapi-generator.
I haven't tested that this works, but you could use NIO's FileSystem module (https://github.com/apple/swift-nio/blob/main/Sources/NIOFileSystem/Docs.docc/NIOFileSystem.md) like this. It has the advantage of never loading the full video into memory at once, instead streaming it from disk and printing a log line on every chunk update.
import OpenAPIRuntime
import Foundation
import OpenAPIURLSession
import NIOFileSystem
let client = Client(
serverURL: URL(string: "https://example.com/api")!,
transport: URLSessionTransport()
)
actor ProgressLogger {
private let totalSize: Int64
private var accumulatedSize: Int64
init(totalSize: Int64) {
self.totalSize = totalSize
self.accumulatedSize = 0
}
func processChunkSize(_ size: Int) {
accumulatedSize += Int64(size)
print("Progress: \(accumulatedSize)/\(totalSize)")
}
}
try await FileSystem.shared.withFileHandle(forReadingAt: FilePath("/foo/video.mp4")) { read in
let chunks = read.readChunks()
let size = try await read.fileHandle.info().size
let progress = ProgressLogger(totalSize: size)
_ = try await client
.upload(
body: .multipartForm(
[
.content(.init(payload: .init(body: "this is a description"))),
.video(
.init(
payload: .init(body: HTTPBody(
chunks
.map { ArraySlice($0.readableBytesView) }
.map { chunk in
await progress.processChunkSize(chunk.count)
return chunk
},
length: .known(size),
iterationBehavior: .single
)),
filename: "video.mp4"
)
)
]
)
)
.accepted
}
from swift-openapi-generator.
This one needs a bit of design, but should be possible.
A simple idea:
- generate a struct for the whole multipart payload
- each property is one content type + payload, decoded as we'd normally decode it if it were a full body
- this struct would not itself be Codable, as we'd generate custom code to decode/encode each of the properties, just like we do for type-safe properties for each operation
from swift-openapi-generator.
Additional reading: https://datatracker.ietf.org/doc/html/rfc7578
from swift-openapi-generator.
There are also headers associated with each part, not just content type + payload: https://github.com/OAI/OpenAPI-Specification/blob/main/versions/3.1.0.md#encoding-object-example
from swift-openapi-generator.
Can also be represented as an array property at the top level, at which point it could lend itself well to being a stream of values?
from swift-openapi-generator.
Should the parts be available as they become available, or all at the end in the generated struct? Seems we have to decide between more type-safety and delivering parts as they're ready instead of only returning them all at the end.
from swift-openapi-generator.
Hi @czechboy0 thank you for taking on this issue, as I'm currently running into a related issue with the generated client while performing a multipart upload.
I'm writing code that updates a user's profile photo, they are submitting the photo Data
after retrieving that value from their library.
let res = try await client.post_sol_users_sol__lcub_userId_rcub__sol_photo(
.init(path: .init(userId: user.userId),
headers: .init(),
body: .multipartForm(data) // image data here
)
)
Unfortunately the server is producing an error when I attempt to complete this operation:
no multipart boundary param in Content-Type
This makes sense, as the client currently hardcodes the content type to multipart/form-data
. From the relevant part of the generated client;
request.body = try converter.setOptionalRequestBodyAsBinary(
value,
headerFields: &request.headerFields,
contentType: "multipart/form-data"
)
Even without fully fleshing out the multipart spec as outlined above, it would be great to address this in a point release of the client.
Thank you for taking the time to work on this!
from swift-openapi-generator.
unfortunately this is expected, as what you're sending is not a valid multipart request.
While I hope to have a proposal up for multipart support this week, and hopefully have everything landed by end of next week, here's a temporary workaround:
- Write a client middleware that adds a constant "boundary" parameter to the content type header before it's sent to the server.
- Change the raw body you're providing by making it look like a multipart request, i.e. boundary to start, part headers section, then your image body, then an ending boundary.
Lmk if you need more details here.
from swift-openapi-generator.
More details.
In your middleware, you'd modify the header from
content-type: multipart/form-data
to
content-type: multipart/form-data; boundary=__X_SWIFT_OPENAPI_GENERATOR_BOUNDARY__
And then the body would need to look something like:
--__X_SWIFT_OPENAPI_GENERATOR_BOUNDARY__
Content-Disposition: form-data; name="photo"
<raw photo bytes here>
--__X_SWIFT_OPENAPI_GENERATOR_BOUNDARY__--
Note that each newline needs to be a CRLF, not just a simple newline.
Hopefully this should be accepted by the server and can serve as a workaround for the new few weeks before proper multipart support lands.
from swift-openapi-generator.
Oh yeah if you only have a single payload, don't use multipart, use "single"part 🙂 (what you're doing, just one body payload).
from swift-openapi-generator.
Proposal: #369
Please, have a read and chime in with questions, comments.
from swift-openapi-generator.
https://forums.swift.org/t/proposal-soar-0009-type-safe-streaming-multipart-support/68331
from swift-openapi-generator.
All the runtime changes landed in main, will be released in 1.0.0-alpha.1 next week.
Generator changes are in a PR: #366.
from swift-openapi-generator.
Runtime changes landed, all going out next week.
from swift-openapi-generator.
@czechboy0 is there a way to track upload progress of a multipart request?
from swift-openapi-generator.
@czechboy0 is there a way to track upload progress of a multipart request?
Yes, a few - using a middleware or using a wrapping async sequence. Before I recommend more details, can you elaborate? Do you want to track the upload progress of the full body (which includes potentially multiple multipart parts), or just the one part? What is the code for uploading it, how are you providing the data?
from swift-openapi-generator.
I'm uploading a large video and a short text. Upload progress of the text content (description) is negligible - I don't care if it is included in overall progres.
guard let data = try? Data(contentsOf: videoURL) else { return .... }
let body = HTTPBody(data)
let multi: MultipartBody<Operations.post_sol_accounts_sol__lcub_account_rcub__sol_content_sol_basic_sol_video.Input.Body.multipartFormPayload> = [
.video(.init(payload: .init(body: body), filename: "video")),
.content(.init(payload: .init(body: HTTPBody(description))))
]
from swift-openapi-generator.
I haven't tested that this works, but you could use NIO's FileSystem module (https://github.com/apple/swift-nio/blob/main/Sources/NIOFileSystem/Docs.docc/NIOFileSystem.md) like this. It has the advantage of never loading the full video into memory at once, instead streaming it from disk and printing a log line on every chunk update.
import OpenAPIRuntime import Foundation import OpenAPIURLSession import NIOFileSystem let client = Client( serverURL: URL(string: "https://example.com/api")!, transport: URLSessionTransport() ) actor ProgressLogger { private let totalSize: Int64 private var accumulatedSize: Int64 init(totalSize: Int64) { self.totalSize = totalSize self.accumulatedSize = 0 } func processChunkSize(_ size: Int) { accumulatedSize += Int64(size) print("Progress: \(accumulatedSize)/\(totalSize)") } } try await FileSystem.shared.withFileHandle(forReadingAt: FilePath("/foo/video.mp4")) { read in let chunks = read.readChunks() let size = try await read.fileHandle.info().size let progress = ProgressLogger(totalSize: size) _ = try await client .upload( body: .multipartForm( [ .content(.init(payload: .init(body: "this is a description"))), .video( .init( payload: .init(body: HTTPBody( chunks .map { ArraySlice($0.readableBytesView) } .map { chunk in await progress.processChunkSize(chunk.count) return chunk }, length: .known(size), iterationBehavior: .single )), filename: "video.mp4" ) ) ] ) ) .accepted }
That worked with almost no changes! Thanks a lot. This should be documented somewhere in readme 🙏
from swift-openapi-generator.
Related Issues (20)
- Support for XML request body and response body HOT 7
- Support schemas.type: string, format: uri generates URL property not String HOT 3
- How to support OAuth2 Password flows? HOT 3
- allOf plus properties HOT 2
- OpenAPI shared components HOT 4
- bug/regression: latest yams results in incomplete generation of types HOT 10
- Logging and HTTPBody HOT 3
- Merging oneOf with many compatible enums of a single value HOT 10
- anyOf encoder does not include discriminator's `parameterName` leading to incorrect encoding HOT 2
- Is there a workaround to deal with slow compilation times for a very large openapi file? HOT 3
- Empty arrays are omitted from query params HOT 5
- Support for `multipart/mixed`? HOT 1
- When a delete path do not have responses content an error generated HOT 2
- Add ability to attach protocols to generated models HOT 9
- Protocol where each `Output` conforms to HOT 3
- Make the generator usable on very big specs HOT 3
- Old and new examples HOT 3
- visionOS can't use HOT 4
- How to convert the binary obtained from the server into a playable Data type? HOT 2
- HTTPTypes linker error if multiple targets use open API generator HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from swift-openapi-generator.