Feature Proposal: Background Tasks

The proposed feature is related to background tasks in Swift. The current implementation of the background API has several issues that need to be addressed to support the execution of requests in the background using RequestDL.

The problem is not limited to the use of URLSessionConfiguration, but it also requires direct communication with the AppDelegate. Additionally, the way each request is identified can be problematic for developers who are using RequestDL.

This proposal needs further review and refinement.

Please feel free to contribute to the development of this solution and share your opinion.

Overview

TBD.

Introduction

RequestDL is a powerful and flexible library for building requests on Apple platforms. The Property protocol of RequestDL is a fundamental part of the library, representing all the properties that can be updated and set internally. However, in some cases, it's useful to abstract the concrete type of a property. This is where type erasure comes in, allowing developers to work with properties in a more generic and flexible way.

Proposal

We propose introducing a new type, AnyProperty, which is a type-erasing wrapper that can represent any Property instance. By using AnyProperty, developers can abstract the concrete type of a property, making it easier to pass properties around and work with them in a more generic way.

Implementation Details

The AnyProperty type will be a struct that conforms to the Property protocol. It will have a single initializer that takes a Content type parameter, which must also conform to the Property protocol.

Conclusion

Introducing AnyProperty would provide a useful way to abstract the concrete type of a property in RequestDL, making it easier to work with properties in a more generic way. This feature is a small but valuable addition to the RequestDL package, and it will be beneficial for developers who need to work with type-erased properties.

Overview

TBD.

Overview

TBD.

Overview

After conducting several revisions on AsyncHTTPClient and gaining a better understanding of URLSession with the Security and Network frameworks, we have discovered some optimizations that Apple has exclusively implemented for URLSession. This piqued our interest in exploring the possibility of integrating URLSession into RequestDL.

The aim of this proposal is to make RequestDL smarter by dynamically switching between URLSession and AsyncHTTPClient based on specific constraints.

Objectives

While we are uncertain whether this solution will be successful or if it will address all our concerns, we want to ensure thorough documentation of our approach. Here are some initial considerations:

• Develop a logic within RequestDL to determine whether URLSession or AsyncHTTPClient should be used.

  This flag should be set to true when it is feasible to execute the specified request using URLSession.

  Note: This functionality should only be available on Darwin platforms.

• Integrate Streams with URLSessionDelegate.

  At first glance, it appears that we can utilize the same async response steps in URLSession, but further investigation is required.

• Leverage RequestTask contexts.

  Since we already have UploadTask, DownloadTask, and DataTask, we can potentially align them with URLSession's upload, download, and task to take advantage of the URLSession API's conventions.

• Adapt SecureConnection to Security or URLSessionDelegate methods.

  We need to explore the possibilities enabled by URLSession within the RequestDL codebase.

  • Server Trust (DefaultTrusts, Trusts, and AdditionalTrust)
  • Client Authorization (Certificates and PrivateKey)
  • Pre-shared Key (PSKIdentity)

Implementation

If the planning phase progresses smoothly, we believe that we can implement these changes incrementally in small contributions to RequestDL. Each contribution will be released as a separate version.

Conclusion

We will update this proposal with our findings once the revision process is complete.

The current documentation for GroupTask is outdated due to changes in GroupResult. This feature proposal aims to update the documentation by providing accurate and consistent information.

Feature proposal: Add DelegateAdaptor as a Property

With this proposal, developers will have the option to add a DelegateAdaptor during the request specification, allowing them to receive exclusive notifications from URLSession.

Although RequestDL may not cover all possible scenarios, this functionality will give developers more control over the progress of the request, which is just as important as the other resources being developed.

Proposed solution

DataTask {
    // Properties specifications
    DelegateAdaptor(someOtherDelegate)
}

Technical Details

Currently, the way the delegate is used may not be straightforward to directly connect with the DelegateAdaptor. Therefore, it is necessary to review this proposal to identify the best way to implement it and make it available to developers.

This resource request aims to complete the documentation of the HeaderGroup by providing clear and consistent details for developers.

Overview

The purpose of this feature proposal is to enhance RequestDL by implementing the UserAgentHeader. However, it should be taken into consideration that there might already be a pre-configured value in the header to align with the standards of other HTTP request libraries.

Note: Following the standards of RequestDL, the User-Agent should be set manually and we'll provide the empty initializer to add the default User-Agent value.

One pattern to be considered when implementing the pre-configured value is:

APP_NAME/APP_VERSION (PLATFORM_NAME PLATFORM_VERSION)

The implementation of the UserAgentHeader should allow for appending the value of the User-Agent if a new value is specified, separated by a blank space. In case developers want to overwrite the User-Agent value entirely, they can do so using the headerStrategy(.setting) method.

Details

• Define a new Swift struct called UserAgentHeader.
• This struct will conform to the Property protocol, ensuring compatibility with other components in RequestDL.
• The struct will have a single initializer that takes a String as a parameter.

Example Usage

To utilize the new UserAgentHeader feature, developers can follow these steps:

1. Create a userAgent property:

   let userAgent = "CustomAgent/19.0"

2. Initialize the UserAgentHeader:

   let userAgentHeader = UserAgentHeader(userAgent)

3. Incorporate the UserAgentHeader into the request task:

   let requestTask = DataTask {
       userAgentHeader
   }

The result of this method will be APP_NAME/APP_VERSION (PLATFORM_NAME PLATFORM_VERSION) CustomAgent/19.0.

Note: This was updated to the following format APP_NAME/APP_VERSION PLATFORM_NAME/PLATFORM_VERSION using the same pattern for all fields in the default user agent value.

Point of Attention

Currently, the HeaderStrategyKey has a default value of .setting. To implement this feature, it will be necessary to make it optional with the default value of nil. This way, in the UserAgentHeader, we can use .adding when the value is nil to align with the idea proposed here.

Note: This was removed and originated a new issue. After revisiting the HeaderStrategy, it makes more sense if its default value was .adding.

By incorporating the UserAgentHeader, developers will be able to customize the value of the User-Agent header for their requests in RequestDL.

Please note that this is a proposed feature and further development and testing are required to ensure seamless integration with RequestDL.

Overview

TBD.

Overview

Currently, RequestDL provides support for Linux and all Apple platforms. The feature proposal aims to assess the feasibility and subsequently implement support for Web Assembly.

Similar to the code validation process for Linux, the objective is to include a directive in the swift.yml workflow to compile and test the code on the Web Assembly platform.

Additional Analysis

After the implementation of the Web Assembly support script, it is highly likely that certain portions of the code will not compile successfully.

If this occurs, a second analysis will be necessary to determine whether it is possible to continue providing support and refactor the code or to cancel this feature proposal due to complete incompatibility.

Overview

TBD.

Overview

The goal is to add additional settings for requests that respect the capabilities of each device. URLSession is an excellent example of the options available to developers.

1. allowsCellularAccess: Bool

   Allows/prevents executing a request using cellular data.

2. allowsExpensiveNetworkAccess: Bool

   Allows/prevents executing a request on a network interface labeled by the operating system as expensive.

3. allowsConstrainedNetworkAccess: Bool

   Allows/prevents executing a request when the data-saving option is active.

4. waitsForConnectivity: Bool

   Waits for the device to be connected to the internet before executing the request.

Points to Consider

In a preliminary assessment, to implement these features in RequestDL, it will be necessary to use the Network framework, which is also used by AsyncHTTPClient to provide waitsForConnectivity.

Therefore, before starting the implementation, it is important to verify if it is possible to achieve the same results using Swift and other available APIs to deliver the features also on Linux. Otherwise, it will be necessary to update the code and make the methods related to the Apple Network framework unavailable.

Describe the bug
When using the RequestDL, the default value for MultiThreadedEventLoopGroup.numberOfThreads is the value of ProcessInfo.processInfo.activeProcessorCount. AsyncHTTPClient uses as default value 1.

To Reproduce
Steps to reproduce the behavior:

1. Specify any request with Session().
2. Sets a breakpoint after request successful response.
3. Run the code and waits for breakpoint stop.
4. See Process by Thread debug option.
5. Count the total of NIO-ELT-1-#* threads.

Expected behavior
The number of NIO-ELT-1-#* threads should be 1 as a default value for requests.

Describe the bug
RequestDL defaults EventLoopGroup is the MultiThreadedEventLoopGroup instance. This is not wrong, but it doesn't take advantage of some features that are available for Apple platforms.

To Reproduce
Steps to reproduce the behavior:

1. Go to SharedSessionProvider.swift file
2. Checks the SharedSessionProvider.group function.

Expected behavior
The right one would be NIOTSEventLoopGroup which is optimized for Apple platforms.

Additional context
Here's the documentation of NIOTSEventLoopGroup that explain why we should use it.

/// An `EventLoopGroup` containing `EventLoop`s specifically designed for use with
/// Network.framework's post-sockets networking API.
///
/// These `EventLoop`s provide highly optimised and powerful networking tools for
/// the Darwin platforms. They have a number of advantages over the regular
/// `SelectableEventLoop` that NIO uses on other platforms. In particular:
///
/// - The use of `DispatchQueue`s to schedule tasks allows the Darwin kernels to make
///     intelligent scheduling decisions, as well as to maintain QoS and ensure that
///     tasks required to handle networking in your application are given appropriate
///     priority by the system.
/// - Network.framework provides powerful tools for observing network state and managing
///     connections on devices with highly fluid networking environments, such as laptops
///     and mobile devices. These tools can be exposed to `Channel`s using this backend.
/// - Network.framework brings the networking stack into userspace, reducing the overhead
///     of most network operations by removing syscalls, and greatly increasing the safety
///     and security of the network stack.
/// - The applications networking needs are more effectively communicated to the kernel,
///     allowing mobile devices to change radio configuration and behaviour as needed to
///     take advantage of the various interfaces available on mobile devices.
///
/// In general, when building applications whose primary purpose is to be deployed on Darwin
/// platforms, the ``NIOTSEventLoopGroup`` should be preferred over the
/// `MultiThreadedEventLoopGroup`. In particular, on iOS, the ``NIOTSEventLoopGroup`` is the
/// preferred networking backend.
@available(OSX 10.14, iOS 12.0, tvOS 12.0, watchOS 6.0, *)
public final class NIOTSEventLoopGroup: EventLoopGroup

Feature proposal: Throwing Errors in Property Initialization

This proposed new feature focuses on the internal functioning of properties. Currently, they accept asynchronous operations during Property initialization. With this proposal, it will be possible to execute commands that can throw errors.

The current implementation looks like this:


public protocol Property {

    static func makeProperty(_ property: Self, _ context: Context) async
}

The goal of this proposal is to allow for the following implementation:

public protocol Property {

    static func makeProperty(_ property: Self, _ context: Context) async throws
}

It would be beneficial to expand this syntax of asynchronous code that can also throw errors to Node, making them all have the same syntax and theoretical usage.

With this feature, developers will be able to execute code that can throw errors, making their code more consistent and complete. As the operation of making a request can already generate errors, it would be beneficial to expand this functionality to properties as well. However, this proposal may require some revisions before implementation.

Feature Proposal: Enhancements to the Session API

The Session in RequestDL is currently implemented by mirroring the settings of URLSessionConfiguration.

We propose to transform the values used in the Session's functions into enums and constants. This will make them clearer and more specific to RequestDL, enabling better documentation and discussions focused exclusively on the library's resources.

Feature Proposal: Modifiers for results as raw Element

The current implementation of RequestDL only allows modifiers to be applied to tasks of type TaskResult<Element> or TaskPrimitive. This limitation can make it challenging for developers to use modifiers effectively.

To address this issue, a new feature has been proposed that would make these modifiers available not only for TaskResult<Element>, but also for the Element type directly. This would provide developers with greater flexibility and simplify the usage of modifiers.

The current limitation can be described as:


DataTask {}

    .extractPayload()
    .decode(Model.self) // Compilation error: expects TaskResult<Data>, not Data

Proposed addition

extension Task where Element == Data {

     public func decode<T: Decodable>(
         _ type: T.Type,
         decoder: JSONDecoder = .init()
    ) -> ModifiedTask<Modifiers.Decode<Self, T>> 
}

Results

By adding this proposed feature, developers will have more flexibility in using modifiers and will no longer be limited to the current modifier usage.

Overview

TBD.

Feature Proposal: Improvements on Certificate Usage

Both ServerTrust and ClientCertificate can be optimized to consider the use of URLCredential and avoid excessive conversions on each request.

This proposal is subject to modifications as it requires a thorough review before its implementation.

After reviewing the concepts of URLCredential and certificate usage on iOS, it would be beneficial to improve the API of both objects to ensure consistent usage.

Overview

Currently, RequestDL doesn't handle changes that may cause issues with already implemented features, which is not a good practice for ensuring the stability of updated versions.

The goal of this update is to implement a workflow that validates the pull request (PR) and checks if there are any changes that could break existing features, providing greater confidence in the modifications made to the codebase. Additionally, this validation will enable safer implementation of minor and patch updates.

Note: This is not a limitation, but rather a clear guideline regarding the change rules in RequestDL. In major versions, maintainers may choose to bypass the validation to make changes that break the existing code. These decisions should be discussed to ensure integrity across versions.

Overview

Currently, RequestDL provides support for Linux and all Apple platforms. The feature proposal aims to assess the feasibility and subsequently implement support for Windows.

Similar to the code validation process for Linux, the objective is to include a directive in the swift.yml workflow to compile and test the code on the Windows platform.

Additional Analysis

After the implementation of the Windows support script, it is highly likely that certain portions of the code will not compile successfully.

If this occurs, a second analysis will be necessary to determine whether it is possible to continue providing support and refactor the code or to cancel this feature proposal due to complete incompatibility.

Describe the bug

Alguns servidores, quando recebem múltiplo valores para uma mesma chave, eles ignoram as demais ficando apenas com o primeiro valor. No SwiftNIO isso não acontece, porém testando em alguns outros fornecedores de backend, o problema ocorre.

To Reproduce
Steps to reproduce the behavior:

1. Inicie o servidor PHP.

   <?php
     foreach ($_SERVER as $name => $value) {
       echo "$name: $value <br>";
     }
   ?>

2. Faça uma requisição com múltiplos headers.

   let data = try await DataTask {
       BaseURL(.http, host: "localhost:xxx")
       HeaderGroup {
           AcceptHeader(.json)
           AcceptHeader(.bmp)
       }
       .headerStrategy(.adding)
   }
   .extractPayload()
   .result()
   
   print(String(data: data, encoding: .utf8)!)

3. See missing image/bmp

Expected behavior

I ran tests using AsyncHTTPClient, SwiftNIO server-side, cURL, URLSession, and PHP server-side to observe the behavior of multiple values for the same header in each framework.

Since both the client and server on SwiftNIO function correctly and follow the default behavior of AsyncHTTPClient, this can be considered a standard variation. Additionally, it is widely known that headers should consolidate their values into a single key separated by commas.

The proposed feature: Implementation of Progress API

This feature is of utmost importance for RequestDL users, as it provides the ability to monitor the progress of a request.

However, this proposal requires further review, as it may impact the development of #18. Moreover, it is not entirely clear how progress can be implemented, especially given the advancements in async/await and even AsyncBytes.

Please feel free to contribute to the development of this solution and share your opinions.

Is your feature request related to a problem? Please describe.
The current implementation of RequestDL utilizes .setting as a default value to handle multiple headers with the same key. However, this approach can be unclear both during implementation and when reading the code.

Describe the solution you'd like
Updates the default strategy value from .setting to .adding.

Feature proposal: Selecting a Certificate in ClientCertificate

The way PKCS 12 works allows more than one certificate to be attached in a single file. However, when initializing the ClientCertificate in RequestDL, there is no way to specify which certificate from this group of certificates to use. It is the application's responsibility to inform which certificate to send to the server.

The proposal is to allow developers to specify which certificate to use in the ClientCertificate, such as certificate at index 0 (default value) or 2. If the RequestDL cannot find the certificate in the correct position, it would be helpful to select at least the last available certificate.

Here's an example of how this proposal could be used:

DataTask {
    // Properties specifications
    ClientCertificate(

        certificateData,
        password: password,
        at: 2
   )

}

With this proposal, the implementation of ClientCertificate becomes more complete, better representing the PKCS 12 format.

Overview

TBD.

Overview

TBD.

Overview

TBD.

Overview

This should be done incrementally in various PRs and milestones. It's expected to be closed at 3.3.0 milestone.

TBD.

Feature Proposal: Implementation of UploadTask

This feature proposal aims to develop the UploadTask, allowing developers to internally use the URLSessionUploadTask with resources for sending any payload with Data.

As a reference, we could look at the operation of BytesTask to provide similar usage and potentially allow for progress monitoring.

Please feel free to contribute to the development of this solution and share your opinions.

Overview

Currently, the RequestDL documentation follows the standard DocC structure, lacking an initial landing page and articles explaining how to get started. With the updates introduced by Apple at WWDC23, there are various useful ways to enhance the documentation.

Project's README

The first change involves modifying the README.md file to contain less information and direct users to the DocC articles already published on GitHub Pages. Additionally, a section should be added for documentation translations, allowing the community to contribute and bring new languages to RequestDL.

Shared Workflow

The second contribution is to implement a workflow similar to the existing one used to publish documentation to GitHub Pages. However, this workflow will be used by developers in their repositories containing translated documentation.

This feature still requires review and further study regarding its feasibility.

Note: This was removed due the fact that Github template repositories works great, but we couldn't design a create solution using these features. For now, this will stay in background priority and can be revisited.

DocC Enhancements

It is necessary to organize all implemented objects into a structured grouping to provide clearer context to the reader for each item. Additionally, pages and articles should be added to DocC, providing better explanations of the features to make it easier for users to understand and get started.

Code Documentation Enhancements

The current code documentation contains errors and can benefit from an improved structure to provide clearer information to the reader about each item.

Feature Proposal: Cookies API Implementation

At present, RequestDL doesn't make use of HTTPCookieStorage. This proposal aims to introduce new and exciting functionalities that can be achieved through a better understanding of cookies. This will enable developers to create more robust solutions.

However, this proposal requires further review.

Please feel free to contribute to the development of this solution and share your opinion.

Describe the bug
After conducting research on the usage of Content-Length: 0, I discovered that some backend services do not support it. In a related topic, the SwiftNIO team clarifies that the library itself should not cause such issues, and therefore, RequestDL should also not cause this problem.

To Reproduce
Steps to reproduce the behavior:

1. Upload some empty Data:

   Payload(data: Data())

2. Verify on the server-side whether the Content-Length: 0 header was received.

Expected behavior
The server shouldn't receive the Content-Length header, because indicates that its value is zero.

Overview

TBD.

Overview

The goal of this feature proposal is to enhance RequestDL by implementing the AcceptCharsetHeader functionality. This will allow developers to specify the value of the request header. The Charset object is already implemented, so the focus will be on complementing it by adding the new header.

Details

• Define a new Swift struct called AcceptCharsetHeader.
• This struct will conform to the Property protocol, ensuring compatibility with other components in RequestDL.
• The struct will have a single initializer that takes a Charset object as a parameter.

Example Usage

To utilize the new AcceptCharsetHeader feature, developers can follow these steps:

1. Create a Charset object:

   let charset = Charset.utf8

2. Initialize the AcceptCharsetHeader:

   let acceptCharsetHeader = AcceptCharsetHeader(charset)

3. Incorporate the AcceptCharsetHeader into the request task:

   let requestTask = DataTask {
       acceptCharsetHeader
   }

By incorporating the AcceptCharsetHeader, developers will be able to customize the value of the Accept-Charset header for their requests in RequestDL.

Please note that this is a proposed feature and requires further development and testing to ensure seamless integration with RequestDL.

Overview

TBD.

Feature proposal: Add Modifiers.ConvertBytesToData

This proposal aims to add a new feature that allows developers to convert AsyncBytes to Data using a Task modifier, regardless of whether the result is an AsyncBytes or a TaskResult<AsyncBytes>.

Although the concept of AsyncBytes already provides a mechanism for handling binary data asynchronously, providing a declarative method for direct conversion to Data can significantly improve code readability and maintainability. Therefore, this proposal is not only syntactically sound but also practical for everyday development tasks.

Proposal

extension Task where Element == AsyncBytes {

    public func convertBytesToData() -> ModifiedTask<__ProposedModifier<Self>
}

Results

The convertBytesToData() method allows for greater flexibility and convenience in processing data, as it enables developers to directly convert AsyncBytes to Data without resorting to additional code or workarounds.

By incorporating this proposed feature into their workflow, developers can streamline their code and make it more readable and efficient, while still retaining the power and versatility of the AsyncBytes library.

For instance, consider the following code snippet that uses convertBytesToData() to decode a Model from BytesTask:

BytesTask {}
    .extractPayload()
    .detach { asyncBytes in

        // Perform some operation on AsyncBytes

    }
    .convertBytesToData()
    .decode(Model.self) // See proposal (#01)

With this concise syntax, developers can easily perform complex data operations in a clear and efficient manner, resulting in faster development times and more maintainable code.

Feature proposal: Improved API for Forms

The way the form API is currently implemented in RequestDL solves some of the problems encountered when using the multipart formdata content type.

This proposal aims to follow RFC 7578, allowing for more customization in the construction of parts, particularly by adding the possibility of including other types of headers for each part in the form.

Currently, the implementation only allows the use of FormData, FormValue, and FormFile.

The goal of this proposal is to implement a single FormPart that allows initialization of the headers and content type to be sent, making all possibilities clearer to developers.

The proposal would allow the following usage:

FormPart(data) {
    Headers.ContentType(.json)

    Headers.Any(“123”, forKey: “id”)
}

Please feel free to contribute to the development of this solution and share your opinions.

Overview

TBD.

Overview

This proposal aims to add verification of the Status Code when the HEAD request used for validating the stored cache returns 304 (Not Modified).

By adding this condition, RequestDL doesn't need to perform any further validation and can rely on the response that confirms the stored cache remains valid and doesn't require updating.

Overview

TBD.

Based on community feedback (thanks Mike Lewis!), we have conducted a technical survey and studies to migrate RequestDL's current request mechanism. Version 2.0 requires significant internal refactoring, with some public methods implemented without much abstraction being deprecated.

The new mechanism to be implemented will be Apple's Async-HTTP-Client, which has support for linux and windows, allowing for greater use of RequestDL. Additionally, this new mechanism contains several interesting features that improve existing functionalities and add new ones.

As version 2.0 is implemented, this feature proposal will be updated.

New features:

• Improved TLS verification;
• Full support for upload/download progress;
• New possibilities for managing received bytes;
• Improvements to task stability.

Overview

TBD.