Giter Club home page Giter Club logo

Comments (5)

dmitry-fa avatar dmitry-fa commented on July 28, 2024

New methods of Blob are suggested:

public void uploadFrom(Path path, BlobWriteOption... options) {...}
public void uploadFrom(Path path, int chunkSize, BlobWriteOption... options) {...}
public void uploadFrom(InputStream input, BlobWriteOption... options) {...}

  /**
   * Uploads the given content to this blob using specified blob write options and the provided
   * chunk size. The default chunk size is 2 MB. Larger chunk sizes may improve the upload
   * performance but require more memory. It could cause OutOfMemoryError or add significant garbage
   * collection overhead. Chunk sizes which are less than 256 KB are not allowed, they will be
   * treated as 256 KB.
   *
   * @param input content to be uploaded
   * @param chunkSize the minimum size that will be written by a single RPC.
   * @param options blob write options
   * @throws StorageException upon failure
   */
  public void uploadFrom(InputStream input, int chunkSize, BlobWriteOption... options) {
    chunkSize = Math.max(chunkSize, 262144); // adjust with MinChunkSize of BaseWriteChannel
    try (WriteChannel writer = storage.writer(this, options)) {
      writer.setChunkSize(chunkSize);
      byte[] buffer = new byte[chunkSize];
      int length;
      while ((length = input.read(buffer)) >= 0) {
        writer.write(ByteBuffer.wrap(buffer, 0, length));
      }
    } catch (IOException e) {
      throw new StorageException(e);
    }
  }

from java-storage.

dmitry-fa avatar dmitry-fa commented on July 28, 2024

The following extension to the Blob class is proposed:

  public Blob uploadFrom(Path path, BlobWriteOption... options)
  public Blob uploadFrom(InputStream input, BlobWriteOption... options)

  /**
   * Uploads the given content to the storage using specified write channel and the given
   * buffer size. The default buffer size is 15 MB. Larger buffer sizes may improve the upload
   * performance but require more memory. It could cause OutOfMemoryError or add significant garbage
   * collection overhead. Buffer sizes which are less than 256 KB are not allowed, they will be
   * treated as 256 KB.
   *
   * <p>Note that this method does not close neither InputStream nor WriterChannel</p>
   *
   * <p>Example of uploading:
   *
   * <pre>{@code
   * Path file = Paths.get("humongous.file");
   * try (InputStream input = Files.newInputStream(file); WriteChannel writer = storage.writer(blobInfo)) {
   *     Blob.upload(input, writer, 150 * 1024 * 1024);
   * }
   *
   * @param input content to be uploaded
   * @param writer channel
   * @param bufferSize size of the buffer to read from input and send over writer
   */
  public static void upload(InputStream input, WriteChannel writer, int bufferSize)

from java-storage.

dmitry-fa avatar dmitry-fa commented on July 28, 2024

Similar extension to the Storage interface is planned to be proposed with a separate PR

from java-storage.

dmitry-fa avatar dmitry-fa commented on July 28, 2024

fix was reverted by #190

from java-storage.

dmitry-fa avatar dmitry-fa commented on July 28, 2024

#214 fixes this issue as well

from java-storage.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.