Comments (4)
athakor
commented on September 2, 2024
@andrew-lenets I have tried it with small size of gzipped objects but unable to reproduced it see below are the code sample and output.
Sample:
public class StorageTest {
private static final Storage storage = StorageOptions.getDefaultInstance().getService();
private static final String bucket = "test-bucket";
public static void main(String[] args) {
List<String> blobList = Arrays.asList("2019/10/10/11/0eaba27bcb.xml.gz", "2019/10/10/11/f74aa52b33.xml.gz", "2019/10/10/11/e7342793de.xml.gz");
Map<String, Integer> expectedFileSizes = new HashMap<>();
expectedFileSizes.put(blobList.get(0), 1385);
expectedFileSizes.put(blobList.get(1), 649);
expectedFileSizes.put(blobList.get(2), 282);
int runs = 15, run = 0;
while (run++ < runs) {
System.out.println("Run: " + run);
Iterable<Blob> blobs = storage.list(bucket, Storage.BlobListOption.prefix("2019/10/10/11")).iterateAll();
for (Blob blob : blobs) {
String name = blob.getName();
if (blobList.contains(name)) {
byte[] content = blob.getContent();
int expectedSize = expectedFileSizes.get(name);
int actualSize = content.length;
System.out.println("\tname: " + name + ", size: " + actualSize);
assertEquals(expectedSize, actualSize, "Run: " + run + ": incorrect size! " + name);
}
}
}
}
}Output:
Run: 1
name: 2019/10/10/11/0eaba27bcb.xml.gz, size: 1385
name: 2019/10/10/11/e7342793de.xml.gz, size: 282
name: 2019/10/10/11/f74aa52b33.xml.gz, size: 649
Run: 2
name: 2019/10/10/11/0eaba27bcb.xml.gz, size: 1385
name: 2019/10/10/11/e7342793de.xml.gz, size: 282
name: 2019/10/10/11/f74aa52b33.xml.gz, size: 649
Run: 3
name: 2019/10/10/11/0eaba27bcb.xml.gz, size: 1385
name: 2019/10/10/11/e7342793de.xml.gz, size: 282
name: 2019/10/10/11/f74aa52b33.xml.gz, size: 649
Run: 4
name: 2019/10/10/11/0eaba27bcb.xml.gz, size: 1385
name: 2019/10/10/11/e7342793de.xml.gz, size: 282
name: 2019/10/10/11/f74aa52b33.xml.gz, size: 649
Run: 5
name: 2019/10/10/11/0eaba27bcb.xml.gz, size: 1385
name: 2019/10/10/11/e7342793de.xml.gz, size: 282
name: 2019/10/10/11/f74aa52b33.xml.gz, size: 649
Run: 6
name: 2019/10/10/11/0eaba27bcb.xml.gz, size: 1385
name: 2019/10/10/11/e7342793de.xml.gz, size: 282
name: 2019/10/10/11/f74aa52b33.xml.gz, size: 649
Run: 7
name: 2019/10/10/11/0eaba27bcb.xml.gz, size: 1385
name: 2019/10/10/11/e7342793de.xml.gz, size: 282
name: 2019/10/10/11/f74aa52b33.xml.gz, size: 649
Run: 8
name: 2019/10/10/11/0eaba27bcb.xml.gz, size: 1385
name: 2019/10/10/11/e7342793de.xml.gz, size: 282
name: 2019/10/10/11/f74aa52b33.xml.gz, size: 649
Run: 9
name: 2019/10/10/11/0eaba27bcb.xml.gz, size: 1385
name: 2019/10/10/11/e7342793de.xml.gz, size: 282
name: 2019/10/10/11/f74aa52b33.xml.gz, size: 649
Run: 10
name: 2019/10/10/11/0eaba27bcb.xml.gz, size: 1385
name: 2019/10/10/11/e7342793de.xml.gz, size: 282
name: 2019/10/10/11/f74aa52b33.xml.gz, size: 649
Run: 11
name: 2019/10/10/11/0eaba27bcb.xml.gz, size: 1385
name: 2019/10/10/11/e7342793de.xml.gz, size: 282
name: 2019/10/10/11/f74aa52b33.xml.gz, size: 649
Run: 12
name: 2019/10/10/11/0eaba27bcb.xml.gz, size: 1385
name: 2019/10/10/11/e7342793de.xml.gz, size: 282
name: 2019/10/10/11/f74aa52b33.xml.gz, size: 649
Run: 13
name: 2019/10/10/11/0eaba27bcb.xml.gz, size: 1385
name: 2019/10/10/11/e7342793de.xml.gz, size: 282
name: 2019/10/10/11/f74aa52b33.xml.gz, size: 649
Run: 14
name: 2019/10/10/11/0eaba27bcb.xml.gz, size: 1385
name: 2019/10/10/11/e7342793de.xml.gz, size: 282
name: 2019/10/10/11/f74aa52b33.xml.gz, size: 649
Run: 15
name: 2019/10/10/11/0eaba27bcb.xml.gz, size: 1385
name: 2019/10/10/11/e7342793de.xml.gz, size: 282
name: 2019/10/10/11/f74aa52b33.xml.gz, size: 649
from java-storage.
andrii-lenets
commented on September 2, 2024
@athakor have you tried with objects like:
Content-Encoding: gzip
Content-Type: text/xml
Also, I suppose file size matters. In our case, the issue is producible on files > 1Mb.
from java-storage.
athakor
commented on September 2, 2024
@andrew-lenets Yes, I have tried it with Content-Encoding: gzip , Content-Type: text/xml as well as files > 1 Mb. But still unable to reproduced.
from java-storage.
athakor
commented on September 2, 2024
Closing this issue due to inactivity.please reopen if the issue persists.
from java-storage.
Related Issues (20)
- Cloud storage v2 API, add support for batch operations HOT 2
- Retrying idempotent requests that return 503 backendError HOT 1
- com.google.cloud.storage.StorageException: java.lang.IllegalStateException: Connection pool shut down when trying to check if bucket exists HOT 5
- CVE-2023-33953 through io.grpc.* (1.56.1) HOT 3
- com.example.storage.QuickstartSampleIT: testQuickstartGrpc failed HOT 1
- com.example.storage.QuickstartSampleIT: testQuickstartGrpcDp failed HOT 1
- Google Cloud Storage library using deprecated javax package HOT 4
- Async / streaming support
- com.example.storage.bucket.PrintPubSubNotificationTest: testPrintBucketPubSubNotification failed HOT 2
- Warning: a recent release failed HOT 2
- Add support for Autoclass v2.1 HOT 1
- grpc compatibility with storage 2.27.x HOT 3
- Google Cloud Storage: unexpected exception after successfully writing to a Google Cloud Storage object HOT 2
- grpc compatibility issue 2.27.1 HOT 9
- GCP cloud storage downloaded file corruption HOT 4
- Improve Virtual Thread compatibility
- Unable to initiate resumable uploads from java sdk: Cloud storage v2 API. HOT 4
- Unable to Get or Refresh access token from Google Credentials instance in java sdk HOT 1
- Novel transient error introduced between 2.27.1 and 2.30.1? HOT 5
- Add Support for includeFolders option in ListObjects HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from java-storage.