rebus-org / rebus.azureblobs Goto Github PK

View Code? Open in Web Editor NEW

3.0 5.0 5.0 1.89 MB

:bus: Azure Blobs-based databus storage for Rebus

Home Page: https://mookid.dk/category/rebus

License: Other

C# 95.38% Batchfile 4.62%

azure-storage azure-blob azure-blob-storage attachment databus sagas

rebus.azureblobs's People

Contributors

Stargazers

Watchers

Forkers

oseku torangel jackss0 kristofdegrave hwdwpghavc

rebus.azureblobs's Issues

Deprecated package

Hi, the 'Microsoft.Azure.KeyVault.Core' package included in this project is deprecated.
Would you kindly upgrade to to a newer package, the package deprecation notice recommends Azure.Security.KeyVault.Secrets?

Could not upload data to blob - The argument must not be empty string

Enabling SendBigMessagesAsAttachments together with StoreInBlobStorage throws an exception that says "The argument must not be empty string"

Our configuration is pretty basic

var dataBusOptions = d.StoreInBlobStorage(configuration.GetConnectionString("AzureStorage"), storageOptions.DataBusContainer);
d.SendBigMessagesAsAttachments(0); // When we approach the limit of Azure Storage Queues we want to transfer to blob storage
d.EnableEncryption(); // Encrypted while stored

if (!storageOptions.AllowAutoCreate)
    dataBusOptions.DoNotCreateContainer();

Full exception

Rebus.Exceptions.RebusApplicationException: Could not create (automatic claim check) attachment for outgoing message with ID 83b2a3fe-a812-4874-9da2-41b747691db5
 ---> System.IO.IOException: Could not upload data to blob named 'data-5c7c7972-b94c-4b53-a249-225b2e69f627.dat' in the 'databus' container
 ---> Microsoft.Azure.Storage.StorageException: The argument must not be empty string.
 ---> System.ArgumentException: The argument must not be empty string.
   at Microsoft.Azure.Storage.Shared.Protocol.HttpRequestMessageFactory.AddMetadata(StorageRequestMessage request, String name, String value)
   at Microsoft.Azure.Storage.Shared.Protocol.HttpRequestMessageFactory.AddMetadata(StorageRequestMessage request, IDictionary`2 metadata)
   at Microsoft.Azure.Storage.Blob.Protocol.BlobHttpRequestMessageFactory.AddMetadata(StorageRequestMessage request, IDictionary`2 metadata)
   at Microsoft.Azure.Storage.Blob.CloudBlockBlob.<>c__DisplayClass145_0.<PutBlobImpl>b__1(RESTCommand`1 cmd, Uri uri, UriQueryBuilder builder, HttpContent cnt, Nullable`1 serverTimeout, OperationContext ctx)
   at Microsoft.Azure.Storage.Core.Executor.Executor.ProcessStartOfRequest[T](ExecutionState`1 executionState, String startLogMessage, CancellationTokenSource timeoutTokenSource)
   at Microsoft.Azure.Storage.Core.Executor.Executor.ExecuteAsync[T](RESTCommand`1 cmd, IRetryPolicy policy, OperationContext operationContext, CancellationToken token)
   --- End of inner exception stack trace ---
   at Microsoft.Azure.Storage.Core.Executor.Executor.ExecuteAsync[T](RESTCommand`1 cmd, IRetryPolicy policy, OperationContext operationContext, CancellationToken token)
   at Microsoft.Azure.Storage.Blob.CloudBlockBlob.UploadFromStreamAsyncHelper(Stream source, Nullable`1 length, AccessCondition accessCondition, BlobRequestOptions options, OperationContext operationContext, AggregatingProgressIncrementer progressIncrementer, CancellationToken cancellationToken)
   at Rebus.AzureBlobs.DataBus.AzureBlobsDataBusStorage.Save(String id, Stream source, Dictionary`2 metadata)
Request Information
RequestID:
RequestDate:
StatusMessage:
ErrorCode:

   --- End of inner exception stack trace ---
   at Rebus.AzureBlobs.DataBus.AzureBlobsDataBusStorage.Save(String id, Stream source, Dictionary`2 metadata)
   at Rebus.Encryption.EncryptingDataBusStorageDecorator.Save(String id, Stream source, Dictionary`2 metadata)
   at Rebus.DataBus.DefaultDataBus.CreateAttachment(Stream source, Dictionary`2 optionalMetadata)
   at Rebus.DataBus.ClaimCheck.DehydrateOutgoingMessageStep.DehydrateTransportMessage(OutgoingStepContext context, TransportMessage transportMessage)
   --- End of inner exception stack trace ---
   at Rebus.DataBus.ClaimCheck.DehydrateOutgoingMessageStep.DehydrateTransportMessage(OutgoingStepContext context, TransportMessage transportMessage)
   at Rebus.DataBus.ClaimCheck.DehydrateOutgoingMessageStep.Process(OutgoingStepContext context, Func`1 next)
   at Rebus.Encryption.EncryptMessagesOutgoingStep.Process(OutgoingStepContext context, Func`1 next)
   at Rebus.Pipeline.Send.SerializeOutgoingMessageStep.Process(OutgoingStepContext context, Func`1 next)
   at Rebus.Pipeline.Send.AutoHeadersOutgoingStep.Process(OutgoingStepContext context, Func`1 next)
   at Rebus.Pipeline.Send.FlowCorrelationIdStep.Process(OutgoingStepContext context, Func`1 next)
   at Rebus.Pipeline.Send.AssignDefaultHeadersStep.Process(OutgoingStepContext context, Func`1 next)
   at Rebus.ServiceProvider.ServiceProviderProviderStep.Process(OutgoingStepContext context, Func`1 next)
   at Rebus.Bus.RebusBus.SendUsingTransactionContext(IEnumerable`1 destinationAddresses, Message logicalMessage, ITransactionContext transactionContext)
   at Rebus.Bus.RebusBus.InnerSend(IEnumerable`1 destinationAddresses, Message logicalMessage)
   at Rebus.Bus.RebusBus.Send(Object commandMessage, IDictionary`2 optionalHeaders)

Save method's ExistsAsync call throws exception with container SAS token

Version: 0.3.0

Issue

I'm trying to give Rebus access to only one container in Azure storage. I'm using a container SAS token for this. I generate the token in PowerShell:

$context = New-AzStorageContext -StorageAccountName '...' -StorageAccountKey '...'
New-AzStorageContainerSASToken -Name '...' -Permission 'rwdl' -Context $context

Rebus connects okay, but when it comes time to write, the Save method throws an exception:

This request is not authorized to perform this operation.

From debugging, it seems the call to ExistsAsync is the problem. I guess that whole section is incompatible with a container SAS token, since it tries to create the container if it doesn't exist.

Questions

It looks like Rebus currently expects account-level storage access...right?
Would a PR with an alternative container SAS implementation make sense? Is there another way to do this?

[DataBus] StorageException with status code 412

Version: 0.1.0

Issue

Hi, I'm having this exact same issue described here

It's happening like twice a day and the blobs are still present in my blob container.

HttpStatusCode: 412
HttpStatusMessage: The condition specified using HTTP conditional header(s) is not met.
ErrorCode: ConditionNotMet
ErrorMessage: ErrorMessage: The condition specified using HTTP conditional header(s) is not met.
RequestId:7063d79d-c01e-000a-31fe-dcb1ae000000
Time:2020-02-06T15:00:07.6518868Z

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.