particular / nservicebus.ravendb Goto Github PK

View Code? Open in Web Editor NEW

10.0 10.0 17.0 12.3 MB

Home Page: https://docs.particular.net/nservicebus/ravendb/

License: Other

C# 99.09% PowerShell 0.91%

nservicebus.ravendb's People

Contributors

Stargazers

Watchers

Forkers

zwyl2001 alainbates spouse71 yonglehou jpattom areicher hanin frankclaassens imonthercks ramonsmits rossjayjones nuvemnine rvaridel atikhan uzbekdev1 grisha-kotler

nservicebus.ravendb's Issues

If index is stale we may miss timeouts

Raised by @johnsimons
Migrated from by Particular/NServiceBus#2133

I have been looking at this issue reported in the mailing list.

So to summarise the issue, the user is using the Scheduler API and creates to schedule tasks, one that runs every 5secs and another one that runs every 1min.
Every so often the 5sec schedule task stops from reoccurring.

Here are the logs

And here is one of the timeouts stored in Raven:

{
  "Destination": {
    "Queue": "Flyt.TiosCtcAgent",
    "Machine": "GMBSANNTID"
  },
  "SagaId": "00000000-0000-0000-0000-000000000000",
  "State": "PD94bWwgdmVyc2lvbj0iMS4wIiA/Pg0KPE1lc3NhZ2VzIHhtbG5zOnhzaT0iaHR0cDovL3d3dy53My5vcmcvMjAwMS9YTUxTY2hlbWEtaW5zdGFuY2UiIHhtbG5zOnhzZD0iaHR0cDovL3d3dy53My5vcmcvMjAwMS9YTUxTY2hlbWEiIHhtbG5zPSJodHRwOi8vdGVtcHVyaS5uZXQvTlNlcnZpY2VCdXMuU2NoZWR1bGluZy5NZXNzYWdlcyI+CjxTY2hlZHVsZWRUYXNrPgo8VGFza0lkPmU2YjZlN2NmLTlmYjMtNDhiZS1iZGUwLTc0NmJhZmE3MmRlNzwvVGFza0lkPgo8TmFtZT5Cb290c3RyYXA8L05hbWU+CjxFdmVyeT5QVDVTPC9FdmVyeT4KPC9TY2hlZHVsZWRUYXNrPgo8L01lc3NhZ2VzPg0K",
  "Time": "2014-05-21T08:18:00.2114720Z",
  "CorrelationId": "2f849e22-d760-4ce6-846b-a33200a9b784\\0",
  "OwningTimeoutManager": "Flyt.TiosCtcAgent",
  "Headers": {
    "NServiceBus.MessageId": "2f849e22-d760-4ce6-846b-a33200a9b784",
    "NServiceBus.CorrelationId": "2f849e22-d760-4ce6-846b-a33200a9b784",
    "NServiceBus.OriginatingEndpoint": "Flyt.TiosCtcAgent",
    "$.diagnostics.originating.hostid": "522caf7d3f53a790fdcc328b6cb0d627",
    "NServiceBus.MessageIntent": "Send",
    "NServiceBus.Version": "4.4.2",
    "NServiceBus.TimeSent": "2014-05-21 08:17:55:211472 Z",
    "NServiceBus.OriginatingMachine": "GMBSANNTID",
    "NServiceBus.ContentType": "text/xml",
    "NServiceBus.EnclosedMessageTypes": "NServiceBus.Scheduling.Messages.ScheduledTask, NServiceBus.Core, Version=4.4.0.0, Culture=neutral, PublicKeyToken=9fc386479f8a226c",
    "CorrId": "2f849e22-d760-4ce6-846b-a33200a9b784\\0",
    "WinIdName": "GMBDOMENE1\\sanntidintegrasjon",
    "NServiceBus.RelatedTo": "5da0ead5-1701-4e2e-9b43-a33200a9b16e",
    "NServiceBus.ConversationId": "a00caaba-1eab-48a7-854f-a33100a5c243",
    "NServiceBus.IsDeferredMessage": "True",
    "NServiceBus.Temporary.DelayDeliveryWith": "00:00:05",
    "NServiceBus.Timeout.Expire": "2014-05-21 08:18:00:211472 Z",
    "NServiceBus.Timeout.RouteExpiredTimeoutTo": "Flyt.TiosCtcAgent@GMBSANNTID",
    "NServiceBus.Timeout.ReplyToAddress": "Flyt.TiosCtcAgent@GMBSANNTID"
  }
}

The only explanation I have is if the index is stale, we could end-up skipping over timeouts.
@synhershko has actually identified this issue in the https://github.com/Particular/NServiceBus.RavenDB repo and has fixed it.

I cannot replicate this myself, but the index staleness is definitely a possibility and that would explain the entries I can see in the log file.

Honor the unique properties in the saga metamodel

The new metamodel in core v6 contains the properties that are unique. The persister should honor this.

This is how the InMemory persister is currently doing it

Particular/NServiceBus#346

Support ISagaFinder integration in SagaPersister

This is currently not tested with the new implementation

Run all tests via both OpenSessionBehavior and ProvidedSessionBehavior

Probably by using a dynamic test fixture, or having a test project for each. Ideally we could run each test (both in Unit and Acceptance) multiple times using different configs

@andreasohlund @johnsimons thoughts?

Instantiate a RavenDB connection based on configurations

A la https://github.com/Particular/NServiceBus/blob/develop/src/NServiceBus.Core/Persistence/Raven/ConfigureRavenPersistence.cs#L227

If Configure.PersistanceForAll was called without a DocumentStore instance as a parameter

Somewhat related to #13

Error message when trying to use RavenDB 3.0 persistence

I installed RavenDB 3.0 yesterday and since then, a local NServiceBus app (using RavenDBPersistence) I've been developing has been throwing the following exception.

An exception of type 'System.InvalidOperationException' occurred in NServiceBus.Host.exe but was not handled in user code
Additional information: The RavenDB server you have specified is detected to be Product version: 3.0.0 / cdc39ac / , Build version: 3528. NServiceBus requires RavenDB version 2.5 build 2900 or higher to operate correctly. Please update your RavenDB server.

I see this is because the IsSufficientVersion() method in ConnectionVerifier.cs is hard-coded to only return True when the product version is equal to 2.5. For any newer versions of RavenDB, it will fail and give a confusing error message suggesting that your version of RavenDB needs updating when if anything, it needs downgrading.

Tests

This issue is to track all tests that we commented out or removed from the core but need to run in the RavenDB scope. Some were targeted to RavenDB persistence but should be generalized to be used by more persistence solutions

Should timeouts share the common session

So in the raven timeout persistence we have

    public void RemoveTimeoutBy(Guid sagaId)
    {
        using (var session = OpenSession())
        { 
            var items = session.Query<TimeoutData>().Where(x => x.SagaId == sagaId);
            foreach (var item in items)
                session.Delete(item);

            session.SaveChanges();
        }
    }

    IDocumentSession OpenSession()
    {
        var session = store.OpenSession();

        session.Advanced.AllowNonAuthoritativeInformation = false;
        session.Advanced.UseOptimisticConcurrency = true;

        return session;
    }

Similar code exists in subscriptions and deduplication

But we have the concept of ISessionProvider to allow us to share the session, which is what the saga persister uses. So why dont the above use the same shared session?

NSB Beta5 - Problem processing timeouts from sagas from previous version

Raised by @indualagarsamy
Migrated from by Particular/NServiceBus#2357

Steps to reproduce
Make sure to use RavenDB 2.5 (In this test, I am running 2.5.0, Build version: 2916)

In the attached sample, run the SagaTimeoutTest_v4_4_0 project. This will request 10 timeouts and wait for 30 seconds and shut down.
Then start the SagaTimeoutTest_v5_0_0 endpoint.
The sagas should process the requested timeouts, but unfortunately, nothing happens.

NOTE: Running the same scenario using SagaTimeoutTest_v4_6_4 endpoint results in an startup error.

Exception when starting endpoint.
Autofac.Core.DependencyResolutionException: An exception was thrown while execut
ing a resolve operation. See the InnerException for details. ---> The RavenDB se
rver you have specified is detected to be Product version: 2.5.0 / 6dce79a, Buil
d version: 2916. NServiceBus requires RavenDB version 2.0 to operate correctly.
Please update your RavenDB server.

Further instructions can be found at:http://particular.net/articles/using-ravend
b-in-nservicebus-installing (See inner exception for details.) ---> System.Inval
idOperationException: The RavenDB server you have specified is detected to be Pr
oduct version: 2.5.0 / 6dce79a, Build version: 2916. NServiceBus requires RavenD
B version 2.0 to operate correctly. Please update your RavenDB server.

The same scenario works correct between 4.4.0 and 4.5.0.

Enforce RavenDB 2.5 build 2900

In the connection verifier we should enforce that build is >= 2900

Fix RavenDB issue with streaming

Streaming with RavenDB 2.5 breaks with an exception when using Embedded and breaking out of a streaming loop. This was fixed today with v2.5.2886 (unstable) and we should upgrade our tests suite to use that one.

It is worth noting if the user will pass an EmbeddedDocumentStore instance to be used by the timeouts module it will just break and not work unless he is using v2.5.2886 or higher. This is an unfortunate edge case but worth knowing and having that documented - hence this issue.

/cc @johnsimons @andreasohlund @dannycohen

Starting using RavenDB via Features

PowerShell script to create a new user

When endpoints are deployed to IIS, the user they run under is a low privilege user, thus the endpoint cannot create a new database in RavenDB at startup time.
Should we create a PowerShell script to automate this process?

create a new database;
create a new user, given the low privilege one;
give permissions to the new user to access the new database

Add API for user to get access to the current session

Same as the old RavenSessionFactory.

See conversation here https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/particularsoftware/2TgCwtC7rxA/KJ6GORJeui0J

Move Gateway stuff to its own assembly

Since gateway is now out of the core, all Gateway related code needs to go to a NServiceBus.RavenDB.Gateway assembly

//cc @SimonCropp @andreasohlund

IPersistMessages is obsolete

Should we remove RavenPersistence since IPersistMessages is now obsolete in favour of IDeduplicateMessages ?

Externalise RavenDB from the core

The RavenDB integration has been extracted (copied) from the core to allow better support of RavenDB version changes moving forward.

RavenDB database is not set up automatically during install

Maybe I have a special setup, maybe I don't. I've been fighting the whole day to get this working.

I have two services which use RavenDB. One uses it for sagas and timeouts, the other one just for second level retries. The saga/timeout service will create its RavenDb database upon installation, the SLR service won't. RavenDB doesn't even show up in the feature list on startup, despite being explicitly configured.

Full reproduction here.

configuration.AssembliesToScan(GetType().Assembly);

configuration
  .Conventions()
  .DefiningCommandsAs(type => type.Namespace != null && type.Namespace.Contains(".Commands"));

configuration.UsePersistence<RavenDBPersistence>();
configuration.UseSerialization<JsonSerializer>();

configuration.EnableFeature<SecondLevelRetries>();
configuration.EnableFeature<TimeoutManager>();

configuration.DisableFeature<Sagas>();
configuration.DisableFeature<AutoSubscribe>();

$ NServiceBus.Host.exe /install
Initializing the installer in the Install AppDomain
2014-12-11 18:37:00.588 INFO  DefaultFactory Logging to 'D:\Code\Repro\bin\Debug\' with level Info
Executing the NServiceBus installers
2014-12-11 18:37:01.011 INFO  NServiceBus.Hosting.Profiles.ProfileManager Activating profile: NServiceBus.Production, NServiceBus.Host, Version=5.0.0.0, Culture=neutral, PublicKeyToken=9fc386479f8a226
c
2014-12-11 18:37:01.319 INFO  NServiceBus.Persistence.PersistenceStartup Activating persistence 'RavenDBPersistence' to provide storage for 'GatewayDeduplication' storage.
2014-12-11 18:37:01.326 INFO  NServiceBus.Persistence.PersistenceStartup Activating persistence 'RavenDBPersistence' to provide storage for 'Timeouts' storage.
2014-12-11 18:37:01.328 INFO  NServiceBus.Persistence.PersistenceStartup Activating persistence 'RavenDBPersistence' to provide storage for 'Sagas' storage.
2014-12-11 18:37:01.332 INFO  NServiceBus.Persistence.PersistenceStartup Activating persistence 'RavenDBPersistence' to provide storage for 'Subscriptions' storage.
2014-12-11 18:37:01.797 WARN  NServiceBus.Licensing.LicenseManager Trial for the Particular Service Platform has expired
2014-12-11 18:37:01.866 INFO  NServiceBus.Features.UnicastBus Number of messages found: 2
2014-12-11 18:37:02.674 INFO  NServiceBus.PerformanceMonitorUsersInstaller Skipped adding user 'AXL\Administrator' to group 'Leistungsüberwachungsbenutzer' because the user is already in group.
2014-12-11 18:37:02.689 INFO  NServiceBus.Features.DisplayDiagnosticsForFeatures ------------- FEATURES ----------------
Name: CriticalErrorHandling
Version: 5.1.2
Enabled by Default: Yes
Status: Enabled
Dependencies: None
Startup Tasks: None

Name: CustomIDataBus
Version: 5.1.2
Enabled by Default: No
Status: Disabled
Deactivation reason: Did not meet one of the dependencies: [DataBus]

Name: DataBus
Version: 5.1.2
Enabled by Default: Yes
Status: Disabled
Deactivation reason: Did not fulfill its Prerequisites:
   -No databus properties was found in available messages

Name: Encryptor
Version: 5.1.2
Enabled by Default: Yes
Status: Disabled
Deactivation reason: Did not fulfill its Prerequisites:
   -No encryption properties was found in available messages

Name: ErrorSubscribers
Version: 5.1.2
Enabled by Default: Yes
Status: Enabled
Dependencies: None
Startup Tasks: None

Name: ForwarderFaultManager
Version: 5.1.2
Enabled by Default: Yes
Status: Enabled
Dependencies: None
Startup Tasks: None

Name: InMemoryFaultManager
Version: 5.1.2
Enabled by Default: No
Status: Disabled
Deactivation reason: Did not meet one of the dependencies:

Name: InstallationSupport
Version: 5.1.2
Enabled by Default: No
Status: Enabled
Dependencies: None
Startup Tasks: None

Name: CriticalTimeMonitoring
Version: 5.1.2
Enabled by Default: No
Status: Enabled
Dependencies: None
Startup Tasks: None

Name: Audit
Version: 5.1.2
Enabled by Default: Yes
Status: Disabled
Deactivation reason: Did not fulfill its Prerequisites:
   -No configured audit queue was found

Name: AutoSubscribe
Version: 5.1.2
Enabled by Default: Yes
Status: Disabled
Deactivation reason: Did not meet one of the dependencies:

Name: MsmqSubscriptionPersistence
Version: 5.1.2
Enabled by Default: No
Status: Disabled
Deactivation reason: Did not meet one of the dependencies:

Name: Scheduler
Version: 5.1.2
Enabled by Default: Yes
Status: Enabled
Dependencies: None
Startup Tasks: None

Name: CustomSerialization
Version: 5.1.2
Enabled by Default: Yes
Status: Disabled
Deactivation reason: Did not fulfill its Prerequisites:
   -CustomSerialization not enable since serialization definition not detected.

Name: ForwardReceivedMessages
Version: 5.1.2
Enabled by Default: No
Status: Disabled
Deactivation reason: Did not meet one of the dependencies:

Name: RegisterHandlersInOrder
Version: 5.1.2
Enabled by Default: Yes
Status: Enabled
Dependencies: None
Startup Tasks: None

Name: SLAMonitoring
Version: 5.1.2
Enabled by Default: No
Status: Enabled
Dependencies: None
Startup Tasks: None

Name: LicenseReminder
Version: 5.1.2
Enabled by Default: Yes
Status: Enabled
Dependencies: None
Startup Tasks: None

Name: Outbox
Version: 5.1.2
Enabled by Default: No
Status: Disabled
Deactivation reason: Did not meet one of the dependencies:

Name: InMemoryGatewayPersistence
Version: 5.1.2
Enabled by Default: No
Status: Disabled
Deactivation reason: Did not meet one of the dependencies: [Gateway]

Name: InMemoryOutboxPersistence
Version: 5.1.2
Enabled by Default: No
Status: Disabled
Deactivation reason: Did not meet one of the dependencies: [Outbox]

Name: InMemorySagaPersistence
Version: 5.1.2
Enabled by Default: No
Status: Disabled
Deactivation reason: Did not meet one of the dependencies: [Sagas]

Name: InMemorySubscriptionPersistence
Version: 5.1.2
Enabled by Default: No
Status: Disabled
Deactivation reason: Did not meet one of the dependencies: [MessageDrivenSubscriptions]

Name: InMemoryTimeoutPersistence
Version: 5.1.2
Enabled by Default: No
Status: Disabled
Deactivation reason: Did not meet one of the dependencies: [TimeoutManager]

Name: TimeoutManagerBasedDeferral
Version: 5.1.2
Enabled by Default: No
Status: Enabled
Dependencies: None
Startup Tasks: None

Name: UnicastBus
Version: 5.1.2
Enabled by Default: Yes
Status: Enabled
Dependencies: None
Startup Tasks: None

Name: BinarySerialization
Version: 5.1.2
Enabled by Default: Yes
Status: Disabled
Deactivation reason: Did not fulfill its Prerequisites:
   -BinarySerialization not enable since serialization definition not detected.

Name: BsonSerialization
Version: 5.1.2
Enabled by Default: Yes
Status: Disabled
Deactivation reason: Did not fulfill its Prerequisites:
   -BsonSerialization not enable since serialization definition not detected.

Name: JsonSerialization
Version: 5.1.2
Enabled by Default: Yes
Status: Enabled
Dependencies: None
Startup Tasks: None

Name: XmlSerialization
Version: 5.1.2
Enabled by Default: Yes
Status: Disabled
Deactivation reason: Did not fulfill its Prerequisites:
   -XmlSerialization not enable since serialization definition not detected.

Name: MsmqTransportConfigurator
Version: 5.1.2
Enabled by Default: No
Status: Enabled
Dependencies: None
Startup Tasks: None

Name: TimeoutManager
Version: 5.1.2
Enabled by Default: No
Status: Enabled
Dependencies: [TimeoutManagerBasedDeferral]
Startup Tasks: None

Name: Sagas
Version: 5.1.2
Enabled by Default: Yes
Status: Disabled
Deactivation reason: Did not meet one of the dependencies:

Name: SecondLevelRetries
Version: 5.1.2
Enabled by Default: Yes
Status: Enabled
Dependencies: [ForwarderFaultManager]
Startup Tasks: None

Name: DataBusFileBased
Version: 5.1.2
Enabled by Default: No
Status: Disabled
Deactivation reason: Did not meet one of the dependencies: [DataBus]

Name: StorageDrivenPublishing
Version: 5.1.2
Enabled by Default: No
Status: Enabled
Dependencies: None
Startup Tasks: None

Name: MessageDrivenSubscriptions
Version: 5.1.2
Enabled by Default: No
Status: Enabled
Dependencies: None
Startup Tasks: None

Prevent user from using stale queries to load sagas

Currently if users forgets the [Unique] attribute in the saga property they correlate on the persister will use a lucene query to find the saga. Since this will return potentially stale data, especially during load, it might cause consistency issues where multiple sagas might get created or updated concurrently. Since this is rarely what the user want we now:

Throw an exception if the unsafe Get is used telling them to put the
unique on)
Proivide a config.UsePersistence().AllowStaleSagaReads()
for users that wan't to take the chance

Strong naming issue

I just added a package to the solution, and Ripple rewrote the csproj of one of the projects to add back some references that were ommitted:

88ce426#diff-1

However, this breaks compilation due to strong naming issues like this one:

System.Exception : Could not enumerate all types for 'Z:\Projects\Particular\NServiceBus.RavenDB\src\NServiceBus.RavenDB.AcceptanceTests\bin\Debug\NServiceBus.ObjectBuilder.Ninject.dll'.
It looks like you may be missing binding redirects in your config file for the following assemblies:
Ninject, Version=3.0.0.0, Culture=neutral, PublicKeyToken=c7192dc5380945e7
Ninject.Extensions.NamedScope, Version=3.0.0.0, Culture=neutral, PublicKeyToken=c7192dc5380945e7
For more information see http://msdn.microsoft.com/en-us/library/7wd6ex19(v=vs.100).aspx

I'm going to revert this change for now until we can figure out what's going on

@andreasohlund @SimonCropp @gbiellem

RavenDB server version checker error message can be misinterpreted

Symptoms

At the moment the error message says:

The RavenDB server you have specified is detected to be Product version: 3.0.0 / cdc39ac / , Build version: 3528. NServiceBus requires RavenDB version 2.5 build 2900 or higher to operate correctly. Please update your RavenDB server.

This needs to be updated to:

The RavenDB server you have specified is detected to be Product version: 3.0.0 / cdc39ac / , Build version: 3528. NServiceBus requires version 2.5 build 2900 (or a higher build number for version 2.5) to operate correctly. Please update your RavenDB server.

Who's affected

Anyone that upgrades Raven to a higher version then 2.5

Get TC working for ravenDB

https://builds.particular.net/project.html?projectId=NServiceBusRavenDB&tab=projectOverview

Create a Raven transport

Raised by @andreasohlund
Migrated from by Particular/NServiceBus#425

To allow us to run without msmq we should consider creating a RavenDB transport

Version conflict with latest packages from nuget

When I set up a new NServiceBus host project by getting latest stables from Nuget, I run into a nasty dependency problem.

Steps to repro:

Create fresh new project
install package NServiceBus.Host
install package NServiceBus.RavenDB
install package NServiceBus.RabbitMQ
configure endpoint as a server, using rabbitmq transport (enpointconfig & app.config)

Doing this, my packages.config looks like this:

<package id="NServiceBus" version="4.6.3" targetFramework="net45" />
<package id="NServiceBus.Host" version="4.6.3" targetFramework="net45" />
<package id="NServiceBus.Interfaces" version="4.6.3" targetFramework="net45" />
<package id="NServiceBus.RabbitMQ" version="1.1.3" targetFramework="net45" />
<package id="NServiceBus.RavenDB" version="1.0.0" targetFramework="net45" />
<package id="RabbitMQ.Client" version="3.2.1" targetFramework="net45" />
<package id="RavenDB.Client" version="2.5.2908" targetFramework="net45" />

When I run, I get

WARN  NServiceBus.Persistence.Raven.RavenUserInstaller [(null)] <(null)> - Failed to add user to raven. Processing will continue
System.MissingMethodException: Method not found: '!!0 Raven.Abstractions.Extensions.JsonExtensions.JsonDeserialization(Raven.Json.Linq.RavenJObject)'.
at NServiceBus.Persistence.Raven.RavenUserInstaller.AddUserToDatabase(String identity, DocumentStore documentStore)
at NServiceBus.Persistence.Raven.RavenUserInstaller.Install(String identity)in c:\BuildAgent\work\31f8c64a6e8a2d7c\src\NServiceBus.Core\Persistence\Raven\RavenUserInstaller.cs:line 42

To me, it appears to be a version conflict, since the public API of Raven.Abstractions has changed in RavenDB release RavenDB-2464 DTC NSB. The signature of extension method JsonDeserialization used inside RavenUserInstaller.AddUserToDatabase has changed:
ravendb/ravendb@e3fcf5b.

I was only able to work around this by compiling NServiceBus.RavenDB against build 2700 of RavenDB.Client.Lightweight and referencing it outside of nuget.

Expose the raven session for users implementing their own sagafinder

Users can now take a dependency on the ISessionProvider to get access to the session managed by the RavenDB shared session support

Add Outbox support using RavenDB as Outbox storage

Ripple being run by VS and breaks dev process

Not sure where to post this, so let it be here

It appears like ripple is being run by VS at build time, so sporadically I get this:

1>------ Build started: Project: NServiceBus.RavenDB, Configuration: Debug Any CPU ------
1>  
1>  ===============================================
1>  String: Error executing ExplodeDownloadedNugets
1>  ===============================================
1>  
1>  An unhandled exception occurred:
1>  Exception:
1>  Access to the path 'Z:\Projects\Particular\NServiceBus.RavenDB\src\packages\NServiceBus\lib\net45\NServiceBus.Core.xml' is denied.
1>  StackTrace:
1>     at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
1>     at System.IO.File.SetAttributes(String path, FileAttributes fileAttributes)
1>     at ripple.FileSystemExtensions.cleanDirectory(String directory, Boolean remove)
1>     at ripple.FileSystemExtensions.cleanDirectory(String directory, Boolean remove)
1>     at ripple.FileSystemExtensions.cleanDirectory(String directory, Boolean remove)
1>     at ripple.FileSystemExtensions.ForceClean(IFileSystem system, String path)
1>     at ripple.Nuget.NugetFile.ExplodeTo(String directory)
1>     at ripple.Steps.ExplodeDownloadedNugets.<Execute>b__0(INugetFile nuget)
1>     at System.Collections.Generic.GenericEnumerableExtensions.Each[T](IEnumerable`1 values, Action`1 eachAction)
1>     at ripple.Steps.ExplodeDownloadedNugets.Execute(RippleInput input, IRippleStepRunner runner)
1>     at ripple.RippleOperation.Execute(Boolean throwOnFailure)
1>  Source:
1>  mscorlib
1>  TargetSite:
1>  Void WinIOError(Int32, System.String)
1>  
1>  
1>  ========================
1>  String: Error occurred: 
1>  ========================
1>  
1>  An unhandled exception occurred:
1>  Exception:
1>  Access to the path 'Z:\Projects\Particular\NServiceBus.RavenDB\src\packages\NServiceBus\lib\net45\NServiceBus.Core.xml' is denied.
1>  StackTrace:
1>     at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
1>     at System.IO.File.SetAttributes(String path, FileAttributes fileAttributes)
1>     at ripple.FileSystemExtensions.cleanDirectory(String directory, Boolean remove)
1>     at ripple.FileSystemExtensions.cleanDirectory(String directory, Boolean remove)
1>     at ripple.FileSystemExtensions.cleanDirectory(String directory, Boolean remove)
1>     at ripple.FileSystemExtensions.ForceClean(IFileSystem system, String path)
1>     at ripple.Nuget.NugetFile.ExplodeTo(String directory)
1>     at ripple.Steps.ExplodeDownloadedNugets.<Execute>b__0(INugetFile nuget)
1>     at System.Collections.Generic.GenericEnumerableExtensions.Each[T](IEnumerable`1 values, Action`1 eachAction)
1>     at ripple.Steps.ExplodeDownloadedNugets.Execute(RippleInput input, IRippleStepRunner runner)
1>     at ripple.RippleOperation.Execute(Boolean throwOnFailure)
1>     at ripple.Commands.RestoreCommand.Execute(RestoreInput input)
1>     at ripple.RippleOperation.CommandExecutionExpression.Execute[TInput,TCommand](TInput input, TCommand command)
1>     at RippleRestoreTask.RestoreTask.InnerExecute(ILogListener logListener, String targetDirectory)
1>     at RippleRestoreTask.RestoreTask.<>c__DisplayClass2.<Execute>b__0()
1>     at Locker.ExecuteLocked(String name, Action action)
1>     at RippleRestoreTask.RestoreTask.Execute()
1>  Source:
1>  mscorlib
1>  TargetSite:
1>  Void WinIOError(Int32, System.String)
1>  
2>------ Build started: Project: NServiceBus.RavenDB.Tests, Configuration: Debug Any CPU ------
2>  NServiceBus.RavenDB.Tests -> Z:\Projects\Particular\NServiceBus.RavenDB\src\NServiceBus.RavenDB.Tests\bin\Debug\NServiceBus.RavenDB.Tests.dll
========== Build: 1 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

Followed by missing DLLs that basically break my IDE. I need to shut down VS, do ripple restore and then resume work.

Any idea what's going on?

@andreasohlund @gbiellem @SimonCropp @jeremydmiller

Support for multiple [Unique] attributes on a single saga data instance

Currently we enforce that a ContainsSagaData instance has only one property decorated with the Unique attribute.

Issues with Raven naming convention

Should we fix Particular/NServiceBus#1039 in this new dll ?

Some acceptance tests are tightly coupled with RavenDB

Some saga tests are tightly coupled with RavenDB stuff (see for example the test and endpoint in When_using_a_received_message_for_timeout).

Since we are breaking RavenDB out of the core, I'd suggest we either use an in-memory storage for sagas by default or move those tests to only be run on RavenDB's suite.

I guess this depends on what we are testing here:

If we test transport and wiring and all that, and the saga concept is abstract, we can just move to using in-memory storage for sagas and then re-run some specific tests from the RavenDB suite as needed (basically, find all those tests that RavenDB is significant for and re-run them specifically after injecting RavenDB as persistence).
If the saga tests are specifically meant for sagas only, so this becomes a persistence-specific test that is being written once generically. Therefore, the Acceptance Tests suite needs some sort of a generic UseCurrentPersistence() method that is called, and that is implemented by every persistence thing.

Either way, the Acceptance Tests suite cannot contain the words RavenDb (neither does the core, but that's for later).

@andreasohlund @SimonCropp @johnsimons Thoughts?

SagaPersistance.GetByQuery

Regarding this code:

        IEnumerable<T> GetByQuery<T>(string property, object value) where T : IContainSagaData
        {
            try
            {
                return sessionProvider.Session.Advanced.LuceneQuery<T>()
                    .WhereEquals(property, value)
                    .WaitForNonStaleResultsAsOfNow();
            }
            catch (InvalidCastException)
            {
                return new[]
                    {
                        default(T)
                    };
            }
        }

https://github.com/Particular/NServiceBus.RavenDB/blob/develop/src/NServiceBus.RavenDB/SagaPersister/RavenSagaPersister.cs#L115-L130

I see 2 issues that need to be tackled:

Why are we WaitingForNonStaleResultsAsOfNow()? That's something that should never be done in production
Why are we catching InvalidCastException - have there been cases where this has thrown?

/cc @andreasohlund @johnsimons @SimonCropp

nuget dependencies when in pre-release

So we are now using nuget (and not ripple) for dependencies and also to pack a nuget package as a build artifact, however it seems to not pick NSB 5-alpha / 5-beta corretly when installed in a project, even with the -Pre flag on.

I tried solving it here 0f7ca07 but to no avail

This is what it looks like when trying to install in a fresh project:

PM> Install-Package NServiceBus.RavenDB -Pre
Attempting to resolve dependency 'Microsoft.CompilerServices.AsyncTargetingPack (≥ 1.0.1.0 && < 2.0.0.0)'.
Attempting to resolve dependency 'NServiceBus (≥ 5.0.0.0 && < 6.0.0.0)'.
Install-Package : Unable to resolve dependency 'NServiceBus (≥ 5.0.0.0 && < 6.0.0.0)'.
At line:1 char:1
+ Install-Package NServiceBus.RavenDB -Pre
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Install-Package], InvalidOperationException
    + FullyQualifiedErrorId : NuGetCmdletUnhandledException,NuGet.PowerShell.Commands.InstallPackageCommand

Ensure compatibility between the old and new raven API

So we have people who are persisting things using the raven parts in the core. now we are externalising those parts into the assembly produced by this repository

We need to ensure that people can move over to the new approach in the simplest possible manner. so ideally with no migration or if there is migration involved document it well or perhaps write a tool to perform this migration.

Produce some integration tests that verify the process
automate as much as possible if it is not seamless
document any manual steps people need to take

Release a RavenDB v3 version once available

Check storage connectivity when created / provided

And report a nice error using Helpers.LogRavenConnectionFailure in case of failure

Investigate and fix flaky acpt tests

1d45f9c

When starting NSB 4.6.1 on Asp.net MVC, RavenDB (403) Forbidden

Raised by @danielHalan
Migrated from by Particular/NServiceBus#2144

Getting following exception when Starting NSB on an ASP.NET MVC Instance, at Configure.With()....ForMvc()...RavenPersistence().RavenSubscriptionStorage()....
.UnicastBus()
.LoadMessageHandlers()
.CreateBus()

And http://xx:8080 is up n running.

System.WebException "The remote server returned an error: (403) Forbidden."

   at System.Net.HttpWebRequest.GetResponse()
   at Raven.Client.Connection.HttpJsonRequest.ReadJsonInternal(Func`1 getResponse) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Connection\HttpJsonRequest.cs:line 341
   at Raven.Client.Connection.HttpJsonRequest.ReadResponseJson() in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Connection\HttpJsonRequest.cs:line 244
   at Raven.Client.Connection.ServerClient.DirectPut(RavenJObject metadata, String key, Nullable`1 etag, RavenJObject document, String operationUrl) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Connection\ServerClient.cs:line 398
   at Raven.Client.Connection.ServerClient.<>c__DisplayClass13.<Put>b__12(String u) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Connection\ServerClient.cs:line 333
   at Raven.Client.Connection.ReplicationInformer.TryOperation[T](Func`2 operation, String operationUrl, Boolean avoidThrowing, T& result) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Connection\ReplicationInformer.cs:line 564
   at Raven.Client.Connection.ReplicationInformer.ExecuteWithReplication[T](String method, String primaryUrl, Int32 currentRequest, Int32 currentReadStripingBase, Func`2 operation) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Connection\ReplicationInformer.cs:line 523
   at Raven.Client.Connection.ServerClient.ExecuteWithReplication[T](String method, Func`2 operation) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Connection\ServerClient.cs:line 174
   at Raven.Client.Connection.ServerClient.Put(String key, Nullable`1 etag, RavenJObject document, RavenJObject metadata) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Connection\ServerClient.cs:line 333

RavenDBPersistence - ConnectionParameters: how to set credentials or an entire connection string?

The RavenDBPersistence adds an extension method, SetDefaultDocumentStore, that has an overload that takes a ConnectionParameters instance:

the name is misleading: ConnectionParameters has nothing to do with DefaultDocumentStore;
seems that there is no way to set credentials, such as username and password, in the connection parameters, the only supported credential info is the ApiKey;

I'm following up a Desk case where the requirement is to manually build, in C#, the connection string and set it at endpoint configuration time. Suggestions?

Normalize naming for Persitance / Storage / Persister classes

@SimonCropp @andreasohlund @johnsimons

Use Raven native id assignment to improve saga inserts perf

Raised by @johnsimons
Migrated from by Particular/NServiceBus#1973

Description

As part of fine tuning SC to handle a high volume of messages we have identified a bottleneck on our ravenDB saga persistence.

The issue is that we auto gen our own docs ids and those ids are not the ideal for Raven inserts, which causes the insertions to get slower as the data grows.

So the solution is to let Raven auto gen its own ids using HiLo.

Implementation

Looking at the current code base, one way I see us moving forward with this is to have a in-memory mapping of NSB id <-> Raven id.
So when we instantiate RavenSagaPersister we perform a query to initialize this in-memory mapping and then we keep it up to date on all inserts/deletes
This would be actually quite simple to spike?

@andreasohlund thoughts?

Transitioning story

Let's maintain a list of all breaking changes that we are going to have to support for people migrating from internal RavenDB to this package:

Sagas are going to have Data stripped from their name, because of the conventions if (IsASagaEntity(t)) { tagName = tagName.Replace("Data", String.Empty); }
All storage types are now a different CLR type (timeouts + subscriptions)
TimeoutData was previously persisted and now is stored via a dedicated persistence class

Incorporate DTC fix

By using RavenDB build 2900 or above, and documentStore.TransactionRecoveryStorage = new IsolatedStorageTransactionRecoveryStorage();

Saga persistence uses any IDocumentStore registered in the container, if there is one, instead of the correct one

The fix is changing DocumentStoreWrapper so that auto-wiring cannot override the value (the now does not have a setter and the store is passed via the constructor and stored in a private field)

Repro

Use following bus config

var configuration = new BusConfiguration();
var appDataStore = new DocumentStore
{
    ConnectionStringName = "RavenDB",
    DefaultDatabase = "ApplicationData"
};

// Raven DataStore for NServiceBus
var nsbDataStore = new DocumentStore
{
    ConnectionStringName = "NServiceBus.Persistence",
    DefaultDatabase = "BookingProcessing"
};

// Set up and build AutoFac container
var builder = new ContainerBuilder();
builder.RegisterInstance<DocumentStore>(appDataStore).As<IDocumentStore>().SingleInstance();
var container = builder.Build();

// Set up NServiceBus
configuration.UseContainer<AutofacBuilder>(customizations => customizations.ExistingLifetimeScope(container));
configuration.UsePersistence<RavenDBPersistence>().SetDefaultDocumentStore(nsbDataStore);
configuration.EnableFeature<Sagas>();

var bus = (UnicastBus) Bus.Create(configuration);
bus.Start();
Type documentStoreWrapper =
    Type.GetType("NServiceBus.RavenDB.Internal.IDocumentStoreWrapper, NServiceBus.RavenDB", true);

var wrapper = bus.Builder.Build(documentStoreWrapper);

The wrapper resolved contains reference to the application document store

Root cause

The registration here registeres an instance as singleton but because such registrations support property auto-wiring, when resolving the store gets updated to the one provided by the user.

Allow to configure multiple document stores for different persistence concerns

Do we want this?

What this means is calling PersistenceForTimeouts, PersistenceForSagas and PersistenceForSubscriptions will each accept an IDocumentStore which will be registered for them and them only.

You can reuse the same document store for all 3 (by calling PersistenceForAll or each individually) but that enables you to provide different stores.

Or we can say we only allow for one doc-store globally and blow up when one method is trying to set a doc-store when one is already set

Thoughts?

/cc @andreasohlund @SimonCropp @johnsimons

New configuration API

Here is what I was thinking:

Configure.Storage.RavenDB(s => {
    s.UseSagas();
    s.UseTimeouts();
    s.Use(documentStore); //the default store
    s.UseSubscriptions();
})

All storages are required

So in the core we default to the in-core RavenDB for all storage types https://github.com/Particular/NServiceBus/blob/4.6.5/src/NServiceBus.Core/Persistence/Raven/ConfigureRavenPersistence.cs#L159

    public static void RegisterDefaults()
    {
        InfrastructureServices.SetDefaultFor<ISagaPersister>(() => Configure.Instance.RavenSagaPersister());
        InfrastructureServices.SetDefaultFor<IPersistTimeouts>(() => Configure.Instance.UseRavenTimeoutPersister());
        InfrastructureServices.SetDefaultFor<IPersistMessages>(() => Configure.Instance.UseRavenGatewayPersister());
        InfrastructureServices.SetDefaultFor<IDeduplicateMessages>(() => Configure.Instance.UseRavenGatewayDeduplication());
        InfrastructureServices.SetDefaultFor<ISubscriptionStorage>(() => Configure.Instance.RavenSubscriptionStorage());
    }

And here https://github.com/Particular/NServiceBus.RavenDB/releases we say to use the new external RavenDB nuget all you need is

Configure.With()
    .DefaultBuilder()
    .RavenDBStorage()

However this means that the default for all the other storages (timeouts, subscriptions etc) will default to the ravendb in the core. which results in

"The RavenDB server you have specified is detected to be Product version: 2.5.0 / 6dce79a, Build version: 2916. NServiceBus requires RavenDB version 2.0 to operate correctly. Please update your RavenDB server. Further instructions can be found at:http://particular.net/articles/using-ravendb-in-nservicebus-installing"

The workaround is to fully define all the storages

 Configure.With()
            .DefaultBuilder()
            .RavenDBStorage()
            .UseRavenDBSagaStorage()
            .UseRavenDBSubscriptionStorage()
            .UseRavenDBTimeoutStorage()
            .UseRavenDBGatewayDeduplicationStorage()
            .UseRavenDBGatewayStorage()

Repro here https://github.com/Particular/BugsRepro/tree/master/RavenDbMissingInjection

Support multi-tenancy

Support the ability of having different DocumentStores for different storage types, e.g. one DocStore for sagas, one for timeouts etc

Obviously, the user could always provide one ShardedDocumentStore instance that is configured to do this, but we can then support various other scenarios.

That will be done by registering DocumentStore instances separately for each Feature.

Thoughts? @andreasohlund @SimonCropp @johnsimons

Issue connecting to prod RavenDB in an AD domain

Who's affected

Any user that utilizes RavenDB persistence;

Symptoms

If the endpoint that depends on the RavenDB persistence is installed on a doman joined machine and the RavenDB server is installed on a machine in the same AD domain, it can happen that the endpoint at start up tries to add a user that is already defined on the server and fails, from that point on each attempt to connect to RavenDB returns a HTTP-500.

Raised by @mauroservienti
Migrated from by Particular/NServiceBus#2118

Customer report - 5060,

Scenario:

NServiceBus RavenDB is installed on a production server joined to an AD environment;
NServiceBus Endpoint is installed on a different server jpoined to the same domain;
the Endpoint is correctly configured to access RavenDB using a valid connection string in the config file;
The Endpoint is using Windows Authentication to connect to RavenDB via a Domain account;

At endpoint startup hapèpens that the connection to RavenDB cannot be established, thus the endpoint fails to set timeout persistance, subscriptions and so on.

The first thing we tries is to move to the ApiKey auth and everything worked as expected stating the RavenDB wan correctly running and no third party issues were influencing the communication.

Digging deeper we discovered in the logs the following:

2014-05-07 12:58:52,246 [1] INFO NServiceBus.Persistence.Raven.RavenUserInstaller [(null)] <(null)> - Adding user '' to raven. Instance:'http://ip:port', Database:''.
2014-05-07 12:58:52,261 [1] WARN NServiceBus.Persistence.Raven.RavenUserInstaller [(null)] <(null)> - Failed to add user to raven. Processing will continue

basically the endpoint at startup tries to add a user that is already defined on the server and fails, I can repro on my side the issue using a server hosted on Azure, frem that point on each attempt to connect to RavenDB return a HTTP500.

I think that we should not try to create the RavenDB user.

Move Raven UoW to use Behaviors

/cc @andreasohlund

RavenUserInstaller should check if connection string has username supplied

Who's affected

Any user that utilizes RavenDB persistence and provides the user name in the connection string.

Symptoms

Basically the endpoint at startup tries to add a user that is already defined on the server and fails, I can repro on my side the issue using a server hosted on Azure, frem that point on each attempt to connect to RavenDB return a HTTP500.

A user should only be added to raven authorization if the connection string does not have a supplied account. If the connection string has something like:

Url=https://domain/;Database= serviceBus;user=john;password=….;

Then it should not be attempted to create a RavenDB user.

Raised by @johnsimons
Migrated from by Particular/NServiceBus#2099

particular / nservicebus.ravendb Goto Github PK

nservicebus.ravendb's People

Contributors

Stargazers

Watchers

Forkers

nservicebus.ravendb's Issues

Symptoms

Who's affected

Description

Implementation

Repro

Root cause

Who's affected

Symptoms

Who's affected

Symptoms

Recommend Projects

Recommend Topics

Recommend Org