particular / nservicebus.ravendb Goto Github PK
View Code? Open in Web Editor NEWHome Page: https://docs.particular.net/nservicebus/ravendb/
License: Other
Home Page: https://docs.particular.net/nservicebus/ravendb/
License: Other
Raised by @johnsimons
Migrated from by Particular/NServiceBus#2133
I have been looking at this issue reported in the mailing list.
So to summarise the issue, the user is using the Scheduler API and creates to schedule tasks, one that runs every 5secs and another one that runs every 1min.
Every so often the 5sec schedule task stops from reoccurring.
Here are the logs
And here is one of the timeouts stored in Raven:
{
"Destination": {
"Queue": "Flyt.TiosCtcAgent",
"Machine": "GMBSANNTID"
},
"SagaId": "00000000-0000-0000-0000-000000000000",
"State": "PD94bWwgdmVyc2lvbj0iMS4wIiA/Pg0KPE1lc3NhZ2VzIHhtbG5zOnhzaT0iaHR0cDovL3d3dy53My5vcmcvMjAwMS9YTUxTY2hlbWEtaW5zdGFuY2UiIHhtbG5zOnhzZD0iaHR0cDovL3d3dy53My5vcmcvMjAwMS9YTUxTY2hlbWEiIHhtbG5zPSJodHRwOi8vdGVtcHVyaS5uZXQvTlNlcnZpY2VCdXMuU2NoZWR1bGluZy5NZXNzYWdlcyI+CjxTY2hlZHVsZWRUYXNrPgo8VGFza0lkPmU2YjZlN2NmLTlmYjMtNDhiZS1iZGUwLTc0NmJhZmE3MmRlNzwvVGFza0lkPgo8TmFtZT5Cb290c3RyYXA8L05hbWU+CjxFdmVyeT5QVDVTPC9FdmVyeT4KPC9TY2hlZHVsZWRUYXNrPgo8L01lc3NhZ2VzPg0K",
"Time": "2014-05-21T08:18:00.2114720Z",
"CorrelationId": "2f849e22-d760-4ce6-846b-a33200a9b784\\0",
"OwningTimeoutManager": "Flyt.TiosCtcAgent",
"Headers": {
"NServiceBus.MessageId": "2f849e22-d760-4ce6-846b-a33200a9b784",
"NServiceBus.CorrelationId": "2f849e22-d760-4ce6-846b-a33200a9b784",
"NServiceBus.OriginatingEndpoint": "Flyt.TiosCtcAgent",
"$.diagnostics.originating.hostid": "522caf7d3f53a790fdcc328b6cb0d627",
"NServiceBus.MessageIntent": "Send",
"NServiceBus.Version": "4.4.2",
"NServiceBus.TimeSent": "2014-05-21 08:17:55:211472 Z",
"NServiceBus.OriginatingMachine": "GMBSANNTID",
"NServiceBus.ContentType": "text/xml",
"NServiceBus.EnclosedMessageTypes": "NServiceBus.Scheduling.Messages.ScheduledTask, NServiceBus.Core, Version=4.4.0.0, Culture=neutral, PublicKeyToken=9fc386479f8a226c",
"CorrId": "2f849e22-d760-4ce6-846b-a33200a9b784\\0",
"WinIdName": "GMBDOMENE1\\sanntidintegrasjon",
"NServiceBus.RelatedTo": "5da0ead5-1701-4e2e-9b43-a33200a9b16e",
"NServiceBus.ConversationId": "a00caaba-1eab-48a7-854f-a33100a5c243",
"NServiceBus.IsDeferredMessage": "True",
"NServiceBus.Temporary.DelayDeliveryWith": "00:00:05",
"NServiceBus.Timeout.Expire": "2014-05-21 08:18:00:211472 Z",
"NServiceBus.Timeout.RouteExpiredTimeoutTo": "Flyt.TiosCtcAgent@GMBSANNTID",
"NServiceBus.Timeout.ReplyToAddress": "Flyt.TiosCtcAgent@GMBSANNTID"
}
}
The only explanation I have is if the index is stale, we could end-up skipping over timeouts.
@synhershko has actually identified this issue in the https://github.com/Particular/NServiceBus.RavenDB repo and has fixed it.
I cannot replicate this myself, but the index staleness is definitely a possibility and that would explain the entries I can see in the log file.
The new metamodel in core v6 contains the properties that are unique. The persister should honor this.
This is how the InMemory persister is currently doing it
This is currently not tested with the new implementation
Probably by using a dynamic test fixture, or having a test project for each. Ideally we could run each test (both in Unit and Acceptance) multiple times using different configs
@andreasohlund @johnsimons thoughts?
If Configure.PersistanceForAll
was called without a DocumentStore instance as a parameter
Somewhat related to #13
I installed RavenDB 3.0 yesterday and since then, a local NServiceBus app (using RavenDBPersistence) I've been developing has been throwing the following exception.
An exception of type 'System.InvalidOperationException' occurred in NServiceBus.Host.exe but was not handled in user code
Additional information: The RavenDB server you have specified is detected to be Product version: 3.0.0 / cdc39ac / , Build version: 3528. NServiceBus requires RavenDB version 2.5 build 2900 or higher to operate correctly. Please update your RavenDB server.
I see this is because the IsSufficientVersion() method in ConnectionVerifier.cs is hard-coded to only return True when the product version is equal to 2.5. For any newer versions of RavenDB, it will fail and give a confusing error message suggesting that your version of RavenDB needs updating when if anything, it needs downgrading.
This issue is to track all tests that we commented out or removed from the core but need to run in the RavenDB scope. Some were targeted to RavenDB persistence but should be generalized to be used by more persistence solutions
So in the raven timeout persistence we have
public void RemoveTimeoutBy(Guid sagaId)
{
using (var session = OpenSession())
{
var items = session.Query<TimeoutData>().Where(x => x.SagaId == sagaId);
foreach (var item in items)
session.Delete(item);
session.SaveChanges();
}
}
IDocumentSession OpenSession()
{
var session = store.OpenSession();
session.Advanced.AllowNonAuthoritativeInformation = false;
session.Advanced.UseOptimisticConcurrency = true;
return session;
}
Similar code exists in subscriptions and deduplication
But we have the concept of ISessionProvider
to allow us to share the session, which is what the saga persister uses. So why dont the above use the same shared session?
Raised by @indualagarsamy
Migrated from by Particular/NServiceBus#2357
Steps to reproduce
Make sure to use RavenDB 2.5 (In this test, I am running 2.5.0, Build version: 2916
)
SagaTimeoutTest_v4_4_0 project
. This will request 10 timeouts and wait for 30 seconds and shut down.SagaTimeoutTest_v5_0_0
endpoint.NOTE: Running the same scenario using SagaTimeoutTest_v4_6_4
endpoint results in an startup error.
Exception when starting endpoint.
Autofac.Core.DependencyResolutionException: An exception was thrown while execut
ing a resolve operation. See the InnerException for details. ---> The RavenDB se
rver you have specified is detected to be Product version: 2.5.0 / 6dce79a, Buil
d version: 2916. NServiceBus requires RavenDB version 2.0 to operate correctly.
Please update your RavenDB server.
Further instructions can be found at:http://particular.net/articles/using-ravend
b-in-nservicebus-installing (See inner exception for details.) ---> System.Inval
idOperationException: The RavenDB server you have specified is detected to be Pr
oduct version: 2.5.0 / 6dce79a, Build version: 2916. NServiceBus requires RavenD
B version 2.0 to operate correctly. Please update your RavenDB server.
The same scenario works correct between 4.4.0 and 4.5.0.
In the connection verifier we should enforce that build is >= 2900
Streaming with RavenDB 2.5 breaks with an exception when using Embedded and breaking out of a streaming loop. This was fixed today with v2.5.2886 (unstable) and we should upgrade our tests suite to use that one.
It is worth noting if the user will pass an EmbeddedDocumentStore instance to be used by the timeouts module it will just break and not work unless he is using v2.5.2886 or higher. This is an unfortunate edge case but worth knowing and having that documented - hence this issue.
When endpoints are deployed to IIS, the user they run under is a low privilege user, thus the endpoint cannot create a new database in RavenDB at startup time.
Should we create a PowerShell script to automate this process?
Same as the old RavenSessionFactory
.
See conversation here https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/particularsoftware/2TgCwtC7rxA/KJ6GORJeui0J
Since gateway is now out of the core, all Gateway related code needs to go to a NServiceBus.RavenDB.Gateway
assembly
Should we remove RavenPersistence
since IPersistMessages
is now obsolete in favour of IDeduplicateMessages
?
The RavenDB integration has been extracted (copied) from the core to allow better support of RavenDB version changes moving forward.
Maybe I have a special setup, maybe I don't. I've been fighting the whole day to get this working.
I have two services which use RavenDB. One uses it for sagas and timeouts, the other one just for second level retries. The saga/timeout service will create its RavenDb database upon installation, the SLR service won't. RavenDB doesn't even show up in the feature list on startup, despite being explicitly configured.
configuration.AssembliesToScan(GetType().Assembly);
configuration
.Conventions()
.DefiningCommandsAs(type => type.Namespace != null && type.Namespace.Contains(".Commands"));
configuration.UsePersistence<RavenDBPersistence>();
configuration.UseSerialization<JsonSerializer>();
configuration.EnableFeature<SecondLevelRetries>();
configuration.EnableFeature<TimeoutManager>();
configuration.DisableFeature<Sagas>();
configuration.DisableFeature<AutoSubscribe>();
$ NServiceBus.Host.exe /install
Initializing the installer in the Install AppDomain
2014-12-11 18:37:00.588 INFO DefaultFactory Logging to 'D:\Code\Repro\bin\Debug\' with level Info
Executing the NServiceBus installers
2014-12-11 18:37:01.011 INFO NServiceBus.Hosting.Profiles.ProfileManager Activating profile: NServiceBus.Production, NServiceBus.Host, Version=5.0.0.0, Culture=neutral, PublicKeyToken=9fc386479f8a226
c
2014-12-11 18:37:01.319 INFO NServiceBus.Persistence.PersistenceStartup Activating persistence 'RavenDBPersistence' to provide storage for 'GatewayDeduplication' storage.
2014-12-11 18:37:01.326 INFO NServiceBus.Persistence.PersistenceStartup Activating persistence 'RavenDBPersistence' to provide storage for 'Timeouts' storage.
2014-12-11 18:37:01.328 INFO NServiceBus.Persistence.PersistenceStartup Activating persistence 'RavenDBPersistence' to provide storage for 'Sagas' storage.
2014-12-11 18:37:01.332 INFO NServiceBus.Persistence.PersistenceStartup Activating persistence 'RavenDBPersistence' to provide storage for 'Subscriptions' storage.
2014-12-11 18:37:01.797 WARN NServiceBus.Licensing.LicenseManager Trial for the Particular Service Platform has expired
2014-12-11 18:37:01.866 INFO NServiceBus.Features.UnicastBus Number of messages found: 2
2014-12-11 18:37:02.674 INFO NServiceBus.PerformanceMonitorUsersInstaller Skipped adding user 'AXL\Administrator' to group 'Leistungsüberwachungsbenutzer' because the user is already in group.
2014-12-11 18:37:02.689 INFO NServiceBus.Features.DisplayDiagnosticsForFeatures ------------- FEATURES ----------------
Name: CriticalErrorHandling
Version: 5.1.2
Enabled by Default: Yes
Status: Enabled
Dependencies: None
Startup Tasks: None
Name: CustomIDataBus
Version: 5.1.2
Enabled by Default: No
Status: Disabled
Deactivation reason: Did not meet one of the dependencies: [DataBus]
Name: DataBus
Version: 5.1.2
Enabled by Default: Yes
Status: Disabled
Deactivation reason: Did not fulfill its Prerequisites:
-No databus properties was found in available messages
Name: Encryptor
Version: 5.1.2
Enabled by Default: Yes
Status: Disabled
Deactivation reason: Did not fulfill its Prerequisites:
-No encryption properties was found in available messages
Name: ErrorSubscribers
Version: 5.1.2
Enabled by Default: Yes
Status: Enabled
Dependencies: None
Startup Tasks: None
Name: ForwarderFaultManager
Version: 5.1.2
Enabled by Default: Yes
Status: Enabled
Dependencies: None
Startup Tasks: None
Name: InMemoryFaultManager
Version: 5.1.2
Enabled by Default: No
Status: Disabled
Deactivation reason: Did not meet one of the dependencies:
Name: InstallationSupport
Version: 5.1.2
Enabled by Default: No
Status: Enabled
Dependencies: None
Startup Tasks: None
Name: CriticalTimeMonitoring
Version: 5.1.2
Enabled by Default: No
Status: Enabled
Dependencies: None
Startup Tasks: None
Name: Audit
Version: 5.1.2
Enabled by Default: Yes
Status: Disabled
Deactivation reason: Did not fulfill its Prerequisites:
-No configured audit queue was found
Name: AutoSubscribe
Version: 5.1.2
Enabled by Default: Yes
Status: Disabled
Deactivation reason: Did not meet one of the dependencies:
Name: MsmqSubscriptionPersistence
Version: 5.1.2
Enabled by Default: No
Status: Disabled
Deactivation reason: Did not meet one of the dependencies:
Name: Scheduler
Version: 5.1.2
Enabled by Default: Yes
Status: Enabled
Dependencies: None
Startup Tasks: None
Name: CustomSerialization
Version: 5.1.2
Enabled by Default: Yes
Status: Disabled
Deactivation reason: Did not fulfill its Prerequisites:
-CustomSerialization not enable since serialization definition not detected.
Name: ForwardReceivedMessages
Version: 5.1.2
Enabled by Default: No
Status: Disabled
Deactivation reason: Did not meet one of the dependencies:
Name: RegisterHandlersInOrder
Version: 5.1.2
Enabled by Default: Yes
Status: Enabled
Dependencies: None
Startup Tasks: None
Name: SLAMonitoring
Version: 5.1.2
Enabled by Default: No
Status: Enabled
Dependencies: None
Startup Tasks: None
Name: LicenseReminder
Version: 5.1.2
Enabled by Default: Yes
Status: Enabled
Dependencies: None
Startup Tasks: None
Name: Outbox
Version: 5.1.2
Enabled by Default: No
Status: Disabled
Deactivation reason: Did not meet one of the dependencies:
Name: InMemoryGatewayPersistence
Version: 5.1.2
Enabled by Default: No
Status: Disabled
Deactivation reason: Did not meet one of the dependencies: [Gateway]
Name: InMemoryOutboxPersistence
Version: 5.1.2
Enabled by Default: No
Status: Disabled
Deactivation reason: Did not meet one of the dependencies: [Outbox]
Name: InMemorySagaPersistence
Version: 5.1.2
Enabled by Default: No
Status: Disabled
Deactivation reason: Did not meet one of the dependencies: [Sagas]
Name: InMemorySubscriptionPersistence
Version: 5.1.2
Enabled by Default: No
Status: Disabled
Deactivation reason: Did not meet one of the dependencies: [MessageDrivenSubscriptions]
Name: InMemoryTimeoutPersistence
Version: 5.1.2
Enabled by Default: No
Status: Disabled
Deactivation reason: Did not meet one of the dependencies: [TimeoutManager]
Name: TimeoutManagerBasedDeferral
Version: 5.1.2
Enabled by Default: No
Status: Enabled
Dependencies: None
Startup Tasks: None
Name: UnicastBus
Version: 5.1.2
Enabled by Default: Yes
Status: Enabled
Dependencies: None
Startup Tasks: None
Name: BinarySerialization
Version: 5.1.2
Enabled by Default: Yes
Status: Disabled
Deactivation reason: Did not fulfill its Prerequisites:
-BinarySerialization not enable since serialization definition not detected.
Name: BsonSerialization
Version: 5.1.2
Enabled by Default: Yes
Status: Disabled
Deactivation reason: Did not fulfill its Prerequisites:
-BsonSerialization not enable since serialization definition not detected.
Name: JsonSerialization
Version: 5.1.2
Enabled by Default: Yes
Status: Enabled
Dependencies: None
Startup Tasks: None
Name: XmlSerialization
Version: 5.1.2
Enabled by Default: Yes
Status: Disabled
Deactivation reason: Did not fulfill its Prerequisites:
-XmlSerialization not enable since serialization definition not detected.
Name: MsmqTransportConfigurator
Version: 5.1.2
Enabled by Default: No
Status: Enabled
Dependencies: None
Startup Tasks: None
Name: TimeoutManager
Version: 5.1.2
Enabled by Default: No
Status: Enabled
Dependencies: [TimeoutManagerBasedDeferral]
Startup Tasks: None
Name: Sagas
Version: 5.1.2
Enabled by Default: Yes
Status: Disabled
Deactivation reason: Did not meet one of the dependencies:
Name: SecondLevelRetries
Version: 5.1.2
Enabled by Default: Yes
Status: Enabled
Dependencies: [ForwarderFaultManager]
Startup Tasks: None
Name: DataBusFileBased
Version: 5.1.2
Enabled by Default: No
Status: Disabled
Deactivation reason: Did not meet one of the dependencies: [DataBus]
Name: StorageDrivenPublishing
Version: 5.1.2
Enabled by Default: No
Status: Enabled
Dependencies: None
Startup Tasks: None
Name: MessageDrivenSubscriptions
Version: 5.1.2
Enabled by Default: No
Status: Enabled
Dependencies: None
Startup Tasks: None
Currently if users forgets the [Unique] attribute in the saga property they correlate on the persister will use a lucene query to find the saga. Since this will return potentially stale data, especially during load, it might cause consistency issues where multiple sagas might get created or updated concurrently. Since this is rarely what the user want we now:
I just added a package to the solution, and Ripple rewrote the csproj of one of the projects to add back some references that were ommitted:
However, this breaks compilation due to strong naming issues like this one:
System.Exception : Could not enumerate all types for 'Z:\Projects\Particular\NServiceBus.RavenDB\src\NServiceBus.RavenDB.AcceptanceTests\bin\Debug\NServiceBus.ObjectBuilder.Ninject.dll'.
It looks like you may be missing binding redirects in your config file for the following assemblies:
Ninject, Version=3.0.0.0, Culture=neutral, PublicKeyToken=c7192dc5380945e7
Ninject.Extensions.NamedScope, Version=3.0.0.0, Culture=neutral, PublicKeyToken=c7192dc5380945e7
For more information see http://msdn.microsoft.com/en-us/library/7wd6ex19(v=vs.100).aspx
I'm going to revert this change for now until we can figure out what's going on
At the moment the error message says:
The RavenDB server you have specified is detected to be Product version: 3.0.0 / cdc39ac / , Build version: 3528. NServiceBus requires RavenDB version 2.5 build 2900 or higher to operate correctly. Please update your RavenDB server.
This needs to be updated to:
The RavenDB server you have specified is detected to be Product version: 3.0.0 / cdc39ac / , Build version: 3528. NServiceBus requires version 2.5 build 2900 (or a higher build number for version 2.5) to operate correctly. Please update your RavenDB server.
Anyone that upgrades Raven to a higher version then 2.5
Raised by @andreasohlund
Migrated from by Particular/NServiceBus#425
To allow us to run without msmq we should consider creating a RavenDB transport
When I set up a new NServiceBus host project by getting latest stables from Nuget, I run into a nasty dependency problem.
Steps to repro:
Doing this, my packages.config looks like this:
<package id="NServiceBus" version="4.6.3" targetFramework="net45" />
<package id="NServiceBus.Host" version="4.6.3" targetFramework="net45" />
<package id="NServiceBus.Interfaces" version="4.6.3" targetFramework="net45" />
<package id="NServiceBus.RabbitMQ" version="1.1.3" targetFramework="net45" />
<package id="NServiceBus.RavenDB" version="1.0.0" targetFramework="net45" />
<package id="RabbitMQ.Client" version="3.2.1" targetFramework="net45" />
<package id="RavenDB.Client" version="2.5.2908" targetFramework="net45" />
When I run, I get
WARN NServiceBus.Persistence.Raven.RavenUserInstaller [(null)] <(null)> - Failed to add user to raven. Processing will continue
System.MissingMethodException: Method not found: '!!0 Raven.Abstractions.Extensions.JsonExtensions.JsonDeserialization(Raven.Json.Linq.RavenJObject)'.
at NServiceBus.Persistence.Raven.RavenUserInstaller.AddUserToDatabase(String identity, DocumentStore documentStore)
at NServiceBus.Persistence.Raven.RavenUserInstaller.Install(String identity)in c:\BuildAgent\work\31f8c64a6e8a2d7c\src\NServiceBus.Core\Persistence\Raven\RavenUserInstaller.cs:line 42
To me, it appears to be a version conflict, since the public API of Raven.Abstractions has changed in RavenDB release RavenDB-2464 DTC NSB. The signature of extension method JsonDeserialization used inside RavenUserInstaller.AddUserToDatabase has changed:
ravendb/ravendb@e3fcf5b.
I was only able to work around this by compiling NServiceBus.RavenDB against build 2700 of RavenDB.Client.Lightweight and referencing it outside of nuget.
Users can now take a dependency on the ISessionProvider
to get access to the session managed by the RavenDB shared session support
Not sure where to post this, so let it be here
It appears like ripple is being run by VS at build time, so sporadically I get this:
1>------ Build started: Project: NServiceBus.RavenDB, Configuration: Debug Any CPU ------
1>
1> ===============================================
1> String: Error executing ExplodeDownloadedNugets
1> ===============================================
1>
1> An unhandled exception occurred:
1> Exception:
1> Access to the path 'Z:\Projects\Particular\NServiceBus.RavenDB\src\packages\NServiceBus\lib\net45\NServiceBus.Core.xml' is denied.
1> StackTrace:
1> at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
1> at System.IO.File.SetAttributes(String path, FileAttributes fileAttributes)
1> at ripple.FileSystemExtensions.cleanDirectory(String directory, Boolean remove)
1> at ripple.FileSystemExtensions.cleanDirectory(String directory, Boolean remove)
1> at ripple.FileSystemExtensions.cleanDirectory(String directory, Boolean remove)
1> at ripple.FileSystemExtensions.ForceClean(IFileSystem system, String path)
1> at ripple.Nuget.NugetFile.ExplodeTo(String directory)
1> at ripple.Steps.ExplodeDownloadedNugets.<Execute>b__0(INugetFile nuget)
1> at System.Collections.Generic.GenericEnumerableExtensions.Each[T](IEnumerable`1 values, Action`1 eachAction)
1> at ripple.Steps.ExplodeDownloadedNugets.Execute(RippleInput input, IRippleStepRunner runner)
1> at ripple.RippleOperation.Execute(Boolean throwOnFailure)
1> Source:
1> mscorlib
1> TargetSite:
1> Void WinIOError(Int32, System.String)
1>
1>
1> ========================
1> String: Error occurred:
1> ========================
1>
1> An unhandled exception occurred:
1> Exception:
1> Access to the path 'Z:\Projects\Particular\NServiceBus.RavenDB\src\packages\NServiceBus\lib\net45\NServiceBus.Core.xml' is denied.
1> StackTrace:
1> at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
1> at System.IO.File.SetAttributes(String path, FileAttributes fileAttributes)
1> at ripple.FileSystemExtensions.cleanDirectory(String directory, Boolean remove)
1> at ripple.FileSystemExtensions.cleanDirectory(String directory, Boolean remove)
1> at ripple.FileSystemExtensions.cleanDirectory(String directory, Boolean remove)
1> at ripple.FileSystemExtensions.ForceClean(IFileSystem system, String path)
1> at ripple.Nuget.NugetFile.ExplodeTo(String directory)
1> at ripple.Steps.ExplodeDownloadedNugets.<Execute>b__0(INugetFile nuget)
1> at System.Collections.Generic.GenericEnumerableExtensions.Each[T](IEnumerable`1 values, Action`1 eachAction)
1> at ripple.Steps.ExplodeDownloadedNugets.Execute(RippleInput input, IRippleStepRunner runner)
1> at ripple.RippleOperation.Execute(Boolean throwOnFailure)
1> at ripple.Commands.RestoreCommand.Execute(RestoreInput input)
1> at ripple.RippleOperation.CommandExecutionExpression.Execute[TInput,TCommand](TInput input, TCommand command)
1> at RippleRestoreTask.RestoreTask.InnerExecute(ILogListener logListener, String targetDirectory)
1> at RippleRestoreTask.RestoreTask.<>c__DisplayClass2.<Execute>b__0()
1> at Locker.ExecuteLocked(String name, Action action)
1> at RippleRestoreTask.RestoreTask.Execute()
1> Source:
1> mscorlib
1> TargetSite:
1> Void WinIOError(Int32, System.String)
1>
2>------ Build started: Project: NServiceBus.RavenDB.Tests, Configuration: Debug Any CPU ------
2> NServiceBus.RavenDB.Tests -> Z:\Projects\Particular\NServiceBus.RavenDB\src\NServiceBus.RavenDB.Tests\bin\Debug\NServiceBus.RavenDB.Tests.dll
========== Build: 1 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========
Followed by missing DLLs that basically break my IDE. I need to shut down VS, do ripple restore and then resume work.
Any idea what's going on?
Currently we enforce that a ContainsSagaData
instance has only one property decorated with the Unique
attribute.
Should we fix Particular/NServiceBus#1039 in this new dll ?
Some saga tests are tightly coupled with RavenDB stuff (see for example the test and endpoint in When_using_a_received_message_for_timeout).
Since we are breaking RavenDB out of the core, I'd suggest we either use an in-memory storage for sagas by default or move those tests to only be run on RavenDB's suite.
I guess this depends on what we are testing here:
Either way, the Acceptance Tests suite cannot contain the words RavenDb (neither does the core, but that's for later).
@andreasohlund @SimonCropp @johnsimons Thoughts?
Regarding this code:
IEnumerable<T> GetByQuery<T>(string property, object value) where T : IContainSagaData
{
try
{
return sessionProvider.Session.Advanced.LuceneQuery<T>()
.WhereEquals(property, value)
.WaitForNonStaleResultsAsOfNow();
}
catch (InvalidCastException)
{
return new[]
{
default(T)
};
}
}
I see 2 issues that need to be tackled:
WaitingForNonStaleResultsAsOfNow()
? That's something that should never be done in productionSo we are now using nuget (and not ripple) for dependencies and also to pack a nuget package as a build artifact, however it seems to not pick NSB 5-alpha / 5-beta corretly when installed in a project, even with the -Pre flag on.
I tried solving it here 0f7ca07 but to no avail
This is what it looks like when trying to install in a fresh project:
PM> Install-Package NServiceBus.RavenDB -Pre
Attempting to resolve dependency 'Microsoft.CompilerServices.AsyncTargetingPack (≥ 1.0.1.0 && < 2.0.0.0)'.
Attempting to resolve dependency 'NServiceBus (≥ 5.0.0.0 && < 6.0.0.0)'.
Install-Package : Unable to resolve dependency 'NServiceBus (≥ 5.0.0.0 && < 6.0.0.0)'.
At line:1 char:1
+ Install-Package NServiceBus.RavenDB -Pre
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Install-Package], InvalidOperationException
+ FullyQualifiedErrorId : NuGetCmdletUnhandledException,NuGet.PowerShell.Commands.InstallPackageCommand
So we have people who are persisting things using the raven parts in the core. now we are externalising those parts into the assembly produced by this repository
We need to ensure that people can move over to the new approach in the simplest possible manner. so ideally with no migration or if there is migration involved document it well or perhaps write a tool to perform this migration.
And report a nice error using Helpers.LogRavenConnectionFailure
in case of failure
Raised by @danielHalan
Migrated from by Particular/NServiceBus#2144
Getting following exception when Starting NSB on an ASP.NET MVC Instance, at Configure.With()....ForMvc()...RavenPersistence().RavenSubscriptionStorage()....
.UnicastBus()
.LoadMessageHandlers()
.CreateBus()
And http://xx:8080 is up n running.
System.WebException "The remote server returned an error: (403) Forbidden."
at System.Net.HttpWebRequest.GetResponse()
at Raven.Client.Connection.HttpJsonRequest.ReadJsonInternal(Func`1 getResponse) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Connection\HttpJsonRequest.cs:line 341
at Raven.Client.Connection.HttpJsonRequest.ReadResponseJson() in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Connection\HttpJsonRequest.cs:line 244
at Raven.Client.Connection.ServerClient.DirectPut(RavenJObject metadata, String key, Nullable`1 etag, RavenJObject document, String operationUrl) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Connection\ServerClient.cs:line 398
at Raven.Client.Connection.ServerClient.<>c__DisplayClass13.<Put>b__12(String u) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Connection\ServerClient.cs:line 333
at Raven.Client.Connection.ReplicationInformer.TryOperation[T](Func`2 operation, String operationUrl, Boolean avoidThrowing, T& result) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Connection\ReplicationInformer.cs:line 564
at Raven.Client.Connection.ReplicationInformer.ExecuteWithReplication[T](String method, String primaryUrl, Int32 currentRequest, Int32 currentReadStripingBase, Func`2 operation) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Connection\ReplicationInformer.cs:line 523
at Raven.Client.Connection.ServerClient.ExecuteWithReplication[T](String method, Func`2 operation) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Connection\ServerClient.cs:line 174
at Raven.Client.Connection.ServerClient.Put(String key, Nullable`1 etag, RavenJObject document, RavenJObject metadata) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Connection\ServerClient.cs:line 333
The RavenDBPersistence adds an extension method, SetDefaultDocumentStore, that has an overload that takes a ConnectionParameters instance:
I'm following up a Desk case where the requirement is to manually build, in C#, the connection string and set it at endpoint configuration time. Suggestions?
Raised by @johnsimons
Migrated from by Particular/NServiceBus#1973
As part of fine tuning SC to handle a high volume of messages we have identified a bottleneck on our ravenDB saga persistence.
The issue is that we auto gen our own docs ids and those ids are not the ideal for Raven inserts, which causes the insertions to get slower as the data grows.
So the solution is to let Raven auto gen its own ids using HiLo.
Looking at the current code base, one way I see us moving forward with this is to have a in-memory mapping of NSB id <-> Raven id
.
So when we instantiate RavenSagaPersister
we perform a query to initialize this in-memory mapping and then we keep it up to date on all inserts/deletes
This would be actually quite simple to spike?
@andreasohlund thoughts?
Let's maintain a list of all breaking changes that we are going to have to support for people migrating from internal RavenDB to this package:
if (IsASagaEntity(t)) { tagName = tagName.Replace("Data", String.Empty); }
By using RavenDB build 2900 or above, and documentStore.TransactionRecoveryStorage = new IsolatedStorageTransactionRecoveryStorage();
The fix is changing DocumentStoreWrapper so that auto-wiring cannot override the value (the now does not have a setter and the store is passed via the constructor and stored in a private field)
var configuration = new BusConfiguration();
var appDataStore = new DocumentStore
{
ConnectionStringName = "RavenDB",
DefaultDatabase = "ApplicationData"
};
// Raven DataStore for NServiceBus
var nsbDataStore = new DocumentStore
{
ConnectionStringName = "NServiceBus.Persistence",
DefaultDatabase = "BookingProcessing"
};
// Set up and build AutoFac container
var builder = new ContainerBuilder();
builder.RegisterInstance<DocumentStore>(appDataStore).As<IDocumentStore>().SingleInstance();
var container = builder.Build();
// Set up NServiceBus
configuration.UseContainer<AutofacBuilder>(customizations => customizations.ExistingLifetimeScope(container));
configuration.UsePersistence<RavenDBPersistence>().SetDefaultDocumentStore(nsbDataStore);
configuration.EnableFeature<Sagas>();
var bus = (UnicastBus) Bus.Create(configuration);
bus.Start();
Type documentStoreWrapper =
Type.GetType("NServiceBus.RavenDB.Internal.IDocumentStoreWrapper, NServiceBus.RavenDB", true);
var wrapper = bus.Builder.Build(documentStoreWrapper);
The registration here registeres an instance as singleton but because such registrations support property auto-wiring, when resolving the store gets updated to the one provided by the user.
Do we want this?
What this means is calling PersistenceForTimeouts
, PersistenceForSagas
and PersistenceForSubscriptions
will each accept an IDocumentStore
which will be registered for them and them only.
You can reuse the same document store for all 3 (by calling PersistenceForAll
or each individually) but that enables you to provide different stores.
Or we can say we only allow for one doc-store globally and blow up when one method is trying to set a doc-store when one is already set
Thoughts?
Here is what I was thinking:
Configure.Storage.RavenDB(s => {
s.UseSagas();
s.UseTimeouts();
s.Use(documentStore); //the default store
s.UseSubscriptions();
})
So in the core we default to the in-core RavenDB for all storage types https://github.com/Particular/NServiceBus/blob/4.6.5/src/NServiceBus.Core/Persistence/Raven/ConfigureRavenPersistence.cs#L159
public static void RegisterDefaults()
{
InfrastructureServices.SetDefaultFor<ISagaPersister>(() => Configure.Instance.RavenSagaPersister());
InfrastructureServices.SetDefaultFor<IPersistTimeouts>(() => Configure.Instance.UseRavenTimeoutPersister());
InfrastructureServices.SetDefaultFor<IPersistMessages>(() => Configure.Instance.UseRavenGatewayPersister());
InfrastructureServices.SetDefaultFor<IDeduplicateMessages>(() => Configure.Instance.UseRavenGatewayDeduplication());
InfrastructureServices.SetDefaultFor<ISubscriptionStorage>(() => Configure.Instance.RavenSubscriptionStorage());
}
And here https://github.com/Particular/NServiceBus.RavenDB/releases we say to use the new external RavenDB nuget all you need is
Configure.With()
.DefaultBuilder()
.RavenDBStorage()
However this means that the default for all the other storages (timeouts, subscriptions etc) will default to the ravendb in the core. which results in
"The RavenDB server you have specified is detected to be Product version: 2.5.0 / 6dce79a, Build version: 2916. NServiceBus requires RavenDB version 2.0 to operate correctly. Please update your RavenDB server. Further instructions can be found at:http://particular.net/articles/using-ravendb-in-nservicebus-installing"
The workaround is to fully define all the storages
Configure.With()
.DefaultBuilder()
.RavenDBStorage()
.UseRavenDBSagaStorage()
.UseRavenDBSubscriptionStorage()
.UseRavenDBTimeoutStorage()
.UseRavenDBGatewayDeduplicationStorage()
.UseRavenDBGatewayStorage()
Repro here https://github.com/Particular/BugsRepro/tree/master/RavenDbMissingInjection
Support the ability of having different DocumentStore
s for different storage types, e.g. one DocStore for sagas, one for timeouts etc
Obviously, the user could always provide one ShardedDocumentStore
instance that is configured to do this, but we can then support various other scenarios.
That will be done by registering DocumentStore
instances separately for each Feature.
Thoughts? @andreasohlund @SimonCropp @johnsimons
Any user that utilizes RavenDB persistence;
If the endpoint that depends on the RavenDB persistence is installed on a doman joined machine and the RavenDB server is installed on a machine in the same AD domain, it can happen that the endpoint at start up tries to add a user that is already defined on the server and fails, from that point on each attempt to connect to RavenDB returns a HTTP-500.
Raised by @mauroservienti
Migrated from by Particular/NServiceBus#2118
Customer report - 5060,
Scenario:
At endpoint startup hapèpens that the connection to RavenDB cannot be established, thus the endpoint fails to set timeout persistance, subscriptions and so on.
The first thing we tries is to move to the ApiKey auth and everything worked as expected stating the RavenDB wan correctly running and no third party issues were influencing the communication.
Digging deeper we discovered in the logs the following:
2014-05-07 12:58:52,246 [1] INFO NServiceBus.Persistence.Raven.RavenUserInstaller [(null)] <(null)> - Adding user '' to raven. Instance:'http://ip:port', Database:''.
2014-05-07 12:58:52,261 [1] WARN NServiceBus.Persistence.Raven.RavenUserInstaller [(null)] <(null)> - Failed to add user to raven. Processing will continue
basically the endpoint at startup tries to add a user that is already defined on the server and fails, I can repro on my side the issue using a server hosted on Azure, frem that point on each attempt to connect to RavenDB return a HTTP500.
I think that we should not try to create the RavenDB user.
/cc @andreasohlund
Any user that utilizes RavenDB persistence and provides the user name in the connection string.
Basically the endpoint at startup tries to add a user that is already defined on the server and fails, I can repro on my side the issue using a server hosted on Azure, frem that point on each attempt to connect to RavenDB return a HTTP500.
A user should only be added to raven authorization if the connection string does not have a supplied account. If the connection string has something like:
Url=https://domain/;Database= serviceBus;user=john;password=….;
Then it should not be attempted to create a RavenDB user.
Raised by @johnsimons
Migrated from by Particular/NServiceBus#2099
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.