Giter Club home page Giter Club logo

serilog-sinks-mongodb's Introduction

mongo icon Serilog.Sinks.MongoDB

NuGet version Downloads Build status

A Serilog sink that writes events as documents to MongoDB.

Package - Serilog.Sinks.MongoDB | Platforms - .NET 4.7.2, .NET Standard 2.0,, .NET Standard 2.1

New in v5.x

  • Output structured MongoDB Bson logs by switching to the .MongoDBBson() extensions. Existing the .MongoDB() extensions will continue to work converting logs to Json and then to Bson.
  • Rolling Log Collection Naming (Thanks to Revazashvili for the PR!). MongoDBBson sink only.
  • Expire TTL support. MongoDBBson sink only.

Configuration

In the examples below, the sink is writing to the database logs with structured Bson. The default collection name is log, but a custom collection can be supplied with the optional CollectionName parameter. The database and collection will be created if they do not exist.

// use Bson structured logs
var log = new LoggerConfiguration()
    .WriteTo.MongoDBBson("mongodb://mymongodb/logs")
    .CreateLogger();

// capped collection using Bson structured logs
var log = new LoggerConfiguration()
    .WriteTo.MongoDBBson("mongodb://mymongodb/logs", cfg =>
    {
        // optional configuration options:
        cfg.SetCollectionName("log");
        cfg.SetBatchPeriod(TimeSpan.FromSeconds(1));

        // create capped collection that is max 100mb
        cfg.SetCreateCappedCollection(100);
    })
    .CreateLogger();

// create sink instance with custom mongodb settings.
var log = new LoggerConfiguration()
	.WriteTo.MongoDBBson(cfg =>
    {
		// custom MongoDb configuration
		var mongoDbSettings = new MongoClientSettings
		{
			UseTls = true,			
			AllowInsecureTls = true,
			Credential = MongoCredential.CreateCredential("databaseName", "username", "password"),
			Server = new MongoServerAddress("127.0.0.1")
		};

		var mongoDbInstance = new MongoClient(mongoDbSettings).GetDatabase("serilog");
		
		// sink will use the IMongoDatabase instance provided
		cfg.SetMongoDatabase(mongoDbInstance);
		cfg.SetRollingInternal(RollingInterval.Month);
    })
	.CreateLogger();

JSON (Microsoft.Extensions.Configuration)

Keys and values are not case-sensitive. This is an example of configuring the MongoDB sink arguments from Appsettings.json:

{
  "Serilog": {
    "MinimumLevel": {
      "Default": "Information",
      "Override": {
        "Microsoft": "Error",
        "System": "Warning"
      }
    },
    "WriteTo": [
      { 
      	"Name": "MongoDBBson", 
        "Args": { 
            "databaseUrl": "mongodb://username:password@ip:port/dbName?authSource=admin",
            "collectionName": "logs",
            "cappedMaxSizeMb": "1024",
            "cappedMaxDocuments": "50000",
            "rollingInterval": "Month"
        }
      } 
    ]
  }
}

Icon

MongoDB icon by Icons8

serilog-sinks-mongodb's People

Contributors

ammachado avatar behroozbc avatar brantw avatar chandmk avatar giacomociti avatar jaben avatar kalahari avatar karthikvr1990 avatar laitnee avatar merbla avatar nblumhardt avatar rdoubleui avatar revazashvili avatar stevewgh avatar tugberkugurlu avatar yallie avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

serilog-sinks-mongodb's Issues

Serilog timestamp local time.

I'm using Serilog MongoDb Sink with ASP.NET Core 2.2. But I have a big problem. All my records are like that. Timestamps values and UtcTimesamps values are always the same. I couldn't find any solution to fix it. I want to use the localtime for timestamp. How can I do that?

"_id" : "5d1a6815e688ed7b7833d870"
"Timestamp": 2019-07-01T20:07:49.285+00:00
"Level": "Information"
"MessageTemplate": "Test"
"RenderedMessage": "Hello"
"Properties": ...
"UtcTimestamp": 2019-07-01 20:07:49Z

Capped option not set on new collection

I'm not able to get a mongo collection with the capped option using the config example from the README.

Below is my configuration:

...
        "WriteTo": [
            {
                "Name": "MongoDB",
                "Args": {
                    "databaseUrl": "mongodb://localhost:27017/site-monitor",
                    "collectionName": "monitoring.logs",
                    "restrictedToMinimumLevel": "Warning",
                    "cappedMaxSizeMb": "50",
                    "cappedMaxDocuments": "1000"
                }
            }
...

Do I miss something or is there a way to debug?

Properties with dot cannot be inserted into MongoDB

Mongo cannot insert documents that contain properties with dots, like so:

{ "some.property": 123 }

Hence there are no logs produced when any of the properties contain dot character, either from injected proeprties, templated message or exception data.

There was a broken fork - https://github.com/DarkXoro/serilog-sinks-mongodb
And proposed (yet no merged) PR two years ago - #45
Previous issue (closed but not resolved) - #29

This bug is almost two (!) years old, I'm willing to fix the previous fork and propose a new PR

Is it possible to add custom column?

Hello,
I'm storing modifications on the records as a change logs on mongodb. I'm wondering if it's possible to add custom columns? Or I want to store my changes as a json object not string. Because I don't want to do json parse on string variable. It's available on sql server but I couldn't find any documentation on mongo side.

image

Thank you.

MongoDBBson Save Data Not enough information

This is Old Data

image
Thi is MongoDBBson Data

{"_id":{"$oid":"61021bc17acd570dcf064263"},"UtcTimeStamp":{"$date":"2021-07-29T03:08:48.779Z"},"MessageTemplate":{"Text":"From:MixerServices:Query_PageList Msg:获取分页对象成功","Tokens":[{"_t":"TextToken","StartIndex":0,"Text":"From:MixerServices:Query_PageList Msg:获取分页对象成功"}]},"RenderedMessage":"From:MixerServices:Query_PageList Msg:获取分页对象成功","Properties":{},"Exception":{"_csharpnull":true}}

Loss Log Level

Code
.WriteTo.MongoDBBson("")
V:5.0.0-dev-00104
Please Add it.

Unable to start application when MongoDB is not available

Hi!

I have an app that uses Serilog.Sinks.MongoDB to log messages to mongodb. This app can't be started if the mongodb instance is not accessible for some reason, for example due to the mongodb instance is down or due to network issues.

The following error is thrown: A timeout occured after 30000ms selecting a server using ... <<removed for brevity>>. The stacktrace is

   at MongoDB.Driver.Core.Clusters.Cluster.ThrowTimeoutException(IServerSelector selector, ClusterDescription description)
   at MongoDB.Driver.Core.Clusters.Cluster.WaitForDescriptionChangedHelper.HandleCompletedTask(Task completedTask)
   at MongoDB.Driver.Core.Clusters.Cluster.WaitForDescriptionChanged(IServerSelector selector, ClusterDescription description, Task descriptionChangedTask, TimeSpan timeout, CancellationToken cancellationToken)
   at MongoDB.Driver.Core.Clusters.Cluster.SelectServer(IServerSelector selector, CancellationToken cancellationToken)
   at MongoDB.Driver.Core.Bindings.ReadPreferenceBinding.GetReadChannelSource(CancellationToken cancellationToken)
   at MongoDB.Driver.Core.Operations.ListCollectionsOperation.Execute(IReadBinding binding, CancellationToken cancellationToken)
   at MongoDB.Driver.OperationExecutor.ExecuteReadOperation[TResult](IReadBinding binding, IReadOperation`1 operation, CancellationToken cancellationToken)
   at MongoDB.Driver.MongoDatabaseImpl.ExecuteReadOperation[T](IReadOperation`1 operation, ReadPreference readPreference, CancellationToken cancellationToken)
   at MongoDB.Driver.MongoDatabaseImpl.ListCollections(ListCollectionsOptions options, CancellationToken cancellationToken)
   at Serilog.Helpers.MongoDbHelper.CollectionExists(IMongoDatabase database, String collectionName)
   at Serilog.Helpers.MongoDbHelper.VerifyCollectionExists(IMongoDatabase database, String collectionName, CreateCollectionOptions collectionCreationOptions)
   at Serilog.Sinks.MongoDB.MongoDBSink..ctor(IMongoDatabase database, Int32 batchPostingLimit, Nullable`1 period, IFormatProvider formatProvider, String collectionName, CreateCollectionOptions collectionCreationOptions)
   at Serilog.LoggerConfigurationMongoDBExtensions.MongoDB(LoggerSinkConfiguration loggerConfiguration, String databaseUrl, String collectionName, LogEventLevel restrictedToMinimumLevel, Int32 batchPostingLimit, Nullable`1 period, IFormatProvider formatProvider)

Is it possible to ignore those connectivity issues when mongodb is down and to not send log messages to mongo for a while? I need my application to be started even if mongodb is not accessible and probably be able to use some fallback logging destination.

Thank you.

Request: Detailed error messages from Log.Logger

Testing this code in different environments:

Log.Logger = new LoggerConfiguration() .MinimumLevel.Debug() .Enrich.FromLogContext() .WriteTo.MongoDB(GetConnectionString()) .CreateLogger();

Normally it works fine:

  1. Create the log collection
  2. Write to it with an ILogger call like logger.LogInformation("hi!");

In Integration Tests it runs through the same code but:

  1. Does not create the log collection
  2. Does not write
  3. Does not generate any errors

Request: can we set a flag to generate meaningful errors?

Memory leak

Software version :
4.1.0-dev-00069 on dotnet core 3.1
step to produce :
1.create a simple console app (C#)
2.Create a mongodb logger with :
_serverobject.EventLogger = new LoggerConfiguration() .MinimumLevel.Debug() .WriteTo.MongoDB("mongodb://127.0.0.1:27017/logs", "SinaConsole", LogEventLevel.Debug) .CreateLogger();
3.start Performance Profiler and select .NET object allocation

Expected :
There should be no memory leak over time.
Observed:
There is a memory leak over the time.
there is no problem with other loggers( .WriteTo.file . etc.)

image

Release v4.1.0

@nblumhardt any issue you see releasing v4.1.0 from current dev as final release before moving to v5.0 for the BsonStructure PR?

mongolab and serlog mongodb sink

I am getting this when attempting to connect to MongoDB hosted on MongoLab.

An exception of type 'System.TimeoutException' occurred in MongoDB.Driver.Core.dll but was not handled in user code
Additional information: A timeout occured after 30000ms selecting a server using CompositeServerSelector{ Selectors = ReadPreferenceServerSelector{ ReadPreference = { Mode = Primary, TagSets = [] } }, LatencyLimitingServerSelector{ AllowedLatencyRange = 00:00:00.0150000 } }. Client view of cluster state is { ClusterId : "1", ConnectionMode : "Automatic", Type : "Unknown", State : "Disconnected", Servers : [{ ServerId: "{ ClusterId : 1, EndPoint : "Unspecified/ds155461.mlab.com:55461" }", EndPoint: "Unspecified/ds155461.mlab.com:55461", State: "Disconnected", Type: "Unknown" }] }.

I am using a connection string like so:

var log = new LoggerConfiguration()
                .WriteTo.MongoDB("mongodb://dbuser:[email protected]:55461/zombie_log", collectionName: "konradSobon")
                .CreateLogger();

...where <dbuser> and <dbpassword> are actual values setup in mongolab.

Is it possible to connect to a remote db using serilog or does it have to be a local db?

Cheers!

V5 regression

Hi,

After updating to V5, my code no longer works.
To reproduce :

 var logger = new LoggerConfiguration()
            .Enrich.FromLogContext()
            .WriteTo.MongoDB(
                context.Database,
                period: TimeSpan.FromSeconds(1),
                batchPostingLimit: 10
                ).WriteTo.Console()
            .CreateLogger();

        for (int i = 0; i <500; i++)
        {
            logger.Information("Tst");
            Thread.Sleep(150);
        }

It breaks in the legacy sink when writing the end of the json document :

payload.Write("]}");

Somehow the json is maflformated.

Thanks

support mongo db password

hi
I use this connection string have password and dont work

{
        "Name": "MongoDB",
        "Args": {
          "databaseUrl": "mongodb://username:password@ip:port/dbName",
          "collectionName": "logs",
          "cappedMaxSizeMb": "1024",
          "cappedMaxDocuments": "50000"
        }
}

need a queue to cache logs

the performance is not good enough,
maybe better to add a queue to cache more logs and write to db once for all.

Sign the Assembly with a Strong Name

When building a strong-named assembly referencing this sink, one will get the warning below during the build.

CSC : warning CS8002: Referenced assembly 'Serilog.Sinks.MongoDB, Version=3.1.0.0, Culture=neutral, PublicKeyToken=null' does not have a strong name.

Signing the assembly with a strong name is a best practice, and would cause no side-effects whatsoever to anyone.

Problem creating collection on mongodb with serilog

I have this code below:
Logger = new LoggerConfiguration() .WriteTo.Console() .WriteTo.File(path: LoggerPath) .WriteTo.MongoDBCapped(database: database, restrictedToMinimumLevel: LogEventLevel.Verbose, collectionName: mongoCollection, batchPostingLimit: 1, period: TimeSpan.Zero).CreateLogger();

And when I go to my mongodb database I can't see the collection created with the custom name.

New organization required for this repository

Hi @serilog/reviewers-mongodb 👋 (... a bit of a form letter - greetings @Jaben :-))

Via serilog/serilog#1627 - we're unbundling the serilog organization to help distribute the effort involved in managing the various Serilog sub-projects. The serilog organization will now only manage the fundamental sinks and other packages that the rest of the ecosystem builds upon.

If this package is actively maintained, it can be moved to a new organization managed by the maintainers. Otherwise, it can move to the serilog-archive parking lot, from where we hope a new community-run fork might spring from in the future 🌷 .

Let me know if you're a maintainer and keen to continue this project under a new org; otherwise, I'll shuffle things around and move this one to the archive.

Thanks!

Register BSON Serializer

Hi guys,
I'm trying to register BSON serializers that MongoDB uses, But this does not work properly and MongoDB does not use serializers.
I guess this is not considered in package configuration.
If this issue is not resolved, I think I can help you implement it.

This is my code:

loggerConfiguration
                .WriteTo.Logger(lc => lc.Filter.With(new ErrorLogEvent())
                    .WriteTo.MongoDB(mongoDBJsonFormatter: jsonFormatter, databaseUrl: connectionString, collectionName: mongoDBLogSetting.ErrorCollection))

                .WriteTo.Logger(lc => lc.Filter.With(new FatalLogEvent())
                    .WriteTo.MongoDB(mongoDBJsonFormatter: jsonFormatter, databaseUrl: connectionString, collectionName: mongoDBLogSetting.FatalCollection))

                .WriteTo.Logger(lc => lc.Filter.With(new InformationLogEvent())
                    .WriteTo.MongoDB(mongoDBJsonFormatter: jsonFormatter, databaseUrl: connectionString, collectionName: mongoDBLogSetting.InformationCollection))

                .WriteTo.Logger(lc => lc.Filter.With(new DebugLogEvent())
                    .WriteTo.MongoDB(mongoDBJsonFormatter: jsonFormatter, databaseUrl: connectionString, collectionName: mongoDBLogSetting.DebugCollection))

                .WriteTo.Logger(lc => lc.Filter.With(new VerboseLogEvent())
                    .WriteTo.MongoDB(mongoDBJsonFormatter: jsonFormatter, databaseUrl: connectionString, collectionName: mongoDBLogSetting.VerboseCollection))

                .WriteTo.Logger(lc => lc.Filter.With(new WarningLogEvent())
                    .WriteTo.MongoDB(mongoDBJsonFormatter: jsonFormatter, databaseUrl: connectionString, collectionName: mongoDBLogSetting.WarningCollection));

BsonDefaults.GuidRepresentationMode = GuidRepresentationMode.V3;

BsonSerializer.RegisterSerializer(typeof(Guid), new GuidSerializer(GuidRepresentation.Standard));
BsonSerializer.RegisterSerializer(typeof(decimal), new DecimalSerializer(BsonType.Decimal128));
BsonSerializer.RegisterSerializer(typeof(decimal?), new NullableSerializer<decimal>(new DecimalSerializer(BsonType.Decimal128)));
BsonSerializer.RegisterSerializer(typeof(IApplicationContext), new AppContextBsonSerializer());

Thank you in advance for your response 🌹

Creating one collection per day

Would be possible to configure the serilog mongodb sink to create one collection per day, and to keep for instance the logs from the last 30 days (the older collections should be removed) ?
Are there any samples available about how to implement such a feature ?

When Serilog writes to MongoDb, what class does it use?

Sorry for a basic question, but I wonder if there is a class in the Serilog libs that is used to fill in the blank.

var r = await _collection.FindAsync<T>(filter);

What class would T be if I am reading from the MongoDB that I just wrote to with Serilog?

Could not load type 'Serilog.Sinks.MongoDB.MongoDBSinkBase' from assembly 'Serilog.Sinks.MongoDB, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' because the parent type is sealed.

It seems that with there is an compatibility issuse with Serilog.Sinks.PeriodicBatching in version >= 3.0.0

On StartUp a exceptiion produces the following message:

"Could not load type 'Serilog.Sinks.MongoDB.MongoDBSinkBase' from assembly 'Serilog.Sinks.MongoDB, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' because the parent type is sealed."

With version 2.3.1 it works as expected.

Used versions:

Working:
Serilog.Sinks.MongoDB Version 5.2.2
Serilog.Sinks.PeriodicBatching Version 2.3.1

Error:
Serilog.Sinks.MongoDB Version 5.2.2
Serilog.Sinks.PeriodicBatching 3.0.0

Full Stacktrace

   at System.Reflection.RuntimeAssembly.GetExportedTypes()
   at System.Reflection.Assembly.get_ExportedTypes()
   at Serilog.Settings.Configuration.ConfigurationReader.<>c.<FindConfigurationExtensionMethods>b__32_1(Assembly a)
   at System.Linq.Enumerable.SelectManySingleSelectorIterator`2.MoveNext()
   at System.Linq.Enumerable.SelectManySingleSelectorIterator`2.MoveNext()
   at System.Linq.Enumerable.WhereEnumerableIterator`1.ToList()
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Serilog.Settings.Configuration.ConfigurationReader.FindEventEnricherConfigurationMethods(IReadOnlyCollection`1 configurationAssemblies)
   at Serilog.Settings.Configuration.ConfigurationReader.ApplyEnrichment(LoggerConfiguration loggerConfiguration)
   at Serilog.Settings.Configuration.ConfigurationReader.Configure(LoggerConfiguration loggerConfiguration)
   at Serilog.Configuration.LoggerSettingsConfiguration.Settings(ILoggerSettings settings)
   at xpergo.HUB.Cloud.Program.<>c.<CreateHostBuilder>b__2_1(HostBuilderContext context, LoggerConfiguration configuration) in C:\Git\xpergo\xpergo.HUB.Cloud\xpergo.HUB.Cloud\Program.cs:line 32
   at Serilog.SerilogHostBuilderExtensions.<>c__DisplayClass3_1.<UseSerilog>b__1(IServiceProvider services)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateServiceAccessor(Type serviceType)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Serilog.SerilogHostBuilderExtensions.<>c__DisplayClass3_1.<UseSerilog>b__3(IServiceProvider services)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateServiceAccessor(Type serviceType)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Microsoft.Extensions.Hosting.HostBuilder.<CreateServiceProvider>b__28_2(IServiceProvider _)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateServiceAccessor(Type serviceType)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Microsoft.Extensions.Hosting.HostBuilder.Build()
   at xpergo.HUB.Cloud.Program.Main(String[] args) in C:\Git\xpergo\xpergo.HUB.Cloud\xpergo.HUB.Cloud\Program.cs:line 64

Could not load file or assembly 'Serilog.FullNetFx, Version=1.5.0.0'

I have .Net 4.5.2 application and I have installed Serilog 2.1.0 and Serilog.Sinks.MongoDB 2.0.19
I have configured sink as
var log = new Seri.LoggerConfiguration()
.WriteTo.MongoDBCapped("mongodb://localhost:27017/test", collectionName: "logs")
.CreateLogger();

However I am getting error
Could not load file or assembly 'Serilog.FullNetFx, Version=1.5.0.0, Culture=neutral, PublicKeyToken=24c2f752a8e58a10' or one of its dependencies. The system cannot find the file specified.":"Serilog.FullNetFx, Version=1.5.0.0, Culture=neutral, PublicKeyToken=24c2f752a8e58a10

MongoDb Sink with X.509 Cert (MongoDB with SSL)

Using C# .Net

Any ideas how to accomplish this? My instance is needs SSL and would like to use serilog rather than log4net. Can I extend MongoDb sink? Will I have to write my own sink to accomplish?

Cheers,

Corey

Remove obsolete dependencies

Currently the 4.0 release still relies on older implementations of formatters, these should be removed/updated

See: https://ci.appveyor.com/project/serilog/serilog-sinks-mongodb/branch/master

C:\projects\serilog-sinks-mongodb\src\Serilog.Sinks.MongoDB\Sinks\MongoDB\MongoDBJsonFormatter.cs(62,33): warning CS0672: Member 'MongoDBJsonFormatter.WriteJsonProperty(string, object, ref string, TextWriter)' overrides obsolete 
member 'JsonFormatter.Write
JsonProperty(string, object, ref string, TextWriter)'. Add the Obsolete attribute to 'MongoDBJsonFormatter.WriteJsonProperty(string, object, ref string, TextWriter)'.
C:\projects\serilog-sinks-mongodb\src\Serilog.Sinks.MongoDB\Sinks\MongoDB\MongoDBJsonFormatter.cs(50,13): warning CS0618: 'JsonFormatter.JsonFormatter(bool, string, bool, IFormatProvider)' is obsolete: 'The omitEnclosingObject 
parameter is obsolete and wil
l be removed in a future Serilog version.'
C:\projects\serilog-sinks-mongodb\src\Serilog.Sinks.MongoDB\Sinks\MongoDB\MongoDBJsonFormatter.cs(79,17): warning CS0618: 'JsonFormatter.WriteJsonProperty(string, object, ref string, TextWriter)' is obsolete: 'Extension of 
JsonFormatter by subclassing is o
bsolete and will be removed in a future Serilog version. Write a custom formatter based on JsonValueFormatter instead. See https://github.com/serilog/serilog/pull/819.'

.NET type System.Uri cannot be mapped to a BsonValue.

when I use masstransit with serilong-sinks-mongodb, the masstranist log the host info,
then the exception happened.

I debug the source code and find out that the Bson convert with url make that exception happened.
image

collectionName is not exposed in class LoggerConfigurationMongoDBExtensions

This is from main page :
The default collection name is 'log', but a custom collection can be supplied with the optional CollectionName parameter.

// creates custom collection `applog`
var log = new LoggerConfiguration()
    .WriteTo.MongoDB("mongodb://mymongodb/logs", collectionName: "applog")
    .CreateLogger();

There is no CollectionName parameter in method MongoDB

public static class LoggerConfigurationMongoDBExtensions
  {
    public static LoggerConfiguration MongoDB(this LoggerSinkConfiguration loggerConfiguration, string databaseUrl, LogEventLevel restrictedToMinimumLevel = LogEventLevel.Verbose, int batchPostingLimit = 50, TimeSpan? period = null, IFormatProvider formatProvider = null)
    {
// ....
// We are limited only to single collection name :(
// public static readonly string DefaultCollectionName = "log";

 return loggerConfiguration.Sink((ILogEventSink) new MongoDBSink(databaseUrl, batchPostingLimit, period1, formatProvider, MongoDBSink.DefaultCollectionName, new CreateCollectionOptions()), restrictedToMinimumLevel);
}

Code fix :

public static LoggerConfiguration MongoDB(this LoggerSinkConfiguration loggerConfiguration, string databaseUrl, LogEventLevel restrictedToMinimumLevel = LogEventLevel.Verbose, int batchPostingLimit = 50, TimeSpan? period = null, IFormatProvider formatProvider = null, string collectionName = MongoDBSink.DefaultCollectionName)
{
....
return loggerConfiguration.Sink((ILogEventSink) new MongoDBSink(databaseUrl, batchPostingLimit, period1, formatProvider, collectionName, new CreateCollectionOptions()), restrictedToMinimumLevel);
}

I need to use different collections in log database.

Race Condition in MongoDbHelper.VerifyCollectionExists

When multiple processes simultaneously execute MongoDbHelper.VerifyCollectionExists with the same collection, more than one process may try to create the collection. This results in an unhanded exception:

Unhandled Exception: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> MongoDB.Driver.MongoCommandException: Command create failed: collection already exists.
   at MongoDB.Driver.Core.WireProtocol.CommandWireProtocol`1.ProcessReply(ConnectionId connectionId, ReplyMessage`1 reply)
   at MongoDB.Driver.Core.WireProtocol.CommandWireProtocol`1.Execute(IConnection connection, CancellationToken cancellationToken)
   at MongoDB.Driver.Core.Servers.Server.ServerChannel.ExecuteProtocol[TResult](IWireProtocol`1 protocol, CancellationToken cancellationToken)
   at MongoDB.Driver.Core.Servers.Server.ServerChannel.Command[TResult](DatabaseNamespace databaseNamespace, BsonDocument command, IElementNameValidator commandValidator, Func`1 responseHandling, Boolean slaveOk, IBsonSerializer`1 resultSerializer, MessageEncoderSettings messageEncoderSettings, CancellationToken cancellationToken)
   at MongoDB.Driver.Core.Operations.CommandOperationBase`1.ExecuteProtocol(IChannelHandle channel, ServerDescription serverDescription, ReadPreference readPreference, CancellationToken cancellationToken)
   at MongoDB.Driver.Core.Operations.CommandOperationBase`1.ExecuteProtocol(IChannelSource channelSource, ReadPreference readPreference, CancellationToken cancellationToken)
   at MongoDB.Driver.Core.Operations.WriteCommandOperation`1.Execute(IWriteBinding binding, CancellationToken cancellationToken)
   at MongoDB.Driver.Core.Operations.CreateCollectionOperation.Execute(IWriteBinding binding, CancellationToken cancellationToken)
   at MongoDB.Driver.OperationExecutor.ExecuteWriteOperation[TResult](IWriteBinding binding, IWriteOperation`1 operation, CancellationToken cancellationToken)
   at MongoDB.Driver.MongoDatabaseImpl.ExecuteWriteOperation[T](IWriteOperation`1 operation, CancellationToken cancellationToken)
   at MongoDB.Driver.MongoDatabaseImpl.CreateCollectionHelper[TDocument](String name, CreateCollectionOptions`1 options, CancellationToken cancellationToken)
   at MongoDB.Driver.MongoDatabaseImpl.CreateCollection(String name, CreateCollectionOptions options, CancellationToken cancellationToken)
   at Serilog.Helpers.MongoDbHelper.VerifyCollectionExists(IMongoDatabase database, String collectionName, CreateCollectionOptions collectionCreationOptions)
   at Serilog.Sinks.MongoDB.MongoDBSink..ctor(IMongoDatabase database, Nullable`1 batchPostingLimit, Nullable`1 period, IFormatProvider formatProvider, String collectionName, CreateCollectionOptions collectionCreationOptions)
   at Serilog.Sinks.MongoDB.MongoDBSink..ctor(String databaseUrlOrConnStrName, Nullable`1 batchPostingLimit, Nullable`1 period, IFormatProvider formatProvider, String collectionName, CreateCollectionOptions collectionCreationOptions)
   at Serilog.LoggerConfigurationMongoDBExtensions.MongoDB(LoggerSinkConfiguration loggerConfiguration, String databaseUrl, String collectionName, LogEventLevel restrictedToMinimumLevel, Nullable`1 batchPostingLimit, Nullable`1 period, IFormatProvider formatProvider)
   at SuMo.Common.Service.CommonStartup.CommonConfigure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
   at SuMo.ClassicImport.Service.Startup.Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
   at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Microsoft.AspNetCore.Hosting.Internal.ConfigureBuilder.Invoke(Object instance, IApplicationBuilder builder)
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.BuildApplication()
   at Microsoft.AspNetCore.Hosting.WebHostBuilder.Build()
   at SuMo.ClassicImport.Service.Program.Main(String[] args)

I will work up a PR for this shortly.

Documentation for Secure Connection Needed

Was really fumbling around when attempting to write to mongo sink with a password protected connection and stumbled on #12 (comment).

Would be really helpful to have this in the documentation that IMongoDatabase can be used (therefore credentials added).

Error in MongoDBJsonFormatter for Exceptions ?

I have an error when Logging exceptions

var exception = new ArgumentNullException("x");
//...
Log.Fatal(exception, "Some message...");

I enabled the SelfLog and that's the message I get

[...]
Exception thrown: 'System.FormatException' in MongoDB.Bson.dll
Exception while emitting periodic batch from Serilog.Sinks.MongoDB.MongoDBSink: System.FormatException: JSON reader was expecting a name but found '{'.
[...]

I set up the logger like this:

Log.Logger = new LoggerConfiguration()
                           .Enrich.WithExceptionDetails(new DestructuringOptionsBuilder().WithDefaultDestructurers().WithRootName("Exception"))
                           .Enrich.WithDemystifiedStackTraces()
                           .Enrich.WithEnvironmentUserName()
                           .Enrich.WithMachineName()
                           .WriteTo.Async(a => a.File(path: "log-mytool.log",
                                                      restrictedToMinimumLevel: LogEventLevel.Verbose,
                                                      outputTemplate: "{Timestamp:yyyy-dd-MM HH:mm:ss K} [{Level}] {Message:lj}{NewLine}{Properties:j}{NewLine}{Exception}",
                                                      rollingInterval: RollingInterval.Day,
                                                      retainedFileCountLimit: 31,
                                                      buffered: true))
                           .WriteTo.MongoDBCapped(databaseUrl: $"mongodb://user:password@localhost:27017/database",
                                           restrictedToMinimumLevel: LogEventLevel.Debug,
                                           period: TimeSpan.Zero,
                                           collectionName: "logs",
                                           cappedMaxSizeMb: 50,
                                           cappedMaxDocuments: 10000,
                                           mongoDBJsonFormatter: new MongoDBJsonFormatter(omitEnclosingObject: false, renderMessage: true))
                          .CreateLogger();

After playing around, I found out that when I use the default MongoDBJsonFormatter or when I just set omitEnclosingObject = true (like in the default) I get this error:

[...]
Exception thrown: 'MongoDB.Bson.BsonSerializationException' in mscorlib.dll
Exception thrown: 'System.AggregateException' in mscorlib.dll
Exception while emitting periodic batch from Serilog.Sinks.MongoDB.MongoDBSink: System.AggregateException: One or more errors occurred. ---> MongoDB.Bson.BsonSerializationException: Element name 'System.Object' is not valid'.
   at MongoDB.Bson.IO.BsonWriter.WriteName(String name)
[...]

Log to file works fine.
Log to mongodb when serializing simple messages with or without params is also working fine

Did I mess up the configuration or does someone have an idea on how to solve this?
I suspect a problem in the MongoDBJsonFormatter where some invalid character is not escaped or replaced correctly ?

NET type System.DateTimeOffset cannot be mapped to a BsonValue.

We are getting following error with following code
ILogger _logger = null;
public MasterController(ILogger logger,)
{
_logger = logger;
}

_logger.LogInformation("testc info");
throw new Exception("test exceptiopn");

Following is the stacktrace:

2022-07-14T05:30:39.8837936Z Exception while emitting periodic batch from Serilog.Sinks.MongoDB.MongoDBSink: System.ArgumentException: .NET type System.DateTimeOffset cannot be mapped to a BsonValue.
at MongoDB.Bson.BsonTypeMapper.MapToBsonValue(Object value)
at Serilog.Helpers.MongoDbDocumentHelpers.ToBsonValue(LogEventPropertyValue value)
at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable1 source, Func2 keySelector, Func2 elementSelector, IEqualityComparer1 comparer)
at Serilog.Sinks.MongoDB.LogEntry.MapFrom(LogEvent logEvent)
at System.Linq.Utilities.<>c__DisplayClass2_03.<CombineSelectors>b__0(TSource x) at System.Linq.Enumerable.SelectEnumerableIterator2.MoveNext()
at MongoDB.Driver.MongoCollectionImpl1.BulkWriteAsync(IClientSessionHandle session, IEnumerable1 requests, BulkWriteOptions options, CancellationToken cancellationToken)
at MongoDB.Driver.MongoCollectionImpl1.UsingImplicitSessionAsync[TResult](Func2 funcAsync, CancellationToken cancellationToken)
at Serilog.Sinks.PeriodicBatching.PeriodicBatchingSink.OnTick()
The thread 0x6230 has exited with code 0 (0x0).
2022-07-14T05:31:19.8832536Z Exception while emitting periodic batch from Serilog.Sinks.MongoDB.MongoDBSink: System.ArgumentException: .NET type System.DateTimeOffset cannot be mapped to a BsonValue.
at MongoDB.Bson.BsonTypeMapper.MapToBsonValue(Object value)
at Serilog.Helpers.MongoDbDocumentHelpers.ToBsonValue(LogEventPropertyValue value)
at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable1 source, Func2 keySelector, Func2 elementSelector, IEqualityComparer1 comparer)
at Serilog.Sinks.MongoDB.LogEntry.MapFrom(LogEvent logEvent)
at System.Linq.Utilities.<>c__DisplayClass2_03.<CombineSelectors>b__0(TSource x) at System.Linq.Enumerable.SelectEnumerableIterator2.MoveNext()
at MongoDB.Driver.MongoCollectionImpl1.BulkWriteAsync(IClientSessionHandle session, IEnumerable1 requests, BulkWriteOptions options, CancellationToken cancellationToken)
at MongoDB.Driver.MongoCollectionImpl1.UsingImplicitSessionAsync[TResult](Func2 funcAsync, CancellationToken cancellationToken)
at Serilog.Sinks.PeriodicBatching.PeriodicBatchingSink.OnTick()
The thread 0x1314 has exited with code 0 (0x0).
2022-07-14T05:32:39.8972089Z Exception while emitting periodic batch from Serilog.Sinks.MongoDB.MongoDBSink: System.ArgumentException: .NET type System.DateTimeOffset cannot be mapped to a BsonValue.
at MongoDB.Bson.BsonTypeMapper.MapToBsonValue(Object value)
at Serilog.Helpers.MongoDbDocumentHelpers.ToBsonValue(LogEventPropertyValue value)
at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable1 source, Func2 keySelector, Func2 elementSelector, IEqualityComparer1 comparer)
at Serilog.Sinks.MongoDB.LogEntry.MapFrom(LogEvent logEvent)
at System.Linq.Utilities.<>c__DisplayClass2_03.<CombineSelectors>b__0(TSource x) at System.Linq.Enumerable.SelectEnumerableIterator2.MoveNext()
at MongoDB.Driver.MongoCollectionImpl1.BulkWriteAsync(IClientSessionHandle session, IEnumerable1 requests, BulkWriteOptions options, CancellationToken cancellationToken)
at MongoDB.Driver.MongoCollectionImpl1.UsingImplicitSessionAsync[TResult](Func2 funcAsync, CancellationToken cancellationToken)
at Serilog.Sinks.PeriodicBatching.PeriodicBatchingSink.OnTick()
The thread 0x1c98 has exited with code 0 (0x0).
2022-07-14T05:35:19.8991323Z Exception while emitting periodic batch from Serilog.Sinks.MongoDB.MongoDBSink: System.ArgumentException: .NET type System.DateTimeOffset cannot be mapped to a BsonValue.
at MongoDB.Bson.BsonTypeMapper.MapToBsonValue(Object value)
at Serilog.Helpers.MongoDbDocumentHelpers.ToBsonValue(LogEventPropertyValue value)
at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable1 source, Func2 keySelector, Func2 elementSelector, IEqualityComparer1 comparer)
at Serilog.Sinks.MongoDB.LogEntry.MapFrom(LogEvent logEvent)
at System.Linq.Utilities.<>c__DisplayClass2_03.<CombineSelectors>b__0(TSource x) at System.Linq.Enumerable.SelectEnumerableIterator2.MoveNext()
at MongoDB.Driver.MongoCollectionImpl1.BulkWriteAsync(IClientSessionHandle session, IEnumerable1 requests, BulkWriteOptions options, CancellationToken cancellationToken)
at MongoDB.Driver.MongoCollectionImpl1.UsingImplicitSessionAsync[TResult](Func2 funcAsync, CancellationToken cancellationToken)
at Serilog.Sinks.PeriodicBatching.PeriodicBatchingSink.OnTick()

How are we supposed to fix it?

Settings for capped collection are not working

I tried to configure settings for capped Logging as below, But neither the number of records in collections are getting limited nor the size of documents in the collection.
I have hundreds of record in my log collection in the repository.
{
"Name": "MongoDBCapped",
"Args": {
"databaseUrl": "mongodb://localhost:27017/test",
"collectionName": "log",
"cappedMaxDocuments": "10",
"cappedMaxSizeMb": "1"
}
}

sink has quit working with no errors reported

I'm running a MongoDB 4.0.9 Community Server.

I've been using Serilog with MongoDb Sink for months with no issues logging to this database under a collection titled Logging.

With seemingly no major changes, other than nuget package updates from time to time, this sink has quit working once I've put it into production. Oddly, the sink is working fine when I run it from my visual studio development environment.

I have confirmed that I've tested the identical connection string on both my development and production machine. It looks like this:
mongodb://[user]:[password]@[host]:27017/Logging?authSource=admin

I've enabled self log in Serilog with this line:
Serilog.Debugging.SelfLog.Enable(msg => File.AppendAllText(selfLogPath, msg));

This has successfully output a warning that I fixed, but nothing else.

Both machines have port 27017 open and I've confirmed I can access the MongoDb database using MongoDb Compass and the above connection string. This works on both machines.

Is there any additional debugging I could perform as to why the production machine would not be logging data to MongoDb?

The code is running fine on the production machine, as data is flowing to-from a MySql database connection within the app. Also, I have File and EventLog sinks successfully running... but the MongoDb sink doesn't work.

This is a .NET Core 3.1 Windows Worker Service application.

Capped collection is not created

I call:

var log = new LoggerConfiguration()
    .WriteTo.MongoDBCapped("mongodb://mymongodb/logs", collectionName: "rollingapplog")
    .CreateLogger();

Expected: MongoDBCapped method creates Capped collection.
Actual: It doesn't create a capped collection. A common collection is created at the moment when some log writes to the collection.

Root cause description:
I have debugged your code and found that MongoDBCapped method verifies if the collection already exists. If not, then it creates a capped collection.
It seems that the following method is broken. database.GetCollection always return some object. And it always returns true even for nonexistent collection.

internal static bool CollectionExists(this IMongoDatabase database, string collectionName)
        {
            var collection = database.GetCollection<BsonDocument>(collectionName);
            return collection != null;
        }

I use:
MongoDB v 4.0.5
MongoDB driver 2.7.3

Serilog.AspNetCore 2.1.1
Serilog.Settings.Configuration 3.0.1
Serilog.Sinks.Console 3.1.1
Serilog.Sinks.ElasticSearch 7.1.0
Serilog.Sinks.MongoDB 4.0.0
Serilog.Sinks.SyslogServer 1.0.10

Multiple MongoDB subloggers without collection race condition

I have two mongodb subloggers logging to the same collection. I am using two to handle some filter logic. If the collection is not created one of the two subloggers loggers doesnt get to log. I assume this is due to the batch emit. It might be reading two mongodb sinks and sees they need to create the collection, adds them to the batch and tries to create it twice. I am new to this so I might just be missing something simple. This should only apply to using two subloggers to the same collection on first batch without a collection made already.

Mongodb_race

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.