adams85 / filelogger Goto Github PK

Hey,

when I try to add JSON file logger for my custom logger provider, it doesn't take into account optionsName when call Configure:

var textBuilder = new JsonFileLogEntryTextBuilder(new JsonWriterOptions { Indented = false });
loggingBuilder.AddJsonFile<ErrorFileLoggerProvider>(textBuilder: textBuilder, configure: options =>
{
    options.RootPath = AppContext.BaseDirectory;
});

AddJsonFile() calls JsonFileLoggerExtensions.ConfigureTextBuilder() under the hood and there optionsName is not passed to Configure method

 builder.Services.Configure(delegate (FileLoggerOptions options)
 {
      options.TextBuilder = textBuilder;
 });

so it configures default options but not the specific for the provider.
The same issue with AddJsonFile method itself

public static ILoggingBuilder AddJsonFile<TProvider>(this ILoggingBuilder builder, FileLoggerContext context = null, JsonFileLogEntryTextBuilder textBuilder = null, Action<FileLoggerOptions> configure = null, string optionsName = null) where TProvider : FileLoggerProvider
        {
            builder.AddFile<TProvider>(context, null, optionsName).ConfigureTextBuilder(textBuilder);
            if (configure != null)
            {
                builder.Services.Configure(configure);
            }

            return builder;
        }

builder.Services.Configure(configure); configures default options but not the providers one.

As a workaround I've added custom provider the following way:

var textBuilder = new JsonFileLogEntryTextBuilder(new JsonWriterOptions { Indented = false });
var optionsName = typeof(ErrorFileLoggerProvider).ToString();
    loggingBuilder.AddFile<ErrorFileLoggerProvider>(optionsName: optionsName);
    builder.Services.Configure(optionsName, (FileLoggerOptions options) =>
    {
        options.TextBuilder = textBuilder;
        options.RootPath = AppContext.BaseDirectory;
    });

I would be really nice if you can fix it.
Thanks for the great logger!

Thank you for sharing your file logger.

I would like to use this in a WinForm app. My business layer uses EF Core, so I have been using Microsoft.Extension.Logging.Console in development and wanted to add file logging.

WinForm apps use the XML-based App.config file instead of appsetting.json, so I was looking to include the logging configuration in App.config.

Configurable settings are normally defined in the VS-generated .Properties.Settings class. I defined an XML node for the logging parameters and loaded them via ConfigurationBuilder().AddXmlStream(stream).Build(). Console logging works OK using those settings, but filelogger raises exceptions when I call CreateLogger().

Do you have any examples or guidance for using XML configuration, particularly within App.config?
Below are some snippets for context, but the simple test project is at WFAppLogger if you want to reproduce the exception.

Program.cs:

using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using System;
using System.IO;
using System.Windows.Forms;
using WFAppLogger.Properties;

namespace WFAppLogger
{
    internal class Program
    {
        // Global Service Collection for dependency injection
        static IServiceCollection services;

        /// <summary>
        /// The main entry point for the application.
        /// </summary>
        [STAThread]
        static void Main()
        {
            // FYI: the roaming configuration that applies to the current user is at
            // %LOCALAPPDATA%\\<Company Name>\<appdomainname>_<eid>_<hash>\<version>\user.config
            //string userConfig = System.Configuration.ConfigurationManager.OpenExeConfiguration(
            //    System.Configuration.ConfigurationUserLevel.PerUserRoamingAndLocal).FilePath;
            // If no user.config file exists, then the application.exe.config file is used
            //string appConfig = System.Configuration.ConfigurationManager.OpenExeConfiguration(
            //    System.Configuration.ConfigurationUserLevel.None).FilePath;

            // Get logging configuration from settings in app.config (or user.config)
            var loggingConfig = Settings.Default.LoggingConfig;
            var stream = new MemoryStream();
            loggingConfig.Save(stream);
            stream.Position = 0;
            var configuration = new ConfigurationBuilder()
                .AddXmlStream(stream)
                .Build();
            var config = configuration.GetSection("Logging");

            // Initialize application logging via dependency injection
            services = new ServiceCollection();
            services.AddLogging(builder =>
            {
                builder.AddConfiguration(config);

                builder.AddConsole();

                builder.AddFile(o => o.RootPath = AppContext.BaseDirectory);
            });

            // Create logger for Program class and log that we're starting up
            var logger = CreateLogger<Program>();
            logger.LogInformation($"Starting {Application.ProductName}...");

            Application.EnableVisualStyles();
            Application.SetCompatibleTextRenderingDefault(false);

            // Run the application
            Application.Run(new Form1(CreateLogger<Form1>()));

            // Log that we're exiting
            logger.LogInformation($"Exiting {Application.ProductName}.");
        }

        /// <summary>
        /// Creates a new Microsoft.Extensions.Logging.ILogger instance using the full name of the given type
        /// </summary>
        /// <typeparam name="T">The class type to create a logger for</typeparam>
        /// <returns>The Microsoft.Extensions.Logging.ILogger that was created</returns>
        public static ILogger<T> CreateLogger<T>()
        {
            // Create and return Logger instance for the given type using global dependency injection for logger factory
            using (ServiceProvider sp = services.BuildServiceProvider())
            { 
                return sp.GetRequiredService<ILoggerFactory>().CreateLogger<T>();
            }
        }
    }
}

Snippet from App.config

<?xml version="1.0" encoding="utf-8"?>
<configuration>
    <configSections>
        <sectionGroup name="userSettings" type="System.Configuration.UserSettingsGroup, System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
            <section name="WFAppLogger.Properties.Settings" type="System.Configuration.ClientSettingsSection, System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" allowExeDefinition="MachineToLocalUser" requirePermission="false" />
        </sectionGroup>
    </configSections>
  <!-- some sections omitted for brevity ... !-->
  <userSettings>
    <WFAppLogger.Properties.Settings>
      <setting name="DefaultLogMessage" serializeAs="String">
        <value>Sample log message</value>
      </setting>
      <setting name="LoggingConfig" serializeAs="Xml">
        <value>
          <Root>
            <Logging>
              <LogLevel>
                <Default>Debug</Default>
                <WFAppLogger>Trace</WFAppLogger>
                <Microsoft>Warning</Microsoft>
                <System>Warning</System>
              </LogLevel>
              <Console>
                <IncludeScopes>true</IncludeScopes>
              </Console>
              <File>
                <IncludeScopes>true</IncludeScopes>
                <BasePath>Logs</BasePath>
                <Files>
                  <Path>&lt;appname&gt;-&lt;counter:000&gt;.log</Path>
                  <MaxFileSize>10000</MaxFileSize>
                </Files>
              </File>
            </Logging>
          </Root>
        </value>
      </setting>
    </WFAppLogger.Properties.Settings>
  </userSettings>
</configuration>
```

Is there an easy way to do SingleLine logging (ie the scope and the log entry are on one line), like SimpleConsole and Serilog do?

Hello,
While using logger I saw that in case of month change, counter do not reset in new path.
In config I've set path to:
"Files": [ { "Path": "Logs/date:yyyy/date:MM/.log" } ]
and it works great for most of the time but while advancing to next month I get more or less:

Logs/2022/02/000.log
...
Logs/2022/02/005.log
(after date change)
Logs/2022/03/005.log

I'm not sure if it's expected behaviour and I guess I can change it for myself in FileLoggerProcessor.cs->UpdateFilePath but it would be great to confirm that with creators.

Thanks,
Surdna

public static int Main(string[] args)
{
    var services = new ServiceCollection()
        .AddLogging(builder =>
        {
            // Add File Logging
            builder.AddFile(o => o.RootPath = AppContext.BaseDirectory);
        });

    var x = services.BuildServiceProvider().GetRequiredService<ILoggerFactory>();

    return 0;
}

with csproj

<Project Sdk="Microsoft.NET.Sdk">

	<PropertyGroup>
		<OutputType>Exe</OutputType>
		<TargetFramework>net6.0</TargetFramework>
		<AssemblyName>arius</AssemblyName>
	</PropertyGroup>

	<ItemGroup>
		<PackageReference Include="Karambolo.Extensions.Logging.File" Version="3.2.1" />
		<PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="6.0.0" />
		<PackageReference Include="Microsoft.Extensions.DependencyInjection" Version="6.0.0" />
		<PackageReference Include="Microsoft.Extensions.Logging" Version="6.0.0" />
	</ItemGroup>

	<ItemGroup>
		<None Update="appsettings.json">
			<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
		</None>
	</ItemGroup>

</Project>

yields this exception on the GetRequiredService

System.MethodAccessException: 'Attempt by method 'Microsoft.Extensions.Logging.Configuration.LoggerProviderConfigurationFactory.GetConfiguration(System.Type)' to access method 'Microsoft.Extensions.Logging.ProviderAliasUtilities.GetAlias(System.Type)' failed.'

When changing the csproj file to the following (note the Hosting io. Configuration.Json, DependencyInjection, Logging) and cleaning the solution, it works as expected

<Project Sdk="Microsoft.NET.Sdk">

	<PropertyGroup>
		<OutputType>Exe</OutputType>
		<TargetFramework>net6.0</TargetFramework>
		<AssemblyName>arius</AssemblyName>
	</PropertyGroup>

	<ItemGroup>
		<PackageReference Include="Karambolo.Extensions.Logging.File" Version="3.2.1" />
		<PackageReference Include="Microsoft.Extensions.Hosting" Version="6.0.0" />
	</ItemGroup>

	<ItemGroup>
		<None Update="appsettings.json">
			<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
		</None>
	</ItemGroup>

</Project>

I would like to be able to enable a setting for always creating next file on program start nomatter how big the file last used is.

I don't want to clean it up regularly, so is there an option to clean up the oldest file, when the maximum number of logs is reached?

I have the following code - nothing in appsettings or anywhere...

                configLogging.AddProvider(new SimpleConsoleLogger())
                    .AddFile(c =>
                    {
                        string dir = Path.Combine(Path.GetTempPath(), "Paros.Logs");
                        if (!Directory.Exists(dir))
                        {
                            Directory.CreateDirectory(dir);
                        }

                        c.RootPath = dir;
                        c.MaxFileSize = 10_000_000;
                        c.FileAccessMode = Karambolo.Extensions.Logging.File.LogFileAccessMode.KeepOpenAndAutoFlush;
                    });

And no logs are being created.

What else am I missing?

What would be the correct values for "TextBuilderType": "MyApp.CustomLogEntryTextBuilder, MyApp"?

It might be me missing something, but if there is no feature for cleaning up old files when rolling to a new file would it be posible to add that? It could be a setting of MaxFiles or something like that.

Maybe add a new variable like in the Path, and replace it to current application name in code?

Thanks,

Could you provide a simple example of how to configure logging, such that the log files are filled with structured logging and the console with human readable logging?
Ideally, these settings would make it happen:

    "Console": {
      "FormatterName": "simple",
      "FormatterOptions": {
        "SingleLine": true
      }
    },
    "File": {
      "FormatterName": "json",
      "FormatterOptions": {
        "SingleLine": true
      },
      "Files": [
        {
          "Path": "default-<date>.log"
        }
      ]
    }

Is Karambolo file logging provider thread safe? What happens if two threads try to log a message at the same time?

When BasePath is relative upwards the current file providers directory then the directory and logs not created.

Sample:

  "logging": {
    "BasePath": "..\\appdata\\logs",
    "EnsureBasePath": true,
    // ...

Hi!

This is just a question, or maybe an enhancement for the docs if it's not just me.

First of all, thank you for this library! Good job! I've been using it for about 6 months, had no problem!

Context:
Library Version: 3.0.0 (I'll probably update soon...)
Target framework: .NET Core 3.0 (Upgrade coming as well...)
It's a complex ASP.NET Core project, with background tasks etc.

So the one thing I've been messing around is setting up my logs to be "just perfect".
I have some Global LogLevel settings:

"Logging": {
    ...
    "LogLevel": {
      "Default": "Trace",
      "Microsoft": "Warning",
      "Microsoft.Hosting.Lifetime": "Information",
      "NHibernate": "Debug"
    },
    ...
}

Then I have a Console provider which is most probably irrelevant.
And lastly I have my File provider settings:

"File": {
      "BasePath": "logs",
      "FileAccessMode": "KeepOpenAndAutoFlush",
      "MaxFileSize": 10485760,
      "CounterFormat": "000",
      "IncludeScopes": true,
      "MaxQueueSize": 1000,
      "Files": [
        {
          "Path": "app-persistence-<counter>.log",
          "MinLevel": {
            "Default": "Trace",
            "NHibernate": "Debug",
            "NHibernate.Type": "Information",
            "NHibernate.Cfg.XmlHbmBinding": "Warning",
            "NHibernate.Cfg.XmlHbmBinding.Binder": "Information",
            "Itg.Persistence": "Trace"
          }
        },
        {
          "Path": "app-<counter>.log",
          "MinLevel": {
            "Default": "Debug",
            "Microsoft": "Warning",
            "Microsoft.AspNetCore.Authentication": "Debug",
            "Microsoft.AspNetCore.Authorization": "Debug",
            "Microsoft.AspNetCore.Server": "Debug",
            "NHibernate": "None",
            "Itg.Persistence": "None",
            "Itg.Services": "Debug",
            "Itg.Services.Shop": "Information",
            "Itg.Mailing": "Information"
          }
        }
      ]
    }

The thing I can't seem to understand is the following:
I'd like my app-persistence-<counter>.log to contain only NHibernate and Itg.Persistence categories, the other one mostly everything else.

On my initial run I had "Default": "None" on my persistence file, which I thought must be right, because I didn't want anything else to land in there, only the ones I specified (and more specific ones!).

Now based on these, after reading through Microsoft's logging tutorial, and this repo's readme, the category "Itg.Persistence.Secondary" should have been written to the persistence file, but it did not! (it's "under" "Itg.Persistence" which I set to "Trace")

The logging worked, as I could find my entries on the console, but nowhere else.
To be exact, this is the line I was looking for:
crit: Itg.Persistence.Secondary.FileKeyValueStore[LoadFailed]
And to be more exact, this line is logged by a queued background task!

Now I set "Default": "Trace" on my persistence file, and here I thought, now everything should show up. Not just the ones I specified. To my surprise the one I was looking for DID show up ("Itg.Persistence.Secondary") BUT not at all everything... This is the only new category in my persistence log. Which is just about the effect I wanted, but I can't imagine these settings are any good...

Could you help me out please? What am I missing? I'm so confused... :(

I have the following configuration:

   services.AddLogging(b =>
            {
                b.AddFile(o =>
                {
                    o.RootPath = AppContext.BaseDirectory;
                    o.IncludeScopes = true;

                    o.Files = new[]
                    {
                        new LogFileOptions
                        {
                            Path = logFilePath
                        }
                    };
                });
                b.AddConfiguration(configuration.GetSection("Logging"));
            });

The initial set of log entries are flushed to the file right away, however, once the program goes parallel and has many different ILogger instances writing concurrently, no new log entries appear in the output file and by adding:

FileLoggerContext.Default.DiagnosticEvent += e => { Debug.WriteLine(e); };

I can see many events indicating file locking issues

' created at 9/23/2021 4:10:29 AM +00:00 to log file "logs\testlog.log" failed. System.IO.IOException: The process cannot access the file '....\Debug\logs\testlog.log' because it is being used by another process.
   at System.IO.FileStream.ValidateFileHandle(SafeFileHandle fileHandle)
   at System.IO.FileStream.CreateFileOpenHandle(FileMode mode, FileShare share, FileOptions options)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, Boolean useAsync)
   at Karambolo.Extensions.Logging.File.PhysicalFileAppender.CreateAppendStream(IFileInfo fileInfo)
   at Karambolo.Extensions.Logging.File.FileLoggerProcessor.WriteEntryAsync(LogFileInfo logFile, FileLogEntry entry, CancellationToken cancellationToken)

I know that it's not something I'm doing locking the file, because if I add

               o.FileAccessMode = LogFileAccessMode.OpenTemporarily;

I don't get any locking exceptions from FileLoggerContext.Default.DiagnosticEvent and new log entries are correctly flushed to the output log file.

I'm running a WEBAPI in ASP.NET CORE 7.0
however I need to code them again in every controller.
The logging service provide by ASP.NET Core can be setting in program.cs and nothing is need to be done in the controller.

public apiController(ILogger logger)
{
var configuration = new ConfigurationBuilder()
.AddJsonFile("appsettings.json")
.Build();

        var services = new ServiceCollection();

        services.AddLogging(builder =>
        {
            builder.AddConfiguration(configuration.GetSection("Logging"));
            // the "standard" provider which logs all messages with severity warning or above to 'warn+err.log' (see appsettings.json for configuration settings)
            builder.AddFile(o => o.RootPath = AppContext.BaseDirectory);
            // a custom one which only logs messages with level information or below to 'info.log'
            builder.AddFile<InfoFileLoggerProvider>(configure: o => o.RootPath = AppContext.BaseDirectory);
        });

        using (ServiceProvider sp = services.BuildServiceProvider())
        {
            // create logger factory
            _logger = sp.GetRequiredService<ILoggerFactory>().CreateLogger<apiController>();
        }
    }

The apicontroller is one of the controller.
In the logging service provice by ASP.NET Core, I just need to use the logger in the parameter.
Am I do anything wrong?

Hello. In my app i need to create additional containers for creating two instances of Rebus.

			var services = new ServiceCollection();
			services.AddLogging(builder =>
			{
				builder.AddConfiguration(_configuration.GetSection("Logging"));
				builder.AddFile(o => o.RootPath = AppContext.BaseDirectory);
			});
			services.AddRebusClient();

			var provider = services.BuildServiceProvider();
			provider.UseRebus();

When I use the same in ConfigureServices((hostContext, services) it works great, but when I use it in ServiceCollection it doesn't write to file. Do you have any ideas? Where did I make mistake?
There is my config file:

I see issue #2 was closed with, do it externally, but I really need some feature of this library to have

• the latest file is always log.0.txt, the next oldest is log.1.txt, etc.
• fixed number of log files, e.g. 5
• rotate on max file size
• when rotate happens, log.0.txt is renamed to log.1.txt
• log.4.txt is deleted before the rename

I can almost write this

protected override bool UpdateFilePath(LogFileInfo logFile, FileLogEntry entry, CancellationToken cancellationToken)
{
    var res = base.UpdateFilePath(logFile, entry, cancellationToken);
    if (logFile.Counter == 0)
    {
        return res;
    }

    const int maxFiles = 5;
    logFile.Counter = maxFiles - 1;
    var p = Path.Combine(logFile.BasePath, this.FormatFilePath(logFile, entry));
    File.Delete(p);

    for (int i = maxFiles; i > 1; i--)
    {
        logFile.Counter = i - 1;
        var p1 = Path.Combine(logFile.BasePath, this.FormatFilePath(logFile, entry));

        logFile.Counter = i - 2;
        var p2 = Path.Combine(logFile.BasePath, this.FormatFilePath(logFile, entry));

        try
        {
            File.Move(p2, p1);
        }
        catch (FileNotFoundException)
        {
        }
    }

    logFile.Counter = 0;
    return base.UpdateFilePath(logFile, entry, cancellationToken);
}

But I cannot actually use this with LogFileAccessMode.KeepOpen as the file is in use.
logFile.Close() is internal...

Assuming I haven't missed something, is it possible to request something like

• change accessibility of logFile.Close()
• or, a logFile close callback? then I could implement the rename in that, and reset counter back to 0

Hi! Absolutely love the convenience of this library!

I do have a question:
Is it possible to customize the Logging Headers that's being outputted to the file (Not the file name)?

Looks like by default it's writing like this:

info: My.Project.Here[0] @ 2021-10-14T00:44:56.8727360-04:00

Is there currently a way to add a custom Date/Time Format, and swap the order of the tags above?

Thanks!

Hi,

Can you add a setting to allow set the start value of the counter? or set its start value to 1 instead of 0?

Thanks,

Hi,
I'm evaluating Karambolo.Extensions.Logging.File.Json and the README states:

The code is based on ConsoleLogger whose full feature set is implemented

But while experimenting, I found some small differences to AddJsonConsole().

First, it looks like Karambolo separates entries with a ',', while the console logger only uses line break.

Second, the JsonConsoleFormatterOptions accept JsonWriterOptions, where we can set 'Indented = false'.

This combination allows us to write the JSON lines (aka NDJSON) format, each log entry on a single line, with the console logger.

Would you implement the changes, or accept patches, to enable JSON lines support?

This is more lika a suggestion.
Maybe this library could benefit from channels.

Here is a blog post I find quite easy to understand: https://ndportmann.com/system-threading-channels/

Channels seem to be a more recent development than the dataflow library. They have a clean API and seem to focus on solving the problem of fast and thread safe queues.

Would be nice to have the possibility to clean the output of the category name:

https://stackoverflow.com/questions/55924730/is-it-possible-to-disable-category-output-in-net-core-consolelogger-and-debuglo

Thanks!