Giter Club home page Giter Club logo

zlogger's Introduction

ZLogger

GitHub Actions Releases

Zero Allocation Text/Structured Logger for .NET and Unity, with StringInterpolation and Source Generator, built on top of a Microsoft.Extensions.Logging.

The usual destinations for log output are Console(Stream), File(Stream), Network(Stream), all in UTF8 format. However, since typical logging architectures are based on Strings (UTF16), this requires additional encoding costs. In ZLogger, we utilize the String Interpolation Improvement of C# 10 and by leveraging .NET 8's IUtf8SpanFormattable, we have managed to avoid the boxing of values and maintain high performance by consistently outputting directly in UTF8 from input to output.

ZLogger is built directly on top of Microsoft.Extensions.Logging. Microsoft.Extensions.Logging is an official log abstraction used in many frameworks, such as ASP.NET Core and Generic Host. However, since regular loggers have their own systems, a bridge is required to connect these systems, and this is where a lot of overhead can be observed. ZLogger eliminates the need for this bridge, thereby completely avoiding overhead.

Alt text

This benchmark is for writing to a file, but the default settings of typical loggers are very slow. This is because they flush after every write. In the benchmark, to ensure fairness, careful attention was paid to set the options in each logger for maximum speed. ZLogger is designed to be the fastest by default, so there is no need to worry about any settings.

The slowness of this default setting is due to I/O, so it can be mitigated by using a faster drive. When taking benchmarks, please note that the results can vary greatly not only on your local (which is probably fast!) but also on drives attached to the cloud and in environments like Docker. One of the good points about the async-buffered setting is that it can reduce the impact of such I/O issues.

ZLogger focuses on the new syntax of C#, and fully adopts Interpolated Strings.

Alt text

This allows for providing parameters to logs in the most convenient form. Also, by closely integrating with System.Text.Json's Utf8JsonWriter, it not only enables high-performance output of text logs but also makes it possible to efficiently output structured logs.

ZLogger also emphasizes console output, which is crucial in cloud-native applications. By default, it outputs with performance that can withstand destinations in cloud log management. Of course, it supports both text logs and structured logs.

ZLogger delivers its best performance with .NET 8 and above, but it is designed to maintain consistent performance with .NET Standard 2.0 and .NET 6 through a fallback to its own IUtf8SpanFormattable.

As for standard logger features, it supports loading LogLevel from json, filtering by category, and scopes, as found in Microsoft.Extensions.Logging. In terms of output destinations, it is equipped with sufficient capabilities for Console, File, RollingFile, InMemory, Stream, and an AsyncBatchingProcessor for sending logs over HTTP and similar protocols.

Table of Contents

Getting Started

This library is distributed via NuGet, supporting .NET Standard 2.0, .NET Standard 2.1, .NET 6(.NET 7) and .NET 8 or above. For Unity, the requirements and installation process are completely different. See the Unity section for details.

dotnet add package ZLogger

Here is the most simple sample on ASP.NET Core.

using ZLogger;

var builder = WebApplication.CreateBuilder(args);

builder.Logging.ClearProviders();
builder.Logging.AddZLoggerConsole();

You can get logger from dependency injection.

@page
@using ZLogger;
@inject ILogger<Index> logger
@{
    logger.ZLogInformation($"Requested path: {this.HttpContext.Request.Path}");
}

This simple logger setup is possible because it is integrated with Microsoft.Extensions.Logging by default. For reference, here's how you would set it up using LoggerFactory:

using Microsoft.Extensions.Logging;
using ZLogger;

using var factory = LoggerFactory.Create(logging =>
{
    logging.SetMinimumLevel(LogLevel.Trace);

    // Add ZLogger provider to ILoggingBuilder
    logging.AddZLoggerConsole();
    
    // Output Structured Logging, setup options
    // logging.AddZLoggerConsole(options => options.UseJsonFormatter());
});

var logger = factory.CreateLogger("Program");

var name = "John";
var age = 33;

// Use **Z**Log method and string interpolation to log message
logger.ZLogInformation($"Hello my name is {name}, {age} years old.");

Normally, you don't create LoggerFactory yourself. Instead, you set up a Generic Host and receive ILogger through dependency injection (DI). You can setup logger by .NET Generic Host(for ASP.NET Core) and if you want to use this in ConsoleApplication, we provides ConsoleAppFramework to use hosting abstraction.

Here is the showcase of providers.

using ZLogger;

var builder = Host.CreateApplicationBuilder();

builder.Logging
    // optional(MS.E.Logging):clear default providers(recommend to remove all)
    .ClearProviders()

    // optional(MS.E.Logging):setup minimum log level
    .SetMinimumLevel(LogLevel.Trace)
    
    // Add to output to console
    .AddZLoggerConsole();

    // Add to output to the file
    .AddZLoggerFile("/path/to/file.log")
    
    // Add to output the file that rotates at constant intervals.
    .AddZLoggerRollingFile(options =>
    {
        // File name determined by parameters to be rotated
        options.FilePathSelector = (timestamp, sequenceNumber) => $"logs/{timestamp.ToLocalTime():yyyy-MM-dd}_{sequenceNumber:000}.log";
        
        // The period of time for which you want to rotate files at time intervals.
        options.RollingInterval = RollingInterval.Day;
        
        // Limit of size if you want to rotate by file size. (KB)
        options.RollingSizeKB = 1024;        
    })    
    
    // Add to output of simple rendered strings into memory. You can subscribe to this and use it.
    .AddZLoggerInMemory(processor =>
    {
        processor.MessageReceived += renderedLogString => 
        {
            System.Console.WriteLine(renderedLogString);    
        };
    })
    
    // Add output to any steram (`System.IO.Stream`)
    .AddZLoggerStream(stream);

    // Add custom output
    .AddZLoggerLogProcessor(new YourCustomLogExporter());
    
    // Format as json
    .AddZLoggerConsole(options =>
    {
        options.UseJsonFormatter();
    })
    
    // Format as json and configure output
    .AddZLoggerConsole(options =>
    {
        options.UseJsonFormatter(formatter =>
        {
            formatter.IncludeProperties = IncludeProperties.ParameterKeyValues;
        });
    })

    // Further common settings
    .AddZLoggerConsole(options =>
    {
        // Enable LoggerExtensions.BeginScope
        options.IncludeScopes = true;
        
        // Set TimeProvider
        options.TimeProvider = yourTimeProvider
    });

Look at the use of loggers and the syntax of ZLog.

using Microsoft.Extensions.Logging;
using ZLogger;

// get ILogger<T> from DI.
public class MyClass(ILogger<MyClass> logger)
{
    // name = "Bill", city = "Kumamoto", age = 21
    public void Foo(string name, string city, int age)
    {
        // plain-text:
        // Hello, Bill lives in Kumamoto 21 years old.
        // json:
        // {"Timestamp":"2023-11-30T17:28:35.869211+09:00","LogLevel":"Information","Category":"MyClass","Message":"Hello, Bill lives in Kumamoto 21 years old.","name":"Bill","city":"Kumamoto","age":21}
        // json(IncludeProperties.ParameterKeyValues):
        // {"name":"Bill","city":"Kumamoto","age":21}
        logger.ZLogInformation($"Hello, {name} lives in {city} {age} years old.");
    
        // Explicit property name, you can use custom format string start with '@'
        logger.ZLogInformation($"Hello, {name:@user-name} id:{100:@id} {age} years old.");
    
        // Dump variables as JSON, you can use custom format string `json`
        var user = new User(1, "Alice");

        // user: {"Id":1,"Name":"Bob"}
        logger.ZLogInformation($"user: {user:json}");
    }
}

All standard .Log methods are processed as strings by ZLogger's Provider. However, by using our unique .ZLog* methods, you can process them at high performance while remaining in UTF8. Additionally, these methods support both text logs and structured logs using String Interpolation syntax.

All logging methods are completely similar as Microsoft.Extensions.Logging.LoggerExtensions, but it has Z prefix overload.

The ZLog* method uses InterpolatedStringHandler in .NET and prepare the template at compile time.

Some special custom formats are also supported. The :@ can be used when you want to explicitly give the structured log a name other than the name of the variable to capture. :json can be used to log the result of JsonSerializing an object.

The @ parameter name specification and format string can be used together.

// Today is 2023-12-19.
// {"date":"2023-12-19T11:25:34.3642389+09:00"}
logger.ZLogDebug($"Today is {DateTime.Now:@date:yyyy-MM-dd}.");

Logging Providers

By adding Providers, you can configure where the logs are output. ZLogger has the following providers.

Type Alias Builder Extension
ZLoggerConsoleLoggerProvider ZLoggerConsole AddZLoggerConsole
ZLoggerFileLoggerProvider ZLoggerFile AddZLoggerFile
ZLoggerRollingFileLoggerProvider ZLoggerRollingFile AddZLoggerRollingFile
ZLoggerStreamLoggerProvider ZLoggerStream AddZLoggerStream
ZLoggerInMemoryProcessorLoggerProvider ZLoggerInMemory AddZLoggerInMemory
ZLoggerLogProcessorLoggerProvider ZLoggerLogProcessor AddZLoggerLogProcessor

All Providers can take an Action that sets ZLoggerOptions as the last argument. As follows.

builder.Logging
    .ClearProviders()

    // Configure options
    .AddZLoggerConsole(options => 
    {
        options.LogToStandardErrorThreshold = LogLevel.Error;
    });
    
    // Configure options with service provider
    .AddZLoggerConsole((options, services) => 
    {
        options.TimeProvider = services.GetService<YourCustomTimeProvider>();
    });

If you are using Microsoft.Extensions.Configuration, you can set the log level through configuration. In this case, alias of Provider can be used. for example:

{
  "Logging": {
    "LogLevel": {
      "Default": "Information"
    },
    "ZLoggerConsoleLoggerProvider": {
      "LogLevel": {
        "Default": "Debug"
      }
    }
  }
}

Each Provider's behavior can be modified using the common ZLoggerOptions. For details, please refer to the ZLoggerOptions section. Additionally, you can customize structured logging (JSON Logging) using the UseFormatter method within these options. For more information on this, check the Formatter Configurations section.

Console

Console writes to the standard output. Console output is not only for development purposes, but also serves as a standard log input port in containerized and cloud environments, making performance critically important. ZLogger has been optimized to maximize console output performance.

logging.AddZLoggerConsole();

If you are using ZLoggerConsoleLoggerProvider, the following additional options are available:

Name Description
bool OutputEncodingToUtf8 Set Console.OutputEncoding = new UTF8Encoding(false) when the provider is created. (default: true)
bool ConfigureEnableAnsiEscapeCode If set true, then configure console option on execution and enable virtual terminal processing(enable ANSI escape code). (default: false)
LogLevel LogToStandardErrorThreshold If set, logs at a higher level than the value will be output to standard error. (default: LogLevel.None)

File

File outputs text logs to a file. This is a Provider that writes to a single file in append mode at high speed.

logging.AddZLoggerFile("log.txt");

If you are using ZLoggerFileLoggerProvider, the following additional options are available:

Name Description
bool fileShared If set true, enables exclusive control of writing to the same file from multiple processes.(default: false)

RollingFile

RollingFile is a Provider that dynamically changes the output file based on certain conditions.

// output to  yyyy-MM-dd_*.log, roll by 1MB or changed date
logging.AddZLoggerRollingFile((dt, index) => $"{dt:yyyy-MM-dd}_{index}.log", 1024 * 1024);

If you are using ZLoggerRollingFileLoggerProvider, the following additional options are available:

Name Description
Func<DateTimeOffset, int, string> fileNameSelector The Func to consturct the file path. DateTimeOffset is date of file open time(UTC), int is number sequence.
RollingInterval rollInterval Interval to automatically rotate files.
int rollSizeKB Limit size of single file. If the file size is exceeded, a new file is created with the sequence number moved up.
bool fileShared If set true, enables exclusive control of writing to the same file from multiple processes.(default: false)

Stream

Stream can output logs to any arbitrary Stream. For example, if you output to a MemoryStream, you can retrieve the rendered results in memory. If you pass a NetworkStream such as TCP, it will write logs over the network.

var ms = new MemoryStream();
logging.AddZLoggerStream(ms);

In-Memory

InMemory allows you to retrieve rendered strings as they are generated. It can be conveniently used for purposes such as accumulating logs in a List<string> or Queue<string> for display on screen.

logging.AddZLoggerInMemory(processor =>
{
    processor.MessageReceived += msg =>
    {
        Console.WriteLine($"Received:{msg}");
    };
});

If you are using ZLoggerInMemoryLoggerProvider, the following additional options are available:

Name Description
string processorKey If specified, InMemoryObservableLogProcessor is registered in the DI container as a keyed service and can be retrieved by name.
Action<InMemoryObservableLogProcessor> configureProcessor Custom actions can be added that use processors instead of DI containers.

LogProcessor

LogProcessor is the most primitive Provider that allows you to customize output on a per-log basis (IZLoggerEntry) by implementing a custom IAsyncLogProcessor.

public interface IAsyncLogProcessor : IAsyncDisposable
{
    void Post(IZLoggerEntry log);
}

For example, a LogProcessor that propagates logs as string events can be written as follows:

public class SimpleInMemoryLogProcessor : IAsyncLogProcessor
{
    public event Action<string>? OnMessageReceived;

    public void Post(IZLoggerEntry log)
    {
        var msg = log.ToString();
        log.Return();
        
        OnMessageReceived?.Invoke(msg);
    }

    public ValueTask DisposeAsync()
    {
        return default;
    }
}
var processor = new SimpleInMemoryLogProcessor();
processor.OnMessageReceived += msg => Console.WriteLine(msg);

logging.AddZLoggerLogProcessor(processor);

Note that IZLoggerEntry is pooled, so you must always call Return().

Here's a more complex example. BatchingAsyncLogProcessor can batch logs together, which is useful for scenarios like sending multiple log lines via HTTP in a single request.

public class BatchingHttpLogProcessor : BatchingAsyncLogProcessor
{
    HttpClient httpClient;
    ArrayBufferWriter<byte> bufferWriter;
    IZLoggerFormatter formatter;

    public BatchingHttpLogProcessor(int batchSize, ZLoggerOptions options)
        : base(batchSize, options)
    {
        httpClient = new HttpClient();
        bufferWriter = new ArrayBufferWriter<byte>();
        formatter = options.CreateFormatter();
    }

    protected override async ValueTask ProcessAsync(IReadOnlyList<INonReturnableZLoggerEntry> list)
    {
        foreach (var item in list)
        {
            item.FormatUtf8(bufferWriter, formatter);
        }
        
        var byteArrayContent = new ByteArrayContent(bufferWriter.WrittenSpan.ToArray());
        await httpClient.PostAsync("http://foo", byteArrayContent).ConfigureAwait(false);

        bufferWriter.Clear();
    }

    protected override ValueTask DisposeAsyncCore()
    {
        httpClient.Dispose();
        return default;
    }
}

In this case, the LogEntry is NonReturnable, so there's no need to call Return().

Formatter Configurations

Both PlainText and JSON can be customized in addition to the standard log formats.

PlainText

logging.AddZLoggerConsole(options =>
{
    // Text format
    // e.g) "2023-12-01 16:41:55.775|Information|This is log message. (MyNamespace.MyApp)
    options.UsePlainTextFormatter(formatter =>
    {
        formatter.SetPrefixFormatter($"{0}|{1}|", (in MessageTemplate template, in LogInfo info) => template.Format(info.Timestamp, info.LogLevel));
        formatter.SetSuffixFormatter($" ({0})", (in MessageTemplate template, in LogInfo info) => template.Format(info.Category));
        formatter.SetExceptionFormatter((writer, ex) => Utf8StringInterpolation.Utf8String.Format(writer, $"{ex.Message}"));
    });
});

You can set Prefix and Suffix individually for text output. For performance reasons, the first argument is a special String Interpolation Template, which is formatted by the lambda expression in the second argument. For properties that can actually be retrieved with LogInfo, refer to LogInfo. It is also possible to retrieve the log file path and line number from LogInfo.

Only LogLevel supports a special format specification. By passing :short, you can get a 3-character log level notation such as TRC, DBG, INF, WRN, ERR, CRI, NON (the length of the beginning matches, making it easier to read when opened in an editor). For Timestamp, there are local | local-longdate | longdate(local, local-longdate, longdate are same, there are alias), utc | utc-longdate, datetime | local-datetime, utc-datetime, dateonly | local-dateonly, utc-dateonly, timeonly | local-timeonly, utc-timeonly. Default is local.

logging.AddZLoggerConsole(options =>
{
    options.UsePlainTextFormatter(formatter =>
    {
        // 2023-12-19 02:46:14.289 [DBG]......
        formatter.SetPrefixFormatter($"{0:utc-longdate} [{1:short}]", (template, info) => template.Format(info.Timestamp, info.LogLevel));
    });
});

SetExceptionFormatter allows you to customize the display when outputting exceptions. This can be easily converted to a string using Utf8String.Format.

JSON

You can flexibly change the JSON output format by modifying the JsonFormatter options. For example, if you set IncludeProperties to only ParameterKeyValues, you will get only the payload JSON. By default, the payload part is output directly without nesting, but if you set PropertyKeyValuesObjectName, you can output the payload JSON to a nested location. It is also possible to add values for arbitrary JSON Objects using AdditionalFormatter.

The following is an example of customization to conform to the Google Cloud Logging format. We have also changed standard key names such as Timestamp.

using System.Text.Json;
using ZLogger;
using ZLogger.Formatters;

namespace ConsoleApp;

using static IncludeProperties;
using static JsonEncodedText; // JsonEncodedText.Encode

public static class CloudLoggingExtensions
{
    // Cloud Logging Json Field
    // https://cloud.google.com/logging/docs/structured-logging?hl=en
    public static ZLoggerOptions UseCloudLoggingJsonFormat(this ZLoggerOptions options)
    {
        return options.UseJsonFormatter(formatter =>
        {
            // Category and ScopeValues is manually write in AdditionalFormatter at labels so remove from include properties.
            formatter.IncludeProperties = Timestamp | LogLevel | Message | ParameterKeyValues;

            formatter.JsonPropertyNames = JsonPropertyNames.Default with
            {
                LogLevel = Encode("severity"),
                LogLevelNone = Encode("DEFAULT"),
                LogLevelTrace = Encode("DEBUG"),
                LogLevelDebug = Encode("DEBUG"),
                LogLevelInformation = Encode("INFO"),
                LogLevelWarning = Encode("WARNING"),
                LogLevelError = Encode("ERROR"),
                LogLevelCritical = Encode("CRITICAL"),

                Message = Encode("message"),
                Timestamp = Encode("timestamp"),
            };

            formatter.PropertyKeyValuesObjectName = Encode("jsonPayload");

            // cache JsonEncodedText outside of AdditionalFormatter
            var labels = Encode("logging.googleapis.com/labels");
            var category = Encode("category");
            var eventId = Encode("eventId");
            var userId = Encode("userId");

            formatter.AdditionalFormatter = (Utf8JsonWriter writer, in LogInfo) =>
            {
                writer.WriteStartObject(labels);
                writer.WriteString(category, logInfo.Category.JsonEncoded);
                writer.WriteString(eventId, logInfo.EventId.Name);

                if (logInfo.ScopeState != null && !logInfo.ScopeState.IsEmpty)
                {
                    foreach (var item in logInfo.ScopeState.Properties)
                    {
                        if (item.Key == "userId")
                        {
                            writer.WriteString(userId, item.Value!.ToString());
                            break;
                        }
                    }
                }
                writer.WriteEndObject();
            };
        });
    }
}

The list of properties is as follows.

Name Description
JsonPropertyNames JsonPropertyNames Specify the name of each key in the output JSON
IncludeProperties IncludeProperties Flags that can specify properties to be output. (default: Timestamp, LogLevel, CategoryName, Message, Exception, ScopeKeyValues, ParameterKeyValues)
JsonSerializerOptions JsonSerializerOptions The options of System.Text.Json
JsonLogInfoFormatter? AdditionalFormatter Action when rendering additional properties based on LogInfo.
JsonEncodedText? PropertyKeyValuesObjectName If set, the key/value properties is nested under the specified key name.
IKeyNameMutator? KeyNameMutator You can set the naming convention if you want to automatically convert key names.
bool UseUtcTimestamp If true, timestamp is output in utc. (default: false)

KeyNameMutator

By default, JSON key names are output as is, so in the following character output, "user.Name" becomes the JSON key name.

var user = new User(1, "Alice");
logger.ZLogInformation($"Name: {user.Name}");

If you set this to formatter.KeyNameMutator = KeyNameMutator.LastMemberName, it becomes Name. If you set this to LastMemberNameLowerFirstCharacter, the first character is replaced with lower-case, resulting in name.

The following is a list of KeyNameMutators provided as standard:

Name Description
LastMemberName Returns the last member name of the source.
LowerFirstCharacter The first character converted to lowercase.
UpperFirstCharacter The first character converted to uppercase.
LastMemberNameLowerFirstCharacter Returns the last member name of the source with the first character converted to lowercase.
LastMemberNameUpperFirstCharacter Returns the last member name of the source with the first character converted to uppercase.

MessagePack

We also support structured logging output in binary format using MessagePack instead of JSON. UseMessagePackFormatter() requires a reference to the additional package ZLogger.MessagePack.

PM> Install-Package ZLogger.MessagePack

logging.AddZLoggerFile("log.bin", options =>
{
    options.UseMessagePackFormatter();
});

MessagePack extension uses MessagePack-CSharp as writer.

The list of properties is as follows.

Name Description
MessagePackSerializerOptions MessagePackSerializerOptions The options of MessagePack-CSharp.
IncludeProperties IncludeProperties Flags that can specify properties to be output. (default: `Timestamp
IKeyNameMutator? KeyNameMutator You can set the naming convention if you want to automatically convert key names.

Custom Formatter

If you want to create a formatter other than the default PlainText, Json, and MessagePack, you can implement IZloggerFormatter to create any custom output.

public interface IZLoggerFormatter
{
    bool WithLineBreak { get; }
    void FormatLogEntry<TEntry>(IBufferWriter<byte> writer, TEntry entry)
        where TEntry : IZLoggerEntry;
}
options.UseFormatter(() => new MyFormatter());

LogInfo

Additional information about when each log was written can be obtained from this LogInfo struct.

Name Description
LogCategory Category The category name set for each logger. And holds JsonEncodedText and utf8 byte sequence representations.
Timestamp Timestamp Timestamp
LogLevel LogLevel LogLevel of Microsoft.Extensions.Logging
EventId EventId EventId of Microsoft.Extensions.Logging
Exception? Exception Exception given as argument when logging.
LogScopeState? ScopeState Additional properties set by ILogger.BeginScope(...) (if ZLoggerOptions.IncludeScopes = true)
object? Context Additional context
string? MemberName Caller MemberName
string? FilePath Caller FilePath
int LineNumber Caller LineNumber

ZLoggerOptions

The following are common option items for all providers.

Name Description
bool IncludeScopes { get; set; } Enable ILogger.BeginScope, default is false.
bool IsFormatLogImmediatelyInStandardLog { get; set; } Fallback of standard logger.Log, message stringify immediately or not. Default is true.
TimeProvider? TimeProvider { get; set; } Gets or sets the time provider for the logger. The Timestamp of LogInfo is generated by TimeProvider's GetUtcNow() and LocalTimeZone when TimeProvider is set. The default value is null, which means use the system standard.
Action<Exception>? InternalErrorLogger { get; set; } InternalErrorLogger is a delegate that is called when an exception occurs in the log writing process (such as a serialization error). The default value is null, which means errors are ignored.
CreateFormatter() Create an formatter to use in ZLoggerProvider.
UseFormatter(Func<IZLoggerFormatter> formatterFactory) Set the formatter that defines the output format of the log.
UsePlainTextFormatter(Action<PlainTextZLoggerFormatter>? configure = null) Use the built-in plain text formatter.
UseJsonFormatter(Action<SystemTextJsonZLoggerFormatter>? configure = null) Use the built-in json formatter. (implementation of System.Text.Json)

By default, UsePlainTextFormatter is set. Also, only one formatter can be set for one provider. If you want to use multiple formatters, you need to add multiple providers.

// Show plain text log for console, json log for file
logging.AddZLoggerConsole(options => options.UsePlainTextFormatter());
logging.AddZLoggerFile("json.log", options => options.UseJsonFormatter());

ZLoggerMessage Source Generator

A log method generator similar to .NET 6's Compile-time logging source generation is bundled as standard.

public static partial class MyLogger
{
    [ZLoggerMessage(LogLevel.Information, "Bar: {x} {y}")]
    public static partial void Bar(this ILogger<Foo> logger, int x, int y);
}

This can achieve the highest performance. It's also possible to use special format specifiers like :json.

Microsoft.CodeAnalysis.BannedApiAnalyzers

Microsoft.CodeAnalysis.BannedApiAnalyzers is an interesting analyzer, you can prohibit the normal Log method and induce the user to call ZLogger's ZLog method.

image

All you have to do is prepare the following configuration.

T:Microsoft.Extensions.Logging.LoggerExtensions;Don't use this, use ZLog*** instead.
T:System.Console;Don't use this, use logger instead.

Global LoggerFactory

Like the traditional log manager, how to get and store logger per type without DI(such as static readonly ILogger logger = LogManager.GetLogger()). You can get ILoggerFactory from IHost before Run and set to the global static loggerfactory store.

using var host = Host.CreateDefaultBuilder()
    .ConfigureLogging(logging =>
    {
        logging.ClearProviders();
        logging.AddZLoggerConsole();
    })
    .Build(); // use Build instead of Run directly

// get configured loggerfactory.
var loggerFactory = host.Services.GetRequiredService<ILoggerFactory>();

LogManager.SetLoggerFactory(loggerFactory, "Global");

// Run after set global logger.
await host.RunAsync();

// -----

// Own static logger manager
public static class LogManager
{
    static ILogger globalLogger = default!;
    static ILoggerFactory loggerFactory = default!;

    public static void SetLoggerFactory(ILoggerFactory loggerFactory, string categoryName)
    {
        LogManager.loggerFactory = loggerFactory;
        LogManager.globalLogger = loggerFactory.CreateLogger(categoryName);
    }

    public static ILogger Logger => globalLogger;

    // standard LoggerFactory caches logger per category so no need to cache in this manager
    public static ILogger<T> GetLogger<T>() where T : class => loggerFactory.CreateLogger<T>();
    public static ILogger GetLogger(string categoryName) => loggerFactory.CreateLogger(categoryName);
}

You can use this logger manager like following.

public class Foo
{
    static readonly ILogger<Foo> logger = LogManager.GetLogger<Foo>();

    public void Foo(int x)
    {
        logger.ZLogDebug($"do do do: {x}");
    }
}

Unity

Installation

ZLogger uses some of the compile time features of C# 10, and ZLogger.Generator uses some of the features of C# 11.

To use them in Unity, needs to check the Unity version and set up the compiler.

  • Unity 2022.2 or newer
    • Standard ZLogger features are available.
    • Unity internally embeds the .NET SDK 6. So C# 10 is available via compiler arguments.
  • Unity 2022.3.12f1 or newer
    • ZLogger source generator available.
    • Unity internaly update .NET SDK 6. So C# 11 features are in preview.

Prerequirements:

  • Install NuGetForUnity
    • Required to install the dlls of ZLogger and its dependencies.
  • Install CsprojModifier
    • Required to develop in the IDE with a new language version.
  • Install ZLogger.Unity package via git url.
    • Add https://github.com/Cysharp/ZLogger.git?path=src/ZLogger.Unity/Assets/ZLogger.Unity to Package Manager

Installation steps:

  1. Setup the C# compiler for unity.

    • Add a text file named csc.rsp with the following contents under your Assets/.
      • -langVersion:10 -nullable
        
    • Note:
      • If you are using assembly definition, put it in the same folder as the asmdef that references ZLogger.
      • If you are using Unity 2022.3.12f1 or newer, you can use langVersion:preview allows parts of C# 11 features.
  2. Setup the C# compiler for your IDE.

    • Add a text file named LangVersion.props with the following contents
      • <Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
          <PropertyGroup>
            <LangVersion>10.0</LangVersion>
            <Nullable>enable</Nullable>
          </PropertyGroup>
        </Project>
    • Open Project Settings and [C# Project Modifier] section under the [Editor].
    • Add the .props file you just created, to the list of [Additional project imports].
    • Note:
      • If you are using assembly definition, add your additional csproj in the list of [The project to be addef for import].
      • If you want to use ZLoggerMessage Source Generator, require Unity 2022.3.12f1 and change to <LangVersion>11</LangVersion>
  3. Install ZLogger nuget package.

    • Open [Nuget] -> [Manage Nuget Packages] in the menu bar.
    • Search ZLogger, and press [Install].

Basic usage

The basic functions of ZLogger are also available in Unity as follows. Use LoggerFactory directly to create loggers.

var loggerFactory = LoggerFactory.Create(logging =>
{
    logging.SetMinimumLevel(LogLevel.Trace);
    logging.AddZLoggerUnityDebug(); // log to UnityDebug
});

var logger = loggerFactory.CreateLogger<YourClass>();

var name = "foo";
logger.ZLogInformation($"Hello, {name}!");

Also supports StructuredLogging(JSON), and FileProvider.

var loggerFactory = LoggerFactory.Create(logging =>
{
    logging.AddZLoggerFile("/path/to/logfile", options =>
    {
        options.UseJsonFormatter();
    });
});

Unity 2022.3.12f1 and enables -langVersion:preview supports Source Generator.

public static partial class LogExtensions
{
    [ZLoggerMessage(LogLevel.Debug, "Hello, {name}")]
    public static partial void Hello(this ILogger<NewBehaviourScript> logger, string name);
}

License

This library is licensed under the MIT License.

zlogger's People

Contributors

0xf6 avatar dependabot[bot] avatar doyasu24 avatar energy0124 avatar github-actions[bot] avatar guitarrapc avatar hadashia avatar happyfacade avatar jackd83 avatar ltrzesniewski avatar makiuchi-d avatar masato-tanaka-actevolve avatar masatoru avatar meilcli avatar monoman avatar neuecc avatar shirakawayohane avatar tetsuzin avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

zlogger's Issues

Improvement: Shared Files

When logging from multiple processes into the same file, the file will be corrupted.
To prevent that, the file could be locked during the actual write, and the lock could be released after that.
I am writing something fairly similar to this and archived over 2 million messages/sec with that approach.

Here is my approach if you want to get some inspiration: https://gist.github.com/JKamsker/0de2d65ba0730d89022e7aea83ca7643

"System.Runtime.CompilerServices.Unsafe.dll" version clash

Maybe not really a ZLogger issue.

We use the native collections from the package unity.collections and that package is already shipped with the "System.Runtime.CompilerServices.Unsafe.dll" but version 4.0.4.1 and ZLogger is shipped with version 4.0.6 hence the version clash. As far as I can see the only workaround would be to use ilmerge to merge the version shipped with ZLogger as internal dll. Or do you see another possible solution for this?

see also: https://answers.unity.com/questions/1697304/how-to-resolve-multiple-precompiled-assemblies-err.html

v2 release task

CodeGenerator Test

  • ToString, alignment, format
  • ToString(IbufferWriter), alignment format
  • IEnumerable out to json?, format json?
  • WriteJsonParameterKeyValues, IEnumerable, Enum, DateTime, DateTimeOffset, Guid, nullable
  • json format in WriteJson paramters

InterpolatedString

  • nullable, IEnumerable recheck

CodeGenerator Impl2

  • varidation more and more
  • varidation test
  • SkipEnabledCheck test
  • FirstExceptoin test
  • FirstLogLevel test
  • More ZLoggerMessage constructor
  • check nullable and GetParameterValue<T>

Packaging

  • ZLogger includes Generator

ReadMe

  • Final Benchmark and Graph
  • Manual

GitHub about corrections.

The current GitHub About text for this repo reads:

Zero Allocation Text/Strcutured Logger for .NET Core and Unity, built on top of a Microsoft.Extensions.Logging.

Should read:

Zero allocation text/structured logger for .NET Core and Unity, built on top of Microsoft.Extensions.Logging.

AddZLoggerConsole doesn't write Visual Studio's output window on WPF Application

It seems that ZLoggerConsoleLoggerProvideruses System.Console's stream.

this.streamWriter = new AsyncStreamLineMessageWriter(Console.OpenStandardOutput(), opt);

I tried to implement IAsyncLogProcessor using System.Diagnostics. It works.
Such like as:

    using System.Threading.Tasks;
    using Microsoft.Extensions.Logging;
    using ZLogger;
    public class SimpleProcessor : IAsyncLogProcessor {
        private readonly ZLoggerOptions _options;
        public SimpleProcessor (ZLoggerOptions options) {
            _options = options;
        }
        public ValueTask DisposeAsync() {
            return default;
        }
        public void Post(IZLoggerEntry log) {
            try {
                var msg = log.FormatToString(_options, null);
                System.Diagnostics.Trace.WriteLine(msg);
            }
            finally {
                log.Return();
            }
        }
    }

Do you have any plans to support this on non custom processor?

Deleting old rolling file logs

RollingFile should have limits, like last month, or 1gb max, to not drain disk space on customers machine. I think this is critical to have if this lib is meant to be used in production environments.

The generated files will not be deleted. If you want to do a log rotation, please do it from outside.

I think this is not good way to deal with log rotation. Especially if you have something misbehaving (generating lots of logs), the only solution is to react before disk fills up.

To solve this from outside it would be really hard to get right, as it cannot be just solved with cron job, it needs active monitoring and dealing with locked files, whereas log process already knows file masks and can correctly compute file sizes and stop logging when attempting to roll.

I think the right thing to do is stop logging, because if something is misbehaving and creating lots of log files probably cause of the problem is much more useful. If logger would remove younger logs, it could hide the cause of problem with lots of gibberish.

Using scopes

Hello!
Is there any chance to show scopes in log entries?

I mean when we using _log.BeginScope() I want to see it in output in Console.
In options I can't find any references for that

Configuring logging from external config

Is there any support or at least some example on how to allow configuring logging parameters or output providers from external config file like appsettings.json?

It is useful for allowing app admins to configure the logging (ex. rolling file pattern, format, size, etc) according to their rules.

Load Assembly faild in unity 2021.3.12(MacOs)/15(Win11)

Assembly 'Library/ScriptAssemblies/Assembly-CSharp.dll' will not be loaded due to errors:
Reference has errors 'Microsoft.Extensions.Logging'.

Assembly 'Library/ScriptAssemblies/Assembly-CSharp-Editor.dll' will not be loaded due to errors:
Reference has errors 'Assembly-CSharp'.

Assembly 'Library/ScriptAssemblies/ZLogger.dll' will not be loaded due to errors:
Reference has errors 'Microsoft.Extensions.Logging'.

Assembly 'Assets/Plugins/Microsoft.Extensions.Logging.dll' will not be loaded due to errors:
Unable to resolve reference 'System.Diagnostics.DiagnosticSource'. Is the assembly missing or incompatible with the current platform?
Reference validation can be disabled in the Plugin Inspector.

Assembly 'Assets/Plugins/Microsoft.Extensions.Logging.Configuration.dll' will not be loaded due to errors:
Reference has errors 'Microsoft.Extensions.Logging'.

FormatException is thrown if the log message string contains "{".

Hello @neuecc .

FormatException is thrown if the log message string contains "{".

Configure the console logger by calling the AddZLoggerConsole method.

await MagicOnionHost.CreateDefaultBuilder()
.UseMagicOnion(options, new ServerPort("localhost", port, credentials))
.ConfigureLogging(logging =>
{
logging.ClearProviders();
logging.AddZLoggerConsole(options =>
{
options.EnableStructuredLogging = false;
}
);
}
)
.RunConsoleAsync().ConfigureAwait(false);

Outputs a string containing "{" to the logger.

logger.ZLogInformation("{abc=1}");

FormatException is thrown.

System.FormatException: Input string was not in a correct format.
at System.Number.ThrowOverflowOrFormatException(ParsingStatus status, TypeCode type)
at Cysharp.Text.FormatParser.Parse(ReadOnlySpan1 format) at Cysharp.Text.ZString.Utf8Format[T1](IBufferWriter1 bufferWriter, String format, T1 arg1)
at ZLogger.Entries.FormatLogEntry2.FormatUtf8(IBufferWriter1 writer, ZLoggerOptions options, Utf8JsonWriter jsonWriter)
at ZLogger.AsyncStreamLineMessageWriter.WriteLoop()

How to format log entry timestamp?

Is it possible to change the log's timestamp format for text logger?
Currently I am using:

var prefixFormat = ZString.PrepareUtf8<string, DateTime>("{0} [{1}] ");
options.PrefixFormatter = (writer, info) => prefixFormat.FormatTo(ref writer, ConvertLoggingLevel(info.LogLevel), info.Timestamp.DateTime.ToLocalTime());

and obtain the following output:

DBUG [11/15/2020 01:22:30] Running ..........

but I would like a different date format like dd-MM-YYYY HH:mm:ss.fff

The Unity logger does not allow to provide a context object

Unity's Debug.Log methods allow the passing of a context GameObject. Whenever such a log message is clicked the appropriate object in the scene hierarchy is highlighted.
The current implementation of the Unity logger in ZLogger does provide and API to pass such a context.

Is it possible to change the flush rate?

My current logger writes to an output stream that i flush once ever 10 seconds to save on IO hits. It looks like ZLogger flushes immediately. It is possible to adjust this (as i suspect this might cause performance issues...or am i wrong?)

DLL overlap with Unity packages

Hey there,
I wanted to give this a try but unfortunately, my project exploded with errors.

The Unity package com.unity.collections already includes System.Runtime.CompilerServices.Unsafe.dll. I tried to see if deleting the one that is included with ZLogger might do the trick, but then it gave these errors:

Assembly 'Assets/Plugins/Microsoft.Extensions.Primitives.dll' will not be loaded due to errors:
Microsoft.Extensions.Primitives references strong named System.Runtime.CompilerServices.Unsafe in a different folder, versions has to match. Assembly references: 4.0.6.0 Found in project: 4.0.4.0.

Then there are many other similar ones:

Assembly 'Assets/Plugins/System.Threading.Tasks.Extensions.dll' will not be loaded due to errors:
System.Threading.Tasks.Extensions references strong named System.Runtime.CompilerServices.Unsafe in a different folder, versions has to match. Assembly references: 4.0.4.1 Found in project: 4.0.4.0.

It then just continues on:

I am not 100% sure on this, but I believe if you make ZLogger into either an official Unity package, or an OpenUPM (Some info on it here), this might be able to be avoided. I believe it lets you contain the dependencies within the package so they do not conflict.

Thanks,
-MH

Multiple files on the fly

Hello,

Firstly, this project looks excellent and seems to compliment your other libraries. I have a few questions

  1. Is there any thought of logging MessagePack serialized byte[] straight to file with headers to speed up reading them? Eg multiple serialized objects written to same file.
  2. Often I need to log to files where the filename is unknown until time of logging. Is it possible to create new filenames when logging? For example, Serilog uses sinks and NLog has variables. Eg something like this...
// Serilog
string logPath = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData), $"SomePath");
Log.Logger = new LoggerConfiguration().WriteTo.Map("Filename", "DefaultFileName.txt", (name, wt) => wt.File($"{logPath}/{name}",
                        outputTemplate: "{Message}{NewLine}",
                        flushToDiskInterval: TimeSpan.FromSeconds(1),
                        encoding: Encoding.UTF8
                        )).CreateLogger();


        [Benchmark]
        public void SerilogWriter2()
        {
            for (int i = 0; i < N; i++)
            {
                for (int m = 0; m < M; m++)
                {
                    Log.ForContext("Filename", $"{m}.txt").Information(s);
                }
            }

            Log.CloseAndFlush();
        }


// NLog 
        [Benchmark(Baseline = true)]
        public void NLogWriter()
        {
            for (int i = 0; i < N; i++)
            {
                for (int m = 0; m < M; m++)
                {
                    loggerBf.WithProperty("VarFilename", $"{m}.txt").Info(s);
                }
            }

            // Important - https://github.com/NLog/NLog/wiki/Tutorial#5-remember-to-flush
            // Flush and close down internal threads and timers
            NLog.LogManager.Shutdown();
        }

InvalidOperationException when DOTNET_ENVIRONMENT is "Development"

When DOTNET_ENVIRONMENT is set to "Development", an exception is thrown during configuration:

System.InvalidOperationException: Cannot resolve scoped service 'Microsoft.Extensions.Options.IOptionsSnapshot`1[ZLogger.ZLoggerOptions]' from root provider.

Simple app to reproduce: (using netcoreapp3.1 and ZLogger v1.1.11)

	class Program : ConsoleAppBase
	{
		readonly ILogger<Program> _log;

		public Program(ILogger<Program> logger)
		{
			_log = logger;
		}

		public static Task Main(string[] args)
		{
			// THIS DOESN'T WORK
			Environment.SetEnvironmentVariable("DOTNET_ENVIRONMENT", "Development");
			
			// THIS WORKS
			//Environment.SetEnvironmentVariable("DOTNET_ENVIRONMENT", "Test");  

			return Host.CreateDefaultBuilder(args)
				.ConfigureLogging((context, logging) =>
				{
					logging.ClearProviders();
					logging.AddZLoggerConsole();
				})
				.RunConsoleAppFrameworkAsync<Program>(args);
		}

		public void Run()
		{
			_log.LogInformation("started");
		}
	}

Prefix for printing DateTime with miliseconds ?

Can anyone please explain how the heck we are supposed to print "dd-MM-YYYY HH:mm:ss.fff" format ?
Yes i read the git section "DateTime" but it doesnt really help me at all...

This is my current code :

                builder.AddZLoggerConsole(options => {
                    
                    // Tips: use PrepareUtf8 to achive better performance.
                    var prefixFormat = ZString.PrepareUtf8<LogLevel, DateTime>("[{0}][{1}]");
                    options.PrefixFormatter = (writer, info) => prefixFormat.FormatTo(ref writer, info.LogLevel, info.Timestamp.DateTime.ToLocalTime());
                });

Which only prints "dd-MM-YYYY HH:mm:ss"... how do we add milliseconds to it ?

Unity 2021.2 link issues

Hi @neuecc, thank you for all your hard work and I tell people about Zlogger whenever I can!

However, it would be great to secure Unity 2021.2 compatibility as it seems there are multiple linking issues unfortunately. We are switching to 2021.2 and will need to drop Zlogger if we can't resolve this.

So first, one removes the 4 libs now in Unity to clear errors:

Microsoft.Bcl.AsyncInterfaces
System.Buffers
System.Memory
System.Threading.Tasks.Extensions

However, then these two errors appear:

Assets/Scripts/ZLogger/Unity/UnityLoggerFactory.cs(40,33): error CS0012: The type 'IAsyncDisposable' is defined in an assembly that is not referenced. You must add a reference to assembly 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'.

Assets/Scripts/ZLogger/Unity/UnityLoggerFactory.cs(41,24): error CS0012: The type 'IAsyncDisposable' is defined in an assembly that is not referenced. You must add a reference to assembly 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'.

Also, there are dozens of these warnings:

Assets/Scripts/ZLogger/Entries/FormatLogEntry.cs(22,32): warning CS0436: The type 'AllowNullAttribute' in 'ZLogger/Shims/NullableAttributes.cs' conflicts with the imported type 'AllowNullAttribute' in 'netstandard, Version=2.1.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. Using the type defined in 'ZLogger/Shims/NullableAttributes.cs'.

Perhaps you could do a distro/release intended for 2021.2? Please let us know how we can support, friend.

IL2CPP build problem with UnityLoggerFactory

Hi, UnityLoggerFactory throws this exception on IL2CPP build (tried both .net standard and framework), any ideas what might be the cause of this ?

ArgumentNullException: Value cannot be null.
Parameter name: obj
System.Threading.Monitor.ReliableEnterTimeout (System.Object obj, System.Int32 timeout, System.Boolean& lockTaken) (at <00000000000000000000000000000000>:0)
Microsoft.Extensions.DependencyInjection.DependencyInjectionEventSource.ServiceProviderBuilt (Microsoft.Extensions.DependencyInjection.ServiceProvider provider) (at <00000000000000000000000000000000>:0)
Microsoft.Extensions.DependencyInjection.ServiceCollectionContainerBuilderExtensions.BuildServiceProvider (Microsoft.Extensions.DependencyInjection.IServiceCollection services, Microsoft.Extensions.DependencyInjection.ServiceProviderOptions options) (at <00000000000000000000000000000000>:0)
ZLogger.UnityLoggerFactory.Create (System.Action`1[T] configure) (at <00000000000000000000000000000000>:0)
IntroSceneManager.InitLogger () (at <00000000000000000000000000000000>:0)
IntroSceneManager.Start () (at <00000000000000000000000000000000>:0)
System.Runtime.CompilerServices.AsyncVoidMethodBuilder.Start[TStateMachine] (TStateMachine& stateMachine) (at <00000000000000000000000000000000>:0)
IntroSceneManager.Start () (at <00000000000000000000000000000000>:0)

How to use customize structured logging

Hello
I would like to customize structedLogging
Is it possible??

Or can I make custom log processor??
your example doesn't work MessageTest.cs
It output nothing...

And I would like to use Payload in custom log processor.

this code is using Prefix, Suffix

builder.AddZLoggerConsole(options =>
{
    options.PrefixFormatter = (writer, info) =>
    {
        var strList = new List<string>()
        {
            $"\"CategoryName\":\"{info.CategoryName}\"",
            $"\"LogLevel\":\"{info.LogLevel}\"",
            $"\"Timestamp\":\"{info.Timestamp.DateTime.ToLocalTime()}\"",
            $"\"Exception\":\"{info.Exception}\"",
        };

        var outStr = String.Join(", ", strList);
        
        ZString.Utf8Format(writer, "{0}, \"Message\":\"", "{" + outStr);
    };
}

this code is using structedLogging
I would like to erase EventID or some values I don't use.

builder.AddZLoggerConsole(options =>
{
    options.PrefixFormatter = (writer, info) =>
    {
        options.EnableStructuredLogging = true;

        options.StructuredLoggingFormatter = (writer, info) =>
        {
            //writer.WriteString(applicationKey, applicationValue);
            info.WriteToJsonWriter(writer);
        };

        options.JsonSerializerOptions = new JsonSerializerOptions
        {
            WriteIndented = false,
            IgnoreNullValues = true,
            Encoder = JavaScriptEncoder.Create(UnicodeRanges.All)
        };
    };
}

Unity Editor 2021 IAsyncDisposaple Multiple dll issue

In Unity Editor 2021.4.3, Zlogger gives two compile error:


Assets\Scripts\ZLogger\AsyncStreamLineMessageWriter.cs(12,69): error CS0433: The type 'IAsyncDisposable' exists in both 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' and 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'

Assets\Scripts\ZLogger\IAsyncLogProcessor.cs(7,43): error CS0433: The type 'IAsyncDisposable' exists in both 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' and 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'

Binary logging support?

I'm investigating ZLogger in the context of wanting to log every event that occurs within a system to a flat file.

Specifically, I am aiming to use a simple CQRS mechanism which logs every event to a file. The aim is to achieve the auditability benefits of event sourcing, but without the additional cost of maintenance (and hardware) required for a full event sourcing solution. For the app's data layer, I am currently happy with a standard PostgreSQL database for both reads and writes, but would like to have a record of everything that happens in the system. Obviously event logging performance would be very important for this use case. Ideally the logs could also be used to reconstruct state at a later date (via projections) if required.

Do you have any plans to implement a binary log file format and, if not, do you have any suggestions as to how to best go about this?

I was thinking about using typeless MessagePack, but ideally one needs to know how to reliably version the serialized types if reconstruction is required. One option might be to include a hash of type definition somehow? Another option might be to include the type schemas in the header of each log file (or in a separate schema file). I'd be grateful for any thoughts or suggestions.

Unity and structured logging

Hi,

Seeing how Unity build doesn't support structured logging yet, is there a recommended workaround to accomplish it through available API?

Additional information in InternalErrorLogger

Would love to see additional information available in the InternalErrorLogger callback.

Currently errors in a log entry, such as a string formatting issue, just give an exception message something like "System.IndexOutOfRangeException: Index was outside the bounds of the array.". In addition, since the processing is being done asynchronously, the call stack of the exception has very little useful information.

So, any additional information that could be provided, such as the raw message before formatting, is critical in narrowing down which of thousands of log entries is the culprit for an internal error.

Format DateTime with custom format?

Hi @neuecc , awesome work on the library!

Just posting this here as something I noticed that is missing on the library... It seems like the formatter doesn't support DateTime formatting, like this for the PrefixFormatter:

logging.AddZLoggerConsole(options => options.PrefixFormatter = (buf, info) => ZString.Utf8Format(buf, "[{0}] [{1:hh:mm:ss}]", info.LogLevel, info.Timestamp.LocalDateTime));

This works for now of course, but it doesn't add the leading 0 on the numbers:

logging.AddZLoggerConsole(options => options.PrefixFormatter = (buf, info) => ZString.Utf8Format(buf, "[{0}] [{1}:{2}:{3}]", GetLogLevelString(info.LogLevel), info.Timestamp.LocalDateTime.Hour, info.Timestamp.LocalDateTime.Minute, info.Timestamp.LocalDateTime.Second));

Separate outputs for loggers

Is it possible to separate loggers output in multiple files?

For example, I want the "ProtoLogger" to write to a file named protocol.log and the rest of the application to write to a different file e.g app.log.

Misuse

Hello @neuecc
I have the following code:

var builder = WebApplication.CreateSlimBuilder(args);
builder.Logging.ClearProviders();
builder.Logging.AddZLoggerConsole();
builder.Services.AddHostedService<StartupService>();

var app = builder.Build();
app.Run();

public class StartupService(ILogger<StartupService> logger) : IHostedService
{
    public Task StartAsync(CancellationToken cancellationToken)
    {
        while (true) logger.ZLogInformation($"yeap");
    }

    public Task StopAsync(CancellationToken cancellationToken) => throw new NotImplementedException();
}

It causes a lot of allocations and memory leaks

image

Unlike standard code:

var builder = WebApplication.CreateSlimBuilder(args);
builder.Logging.ClearProviders();
builder.Logging.AddConsole();
builder.Services.AddHostedService<StartupService>();

var app = builder.Build();
app.Run();

public class StartupService(ILogger<StartupService> logger) : IHostedService
{
    public Task StartAsync(CancellationToken cancellationToken)
    {
        while (true) logger.LogInformation($"yeap");
    }

    public Task StopAsync(CancellationToken cancellationToken) => throw new NotImplementedException();
}

image

Where did I go wrong?

PS. .NET 8. ZLogger 2.0.0

Structured logging for Unity

Is there any plan to add this in the future?
What is the problem with integrating structured logging in Unity?

CamelCase for structured logs

Is it possible to format camel-case the json output for structured logs?

{
  "CategoryName": "PlayGround.Program",
  "LogLevel": "Information",
  "EventId": 0,
  "EventIdName": null,
  "Timestamp": "2020-11-14T10:11:49.940606+00:00",
  "Exception": null,
  "Message": "Original...",
  "Payload": null
}

The Unity logger does not allow to provide a context object

Issue was closed by a bot for being stale, but it feels pretty important so I'm opening it again as a new issue:

Unity's Debug.Log methods allow the passing of a context GameObject. Whenever such a log message is clicked the appropriate object in the scene hierarchy is highlighted.
The current implementation of the Unity logger in ZLogger does provide and API to pass such a context.

From:
#45 (comment)

Worker support?

Is there a way to use ZLogger in the .NET 5-based worker service?

The readme says we need to use ConsoleFramework but isn't both ASP.NET and Console apps in NET 3.1+ use the same IHostBuilder supporting the same ConfigureLogging method?

Exclude ZLogger functions from stack trace

Hello,

We are surveying ZLogger as a replacement of our Unity logger. The string formatting GC allocation is indeed smaller, but the GC allocation for stack trace is larger since FormatLogState has many different overloads and all of them are printed to the stack trace. I wonder if there is a way or workaround to exclude ZLogger functions from Unity stack trace? I tried tagging ZLogger functions with DebuggerHiddenAttribute but it doesn’t work.

Thank you.

UnityEngine.Debug:Log (object)
ZLogger.Providers.UnityDebugLogProcessor:Post (ZLogger.IZLoggerEntry) (at Packages/[email protected]/Unity/ZLoggerUnityLoggerProvider.cs:56)
ZLogger.AsyncProcessZLogger:Log<ZLogger.Entries.FormatLogState`7<object, int, int, int, int, int, int>> (Microsoft.Extensions.Logging.LogLevel,Microsoft.Extensions.Logging.EventId,ZLogger.Entries.FormatLogState`7<object, int, int, int, int, int, int>,System.Exception,System.Func`3<ZLogger.Entries.FormatLogState`7<object, int, int, int, int, int, int>, System.Exception, string>) (at Packages/[email protected]/AsyncProcessZLogger.cs:30)
Microsoft.Extensions.Logging.Logger:<Log>g__LoggerLog|12_0<ZLogger.Entries.FormatLogState`7<object, int, int, int, int, int, int>> (Microsoft.Extensions.Logging.LogLevel,Microsoft.Extensions.Logging.EventId,Microsoft.Extensions.Logging.ILogger,System.Exception,System.Func`3<ZLogger.Entries.FormatLogState`7<object, int, int, int, int, int, int>, System.Exception, string>,System.Collections.Generic.List`1<System.Exception>&,ZLogger.Entries.FormatLogState`7<object, int, int, int, int, int, int>&)
Microsoft.Extensions.Logging.Logger:Log<ZLogger.Entries.FormatLogState`7<object, int, int, int, int, int, int>> (Microsoft.Extensions.Logging.LogLevel,Microsoft.Extensions.Logging.EventId,ZLogger.Entries.FormatLogState`7<object, int, int, int, int, int, int>,System.Exception,System.Func`3<ZLogger.Entries.FormatLogState`7<object, int, int, int, int, int, int>, System.Exception, string>)
Microsoft.Extensions.Logging.Logger`1<Logger>:Microsoft.Extensions.Logging.ILogger.Log<ZLogger.Entries.FormatLogState`7<object, int, int, int, int, int, int>> (Microsoft.Extensions.Logging.LogLevel,Microsoft.Extensions.Logging.EventId,ZLogger.Entries.FormatLogState`7<object, int, int, int, int, int, int>,System.Exception,System.Func`3<ZLogger.Entries.FormatLogState`7<object, int, int, int, int, int, int>, System.Exception, string>)
ZLogger.ZLoggerExtensions:ZLog<int, int, int, int, int, int> (Microsoft.Extensions.Logging.ILogger,Microsoft.Extensions.Logging.LogLevel,Microsoft.Extensions.Logging.EventId,System.Exception,string,int,int,int,int,int,int) (at Packages/[email protected]/ZLoggerExtensions.cs:1463)
ZLogger.ZLoggerExtensions:ZLogDebug<int, int, int, int, int, int> (Microsoft.Extensions.Logging.ILogger,string,int,int,int,int,int,int) (at Packages/[email protected]/ZLoggerExtensions.cs:1534)
Logger:ZLog () (at Assets/Logger.cs:46)
Logger:Update () (at Assets/Logger.cs:35)

Possible symbol name collisions in generated code

I noticed that the generator can use the wrong type in the emitted code when the enclosing namespace defines a LogLevel enum for instance:

image

This may also be caused by several other types such as JsonEncodedText, Unsafe or EventId for instance.

Adding global:: with the full namespace everywhere would fix this, but I'm not sure if you want to do that given the chance for this problem to occur is quite low, so feel free to close this issue. I wanted to notify you just in case.

The problem can be bypassed by the user by generating the messages in a separate namespace.

Hello. In Unity 2018.4.7,ZLogger GC Alloc more than Unity Debug.Log ?

Hello, I compare ZLogger with Unity Debug.Log in Unity Profile . And I get a result like this
ZLogger

Here is my test code:

public class ZLogTest : MonoBehaviour
{
    static readonly ILogger<ZLogTest> zLogger = LogManager.GetLogger<ZLogTest>();
    void Start()
    {
        UnityEngine.Profiling.Profiler.BeginSample("zLogger");
        for (int i = 0; i < 10; i++)
        {
            zLogger.ZLogDebug("0000000000000000");
        }
        UnityEngine.Profiling.Profiler.EndSample();

        UnityEngine.Profiling.Profiler.BeginSample("Debug.Log");
        for (int i = 0; i < 10; i++)
        {
            Debug.Log("00000000000000000000011");
        }
        UnityEngine.Profiling.Profiler.EndSample();

        UnityEngine.Profiling.Profiler.BeginSample("zLogger format");
        for (int i = 0; i < 10; i++)
        {
            zLogger.ZLogDebug("foo{0} bar{1}", 10, 20);
        }
        UnityEngine.Profiling.Profiler.EndSample();

        UnityEngine.Profiling.Profiler.BeginSample("Debug.Log format");
        for (int i = 0; i < 10; i++)
        {
            Debug.LogFormat("foo{0} bar{1}", 10, 20);
        }
        UnityEngine.Profiling.Profiler.EndSample();
        int x = 10, y = 30, z = 40;

        UnityEngine.Profiling.Profiler.BeginSample("x+");
        var str1 = "x:" + x + " y:" + y + " z:" + z;
        UnityEngine.Profiling.Profiler.EndSample();


        UnityEngine.Profiling.Profiler.BeginSample("ZString.Concat");
        var str2 = ZString.Concat("x:", x, " y:", y, " z:", z);
        UnityEngine.Profiling.Profiler.EndSample();


        UnityEngine.Profiling.Profiler.BeginSample("string.Format");
        var str3 = string.Format("x:{0} y:{1} z:{2}", x, y, z);
        UnityEngine.Profiling.Profiler.EndSample();


        UnityEngine.Profiling.Profiler.BeginSample("ZString.Format");
        var str4 = ZString.Format("x:{0} y:{1} z:{2}", x, y, z);
        UnityEngine.Profiling.Profiler.EndSample();
    }
}

Are my results wrong? I am confused.

Unity editor will not response if I used AddZLoggerRollingFile and dispose

//Unity ver: 2019.1.14
// step 1 : use AddZLoggerRollingFile
// step 2: use log print(ZLogTrace, ZLogWarning
// step 3: use dispose
// operation: change unity run mode from play to stop
// result: unity not response

static LogManager()
    {
        // Standard LoggerFactory does not work on IL2CPP,
        // But you can use ZLogger's UnityLoggerFactory instead,
        // it works on IL2CPP, all platforms(includes mobile).
        loggerFactory = UnityLoggerFactory.Create(builder =>
        {
            // or more configuration, you can use builder.AddFilter
            builder.SetMinimumLevel(LogLevel.Trace);

            // AddZLoggerUnityDebug is only available for Unity, it send log to UnityEngine.Debug.Log.
            // LogLevels are translate to
            // * Trace/Debug/Information -> LogType.Log
            // * Warning/Critical -> LogType.Warning
            // * Error without Exception -> LogType.Error
            // * Error with Exception -> LogException
            builder.AddZLoggerUnityDebug();

            // and other configuration(AddFileLog, etc...)

        //TODO step 1
  	builder.AddZLoggerRollingFile((offset, i) => $"Debug_Logs/{DateTime.Now.ToLocalTime():yyyy-MM-dd}_{i:000}.log",  x => x.ToLocalTime().Date, 1024);
        });

        globalLogger = loggerFactory.CreateLogger("Global");

        Application.quitting += () =>
        {
            // when quit, flush unfinished log entries.
            //TODO step 3
            loggerFactory.Dispose();
        };
    }

Zlogger throws exception when using Unity Editor Playback Options

Hey there,

Just an observation, but I'm using a Global logging manager like the example provided in the Readme, and I get NullReference errors when stopping and re-entering play mode.

Here is an example stack trace:

[Exception] NullReferenceException: Object reference not set to an instance of an object
ZLoggerExtensions.ZLogInformation() at Library/PackageCache/[email protected]/ZLoggerExtensions.cs:4757
4755:   public static void ZLogInformation(this ILogger logger, EventId eventId, Exception exception, string message)
4756:   {
-->4757:       logger.Log(LogLevel.Information, eventId, new MessageLogState<object>(null, message), exception, (state, ex) =>
4758:       {
4759:           return state.Message;

ZLoggerExtensions.ZLogInformation() at Library/PackageCache/[email protected]/ZLoggerExtensions.cs:4742
4740:   public static void ZLogInformation(this ILogger logger, string message)
4741:   {
-->4742:       ZLogInformation(logger, default(EventId), default(Exception), message);
4743:   }

AppManager.Start() at /XXXX/Managers/App/AppManager.cs:84
82:   public void Start()
83:   {
-->84:       _logger.ZLogInformation("----- AppManager Initialized");
86:       // userContainer.SetActive(true);

Maybe this is expected, but it would be cool if it worked with the editor play options.

Typo in the "about" text to the repository

Hello, you have a typo in the "about" section to the repository:

Zero Allocation Text/Strcutured Logger for .NET Core and Unity, built on top of a Microsoft.Extensions.Logging.

Ambiguous method call: GetTypeInfo(System.Type)

Full error:
UnityLoggerFactory.cs(200, 56): [CS0121] The call is ambiguous between the following methods or properties: 'System.Reflection.TypeExtensions.GetTypeInfo(System.Type)' and 'System.Reflection.IntrospectionExtensions.GetTypeInfo(System.Type)'

This is in Unity 2019.4.13f1 running .NET 4.x, using ZLogger.Unity.1.6.0 and ZString.Unity.2.4.2.

Strong name

ZLogger can't be used in a strong named assembly right now which is very frustrating

Support for named arguments

Would be nice to have support for named arguments on the format method, without having to rewrite all log messages. I guess in a first pass, they could all be interpreted as incrementing argument numbers, as they're usually not repeated in the log message.

 //this works

Logger.ZLogInformation("Log value {0}", 1234);

 //this throws "Input string was not in a correct format." in Cysharp.Text.FormatParser.Parse
Logger.ZLogInformation("Log value {VALUE}", 1234);

How To Add a custom formatter?

exact example:

In Unity, when you have a vector3, and I try to log that using zlogger, it looks like
(0.1, 0.8, 0.1)

even if the true value is
(0.112341, 0.8214152, 0.1123415)

I have an extension Vector3.ToNiceString()

public static string ToNiceString(this Vector3 vec) {
        return $"{{{(double)vec.x},{(double)vec.y},{(double)vec.z}}}";
    }

but obviously using this would create garbage. How would I achieve the same result using the zlogger, but without writing it out in long form like

LogInformation(format, "(", vector3.x, "," ,vector3.y, "," ,vector.z, ")")

I have already been able to adjust the float logging to log long form (cast to double first), but don't know how to do something more complicated as above.

in other words

I'd like to be able to do

ElevenLogger.MPMatchLogger?.Log("Received ball toss from opponent: ",v);

instead of

ElevenLogger.MPMatchLogger?.Log("Received ball toss from opponent: (", v.x, ", ",v.y, ", ", v.z,")");

and achieve the same result

Is it possible to add thread ID and call stack information to the log output format?

Looking at the code, it seems that only the following information can be added to the log text, but I want to leave the thread ID and callstack information as well.

public readonly struct LogInfo
{
public readonly string CategoryName;
public readonly DateTimeOffset Timestamp;
public readonly LogLevel LogLevel;
public readonly EventId EventId;
public readonly Exception? Exception;
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.