Giter Club home page Giter Club logo

workloadtools's Introduction

WorkloadTools

WorkloadTools is a collection of tools to collect, analyze and replay SQL Server workloads, on premises and in the cloud .

Download

Go to the release page and download the msi installer for your target bitness (x86 or x64)

Documentation

If you're looking for detailed documentation on the individual tools, please have a look at the Wiki

If you're looking for usage scenarios and examples, see the posts tagged WorkloadTools at my blog

SqlWorkload

SqlWorkload is a command line tool to start workload collection, analyze the collected data and replay the workload to a target machine, all in real-time.

SqlWorkload can connect to a SQL Server instance and capture execution related events via SqlTrace or Extended Events. These events are processed and passed to "consumers" that can replay the events to a target instance in real-time and analyze the statements. All the batches are "normalized" (parameters and constants are stripped away) and metrics are calculated on each normalized batch, like cpu, duration, reads and writes.

During the analysis, additional metrics are captured and saved regularly to the analysis database:

  • cpu usage
  • wait stats

Replaying and analyzing a production workload in test

If you want to compare the execution of the same workload on two different machines, you can point a first instance of SqlWorkload to your production server: SqlWorkload will analyze the workload and write the metrics to a database of your choice. It will also replay the workload to a test server, where you can point a second instance of SqlWorkload to obtain the same metrics. This second instance of SqlWorkload will not perform the replay, but it will only perform the workload analysis and write it to the same database where you stored the metrics relative to production (possibly on a different schema).

Once you have captured and replayed the workload for a representative enough time, you can stop the two instances of SqlWorkload and analyze the data using the included Workload Analyzer or PowerBI dashboard.

Command line switches

SqlWorkload accepts two command line switches:

--Log Path to the log file

--File Path to the .JSON configuration file

In fact, SqlWorkload supports a multitude of parameters and specifying them all in the command line can become really tedious. For this reason, SqlWorkload supports .JSON configuration files.

This is a sample configuration file. Please refer to the documentation to see the full list of available configuration options.

{
    "Controller": {

        // The Listener section describes how to capture the events
        "Listener":
        {
            // The main parameter here is the class type of the Listener
            "__type": "ExtendedEventsWorkloadListener",

            // The ConnectionInfo describes how to connect the Listener
            "ConnectionInfo":
            {
                "ServerName": "SQLDEMO\\SQL2014",
                // If you omit the UserName/Password, Windows authentication
                // will be used
                "UserName": "sa",
                "Password": "P4$$w0rd!"
            },

            // Filters for the workload
            "DatabaseFilter": "DS3",
            "ApplicationFilter" : "SomeAppName",
            "HostFilter" : "MyComputer",
            "LoginFilter": "sa"
        },

        // This section contains the list of the consumers
        // The list can contain 0 to N consumers of different types
        "Consumers":
        [
            {
                // This is the type of the consumer
                "__type": "ReplayConsumer",

                // The same considerations for ConnectionInfo
                // valid for the Listener apply here as well
                "ConnectionInfo":
                {
                    "ServerName": "SQLDEMO\\SQL2016",
                    "DatabaseName": "DS3",
                    "UserName": "sa",
                    "Password": "P4$$w0rd!"
                }
            },
            {
                // Here is another example with the AnalysisConsumer
                "__type": "AnalysisConsumer",

                // ConnectionInfo
                "ConnectionInfo": 
                {
                    "ServerName": "SQLDEMO\\SQL2016",
                    "DatabaseName": "DS3",
                    // This "SchemaName" parameter is important, because it 
                    // decides where the analysis data is written to
                    "SchemaName": "baseline",
                    "UserName": "sa",
                    "Password": "P4$$w0rd!"
                },

                // This decides how often the metrics are aggregated and 
                // written to the target database
                "UploadIntervalSeconds": 60
            }
        ]
    }
}

WorkloadViewer

WorkloadViewer is a GUI tool to analyze the data collected by the WorkloadAnalysisTarget in a SQL Server database. It shows metrics about the workload, relative to the beginning of the capture (in minutes).

Here are some screenshots of WorkloadViewer.

Workload

The three charts in the "Workload" tab show an overview of the workload analysis: CPU, Duration and Batches/sec. Two workloads can be compared by displaying independent series (Baseline and Benchmark) for each workload.

SqlWorkload analysis Overview

Queries

This tab displays information about the queries and how they relate to the workload. For a single workload analysis, it shows the most expensive queries. When comparing two workloads, it can be used to identify query regressions.

SqlWorkload regressed queries

Query Details

Double clicking a query in the "Queries" tab takes you to the "Query Details" tab, where you can see the text of the selected query, specific statistics by application, database, host and login and the average duration in a chart.

SqlWorkload query detail

ConvertWorkload

ConvertWorkload is a command line tool to convert existing trace files to the internal SQLite format used by WorkloadTools. In the future, ConvertWorkload will also support conversion from existing Extended Events files.

Why converting trace file to a different intermediate format instead of supporting it directly? Trace files can be read using an API that works only in x86. While WorkloadTools can work in x86, using x64 builds is highly recommended, due to the possible high memory usage when capturing intensive workloads. Using a x86 API would have excluded the functionality from the x64 builds, hence using an external tool to convert trace files seems much more appropriate.

Command line switches

-I or --Input               The input file (trace or extended events) to convert

-O or --Output              The output SQLite file to write

-L or --Log                 Specifies where to save the log file

-A or --ApplicationFilter   Application filter to apply while converting the source file

-D or --DatabaseFilter      Database filter to apply while converting the source file

-H or --HostFilter          Host filter to apply while converting the source file

-U or --LoginFilter         Login filter to apply while converting the source file

workloadtools's People

Contributors

a-teece avatar jeffbarnard avatar mcflyamorim avatar silent1984 avatar spaghettidba avatar thed000d avatar trovalo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

workloadtools's Issues

CPU is not in the same unit measure as duration.

For all types of listeners, make sure that CPU and duration are captured in the same unit measure and converted to milliseconds, because that's what's on the tin (analysis consumer writes both to columns that have "_ms" in the name, so I'd expect milliseconds)

Is there a way to link back from replay warnings to the event_sequence column in sqlite?

Hi Gianluca,

this relates to the latest release v1.2.13. Workload capture was performed with release v1.2.11.
In the replay log (console output) I read errors loke the following

2019-04-11 08:47:29.3007 - Warn - WorkloadTools.Consumer.Replay.ReplayWorker : Worker [278] - Error: 
INSERT failed because the following SET options have incorrect settings: 'QUOTED_IDENTIFIER'. Verify that SET options are correct for use with indexed views and/or indexes on computed columns and/or query notifications and/or xml data type methods.

This seems to relate to filtered indexes and I get quite alarmed by these errors regarding production (where nobody seems to notify it yet).
Is there a way to trace such an entry back to the event_sequence in the SQLite database? The analysis database contains a table called Errors but it's empty in my case. I currently parse the replay errors from the console log using regular expression but of course it would be nice to have it in a database (obviously I could put it there myself...so no offense to you).

If not, that would prove really helpful. I think of setting up an XEvent session tracking this error during replay in order to determine the culprit SQL but thats a bit more legwork.

Thanks in advance for your help

Martin

Exception reading event Collection was modified

Hi Gianluca,

this relates to the latest release v1.2.13. Workload capture was performed with release v1.2.11.
I performed a workload replay and get the following error message right at the end of the console output:

2019-04-11 08:52:48.8148 - Error - WorkloadTools.WorkloadController : Exception reading event
2019-04-11 08:52:48.8304 - Error - WorkloadTools.WorkloadController : Collection was modified; enumeration operation might not execute.
2019-04-11 08:52:48.8304 - Error - WorkloadTools.WorkloadController :    at System.Data.RBTree`1.RBTreeEnumerator.MoveNext()
   at System.Linq.Enumerable.<CastIterator>d__97`1.MoveNext()
   at System.Linq.Lookup`2.Create[TSource](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer)
   at System.Linq.GroupedEnumerable`3.GetEnumerator()
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at WorkloadTools.Util.DataUtils.ToDataTable[T](IEnumerable`1 items) in C:\GitHub\WorkloadTools\WorkloadTools\Util\DataUtils.cs:line 29
   at WorkloadTools.Consumer.Analysis.WorkloadAnalyzer.WriteExecutionDetails(SqlConnection conn, SqlTransaction tran, Int32 current_interval_id) in C:\GitHub\WorkloadTools\WorkloadTools\Consumer\Analysis\WorkloadAnalyzer.cs:line 513
   at WorkloadTools.Consumer.Analysis.WorkloadAnalyzer.WriteToServer() in C:\GitHub\WorkloadTools\WorkloadTools\Consumer\Analysis\WorkloadAnalyzer.cs:line 344
   at WorkloadTools.Consumer.Analysis.AnalysisConsumer.Dispose(Boolean disposing) in C:\GitHub\WorkloadTools\WorkloadTools\Consumer\Analysis\AnalysisConsumer.cs:line 42
   at WorkloadTools.WorkloadController.Stop() in C:\GitHub\WorkloadTools\WorkloadTools\WorkloadController.cs:line 115
   at WorkloadTools.WorkloadController.Run() in C:\GitHub\WorkloadTools\WorkloadTools\WorkloadController.cs:line 50
2019-04-11 08:52:48.8304 - Info - SqlWorkload.Program : Controller stopped.

From the contents of a logging table in the database I managed to figure out that the replay stopped at EventSequence 5709384 and missed approximately 43.000 events which are 6 minutes worth of capture (so in this case nothing dramatic regarding a 10 hour workload).

Thanks for looking into it

Martin

ProfilerWorkloadListener : Exception has been thrown by the target of an invocation.

Ever since Null or Not null release in I am getting following error :

Info - SqlWorkload.Program : SqlWorkload, Version=1.0.6.0, Culture=neutral, PublicKeyToken=null 1.0.6
Info - SqlWorkload.Program : Reading configuration from 'C:\Release\WorkloadTools\SqlWorkload.json'Info - WorkloadTools.Listener.Trace.TraceServerWrapper : SMO Version: Microsoft.SqlServer.ConnectionInfoExtended, Version=14.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91
Error - WorkloadTools.Listener.Trace.ProfilerWorkloadListener : Exception has been thrown by the target of an invocation.
Error - WorkloadTools.Listener.Trace.ProfilerWorkloadListener : Failed to initialize object as reader.
Error - WorkloadTools.WorkloadController : Uncaught Exception
Error - WorkloadTools.WorkloadController : at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.RuntimeType.InvokeMember(String name, BindingFlags bindingFlags, Binder binder, Object target, Object[] providedArgs, ParameterModifier[] modifiers, CultureInfo culture, String[] namedParams)
at WorkloadTools.Listener.Trace.TraceServerWrapper.InitializeAsReader(SqlConnectionInfoWrapper connectionInfo, String TraceDefinition)
in C:\progetti\WorkloadTools\WorkloadTools\Listener\Trace\TraceServerWrapper.cs:line 94
at WorkloadTools.Listener.Trace.ProfilerWorkloadListener.Initialize() in C:\progetti\WorkloadTools\WorkloadTools\Listener\Trace\ProfilerWorkloadListener.cs:line 62
at WorkloadTools.WorkloadController.Run() in C:\progetti\WorkloadTools\WorkloadTools\WorkloadController.cs:line 34
Info - SqlWorkload.Program : Controller stopped.

Add dialogs to setup

Setup is now completely silent. Add dialogs to ask for permission to install and notify when setup is over.

Analysis via stored procedure

At the present moment, the views required to extract the data for the analysis are hardcoded for the "baseline" and "replay" schemas, while the schema is instead free in the input parameters. A stored procedure should be added to the analysis database and should be able to create the views for the analysis in the correct schemas or, in alternative, run the comparison query using the supplied values for the schema.
This stored procedure should be added automatically whenever the analysis consumer starts writing to an analysis database.

01/01/0001 00:00:00 oldest command date

During the replay sometimes an error similar to the following is logged:

2019-04-12 01:33:14.4887 - Info - WorkloadTools.Consumer.Replay.ReplayConsumer : 01/01/0001 00:00:00 oldest command date

The error is risen periodically and after a while the tool completely stops replaying and just logs this error (as if there's nothing to replay).

More info:

  1. 16 CPUs on the replaying server all at 100% usage
  2. 500 workers configured in Workloadtools, around 300 used
  3. many errors during replay (e.g. trying inserting NULLs, missing procedures)

Replay of backup operations

Workloads may contain backup commands that overwrite backup files on network shares when replayed on other instances assuming these have the necessary permissions and the backup commands include INIT/FORMAT options.

This is not a bug per se, just an issue because people may not think that it could happen and accidentally overwrite production backups, so it may be enough to put a big warning in the documentation and/or issue a warnings in log files.

Backup commands may be filtered as any other command by providing a custom XE session and we may add one as an example.

Invalid operation. The connection is closed.

During the replay sometimes an error similar to the following is logged:

2019-04-11 18:10:38.7648 - Error - WorkloadTools.Consumer.Replay.ReplayWorker : Worker [311] - Error: Invalid operation. The connection is closed.
2019-04-11 18:10:38.7648 - Error - WorkloadTools.Consumer.Replay.ReplayWorker : at System.Data.ProviderBase.DbConnectionClosed.ChangeDatabase(String database)
at System.Data.SqlClient.SqlConnection.ChangeDatabase(String database)
at WorkloadTools.Consumer.Replay.ReplayWorker.ExecuteCommand(ReplayCommand command) in C:\Users\gianl\Source\Repos\WorkloadTools\WorkloadTools\Consumer\Replay\ReplayWorker.cs:line 234

The error will continue to be be risen until the worker gets disposed and needs to be investigated.

More info:

  1. happens also with a small workload (e.g. 200-300 batches/sec)
  2. 16 CPUs on the replaying server all at 100% usage
  3. 500 workers configured in Workloadtools
  4. many errors during replay (e.g. trying inserting NULLs, missing procedures)

Unable to write workload analysis.

I am getting following errors on trace replay:

Info - WorkloadTools.Consumer.Analysis.WorkloadAnalyzer : 0 rows aggregated
Info - WorkloadTools.Consumer.Analysis.WorkloadAnalyzer : 0 rows written
Warn - WorkloadTools.Consumer.Analysis.WorkloadAnalyzer : Unable to write workload analysis.
Warn - WorkloadTools.Consumer.Analysis.WorkloadAnalyzer : Value cannot be null.
Info - WorkloadTools.Consumer.Analysis.WorkloadAnalyzer : 0 rows aggregated
Info - WorkloadTools.Consumer.Analysis.WorkloadAnalyzer : 0 rows written
Warn - WorkloadTools.Consumer.Analysis.WorkloadAnalyzer : Unable to write workload analysis.
Warn - WorkloadTools.Consumer.Analysis.WorkloadAnalyzer : Value cannot be null.
Error - WorkloadTools.Consumer.Analysis.WorkloadAnalyzer : Unable to write workload analysis info to the destination database.

Replay controller stops on error.

Hi,

I'm trying to use SqlWorkload in order to benchmark and validate performance of a new production Sql Server instance before migrating from the current cluster.

Both servers are using Sql server 2014 Enterprise; baseline server is the master node of an AlwaysOn cluster, while replay server is a single server.

I'm able to run analysis + replay on baseline server, a few error pops up but the workload seems to be running just fine.
If I run analysis on replay server, everything works fine with current test workload, but as soon as replay consumer starts to push workload from baseline, the analyzer throws an error, and stops working:

2019-04-10 11:39:43.1492 - Error - WorkloadTools.Listener.ExtendedEvents.StreamXEventDataReader : Error converting XE data from the stream: Unable to cast object of type 'System.Byte[]' to type 'System.String'.
2019-04-10 11:39:43.1492 - Error - WorkloadTools.Listener.ExtendedEvents.StreamXEventDataReader :     client_app_name       : WorkloadTools-ReplayWorker
2019-04-10 11:39:43.1492 - Error - WorkloadTools.Listener.ExtendedEvents.StreamXEventDataReader :     database_name         : msdb
2019-04-10 11:39:43.1492 - Error - WorkloadTools.Listener.ExtendedEvents.StreamXEventDataReader :     client_hostname       : WSR02XJBW
2019-04-10 11:39:43.1492 - Error - WorkloadTools.Listener.ExtendedEvents.StreamXEventDataReader :     server_principal_name : skylab
2019-04-10 11:39:43.1492 - Error - WorkloadTools.Listener.ExtendedEvents.StreamXEventDataReader :     session_id            : 101
2019-04-10 11:39:43.1492 - Error - WorkloadTools.Listener.ExtendedEvents.ExtendedEventsWorkloadListener : Unable to cast object of type 'System.Byte[]' to type 'System.String'.
2019-04-10 11:39:43.1492 - Error - WorkloadTools.Listener.ExtendedEvents.ExtendedEventsWorkloadListener :    at WorkloadTools.Listener.ExtendedEvents.StreamXEventDataReader.ReadEvents() in C:\GitHub\WorkloadTools\WorkloadTools\Listener\ExtendedEvents\StreamXEventDataReader.cs:line 165
   at WorkloadTools.Listener.ExtendedEvents.ExtendedEventsWorkloadListener.ReadEvents() in C:\GitHub\WorkloadTools\WorkloadTools\Listener\ExtendedEvents\ExtendedEventsWorkloadListener.cs:line 243
2019-04-10 11:39:43.1802 - Info - WorkloadTools.Listener.ExtendedEvents.ExtendedEventsWorkloadListener : Extended Events session [sqlworkload] stopped successfully.
2019-04-10 11:39:48.6488 - Info - WorkloadTools.Listener.ExtendedEvents.ExtendedEventsWorkloadListener : Extended Events session [sqlworkload] stopped successfully.
2019-04-10 11:39:48.8051 - Info - WorkloadTools.Consumer.Analysis.WorkloadAnalyzer : 178 rows aggregated
2019-04-10 11:39:48.8051 - Info - WorkloadTools.Consumer.Analysis.WorkloadAnalyzer : 77 rows written
2019-04-10 11:39:48.8213 - Info - WorkloadTools.Consumer.Analysis.WorkloadAnalyzer : Performance counters written
2019-04-10 11:39:48.8519 - Info - WorkloadTools.Consumer.Analysis.WorkloadAnalyzer : Wait stats written
2019-04-10 11:39:48.8675 - Info - SqlWorkload.Program : Controller stopped.

I'm running version 1.2.13 both for analysis+replay on baseline server, and for analysis on replay server.

I tried downgrading the replay analyser to version 1.2.11; error is still thrown but the analyzer continues working properly.

Investigate massive memory leak

When running for 15 minutes or more, it ends up using a lot of memory. Looks like Strings are the most allocated objects (sql statements).

Download and display execution plans

Execution plans can be captured by enabling causality tracking and capturing the plans using the guid as a correlation between the multiple plans and the batch. The application can request and save plans only when not already saved

Empty replay schema

The code refers to replay schema and objects, however, those objects are not created automatically as the baseline tables are.
My understanding of the intended implementation is to have 2 instances (source and replay) to capture the traffic on the source and replay it on secondary. The to use the baseline schema and tables to capture run times on origin server. And to use Replay schema to capture Replay durations.

I created The replay schema and tables using individual scripts but there is nothing that populates this schema. However, the analyses view is querying this schema, but since the replay tables are not created it is failing to execute.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.