integerman / gitstractor Goto Github PK

A library for visualizing the commits, authors, and files of any git repository

License: MIT License

Jupyter Notebook 99.66% Python 0.05% C# 0.29% Dockerfile 0.01%

code-analysis data-visualization git repository-management static-code-analysis utilities data-analysis visualization dotnet powerbi

gitstractor's Introduction

GitStractor - Git Repository Analysis Tool

Project by Matt Eland (@IntegerMan)

This project is built for extracting commit, author, and file data from local git repositories in order to visualize repository history and trends to provide insight to software development teams and their stakeholders.

Here are a few examples of the types of visualizations that can be generated using GitStractor:

Stacked bar chart of # of commits per month by day of week:

Tree map of the # of commits per file:

Scatter plot of files in each commit:

Disclaimer

This project is in technical preview and will have bugs, inaccuracies, and numerous other issues. Use at your own risk.

Additionally, this project is intended to help you get a sense of the overall trends in your source code. It should not be used for performance evaluation purposes as its data is not yet known to be reliable and git history is not a good indicator of an indivual's performance.

Usage Instructions

GitStractor currently has two components:

A command line tool for extracting data from a local git repository
A Jupyter notebook for visualizing the extracted data

Data is extracted using the GitStractor-Extract tool. You can build this by opening GitStractor\GitStractor.sln in Visual Studio and setting GitStractor-Extract as the startup project. See Extracting Commit Data below for more information.

Once data is extracted, you can visualize it in a Jupyter Notebook. See Visualizing Commit Data in Jupyter Notebooks below for more information.

Extracting Commit Data

To get started, you'll need to run the GitStractor project in the GitStractor\GitStractor.sln solution.

You can either run the program from the command line once you've built it, or you can customize launchSettings.json to meet your needs.

Some common usage scenarios:

Extract git information from the git repository at C:\Dev\Interactive and store the resulting CSV files in C:\GitStractor

GitStractor-Extract --source C:\Dev\Interactive --destination C:\GitStractor

Extract git information from the git repository at C:\Dev\Interactive and store the resulting CSV files in C:\GitStractor. Ignore .gif, .txt, .json, and .d.ts files.

GitStractor-Extract -s C:\Dev\Interactive -d C:\GitStractor --ignore .gif,.txt,.json,.d.ts

Extract git information from the git repository at C:\Dev\Interactive and store the resulting CSV files in C:\GitStractor. Do not analyze commits off the current branch. This will analyze merge commits and may misattribute commits to the user that merged them. It is also considerably faster.

GitStractor-Extract -s C:\Dev\Interactive -d C:\GitStractor --includebranches false

Extract git information from the git repository at C:\Dev\Interactive and store the resulting CSV files in C:\GitStractor. Uses an authormap.json file to rename or merge together users. This is handy when you have users that have committed under different E-Mail addresses or changed their names.

GitStractor-Extract -s C:\Dev\Interactive -d C:\GitStractor --authormap C:\Dev\AuthorMap.json

An AuthorMap.json file should be structured like this:

[
    {
        "name": "Matt Eland",
        "emails": ["[email protected]", "[email protected]"]
    },
    {
        "name": "GitStractor",
        "emails": ["[email protected]"],
        "bot": true
    }
]

Visualizing Commit Data in Jupyter Notebooks

To view the full range of data visualizations in Jupyter Notebooks, open the GitStractor.ipynb Jupyter notebook in the Notebooks folder.

Once there, change the project_name variable to reflect your project and change the data_dir to indicate the directory your GitStractor .csv files from the prievious step are located.

From there, run the notebook from top to bottom to generate recommendations.

In order to work with Jupyter Notebooks, I recommend you install Anaconda and VS Code.

What's Next?

Future efforts on this project will focus on:

Creating a desktop application for extracting and visualizing data
Expanding the range of visualizations available in the Jupyter notebook
Improving the user experience pulling data from larger repositories
Adding machine learning capabilities for commit classification and clustering

If you'd like to submit a feature request or view the current backlog, please visit the GitHub Issues tab

Contact

Contact Matt Eland for general questions and feedback.

Please open an issue for enhancement requests and bug reports.

gitstractor's People

Contributors

Stargazers

Watchers

gitstractor's Issues

Visualize time data in better ways

It'd be nice to be able to either constrain time windows of certain charts or animate certain graphs by time periods.

Mark authors as no longer on project

I'd like to be able to specify via authormap or auto-detect contributors who are not active on a project (no commits in 3 months) so that we can highlight areas that are owned by people who are no longer on the project.

GitStractor Desktop - Integrate Extract Processes

The GitStractor Desktop utility should provide a convenient method for people to extract insights from local repositories and should do so by using the updated system.

Updated Jupyter Notebook for Displaying Commit Information

I want a good Jupyter Notebook that works with standardized GitStractor files to visualize information about commit activity.

Existing notebooks do this, but not on the current file format and not with sufficient contextual information.

GitStractor Desktop - File Trends

I'd like to be able to visualize file trends from GitStractor Desktop so I can understand how my code is structured.

Power BI Template - Commits

The Power BI template needs to be updated to work with more recent commit formats so people who don't want to use Jupyter Notebooks can use the application.

Ignore Certain Files

I want to be able to provide a series of regular expressions or file extensions that a filename will be evaluated against. If any of these match, the file will not be tracked in the files or commit files observers. This will help me ignore irrelevant files during the review process.

Extract - User Mapping

When mapping long projects, it's normal to see multiple authors with slightly different names or E-Mail addresses. Presently these are aggregated separately leading to messy reports and inaccurate aggregation metrics.

I'd like the ability to provide a JSON mapping from E-Mail addresses to new E-Mail addresses. Additionally, it'd be nice to be able to customize the display name of the user when using this approach so the preferred name is used.

GitStractor Desktop - Integrate Acquire Process

The GitStractor Desktop app should allow me to locally clone remote public repositories by their URL.

Visualize Work Items in relevant graph types

Once we track work items, these work items will allow for more information to be displayed, such as a work items per file, work item type breakdown, etc.

Updated Jupyter Notebook for Displaying Author Information

I want a good Jupyter Notebook that works with standardized GitStractor files to visualize information about commit authors.

Existing notebooks do this, but not on the current file format and not with sufficient contextual information.

Track Lines Added / Removed at a Per-Commit Level

Tracking the # lines added and removed at each commit and file commit enables a lot more interesting visualizations with scatter plots in particular and is a legacy feature I rather enjoyed. This also will help compare different author behavior profiles.

Aggregate Data for Files

I'd like to be able to use gitstractor to show aggregate information about files such as:

Date Created
Last Date Modified
Creating Author
Last Modifying Author
Num. Commits
Num. Authors

This likely is a "second-pass" dataset or something that could be calculated from the existing files datasets