Giter Club home page Giter Club logo

p4harmonize's Introduction

p4harmonize

Important Updates

  • As of 2023-05-16, this depot has been transferred from proletariatgames/p4harmonize to danbrakeley/p4harmonize.
  • As of v0.5.0, bash will no longer be required to run p4harmonize on Windows.
  • Also, the output now uses colors to much better effect (see example image below).

Overview

p4harmonize is a tool for mirroring a stream's head revision from one perforce server to another. This includes reconciling files, fixing differences in file name/path capitalization, fixing the file type, and fixing improperly checked in %AppleDouble files.

p4harmonize was built with Unreal Engine source in mind, where the Epic licensee perforce server is used as the source, and a dedicated stream on a project's perforce server is used as the destination. It is intended to be used with a setup similar to the one recommended by Epic. For example, a recent Unreal project I worked on had the following setup:

name description
//proj/engine_epic exact copy of a specific UE version from Epic's perforce server (p4harmonize targets this stream, no project-specific changes should ever end up here)
//proj/engine_merge dedicated space for a human to merge main with new engine drops; ideally QA will sign off on a build from this branch before the merged engine changes are brought to main
//proj/main our mainline; lots of people work here, so a broken build can be very costly

The output of v0.5.0 when there are no changes

Case-sensitivity

Perforce servers can run in case sensitive or case insensitive modes. When the destination server is running in case insensitive mode, file casing issues can't be fixed with a single move command. Instead, files must first be deleted, then re-added with the correct case. p4harmonize supports doing this work, however it requires the user to run p4harmonize twice. If you end up in this situation, p4harmonize will explain what to do as the first run finishes.

Install

You can download the latest Windows executable from the releases page, or you can build it yourself.

To build your own, see the Developement Setup section below.

Usage

p4harmonize requires a configuration TOML file, and by default will look for config.toml in the current directory. This can be overridden by passing -config <file>.

Here's an example config.toml file:

# source is the perforce server you are pulling from, ie Epic's licensee server.
[source]
p4port = "ssl:perforce.example.com:1667"
p4user = "user"
p4client = "user-UE4-Release-Latest-Minimal" # this needs to exist before running p4harmonize

# destination is the perforce server you want to update so that it matches the source
[destination]
p4port = "perforce.local:1666"
p4user = "localuser"
new_client_name = "localuser-harmonize"   # this will be created by p4harmonize
new_client_root = "d:/p4/local/harmonize" # this will be created by p4harmonize
new_client_stream = "//test/engine_epic"  # this needs to already exist

p4harmonize connects to each server, requests file lists from each, and determines what work needs to be done. If everything is already in sync, then it quickly reports the status and stops. If there is work to be done, then it creates a changelist and begins adding its fixes to it.

While it runs, it outputs status updates and every individual p4 command it is running so you can follow along.

When it is done, there will be a changelist that must be submitted by hand, giving you a chance to verify the work.

p4harmonize will never submit a changelist on its own.

Runtime requirements

p4harmonize requires the following commands to be in your path:

Development setup

  • Ensure you have all the Runtime requirements
  • Ensure you have a recent version of Go (1.19+)
    • Ensure that $GOPATH/bin is added to your $PATH environment variable.
  • To run all the tests, you'll want to have Docker installed
  • Clone the github repo:
    git clone https://github.com/danbrakeley/p4harmonize.git
    
  • Open a bash session (works with git-bash on Windows) in the p4harmonize folder that git created in the previous step and run scripts/setup.sh
    • This ensures Go is installed and ready
    • This builds Mage into your $GOPATH/bin folder

Build/run

p4harmonize uses Mage to automate building and testing tasks. If you've never used Mage before, running mage with no arguments will display all the build targets found in the magefile.go file in the current folder. It should look something like this:

$ mage
Targets:
  build       tests and builds the app (output goes to "local" folder)
  longTest    runs a fresh build of p4harmonize against test files in docker-hosted perforce servers.
  run         runs unit tests, builds, and runs the app
  testDown    brings down and removes the docker containers started by TestUp.
  testUp      brings up fresh perforce servers via Docker, each with a super user named "super" (no password).

Mage usage notes

  • If your shell can't find mage
    • make sure your $GOPATH/bin folder is in your $PATH
    • make sure there's a mage executable in your $GOPATH/bin folder
    • if mage is missing, then re-run scripts/setup.sh
  • Mage targets are not case sensitive, so mage longTest and mage longtest will run the same target.

Contributing

Before opening a pull request, please run mage longtest and make sure it is passing.

If your PR includes performance improvements, please include some benchmarks that show the benefits of your changes.

Special Thanks!

Thanks to Bohdon Sayre and Jørgen P. Tjernø for contributing time and code to help me fix my bugs and dramatically improve performance!

TODO:

  • run longtest via github actions?
  • test on a Mac (maybe with github actions?)

p4harmonize's People

Contributors

danbrakeley avatar jorgenpt avatar bohdon avatar

Stargazers

Marwan Hilmi avatar  avatar wzl avatar James Burns avatar  avatar  avatar Sean Saleh avatar Per Hugoson avatar Fabien Poupineau avatar Chen Tao avatar Vitor Brito avatar Phi Hung Nguyen avatar  avatar scorza avatar  avatar Joe Mukai avatar Felipe Reis avatar ray avatar

Watchers

 avatar James Cloos avatar Joe Mukai avatar  avatar

Forkers

bohdon jorgenpt perh

p4harmonize's Issues

Odd quoting in output

It looks like all the double quotes in p4harmonize's output are escaped in the log output, which is a little confusing. For example:

2023.12.15-20:13:45   --> Exec: p4 -p ssl:perforce.example.com:1666 -u jorgen -c ec2_ondemand_engine_sync_harmonize -x \"/tmp/p4harmonize_delete_3938411009.txt\" delete -c 45400
2023.12.15-20:13:45   --> Exec: p4 -p ssl:perforce.example.com:1666 -u jorgen -c ec2_ondemand_engine_sync_harmonize add -c 45400 -t text -I -f \"/home/ubuntu/ws/org-harmonize/Engine/Source/ThirdParty/CEF3/Build/linux/Toolchain-clang.cmake\"
2023.12.15-20:13:45   --> //proj/engine/Engine/Source/ThirdParty/CEF3/Build/linux/Toolchain-clang.cmake#3 - opened for add
2023.12.15-20:13:45   --> Exec: p4 -p ssl:perforce.example.com:1666 -u jorgen -c ec2_ondemand_engine_sync_harmonize revert -a -c 45400 \"/home/ubuntu/ws/org-harmonize/...\"
2023.12.15-20:13:45 Success! All changes are waiting in CL #45400. Please review and submit when ready.
2023.12.15-20:13:45 Remember to delete workspace \"ec2_ondemand_engine_sync_harmonize\"
2023.12.15-20:13:45 and local folder \"/home/ubuntu/ws/org-harmonize\"

Both the arguments to p4 delete are escaped in the terminal and the workspace name & local folder in the final status message, along with other messages. I wouldn't expect backslashes here -- it makes it hard to copy-paste the output to e.g. run a command manually or copy the path name.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.