Private developer build facility

Table of Contents:

• [What is pbuild] (#what-is-pbuild)
• [Assumptions] (#assumptions)
• [Command Quialifiers to pbuild] (#command-qualifiers-to-pbuild)
• [Environment Variables] (#environment-variables)
• [Valid Projects] (#valid-projects)
• [Settings that Modify Behavior] (#settings-that-modify-behavior)
• [Per-Project Configuration Options] (#per-project-configuration-options)
• [Output description for Progress Setting] (#output-description-for-progress-setting)
• [Support for testrun attributes and names] (#support-for-testrun-attributes-and-names)
• [Keyboard Input] (#keyboard-input)
• [Code of Conduct] (#code-of-conduct)

pbuild (private build) is a utility that allows a developer to verify code changes compile and run unit tests on a wide variety of different platforms simultaneously.

pbuild will allow you to do a build across all our platforms using your git clones under your own accounts. So, since you’re using your own account, if you have a problem on a host, you simply need to go to the appropriate directory and everything is left intact to work on resolving the problem. pbuild supports checking out a branch in git if you ask it to, and making a target of your choice (default target is 'testrun'). There are other options as well. pbuild supports running on any or all of your development hosts.

pbuild is driven by an initialization file to define the hosts to use. A sample configuration file can be found in the same location as pbuild itself, and should contain config_sample with either config_hosts_core or config_hosts_nip appended to that, depending on what s closer to the machine list that you want. The configuration file is normally stored in ~/.pbuild, but this can be customized via environment variable PBUILD.

pbuild, by default, stores log files from the builds in your home directory (~/). This can be customized via the configuration file. See the sample configuration file for details.

pbuild fully supports terminating the builds via ^C. Upon termination, pbuild will kill all child processes, causing remote build operations to terminate.

To run pbuild, you can do "python pbuild.py" with any command line options.

While running, pbuild will show status like this:

Tag                           Host Name                     Status

aix_5.3                       scxaix1                       Done (18:55)
aix_6.1                       scxaix3-3                     Done (23:30)
hp_v2_ia64                    scxhpi10                      Done (21:45)
hp_v2_risc                    scxhpr3                       Done (24:55)
hp_v3_ia64                    scxhpi1
hp_v3_risc                    scxhpr2                       Done (27:00)
mac_10.5                      scx-mac04                     Done (07:15)
redhat_4_x86                  scxrhel4-01                   Done (23:50)
redhat_5_x86                  scxcore-rhel50-01             Done (11:55)
sun_5.10_sparc                scxsun12                      Done (24:40)
sun_5.10_x86                  scxcore-solx86-01             Done (23:50)
sun_5.8_sparc                 scxsun03-s8
sun_5.9_sparc                 scxsun14                      Done (24:30)
suse_10_x86                   scxcore-suse01                Done (12:35)
suse_9_x86                    scxsles9-01

Elapsed Time:  27:45

In this example, hosts scxhpi1, scxsun03-s8, and scxsles9-01 are still building. All other hosts have completed (with time to complete). If a build fails, status is "Failed" (bolded).

###Assumptions:

1. For simplicity, pbuild assumes that you're using certificates everywhere. This means that you can type "ssh " from one Unix/Linux host to go directly to some other Unix/Linux host without username/password prompts.
   
   pbuild, when starting, will verify that all required SSH keys are stored in your ~/.ssh/known_hosts file. If you have modified your configuration file, pbuild will run this verification step again to double check that all your hosts can be reached. This can be forced with --initialize.

2. pbuild assumes that you're using 'bash' as your shell. If important, it can be changed to support other shells. But to date it has no such support.

Please do pbuild --help for latest/greatest information:

Usage: pbuild.py [option-list] [host-list]   (use --help or -h for help)

pbuild will allow you to do a build across all platforms using your own git
clones. If a build problem occurs, all files are left completely intact,
simplifying the steps you must perform to replicate the problem. pbuild
supports a wide variety of options, specified below.  For more information,
view the README file checked in with pbuild.

Options:
  -h, --help            show this help message and exit
  --abortOnError        Immediately aborts when the first remote build fails
                        (for future - do not use yet)
  --attributes=TEST_ATTRS
                        Specifies the unit test attributes that you wish to
                        use to restrict unit tests
  -b BRANCH, --branch=BRANCH
                        Selects the branch for the top level project or
                        superproject
  -d, --debug           Build targets in DEBUG mode
  --clone               Forces a new clone of the repository, even if
                        repository already exists
  --command=COMMAND     Executes the specified command string rather than
                        buiding/running a command script to perform a remote
                        build
  --exclude=EXCLUDE     Overrides default exclude list from configuration file
                        (if any); comma-separated list of hosts to exclude
                        from the build
  --initialize          Verify public keys in known_hosts file
  -l, --list            List host configuration information and exit
  --logdir=LOGDIR       Overrides 'logdir' from configuration file
  --logdir_prior=LOGDIR_PRIOR
                        Overrides 'logdir_prior' from configuration file
  --nodebug             Build targets in NODEBUG mode
  --nocurses            Disable curses for dynamic screen updating (may be
                        useful for diagnostic purposes)
  --select=SELECT       Select specification to build (only build hosts with
                        this select specification)
  --settings=SETTINGS   Overrides default settings from program and
                        configuration file (i.e. 'ShowSummary,LogFile')
  -s SUBPROJECT, --subproject=SUBPROJECT
                        Comma-separated list of subproject:branch pairs to
                        select branches in subprojects, like "opsmgr:jeff-
                        sun,pal:jeff-sun"
  -t TARGET, --target=TARGET
                        Specifies the build target to use for the 'make'
                        command (default: 'testrun').  Set target as empty
                        string to avoid the 'make' step (useful to, say, undo
                        changes on a host)
  --tests=TESTS         Specifies the subset of unit tests that you wish to
                        run (if the target includes "testrun"); may be a
                        comma-separated list

Environment variables that modify and control PBUILD behavior:

Projects are specified in per-project configuration options as well as host entries. Valid projects are defined as follows:

Settings are set to "reasonable defaults for most people" automatically. By that, I mean that settings are the most logical and trouble-free settings, but not necessarily the "nicest" settings.

Settings can be overridden by the configuration file, or by the command line qualifier "--settings". A setting can be prefixed by "no" to disable it; if it is not prefixed by "no", then the setting is enabled.

Settings that can be controlled:

Default settings are:

CheckValidity, Debug, DeleteLogfiles, NoDiagnoseErrors, NoLogfileRename, NoLogfileSelect, Progress, SummaryScreen

PBUILD supports per-project configuration options. This is useful if you need to set project-specific options. Two such options are currently supported:

make_target : project : options
configure_options : project : options

Spaces around the : separators in the above example are optional. Note that fields (project or option fields) must not contain any : characters.

If you run pbuild with the Progress setting (the default), then pbuild will display a line under the status column like:

  - <operation> <lines>

For example, you may see a line like:

  - make pal (619)

This means that the build is currently executing the make pal step, and that 619 lines of log text have been received since the make pal step was started.

Note that if the first byte is a ? rather than a -, as in:

  ? make pal (619)

this means that no log text has been received for 31 seconds or more (in other words, the host does not appear to be responding). Once the host performs additional work, the ? will change back to a -.

Qualifier -attributes can be used to only run certain tests with attributes set in the test source files. Prepend a - to include the opposite setting.

Today, only attribute SLOW is defined in tests, and is defined to be any test that takes "a while" to run.

For example: -attributes=-SLOW will only run tests that are not marked as slow tests

Qualifer -tests can be used to restrict tests that are run, or to ignore tests that are run. Prepend a - to a test name to ignore a certain test.

For example: -tests=-atomic will run all tests except those with atomic in their name. -tests=atomic,condition will only run tests with atomic or condition in their names

Configuration tag test_attributes will form default attribute specification. Configuration tag test_list will form the default unit test specification.

Once every poll interval the keyboard is polled for input. The following characters are recognized:

This project has adopted the [Microsoft Open Source Code of Conduct] (https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ] (https://opensource.microsoft.com/codeofconduct/faq/) or contact [email protected] with any additional questions or comments.