The script supports building given list of sketches for a given set of boards. The builds can happen multiple times (for different compilers, settings, etc.) producing multiple buildsets.
Furthermore, the script can output the build results, including comparisons between buildsets, in CSV format that can be processed using e.g., Google Fusion Tables, Excel, etc.
To build sketches, run the build
subcommand. Pass the .ino filename
(just one per sketch for sketches with multiple .ino files) of each
sketch as a commandline argument.
By default, it builds for the Arduino uno. Use the --boards
option to
specify other boards.
For example:
arduino-mass-builder.py build --boards arduino:avr:uno,arduino:avr:leonardo \
01.Basics/BareMinimum/BareMinimum.ino 01.Basics/Blink/Blink.ino
Build results are placed in the result
directory by default, pass
--results-dir
to put them somewhere else.
You can run this command mutiple times with different sketches or boards and the results will just be merged together.
Add -v
or --verbose
before the build
command to enable verbose
mode.
See arduino-mass-builder.py --help
and arduino-mass-builder.py build --help
for more options.
To create a report (CSV file) from existing build results, run the
report
subcommand.
For example:
arduino-mass-builder.py report
Build results are read from the result
directory by default, pass
--results-dir
to get them somewhere else. The CSV file is also output
into that directory, in the report
subdirectory.
The CSV generated by the build example above looks like this (extra whitespace added for display here):
Buildset , Sketch , Board , Status , Program size , Data size
base , 01.Basics/BareMinimum , arduino:avr:leonardo , OK , 4262 , 157
base , 01.Basics/BareMinimum , arduino:avr:uno , OK , 450 , 9
base , 01.Basics/Blink , arduino:avr:leonardo , OK , 4798 , 157
base , 01.Basics/Blink , arduino:avr:uno , OK , 1030 , 9
See arduino-mass-builder.py --help
and arduino-mass-builder.py report --help
for more options.
Sometimes, you might want to build the same set of sketches, for the same set of boards multiple times, but with changes in the Arduino IDE or core files applied. To supports this, the script can distinguish between multiple "buildsets".
You can specify which buildset a build belongs to by using the
--buildset
option. If not specified, the buildset name is set to
base
.
For example, to find out the effects of a different compile flags, we modified
platform.txt to use -O3
instead of -Os
. Then, we ran another buildset:
arduino-mass-builder.py build --buildset o3 --boards arduino:avr:uno,arduino:avr:leonardo \
01.Basics/BareMinimum/BareMinimum.ino 01.Basics/Blink/Blink.ino
After regenerating the report:
arduino-mass-builder.py report
The CVS now looks like this. Note that there are now delta columns in there.
Also note that everything compares with the base
buildset by default, this
can be changed using the --base-set
option.
Buildset , Sketch , Board , Status , Program size , Data size , Δ status , Δ program size , Δ data size , In base buildset
base , 01.Basics/BareMinimum , arduino:avr:leonardo , OK , 4262 , 157 , Is base , 0 , 0 , Yes
base , 01.Basics/BareMinimum , arduino:avr:uno , OK , 450 , 9 , Is base , 0 , 0 , Yes
base , 01.Basics/Blink , arduino:avr:leonardo , OK , 4798 , 157 , Is base , 0 , 0 , Yes
base , 01.Basics/Blink , arduino:avr:uno , OK , 1030 , 9 , Is base , 0 , 0 , Yes
o3 , 01.Basics/BareMinimum , arduino:avr:leonardo , OK , 5052 , 157 , Modified , 790 , 0 , No
o3 , 01.Basics/BareMinimum , arduino:avr:uno , OK , 460 , 9 , Modified , 10 , 0 , No
o3 , 01.Basics/Blink , arduino:avr:leonardo , OK , 5626 , 157 , Modified , 828 , 0 , No
o3 , 01.Basics/Blink , arduino:avr:uno , OK , 1146 , 9 , Modified , 116 , 0 , No
This shows that adding -O3
changes the resulting binaries
("Modified"), and increases the program size (increase is a lot bigger
for the leonardo), but doesn't affect the RAM size.
To visualize the results, the data can be imported into Google Fusion Tables, or plain Excel. As an example for Fusion Tables, see this document. To re-use the same graphs, you can best make a copy of the document, clear out the data (Edit -> Delete all rows) and import your own (File -> Import more rows).
This script and accompanying docs are written by Matthijs Kooijman, and are made available under the MIT license.
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.