buildfoundation / mainframer Goto Github PK

Great project! The first idea that came to my mind is: how much time I will save? Please, publish a simple comparison in the readme. Let's say "a project that takes 2 minutes to be assebled on my computer, takes 1 minute to be built in a server with 4 cores and 4 GB's". Or... close this issue if you think that it doesn't have sense :-)

Thanks for the work!

https://twitter.com/ckesc/status/812006350385541120

This will help keeping remote home directory ~ clean and easy to navigate through in case of many projects.

Hey there!

First of all I want to start by saying what a wonderful tool this is. It will save us hours if not days in compile time!

We support a project with a large number of build variants and it's a bit annoying to have to edit the run configuration each time we want to switch variants. I had a short-term solution of using an Android Studio macro $Prompt$ to prompt the user which variant they wanted to build, but that must be done each launch.

I dug through the Android Studio files and I found that the module's .iml file contains the build variant. Since the .iml file is XML, we can easily parse out the build variant for our own use.

Here's some quick and dirty parsing code I wrote:

xmllint --xpath "string(//option[@name='ASSEMBLE_TASK_NAME']/@value)" app/app.iml

Output

assemble[my flavor]Debug

From here, we can pass this to the mainframer.sh script. Here's what I have:

Parameters:

-c "bash mainframer.sh ./gradlew :app:$(xmllint --xpath \"string(//option[@name='ASSEMBLE_TASK_NAME']/@value)\" app/app.iml) -Pandroid.enableBuildCache=true -PminSdk=23"

Hopefully this is of use to someone else! Thanks!

rm, tar, etc, let's try to parallel everything that is possible to parallel.

https://github.com/koalaman/shellcheck

At the moment script does not work with uncompressed tars. If you're able to work with remote-build-machine at speeds near to 1 GB/s you actually lose time on compression.

lines like
--exclude='artifacts' \
or
--exclude='kotlin' \
show that it is still bound to a particular project.

It might me a good idea to extract it.

As separate files?
.localignore
.remoteignore
?

Example ignore:

sample

Will ignore not only root (in the project) file or directory with name sample, but also any file or directory with name sample in any level of subfolders in the project.

This is not how file/directory patterns work in .gitignore and other commonly used tools.

Currently we use awk in a way that sometimes leads to wrong results, for example we will read even commented out properties like:

#remote_build.machine=value1
remote_build.machine=value2

Both values will be grepped for remote_build.machine.

Since 2.x will already be breaking #19, we can also extract mainframer config parameters to a separate file since Android Studio adds not very funny header to local.properties (though I've never saw such behavior):

## This file is automatically generated by Android Studio.
# Do not modify this file -- YOUR CHANGES WILL BE ERASED!

Something like .mainframerconfig should be fine I think.

When remote host uncompressing tar it logs a lot of warnings like

tar: Ignoring unknown extended header keyword 'SCHILY.dev'
tar: Ignoring unknown extended header keyword 'SCHILY.ino'
tar: Ignoring unknown extended header keyword 'SCHILY.nlink'

More about this problem:
http://superuser.com/questions/318809/linux-os-x-tar-incompatibility-tarballs-created-on-os-x-give-errors-when-unt

Solution:
use GNU tar (gtar) in mainframer

My remote machine skd path is different than my local one.
I would like to be able to set the remote sdk path separately so my local machine will still work as intended.

For version 1.1.1 I passed the sdk path as an argument 'ANDROID_HOME=path/to/sdk'
Since 1.1.2 this workaround no longer work

Maybe even print mainframer v1.0.2 or something like this in the beginning, this will help investigate reports with and without logs from our users.

pigz works in parallel and in our case does compression faster for a large number of output files.

We can try to ssh to the same machine that runs CI build (Travis, CircleCI, etc) and then run some Gradle build and check its results.

The PR #47 introduce the use of grep to parse the local.properties file, but it fail to open the file when the path get a space in it.

bash -n scriptname

Hey guys, it's being really amazing how much my old 2010 MBP is not freezing anymore due to this project and remote builds, thanks for everything 👍

Does anyone was able to run the project using the feature of Instant Run? I think it would be a really great feature, since it does reduce a lot the time of builds (at least for me is pretty evident the difference).

I have checked some discussions around, and have found this one (https://code.google.com/p/android/issues/detail?id=199609) in Google Discussions, saying that they don't give support to Instant Run outside of the Android Studio IDE. Does anyone know some workaround for this?

Thanks for the help, and one more time for the amazing project!

Currently we use project_for_remote_build.tar as a name of archive which is not great if you'll try to build several project in parallel using same remote user.

So IDE won't have a long pause while no build files are available.

In v2.x we'll add several files #19, #51 to allow customizations and integrations with basically any build system, also we're planning to move away from local.properties #50.

With that in mind, there is an option to put all mainframer configs into a separate folder (eg mainframer) where we could store all required files:

• ignorelocal (should be under VCS in most cases)
• ignoreremote(should be under VCS in most cases)
• personalconfig (or config (should NOT be under VCS in most cases)

Similar to:

if [ -z "$BUILD_COMMAND" ]; then
	echo "Please pass build command as argument(s) to the script."
	exit 1
fi

My output is...

........................
BUILD SUCCESSFUL

Total time: 2 mins 39.252 secs

• tar -c --exclude=kotlin --exclude=tmp build/ '*/build'
  tar: */build: Cannot stat: No such file or directory
  tar: Exiting with failure status due to previous errors

Process finished with exit code 2

And apk not installs on device

Though in our case we generate about 10k build files on each build and rsync might be slower than simple tar + scp.

See this discussion for more info: https://twitter.com/steventannz/status/811460935101493248