Giter Club home page Giter Club logo

buildout.plonetest's Introduction

This repository contains a number of configuration files for using zc.buildout to quickly set up a testing/development environment for your package.

A good example of how to use this, is bobtemplates.plone. Follow the instructions there to create a Plone add-on, and look at the created files, especially the various buildout configs and .travis.yml.

The intended usage is to create a buildout.cfg like:

[buildout]
extends = https://raw.githubusercontent.com/collective/buildout.plonetest/master/test-5.x.cfg
package-name = plone.app.foo

Create a virtualenv and run buildout. This should give you a bin/test script, which can be used to run your package's tests. If your tests have additional dependencies, you should declare them via the extras_require parameter of setuptools.setup and refer to the key used using the package-extras variable:

[buildout]
extends = https://raw.githubusercontent.com/collective/buildout.plonetest/master/test-5.x.cfg
package-name = plone.app.foo
package-extras = [test]

If you want to do continuous integration with Travis CI you can use this same buildout config. And a .travis.yml like:

version: ~> 1.0
import: collective/buildout.plonetest:travis/default.yml
python: "2.7"
env: PLONE_VERSION=5.2

Attention!

This repository also has travis-*.cfg files, that try to be faster by downloading the Plone universal installer. This is no longer recommended, because Travis meanwhile has better caching support. Only the Travis config files for Plone 5.1 and earlier have been kept, to avoid breaking add-ons. In your own buildout config, replace travis-*.cfg with test-*.cfg and you should be fine. Also take over the cache setting from the .travis.yml file above.

Include the following in your buildout configuration to create an i18ndude helper script to update the po files of your product:

[buildout]
extends =
    https://raw.githubusercontent.com/collective/buildout.plonetest/master/test-5.x.cfg
    https://raw.githubusercontent.com/collective/buildout.plonetest/master/qa.cfg
package-name = plone.app.foo
package-extras = [test]
parts+=
    i18ndude
    rebuild_i18n-sh

After running bin/buildout you will find a bin/rebuild_i18n.sh; run the script and the po files will be updated.

Domain name is taken from the ${buildout:package-name} variable. The plone domain is also included.

Attention!

Originally, the rebuild_i18n-sh part used a filesystem template. This is deprecated and no longer updated.

Update your .travis.yml file with the following:

before_script:
  - export DISPLAY=:99.0
  - sh -e /etc/init.d/xvfb start

The buildout.cfg file in your package should look like this:

[buildout]
extends =
    https://raw.githubusercontent.com/collective/buildout.plonetest/master/test-5.x.cfg

package-name = collective.foo
package-extras = [test]

[versions]
setuptools = 41.2.0
zc.buildout = 2.13.2

The versions are optional, but they can help in avoiding restarts when buildout tries to upgrade itself to a version pinned in Plone. It is fine to start without them, but when you run into problems in Travis CI, consider adding them. In this way, all Plone versions use the same buildout and setuptools versions.

These versions match a requirements.txt like this:

setuptools==41.2.0
zc.buildout==2.13.2

The .travis.yml file should look like this:

version: ~> 1.0
import: collective/buildout.plonetest:travis/default.yml
matrix:
  include:
    - python: "2.7"
      env: PLONE_VERSION="4.3"
    - python: "2.7"
      env: PLONE_VERSION="5.1"
    - python: "2.7"
      env: PLONE_VERSION="5.2"
    - python: "3.7"
      env: PLONE_VERSION="5.2"

The trick here is to replace the extended configuration with the right one using the sed command.

Caution!

The following configurations are experimental and may change at any time.

If you want to add Quality Assurance to your continuous integration you can update your buildout.cfg file like:

[buildout]
extends =
    https://raw.githubusercontent.com/collective/buildout.plonetest/master/test-5.x.cfg
    https://raw.githubusercontent.com/collective/buildout.plonetest/master/qa.cfg
package-name = plone.app.foo
package-extras = [test]
package-min-coverage = 80
parts+=
    createcoverage
    coverage-sh
    code-analysis

and add and commit .coveragerc file (see example at https://github.com/plone/plone.recipe.codeanalysis/blob/master/.coveragerc)

and update your .travis.yml like:

language: python
python: 2.7
cache:
  pip: true
  directories:
    - eggs
env:
  - TARGET=test
  - TARGET=coverage.sh
before_install:
  - virtualenv -p `which python` .
  - bin/pip install -r requirements.txt
  - bin/buildout -N -t 3 annotate
install:
  - bin/buildout -N -t 3
script:
  - bin/$TARGET

buildout.plonetest's People

Contributors

agitator avatar ale-rt avatar annegilles avatar arhell avatar datakurre avatar do3cc avatar gforcada avatar gotcha avatar hvelarde avatar idgserpro avatar jensens avatar kroman0 avatar laulaz avatar mauritsvanrees avatar mrtango avatar pbauer avatar petschki avatar rodfersou avatar tdesvenain avatar thet avatar thomasmassmann avatar tisto avatar tomgross avatar toutpt avatar vincentfretin avatar wesleybl avatar witsch avatar zopyx avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

buildout.plonetest's Issues

Inconsistency between buildout.plonetest and buildout.jenkins test eggs

We were having some problems when testing some packages in our local infrastructure that we would like do discuss here in buildout.plonetest.

When running bin/buildout -n && bin/test with our buildout

[buildout]
extends =
    https://raw.github.com/collective/buildout.plonetest/master/test-4.3.x.cfg
    https://raw.github.com/collective/buildout.plonetest/master/qa.cfg
    https://raw.github.com/plone/plone.app.robotframework/master/versions.cfg

everything went fine. But when running bin/buildout -c jenkins.cfg -n && bin/jenkins-test with our jenkins.cfg:

[buildout]
extends =
    buildout.cfg
    https://raw.github.com/plone/buildout.jenkins/master/jenkins.cfg

We got "ImportError: No module named CMFPlacefulWorkflow". Although this is fixed in plone.app.upgrade in plone/plone.app.upgrade@0da7c6c, we were trying to figure out why the error was only happening when using jenkins.cfg and not when using cfgs from buildout.plonetest alone.

It seems the inconsistency is that in https://github.com/collective/buildout.plonetest/blob/master/test-4.3.x.cfg we have Plone and plone.app.upgrade in eggs:

[test]
recipe = zc.recipe.testrunner
defaults = ['-s', '${buildout:package-name}', '--auto-color', '--auto-progress']
eggs =
    Plone
    plone.app.upgrade
   (...)

...but we don't have them in jenkins.cfg. And adding them to the namespace is really what the fix is about (since Plone has Products.CMFPlacefulWorkflow in setup.py).

This is a tricky subject, when asking @tomgross about "easing" needed imported stuff in plone.app.testing, he convinced us that "the general philosophy of unittests is to start with the minimal possible configuration and add stuff as needed" is the best approach, so we're opening this issue to try solve this inconsistency between plonetest and jenkins. Our questions:

  • Should we remove "Plone" and "plone.app.upgrade" in https://github.com/collective/buildout.plonetest/blob/master/test-4.3.x.cfg to follow that principle?
  • Should we add "Plone" as dependency in every plone product we have in install_requires at setup.py, and suggest that as a good practice in docs?
  • Should we add "Plone" as dependency only in extras_require['test'] and change all skeleton templates to follow it?
  • Should we add "Plone" just in our buildout.cfg?
  • Should we add "Plone" to jenkins.cfg?

Following the testing principle, we vote for the first, specially because not having it in jenkins was what made it able to detect this dependency problem.

Still don't know the full implications of choosing the first one so we need your support.

Improve documentation to not use travis.cfg anymore

It should no longer be needed to use the various travis*.cfg files. Those files were meant as a speed-up, by downloading the unified installer. But Travis has better caching support since a few years. Probably the bobtemplates.plone setup can be used as basis.

But in my opinion, the travis*.cfg should stay, to avoid sudden breakage in add-ons. A deprecation warning at the top of these files could be good though.

See also issue #10.

Use option 'accept-buildout-test releases = true' the buildout to avoid problems with versions of recipes.

The version of zc.buildout used by Plone 4 is 1.7.1. In this version, by default, buildout uses final versions when installing recipes. But time to install the 'part' that uses the recipe it uses non-final versions (a1, b1, etc.).This causes inconsistencies if non-final version is not compatible with the final version.

An example this inconsistency is the plone.recipe.codeanalysis recipe. If you do not pining the plone.recipe.codeanalysis the buildout will download the final version (currently 1.1.1) when installing the recipe, but in time to install the part 'code-analysis', it will download the non-final version (2.0b1 currently). To see use in buildout:

[buildout]
extends =
    https://raw.github.com/collective/buildout.plonetest/master/test-4.3.x.cfg
    https://raw.github.com/collective/buildout.plonetest/master/qa.cfg
    https://raw.github.com/plone/plone.app.robotframework/master/versions.cfg
...

and do:

rm -rf /path/to/download-cache/dist/plone.recipe.codeanalysis-*
rm -rf /path/to/eggs-directory/eggs/plone.recipe.codeanalysis-*
./bin/buildout

The result is:

/home/user/cache-python/eggs/setuptools-18.0.1-py2.7.egg/pkg_resources/__init__.py:203: RuntimeWarning: You have iterated over the result of pkg_resources.parse_version. This is a legacy behavior which is inconsistent with the new version class introduced in setuptools 8.0. In most cases, conversion to a tuple is unnecessary. For comparison of versions, sort the Version instances directly. If you have another use case requiring the tuple, please file a bug with the setuptools project describing that need.
Develop: '/home/user/sites/mysite/zeocluster/src/myproduct/.'
/home/user/cache-python/eggs/setuptools-18.0.1-py2.7.egg/setuptools/dist.py:285: UserWarning: Normalizing '1.1.1.dev' to '1.1.1.dev0'
Getting distribution for 'plone.recipe.codeanalysis'.
warning: no previously-included files matching '*.pyc' found anywhere in distribution
Got plone.recipe.codeanalysis 1.1.1.
Uninstalling code-analysis.
Unused options for buildout: 'package-min-coverage'.
Updating instance.
Updating test.
Installing code-analysis.
Getting distribution for 'plone.recipe.codeanalysis'.
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '**/.DS_Store' found anywhere in distribution
warning: no previously-included files matching '**/__pycache__' found anywhere in distribution
Got plone.recipe.codeanalysis 2.0b1.
Generated script '/home/user/sites/mysite/zeocluster/src/myproduct/bin/code-analysis'.
Generated script '/home/user/sites/mysite/zeocluster/src/myproduct/bin/flake8'.
Generated script '/home/user/sites/mysite/zeocluster/src/myproduct/bin/code-analysis-flake8'.
Generated script '/home/user/sites/mysite/zeocluster/src/myproduct/bin/code-analysis-jshint'.
Generated script '/home/user/sites/mysite/zeocluster/src/myproduct/bin/code-analysis-jscs'.
Generated script '/home/user/sites/mysite/zeocluster/src/myproduct/bin/code-analysis-csslint'.
Generated script '/home/user/sites/mysite/zeocluster/src/myproduct/bin/code-analysis-zptlint'.
Generated script '/home/user/sites/mysite/zeocluster/src/myproduct/bin/code-analysis-deprecated-aliases'.
Generated script '/home/user/sites/mysite/zeocluster/src/myproduct/bin/code-analysis-clean-lines'.
Generated script '/home/user/sites/mysite/zeocluster/src/myproduct/bin/code-analysis-pep3101'.
Generated script '/home/user/sites/mysite/zeocluster/src/myproduct/bin/code-analysis-imports'.
Generated script '/home/user/sites/mysite/zeocluster/src/myproduct/bin/code-analysis-debug-statements'.
Generated script '/home/user/sites/mysite/zeocluster/src/myproduct/bin/code-analysis-find-untranslated'.
Generated script '/home/user/sites/mysite/zeocluster/src/myproduct/bin/code-analysis-hasattr'.
Unable to create git pre-commit hook, this does not seem to be a git repository.
Updating omelette.
omelette: (While processing egg setuptools) Package '_markerlib' is zipped.  Skipping.
omelette: (While processing egg setuptools) Package 'easy_install' is zipped.  Skipping.
omelette: (While processing egg setuptools) Package 'pkg_resources' is zipped.  Skipping.
omelette: (While processing egg setuptools) Package 'setuptools' is zipped.  Skipping.
Updating node.

Note that it downloads the two versions (1.1.1 when installing the recipe, and 2.0b1 to install the part 'code-analysis').These two versions are incompatible.

The source code-analysis-clean-lines is something like:

#!/home/user/pythons/python279-virtualenv/bin/python

import sys
sys.path[0:0] = [
    '/home/user/cache-python/eggs/plone.recipe.codeanalysis-2.0b1-py2.7.egg',
    '/home/user/cache-python/eggs/zc.recipe.egg-1.3.2-py2.7.egg',
    '/home/user/cache-python/eggs/zc.buildout-1.7.1-py2.7.egg',
    '/home/user/cache-python/eggs/setuptools-18.0.1-py2.7.egg',
    '/home/user/cache-python/eggs/flake8-2.4.1-py2.7.egg',
    '/home/user/cache-python/eggs/check_manifest-0.25-py2.7.egg',
    '/home/user/cache-python/eggs/mccabe-0.3.1-py2.7.egg',
    '/home/user/cache-python/eggs/pep8-1.5.7-py2.7.egg',
    '/home/user/cache-python/eggs/pyflakes-0.8.1-py2.7.egg',
    ]


import plone.recipe.codeanalysis

if __name__ == '__main__':
    sys.exit(plone.recipe.codeanalysis.code_analysis_clean_lines({...

Note that the egg used in sys.path is plone.recipe.codeanalysis-2.0b1-py2.7.egg. But plone.recipe.codeanalysis.code_analysis_clean_lines method does not exist in version 2.0b1 but in 1.1.1. When you run a ./bin/code-analysis-clean-lines get the error:

AttributeError: 'module' object has no attribute 'code_analysis_clean_lines'

This occurs in other code analysis scripts. And this inconsistency may occur in other recipes.

My suggestion is to use the option:

accept-buildout-test-releases = true

the buildout. With accept-buildout-test-releases = true buildout will download the non-final version when installing recipes, avoiding inconsistencies.

Plone 5 uses zc.buildout 2.2.5. In this version, the inconsistency above does not occur. However, by default, it downloads the final version (in the case above, 1.1.1). To use the no-final version, we would have to use the option:

prefer-final = false

Endless loop with test-4.x.cfg

When trying to install from this simple travis.cfg-file,

[buildout]
extends =
    https://raw.github.com/collective/buildout.plonetest/master/test-4.x.cfg
package-name = collective.blogging

installation ends in an endless loop:

$ bin/buildout -c travis.cfg
Upgraded:
  zc.buildout version 1.7.1,
  setuptools version 26.1.1;
restarting.
Upgraded:
  zc.buildout version 1.7.1,
  setuptools version 26.1.1;
restarting.
…

How tp reproduce

$ mkdir p4 ; cd p4
$ virtualenv -p `which python2.7` .
$ . bin/activate
$ python --version
Python 2.7.16
$ pip list
pip        19.3.1
setuptools 41.4.0
wheel      0.33.
$ pip install zc.buildout
$ bin/buildout -c travis.cfg

qa.cfg fails with test-4.x.cfg - using wrong version of configparser

When trying to install from this simple travis.cfg-file,

[buildout]
extends =
    https://raw.github.com/collective/buildout.plonetest/master/test-4.x.cfg
    https://raw.github.com/collective/buildout.plonetest/master/qa.cfg
package-name = collective.blogging

this fails with

Got flake8 3.3.0.
Getting distribution for 'check-manifest'.
Got check-manifest 0.40.
Getting distribution for 'configparser'.
/usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'use_scm_version'
  warnings.warn(msg)
zip_safe flag not set; analyzing archive contents...
backports.__init__: module references __path__
Installing configparser 4.0.2
Caused installation of a distribution:
UNKNOWN 0.0.0
with a different project name.
Installing configparser 4.0.2
Caused installation of a distribution:
UNKNOWN 0.0.0
with a different version.
Got None.
While:
  Installing.
  Getting section code-analysis.
  Initializing section code-analysis.
  Installing recipe plone.recipe.codeanalysis.
Error: There is a version conflict.
We already have: UNKNOWN 0.0.0
but flake8 3.3.0 requires 'configparser'.

How tp reproduce

$ mkdir p4 ; cd p4
$ virtualenv -p `which python2.7` .
$ . bin/activate
$ python --version
Python 2.7.16
# Work around https://github.com/collective/buildout.plonetest/issues/53
$ pip install --force-reinstall setuptools==26.1.1
$ pip install zc.buildout==1.7.1
$ bin/buildout -c travis.cfg

When removing the qa.cfg line, buildout passes (but qa parts are missing, of course).

Analyzes

This seems to be caused be configparser requiring a newer version for setuptool, which is able to read metadata from setup.cfg. In version 3.5.3 setup.py is the last version where the metadata included in setup.py.

Possibe solution

Add this to qa.cfg

[versions]
configparser=3.5.3

This might affect other versions, too if setuptools is pinned to some [setuptools version < 30.3.0 (8 Dec 2016)][https://setuptools.readthedocs.io/en/latest/setuptools.html#id51]

Add Pillow in Plone 4.2 eggs

Plone 4.2 uses plone.scale 1.2.2. plone.scale 1.2.2 uses the PIL but not declared as a dependency. When you run the tests with Plone 4.2 get the error:

...

  File "/home/user/cache-python/eggs/plone.scale-1.2.2-py2.7.egg/plone/scale/scale.py", line 2, in <module>
    import PIL.Image
ZopeXMLConfigurationError: File "/home/user/sites/brasil.gov.vcge/src/brasil/gov/vcge/configure.zcml", line 9.2-9.37
    ZopeXMLConfigurationError: File "/home/user/cache-python/eggs/collective.z3cform.widgets-1.0rc1-py2.7.egg/collective/z3cform/widgets/configure.zcml", line 13.4-13.45
    ZopeXMLConfigurationError: File "/home/user/cache-python/eggs/plone.app.dexterity-1.2.2-py2.7.egg/plone/app/dexterity/configure.zcml", line 17.4-17.52
    ZopeXMLConfigurationError: File "/home/user/cache-python/eggs/plone.formwidget.namedfile-1.0.5-py2.7.egg/plone/formwidget/namedfile/configure.zcml", line 10.4-10.41
    ZopeXMLConfigurationError: File "/home/user/cache-python/eggs/plone.namedfile-1.0.6-py2.7.egg/plone/namedfile/configure.zcml", line 25.4-25.74
    ZopeXMLConfigurationError: File "/home/user/cache-python/eggs/plone.namedfile-1.0.6-py2.7.egg/plone/namedfile/scaling.zcml", line 8.2-14.6
    ImportError: No module named PIL.Image

So, we most add Pillow in eggs of Plone 4.2.

buildout-cache doesn't seem to work if plone dependencies are declared

I am trying to use the 'buildout-cache' optimisation to speed up my travis builds. However, because my egg declares dependencies on plone eggs, and it is marked as 'develop', buildout downloads all the dependencies before it gets to the 'download' and 'install' parts of buildout-cache.cfg

The result is that I don't seem to get any speed benefit at all!

Am I doing something wrong?

Here is the travis buildout I am using:

[buildout]
extends = 
    https://raw.github.com/collective/buildout.plonetest/master/travis-4.x.cfg
package-name = ploneintranet.socialcontent
package-extras = [test]

parts +=
    flake8
    createcoverage

[flake8]
recipe = zc.recipe.egg
eggs = flake8
       ${instance:eggs}
entry-points =
    flake8=flake8.run:main

[createcoverage]
recipe = zc.recipe.egg
eggs = createcoverage

Pin zc.recipe.cmmi to 1.3.6.

Since zc.recipe.cmmi 2.0.0 in 2017/06/21, it asks for the newest zc.buildout release and not all installations can afford that, so let's pin it to 1.3.6. This version asks for at least zc.buildout 1.4, but since http://dist.plone.org/release/4.0-latest/versions.cfg asks for 1.4.4, no harm done.

For more info:

https://community.plone.org/t/zc-recipe-cmmi-2-0-0-released-today-21-06-2017-will-break-all-buildouts-from-plone-because-it-asks-for-zc-buildout-2-9-4-how-should-we-behave/4412/

GitHub Actions instead of Travis?

Since Travis is giving less free minutes to public repositories and teams, collective projects may need to switch to GitHub Actions. It would be useful if we can come up with a blueprint for that.

Might need changes in qa.cfg for i18ndude

Yesterday I released i18ndude 5. This requires zope.tal 4.3.0 as minimum. This may cause an error on Travis for Plone 4. See https://travis-ci.org/smcmahon/Products.PloneFormGen/jobs/348254762:

While:
  Installing i18ndude.
Error: The requirement ('zope.tal>=4.3.0') is not allowed by your [versions] constraint (3.5.2)

PloneFormGen branch 1.7-maintenance has this in travis.cfg:

[buildout]
extends =
    https://raw.github.com/collective/buildout.plonetest/master/travis-4.x.cfg
    https://raw.github.com/collective/buildout.plonetest/master/qa.cfg

I would expect that the i18ndude pin would be taken over from travis-4.x.cfg which should in the end take the pin from http://dist.plone.org/release/4.3-latest/versions.cfg, which actually pins it to 4.0.1. So I don't yet get why it gets the newer 5.0.1 version.

For now, I have fixed it in PloneFormGen by pinning i18ndude to 4.3.

If more builds from add-ons start failing, we may need to take action. It would be a pity to pin it to version 4.3 for all builds though: Plone 5 should get the latest.

QA pins

When using qa.cfg we miss quite a lot of pins.
That causes the GHA to break all of a sudden.

It took me quite a lot of time to add the proper pins that span through Plone 5.1/Python2.7 to Plone 6, so I am sharing this here:

Probably those pins should be added to the relevant plone-*.cfg files.

rebuild_i18n-sh throws an error if it doesn't have plone domain for translation in the package when running i18ndude sync

Create a new plone 4.3 package in plonecli (like teste.teste). Add the the part rebuild_i18n-sh and your package-name to the buildout section. Run rebuild_i18n-sh:

./src/teste/teste/locales/en/LC_MESSAGES/teste.teste.po: 0 added, 0 removed
No entries for domain "plone".Type i18ndude -h<Enter> to see the help.Warning: ./src/teste/teste/locales/*/LC_MESSAGES/plone.po is not a file or is ignored.I/O Error: [Errno 2] No such file or directory: './src/teste/teste/locales/plone.pot'Type i18ndude -h<Enter> to see the help.%

Since there's no plone domain, plone.pot is not created and this error is shown, with $? in the shell being 1. We need to check if plone.pot exists before calling sync:

for po in $LOCALES/*/LC_MESSAGES/plone.po; do

make it easier to test travis locally

If you have a buildout-cache like the following you avoid having to have new copies of packages you already have in your buildoutcache

[buildout]
plone-series =
basename = Plone-${versions:Plone}-UnifiedInstaller
filename = ${buildout:basename}.tgz
allow-hosts =
    *.plone.org
    *.python.org
    docutils.sourceforge.net
    effbot.org
    prdownloads.sourceforge.net
    *.google.com
    *.googlecode.com

[download]
recipe = hexagonit.recipe.download
url = https://launchpad.net/plone/${buildout:plone-series}/${versions:Plone}/+download/${buildout:filename}

[install]
recipe = collective.recipe.cmd
on_install = true
cmds = 
  tar jxvf ${download:location}/${buildout:basename}/packages/buildout-cache.tar.bz2 1>/dev/null
  mv -n ${buildout:directory}/buildout-cache/eggs/* ${buildout:eggs-directory}
  mv -n ${buildout:directory}/buildout-cache/downloads/* ${buildout:download-cache}


[versions]
# avoids CalledProcessError; see: https://github.com/collective/collective.recipe.cmd/issues/7
collective.recipe.cmd = 0.6

Why for plone domain we specify only $SOURCE/configure.zcml $SOURCE/profiles/default in rebuild_i18n-sh? (TODO: Search for profiles and *.zcml in the script)

For reference:

bin/i18ndude rebuild-pot --pot $LOCALES/plone.pot --create plone $merge_plone $SOURCE/configure.zcml $SOURCE/profiles/default

In https://maurits.vanrees.org/weblog/archive/2010/10/i18n-plone-4:

Domains in GenericSetup xml files
workflows: use the plone domain

is this why we only search for plone domain in the profile? Or there are more motivations for treating plone domain different from the package? Which situations are you trying to avoid by being specific to configure.zcml and profiles/default?

bin/i18ndude rebuild-pot --pot $LOCALES/$I18NDOMAIN.pot $merge --create $I18NDOMAIN $SOURCE

TODO: #49 (comment)

Find correct locales folder for the project

Need to review this last change, the grep to filter recently added for the directory works because the '.' means anything in regex.

I can't imagine all regex possibilities, but the package name could add another bug in this script.

My suggestion is to use the -F option of grep to make it works without regex comparison.

Use --py3k for code-analysis using zc.recipe.command

If plone/plone.recipe.codeanalysis#219 isn't accepted, do you think it's feasible to create a hacky command in qa.cfg using a zc.recipe.command? A variable namedpylint-py3k-pre-commit-hook can be created in qa.cfg (false by default) and a little zc.recipe.command snippet:

[buildout]
index = https://pypi.org/simple/
package-min-coverage = 80
pylint-py3k-pre-commit-hook = false
parts +=
    code-analysis
    i18ndude
    pylint-py3k
    pylint-py3k-pre-commit-hook

(...)

[pylint-py3k]
recipe = zc.recipe.egg
eggs =
    pylint
    ${buildout:eggs}

[pylint-py3k-pre-commit-hook]
recipe = plone.recipe.command
stop-on-error = yes
command =
    pre_commit_file="${buildout:directory}/.git/hooks/pre-commit"
    if ${buildout:pylint-py3k-pre-commit-hook}; then
        if ! grep -q 'py3k' "$pre_commit_file"; then
            echo "\nbin/pylint --py3k --disable=no-absolute-import ${code-analysis:directory}" >> "$pre_commit_file"
        fi
    fi
    # XXX: Else is getting 'not found' when executed. Don't know why so I'm using two ifs.
    if ! ${buildout:pylint-py3k-pre-commit-hook}; then
        sed -i '/py3k/d' "$pre_commit_file"
    fi

Add precompile to qa.cfg?

Since in qa.cfg we already have "${buildout:package-name}", what about adding this part so packages that rely on this for testing can just add a precompile part after extending qa.cfg?

[precompile]
# Needed to create *.mo for tests.
recipe = plone.recipe.precompiler
eggs = ${buildout:package-name}
compile-mo-files = true

@mauritsvanrees thoughts?

tar: buildout-cache/eggs/setuptools-0.6c11-py2.7.egg: Cannot open: File exists

I got this issue on plone.app.multilingual & though I've got past it there, wonder if it might affect other packages using buildout-cache.cfg from here.

The error on Travis is:

Installing download.
Downloading https://launchpad.net/plone/4.3/4.3.2/+download/Plone-4.3.2-UnifiedInstaller.tgz
download: Extracting package to /home/travis/build/plone/plone.app.multilingual/parts/download
Installing install.
tar: buildout-cache/eggs/setuptools-0.6c11-py2.7.egg: Cannot open: File exists
tar: Exiting with failure status due to previous errors
While:
  Installing install.

full travis log is here

basically it seems python bootstrap was putting setuptools etc in buildout-cache/eggs/ and then tar xv doesn't handle overwriting files (at least on Ubuntu 12.04), so I rewrote the install part. Does this seem useful here?

Improve rebuild_i18n-sh for plone-manual.pot entries?

In some projects you may have manual entries to plone translation msgids, specially if you're customizing some views. In this situation, it's normal to have a plone-manual.pot that is merged with the plone.pot that is rebuilt when you run i18ndude. Some examples:

https://github.com/collective/ecreall.trashcan/blob/a33a327d62e9502f15a0907a593e68747b74a6d6/ecreall/trashcan/locales/update.sh

https://github.com/plone/plone.app.workflowmanager/blob/01ef8f6c581e593eb146bd45ad291d3939d8ceb5/rebuild_i18n.sh

If you don't use the manual approach and want to translate something in the Plone namespace and have a file in your package using plone domain, the msgids are removed from plone.pot, thus the need of a plone-manual.pot.

What do you think @mauritsvanrees? We can try adding this with a PR.

travis-5.x.cfg is broken

The build downloads 5.0a2 but tries to extract 5.0a3; this fails.

Downloading https://launchpad.net/plone/5.0/5.0a2/+download/Plone-5.0a2-UnifiedInstaller.tgz
download: Extracting package to /home/travis/build/cosent/plonesocial.activitystream/parts/download
Installing install.
tar (child): /home/travis/build/cosent/plonesocial.activitystream/parts/download/Plone-5.0a3-UnifiedInstaller/packages/buildout-cache.tar.bz2: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now

Full traceback see e.g. https://travis-ci.org/cosent/plonesocial.activitystream/builds/53361711
This is a package I just switched from 4.x to 5.x.

rebuild_i18n-sh doesn't work if I don't have a manual.pot in my package

Create a new plone package using bobtemplates.plone. (plonecli: 1.1 bobtemplates.plone: 4.0.4)

If I try to use the rebuild_i18n-sh part, it doesn't work.

This happens because when calling i18ndude, we take into account that manual.pot exists, but it's not always the case.

bin/i18ndude rebuild-pot --pot $LOCALES/$I18NDOMAIN.pot --merge $LOCALES/manual.pot --create $I18NDOMAIN $SOURCE

% bin/rebuild_i18n.sh
I/O Error: [Errno 2] No such file or directory: './locales/manual.pot'
Type i18ndude -h<Enter> to see the help.
./locales/en/LC_MESSAGES/my.package.po: 0 added, 0 removed

If I add an empty manual.pot as a workaround, it works:

% bin/rebuild_i18n.sh
./portal/locales/en/LC_MESSAGES/my.package.po: 22 added, 0 removed

assert newdist is not None # newloc above is missing our dist?!

When I use bootstrap.py on Plone 5 with a python without setuptools, the following error occurs if the cache is empty:

While:
  Installing.
  Getting section code-analysis.
  Initializing section code-analysis.
  Installing recipe plone.recipe.codeanalysis[recommended].
  Getting distribution for 'configparser'.

An internal error occurred due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
  File "/home/user/cache-python/eggs2/zc.buildout-2.13.2-py2.7.egg/zc/buildout/buildout.py", line 2174, in main
    getattr(buildout, command)(args)
  File "/home/user/cache-python/eggs2/zc.buildout-2.13.2-py2.7.egg/zc/buildout/buildout.py", line 701, in install
    [self[part]['recipe'] for part in install_parts]
  File "/home/user/cache-python/eggs2/zc.buildout-2.13.2-py2.7.egg/zc/buildout/buildout.py", line 1324, in __getitem__
    options._initialize()
  File "/home/user/cache-python/eggs2/zc.buildout-2.13.2-py2.7.egg/zc/buildout/buildout.py", line 1432, in _initialize
    self.initialize()
  File "/home/user/cache-python/eggs2/zc.buildout-2.13.2-py2.7.egg/zc/buildout/buildout.py", line 1438, in initialize
    recipe_class = _install_and_load(reqs, 'zc.buildout', entry, buildout)
  File "/home/user/cache-python/eggs2/zc.buildout-2.13.2-py2.7.egg/zc/buildout/buildout.py", line 1388, in _install_and_load
    allow_hosts=buildout._allow_hosts
  File "/home/user/cache-python/eggs2/zc.buildout-2.13.2-py2.7.egg/zc/buildout/easy_install.py", line 957, in install
    return installer.install(specs, working_set)
  File "/home/user/cache-python/eggs2/zc.buildout-2.13.2-py2.7.egg/zc/buildout/easy_install.py", line 730, in install
    for dist in self._get_dist(req, ws):
  File "/home/user/cache-python/eggs2/zc.buildout-2.13.2-py2.7.egg/zc/buildout/easy_install.py", line 574, in _get_dist
    dists = [_move_to_eggs_dir_and_compile(dist, self._dest)]
  File "/home/user/cache-python/eggs2/zc.buildout-2.13.2-py2.7.egg/zc/buildout/easy_install.py", line 1776, in _move_to_eggs_dir_and_compile
    assert newdist is not None  # newloc above is missing our dist?!
AssertionError

This is because of the use of newest = false made in PR #37. When we run buildout, the setuptools aren't updated, and is used version 33.1.1, "pinned" by ez_setup.py, and not the Plone 5 version. configparser requires newer versions of setuptools, giving the error.

Steps to reproduce:

git clone https://github.com/collective/collective.cover
cd collective.cover
vi buildout.cfg
# Change to Plone 5.1 in extends
virtualenv --no-setuptools .
./bin/python bootstrap.py
# simulate empty cache
./bin/buildout buildout:eggs-directory=/tmp/eggs buildout:download-cache=/tmp/downloads

I think we should not use newer = false

@mauritsvanrees @hvelarde

Add CI test for this very repo

When I tried to reproduce #53 only four month later, this failed. I suggest adding CI testing for this very repo - following the instructions from the readme - to detect issues due to e.g. changes to the exteneds files and to ensure one can actually set to a testing environment for the respective Plone versions.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.