https://github.com/marco-c/crashcorrelations/blob/master/addons.py

Buildhub is the canonical source for product/version/build information:

https://mozilla-services.github.io/buildhub/

Much of what ProductVersions is doing should be done with Buildhub. ProductVersions is deprecated and slated to be removed soon.

If users want to access product/version/build information through libmozdata, then adding support for Buildhub seems prudent.

Waiting to get back admin access to the repo and then I can enable Travis.

The phabricator client should support blocking all write calls (mostly Harbormaster & comments) to be able to cleanly use it in shadow mode.

The new service should be at activedata-public.devsvcprod.mozaws.net.

It's deprecated.

We can do this easily without polluting bugzilla.mozilla.org by using the new class introduced in #46.

libmozdata uses Buildhub:

Buildhub is deprecated and will be decommissioned in July. See mozilla-services/buildhub#532 for details.

There are API docs here:

https://buildhub2.readthedocs.io/en/latest/user.html

This issue covers migrating pushlog-addon to Buildhub2.

We're opening this issue because your project has used Travis CI within the last 6 months. If you have already migrated off it, you can close and ignore this issue.

Travis CI is ending free builds on public repositories. travis-ci.com stopped providingthem in early November, and travis-ci.org will stop after December 31, 2020. To avoid disruptions to your workflows, you must migrate to another CI service.

For production use cases, we recommend switching to CircleCI. This service is already widely used within Mozilla. There is a guide to migrating from Travis CI to CircleCI available here.

For non production use cases, we recommend either CircleCI or Github Actions. There is a guide to migrating from Travis CI to Github Actions available here. Github Actions usage within Mozilla is new, and you will have to work with our github administrators to enable specific actions following this process.

If you have any questions, reach out in #github-admin:mozilla.org on matrix.

FYI: The following changes were made to this repository's wiki:

• defacing spam has been removed

• Restricting write access to contributors is strongly encouraged. Please make that change (documentation).

These were made as the result of a recent automated defacement of publically writeable wikis.

We only use a subset of the fields, we don't need to get them all from Bugzilla.

The fields are: bug_id, creator, raw_text, id, author, tags, text, is_private, time, creation_time, attachment_id.

This is a problem that occurs very often in many of our projects. Having a shared mapping file will make it easier for everyone.

Users of the library don't care about the tests.

https://product-details.mozilla.org/1.0/firefox_versions.json now contains FIREFOX_NIGHTLY, we should use it instead of doing FIREFOX_AURORA + 1.

ProductVersions is deprecated and I'm removing it in the next week or two. We should remove it from libmozdata, too.

During the merge days, we might end up with the version for Aurora being the same for 'Beta'.

Traceback (most recent call last):

  File "/home/travis/build/mozilla/libmozdata/tests/test_versions.py", line 16, in test_versions

    self.assertTrue(v['esr'] <= v['release'] <= v['beta'] < v['aurora'] < v['nightly'])

AssertionError: False is not true

We should modify that assertion to always use <= instead of <.

Root cause of mozilla/release-services#1698

Our upliftbot currently fails to process 100% of Bugzilla uplift request comments:

uplift_bot.sync: Failed to add bug #1411681 : Invalid response from post https://uplift.shipit.mozilla-releng.net/bugs : b'{"detail":"Missing uplift comment in analysis","instance":"about:blank","status":500,"title":"Missing uplift comment in analysis","type":"about:blank"}\n'

I suspect this is due to the recent change in uplift request comment templates.

Comments now look like this: https://bugzilla.mozilla.org/show_bug.cgi?id=1510281#c4

The first line includes the text [Beta/Release Uplift Approval Request] or [ESR Uplift Approval Request], depending on the request.

I can see that libmozdata still expects the text Approval Request Comment, which is no longer used:

More details: https://bugzilla.mozilla.org/show_bug.cgi?id=1297966.

https://bugzilla.mozilla.org/show_bug.cgi?id=1263973 should be fixed first.

This should work better than our current approach of parsing the contents of the page https://wiki.mozilla.org/Modules/All.

When retrieving many fields, it makes sense to have 100 as the chunk size. When you are retrieving a subset of fields (e.g. only the ID), a larger chunk size would be better.

We can use the python-hglib Python library.

https://bugzilla.mozilla.org/show_bug.cgi?id=1475666

Probably best to use "default" everywhere since "tip" isn't all that well defined and predictable.

Regression from #141.

params is undefined at

They slow down the installation of libmozdata a lot and are only used in a couple of files.

This way when we already have a bug object with the required properties (resolution, dupe_of, etc.), we can avoid an unneeded query.

This optimization would be useful in the code to get the top crashes and related bugs here: https://github.com/calixteman/clouseau/blob/6a3d58887967093b6cfb06268feebc8822922a77/clouseau/stability/crashes.py#L178.

For example, for the beta channel on 2016-12-08.

    khours = Redash.get_khours(utils.get_date_ymd(start_date), utils.get_date_ymd(end_date), channel, versions, 'Firefox')
  File "/home/marco/Documenti/workspace/libmozdata/libmozdata/redash.py", line 132, in get_khours
    rows = Redash.__get_rows(channel, versions, rows)
  File "/home/marco/Documenti/workspace/libmozdata/libmozdata/redash.py", line 70, in __get_rows
    v = row['build_version']
KeyError: 'build_version'

bug_ids = # LIST OF 1500 BUGS    

Bugzilla.BUGZILLA_CHUNK_SIZE = 1000

bugs = {}

def bughandler(bug):
    bugs[bug["id"]] = bug["last_change_time"]

Bugzilla(
    bugids=bug_ids,
    bughandler=bughandler,
    include_fields=["id", "last_change_time"],
).get_data().wait()

print(len(bugs))

This snippets doesn't raise any error, but only 500 bugs are actually returned (as the first of the two chunked queries fails because the query URL is too long).

After https://bugzilla.mozilla.org/show_bug.cgi?id=1347465 is fixed, everything will be on p-d.

We should use: PUT /rest/bug/attachment/(attachment_id)

Here's the log of the Travis build which is failing to install libmozdata 0.1.17.

127187447.log.zip

As of January 1 2019, Mozilla requires that all GitHub projects include this CODE_OF_CONDUCT.md file in the project root. The file has two parts:

1. Required Text - All text under the headings Community Participation Guidelines and How to Report, are required, and should not be altered.
2. Optional Text - The Project Specific Etiquette heading provides a space to speak more specifically about ways people can work effectively and inclusively together. Some examples of those can be found on the Firefox Debugger project, and Common Voice. (The optional part is commented out in the raw template file, and will not be visible until you modify and uncomment that part.)

If you have any questions about this file, or Code of Conduct policies and procedures, please see Mozilla-GitHub-Standards or email [email protected].

(Message COC001)

import bugzilla
# change config

The token is not the one we've in the new config.

Some documentation here: https://wiki.mozilla.org/MozillaQualityAssurance:Build_Ids#For_any_build_of_Firefox.2C_SeaMonkey.2C_Thunderbird_or_Sunbird_which_you_are_about_to_download_from_the_ftp.mozilla.org_server.

E.g. https://hg.mozilla.org/mozilla-central/json-mozbuildinfo/829d3be6ba64?p=dom/media/MediaFormatReader.cpp or https://hg.mozilla.org/mozilla-central/json-mozbuildinfo/default?p=dom/media/MediaFormatReader.cpp.

Maybe in modules.py.

There are some flake8 issues which we need to fix:

$ flake8 .
./libmozdata/connection.py:101:17: E722 do not use bare except'
./libmozdata/handler.py:74:9: E741 ambiguous variable name 'l'
./tests/test_config.py:25:9: E722 do not use bare except'
./tests/test_config.py:30:9: E722 do not use bare except'
./tests/test_config.py:35:9: E722 do not use bare except'
./tests/test_config.py:40:9: E722 do not use bare except'
./tests/test_config.py:45:9: E722 do not use bare except'
./tests/test_config.py:51:9: E722 do not use bare except'
./tests/test_config.py:56:9: E722 do not use bare except'

CC @calixteman

Recently, we removed the Platforms API endpoint. It's no longer available and should get removed from libmozdata.

We removed ADI data from Socorro and the /api/ADI endpoint no longer works. libmozdata should be updated accordingly.

Home directory first, then current working directory, etc..

We should rename the file obviously (libmozdata.ini maybe?), because we don't want to force users to put a config.ini file in their home directory.

A lot of pending PRs open for years in the responses repo, requests-mock looks well maintained instead.

The current modules.json file is 4 years old. It needs to be updated. I couldn't figure out how it was generated in the first place, so it's unclear how to update it. Further, this module should be updated on a regular basis.

How was modules.json generated?

Is there a better source for this information? Could it or something like it be generated from moz.build files in mozilla-central and parked somewhere?

The test is prone to timeout. Maybe if we had mocks for it it wouldn't timeout so often.