mozilla / bugbot Goto Github PK
View Code? Open in Web Editor NEWA Mozilla release management tool to send reminders to Firefox developers and improve Bugzilla metadata
License: BSD 3-Clause "New" or "Revised" License
A Mozilla release management tool to send reminders to Firefox developers and improve Bugzilla metadata
License: BSD 3-Clause "New" or "Revised" License
Also:
* https://bugzil.la/1339859 - [meta] Very slow width resize action when multiple tabs are pinned using compact themes
- affected versions: 53, 54, 54
When the bug is closed and leave_open is still there, just automatically remove leave_open from the bug.
https://github.com/marco-c/bugbug uses several rules (based on keywords, flags and so on) to detect bugs vs non-bugs.
For the 380 bugs that are in that repository, its precision was 90%.
We should consider porting the urllib and urllib2 code to python requests perhaps this could be a contribution opportunity or good first bug.
Idea for an enhancement. For bugs which are unassigned, we could send the reminder emails to either the watch user for the bug's Bugzilla component, or to suggested reviewers for the component, or both. The watch user is probably a better idea here. I don't want to annoy our reviewers.
Release management should have alerts for bugs which are nominated for tracking which don’t have a decision after N days. N=0 for release (immediate alerting) N=0 for beta when we’re close to a release, but N=5 when there is not a release imminent.
We should have all the issues impacting 45 first, 46 second etc
For now, it is sorted by bug id
In bug 1180419, Marco was notified with this information:
== Bugs Tracked for ESR31 ==
* https://bugzil.la/1180419 - - (assigned to Marco Bonardo [::mak])
However, the bug was only affecting ESR38 and never ESR31. I guess this tool does not handle correctly the period when we have two ESR releases running in parallel.
Currently, we have:
== Notify release managers when bugs are marked fixed in nightly but still affected for aurora, beta or release ==
* https://bugzil.la/1116036 - - (assigned to Mark Finkle (:mfinkle))
* https://bugzil.la/1192910 - - (assigned to Kartikaya Gupta (email:[email protected]))
We should have
* https://bugzil.la/1116036 - - (assigned to Mark Finkle (:mfinkle))
Fixed 48, affected: 45, 46 and 47
So that release manager have a quick understanding of the issues affecting the releases.
Traceback (most recent call last):
File "/home/sylvestre/dev/mozilla/relman-auto-nag/auto_nag/scripts/email_nag.py", line 420, in <module>
cc_only=options.cc_only)
File "/home/sylvestre/dev/mozilla/relman-auto-nag/auto_nag/scripts/email_nag.py", line 105, in generateEmailOutput
'affected': bug.get_fx_affected_versions()
File "/home/sylvestre/dev/mozilla/relman-auto-nag/auto_nag/bugzilla/models.py", line 124, in get_fx_affected_versions
if self.api_data['cf_status_firefox' + version] == 'affected':
KeyError: u'cf_status_firefox59'
I guess it is because fx 59 flags have been removed too early
https://bugzilla.mozilla.org/show_bug.cgi?id=1459643
We should run:
Further prioritize bugs within groups by keyword tagging (sec/crash/mlk/perf/intermittend-failure) and tracking flag. This will increase the likelihood of a follow up. E.g. I want to know about topcrash fixes that have landed in Nightly and not in 47 before other non-sec, non-crash, non-regression bugs.
Per announcement from Gerv below we should consider porting bztools to the new native REST API in the future but perhaps this would be a good first bug for a contributor.
"This service is due to be decommissioned. Fortunately, Bugzilla has a new native REST API which works somewhat similarly that you could use instead:
https://wiki.mozilla.org/Bugzilla:REST_API
Moving to that new API would require some porting work. If that seems like too much effort, fear not, because there is also a BzAPI compatibility layer built on top of the new API. Because it's compatible, using that should simply require a change of API endpoint.
Change:
https://api-dev.bugzilla.mozilla.org/latest/
(or https://api-dev.bugzilla.mozilla.org/1.3/)
to
https://bugzilla.mozilla.org/bzapi/"
It should be categorized by release versions, so group everything into affecting:
Should be
https://bugzil.la/1455269
(https links on github don't have the protocol displayed)
We should add some numbers like:
This email contains all the bugs which has been fixed in nightly but still affected in other channels.
51 - X bugs
52 - Y bugs
53 - Z bugs
It's best practice to set an informative User Agent when making requests to an external API, and helps those maintaining the API track the sources of inefficient queries/heavy load :-)
https://github.com/mozilla/relman-auto-nag/blob/master/auto_nag/bugzilla/agents.py
When a bug has a patch and is affected the developer gets a nag email. Sometimes, that's a sec bug, and it is either waiting for sec-approval, or it has sec-approval but for a particular date in the future, often a couple of weeks after a release cycle starts. That means a developer might get daily nag emails on a security bug for several weeks, annoyingly and pointlessly.
Kind of a corner case here but for the devs who fix a lot of security issues, maybe we could fix this by parsing the whiteboard field and looking for something that looks like a check-in date. Example: [checkin on 6/26] . But it should not detect "checkin-needed".
This job is running on container-based infrastructure, which does not allow use of 'sudo', setuid and setguid executables.
If you require sudo, add 'sudo: required' to your .travis.yml
See http://docs.travis-ci.com/user/workers/container-based-infrastructure/ for details.
As travis suggests, we should move to the new infra.
In the report, include bugs that are fixed on Aurora channel but not on Beta to catch potential backouts-kind-of-bugs.
Traceback (most recent call last):
File "/home/sylvestre/dev/mozilla/relman-auto-nag/auto_nag/scripts/email_nag.py", line 451, in <module>
dryrun=options.dryrun)
File "/home/sylvestre/dev/mozilla/relman-auto-nag/auto_nag/mail.py", line 61, in sendMail
mailserver = smtplib.SMTP(SMTP, PORT)
File "/usr/lib/python2.7/smtplib.py", line 256, in __init__
(code, msg) = self.connect(host, port)
File "/usr/lib/python2.7/smtplib.py", line 318, in connect
(code, msg) = self.getreply()
File "/usr/lib/python2.7/smtplib.py", line 369, in getreply
raise SMTPServerDisconnected("Connection unexpectedly closed")
smtplib.SMTPServerDisconnected: Connection unexpectedly closed
git revert 9314a38
fixed my issue
For example, in https://bugzilla.mozilla.org/show_bug.cgi?id=1194706 , Liz had to ping the developer. The developer should receive a notification.
In the RelMan attention needed emails, it would be useful to have a link to this repo so we can easily report any issues or suggestions.
In https://github.com/mozilla/relman-auto-nag/tree/master/auto_nag/scripts, we have a bunch of scripts.
Because @calixteman and myself are lazy, we have been duplicating the code.
Now that we are using it more and more, we shouldn't be doing that anymore.
For example, https://github.com/mozilla/relman-auto-nag/blob/master/auto_nag/scripts/no_assignee.py#L118
get_email is duplicated, ditto for send_email, get_login_info, etc
We should create a common file providing these functions with argument to manage the differences (mostly strings it seems)
Flake8 is finding a bunch of issues wrt the python code. We should fix them: (but we don't care about the max line length check)
$ flake8 --max-line-length=10800 scripts/ bugzilla/ templates/
scripts/attach.py:8:1: F401 'Bug' imported but unused
scripts/attach.py:10:1: F401 'qs' imported but unused
scripts/attach.py:14:1: E302 expected 2 blank lines, found 1
scripts/attach.py:35:16: E128 continuation line under-indented for visual indent
scripts/attach.py:91:1: E302 expected 2 blank lines, found 1
scripts/automated_release_emails.py:23:9: E128 continuation line under-indented for visual indent
scripts/automated_release_emails.py:24:9: E128 continuation line under-indented for visual indent
scripts/automated_release_emails.py:25:9: E128 continuation line under-indented for visual indent
scripts/automated_release_emails.py:26:9: E128 continuation line under-indented for visual indent
scripts/automated_release_emails.py:27:9: E128 continuation line under-indented for visual indent
scripts/automated_release_emails.py:28:9: E128 continuation line under-indented for visual indent
scripts/b2g_query_creator.py:35:22: E261 at least two spaces before inline comment
scripts/b2g_query_creator.py:87:127: E231 missing whitespace after ','
scripts/b2g_query_creator.py:88:125: E231 missing whitespace after ','
scripts/b2g_query_creator.py:89:118: E231 missing whitespace after ','
scripts/b2g_query_creator.py:90:116: E231 missing whitespace after ','
scripts/b2g_query_creator.py:91:119: E231 missing whitespace after ','
scripts/b2g_query_creator.py:92:117: E231 missing whitespace after ','
scripts/b2g_query_creator.py:93:134: E231 missing whitespace after ','
scripts/b2g_query_creator.py:94:132: E231 missing whitespace after ','
scripts/email_nag.py:77:35: W601 .has_key() is deprecated, use 'in'
scripts/email_nag.py:88:31: W601 .has_key() is deprecated, use 'in'
scripts/email_nag.py:92:34: W601 .has_key() is deprecated, use 'in'
scripts/email_nag.py:100:25: E126 continuation line over-indented for hanging indent
scripts/email_nag.py:126:5: E303 too many blank lines (2)
scripts/email_nag.py:142:9: E128 continuation line under-indented for visual indent
scripts/email_nag.py:143:9: E128 continuation line under-indented for visual indent
scripts/email_nag.py:144:9: E128 continuation line under-indented for visual indent
scripts/email_nag.py:145:9: E128 continuation line under-indented for visual indent
scripts/email_nag.py:146:9: E128 continuation line under-indented for visual indent
scripts/email_nag.py:150:19: E231 missing whitespace after ','
scripts/email_nag.py:153:21: E231 missing whitespace after ','
scripts/email_nag.py:153:25: E231 missing whitespace after ','
scripts/email_nag.py:153:34: E231 missing whitespace after ','
scripts/email_nag.py:153:43: E231 missing whitespace after ','
scripts/email_nag.py:162:33: E231 missing whitespace after ','
scripts/email_nag.py:183:13: E128 continuation line under-indented for visual indent
scripts/email_nag.py:185:13: E128 continuation line under-indented for visual indent
scripts/email_nag.py:187:13: E128 continuation line under-indented for visual indent
scripts/email_nag.py:189:13: E128 continuation line under-indented for visual indent
scripts/email_nag.py:190:13: E128 continuation line under-indented for visual indent
scripts/email_nag.py:192:13: E128 continuation line under-indented for visual indent
scripts/email_nag.py:193:13: E128 continuation line under-indented for visual indent
scripts/email_nag.py:195:13: E128 continuation line under-indented for visual indent
scripts/email_nag.py:196:13: E128 continuation line under-indented for visual indent
scripts/email_nag.py:197:13: E128 continuation line under-indented for visual indent
scripts/email_nag.py:199:13: E128 continuation line under-indented for visual indent
scripts/email_nag.py:200:13: E128 continuation line under-indented for visual indent
scripts/email_nag.py:202:13: E128 continuation line under-indented for visual indent
scripts/email_nag.py:203:13: E128 continuation line under-indented for visual indent
scripts/email_nag.py:205:13: E128 continuation line under-indented for visual indent
scripts/email_nag.py:207:13: E128 continuation line under-indented for visual indent
scripts/email_nag.py:209:13: E128 continuation line under-indented for visual indent
scripts/email_nag.py:211:13: E128 continuation line under-indented for visual indent
scripts/email_nag.py:213:13: E128 continuation line under-indented for visual indent
scripts/email_nag.py:242:37: W601 .has_key() is deprecated, use 'in'
scripts/email_nag.py:249:20: W601 .has_key() is deprecated, use 'in'
scripts/email_nag.py:252:20: W601 .has_key() is deprecated, use 'in'
scripts/email_nag.py:255:22: W601 .has_key() is deprecated, use 'in'
scripts/email_nag.py:258:17: E265 block comment should start with '# '
scripts/email_nag.py:277:24: W601 .has_key() is deprecated, use 'in'
scripts/email_nag.py:280:21: E126 continuation line over-indented for hanging indent
scripts/email_nag.py:280:26: E203 whitespace before ':'
scripts/email_nag.py:285:17: E121 continuation line under-indented for hanging indent
scripts/email_nag.py:287:35: W601 .has_key() is deprecated, use 'in'
scripts/email_nag.py:288:50: W601 .has_key() is deprecated, use 'in'
scripts/email_nag.py:296:64: E203 whitespace before ','
scripts/email_nag.py:304:21: E126 continuation line over-indented for hanging indent
scripts/email_nag.py:304:26: E203 whitespace before ':'
scripts/email_nag.py:308:22: E121 continuation line under-indented for hanging indent
scripts/email_nag.py:309:17: E121 continuation line under-indented for hanging indent
scripts/email_nag.py:323:43: W601 .has_key() is deprecated, use 'in'
scripts/email_nag.py:333:34: W601 .has_key() is deprecated, use 'in'
scripts/email_nag.py:368:44: W601 .has_key() is deprecated, use 'in'
scripts/email_nag.py:373:46: W601 .has_key() is deprecated, use 'in'
scripts/email_nag.py:375:52: W601 .has_key() is deprecated, use 'in'
scripts/email_nag.py:377:58: W601 .has_key() is deprecated, use 'in'
scripts/email_nag.py:379:56: W601 .has_key() is deprecated, use 'in'
scripts/email_nag.py:388:61: W601 .has_key() is deprecated, use 'in'
scripts/email_nag.py:391:60: W601 .has_key() is deprecated, use 'in'
scripts/email_nag.py:403:21: E126 continuation line over-indented for hanging indent
scripts/email_nag.py:407:27: E251 unexpected spaces around keyword / parameter equals
scripts/email_nag.py:407:29: E251 unexpected spaces around keyword / parameter equals
scripts/email_nag.py:408:32: E251 unexpected spaces around keyword / parameter equals
scripts/email_nag.py:408:34: E251 unexpected spaces around keyword / parameter equals
scripts/email_nag.py:419:20: W601 .has_key() is deprecated, use 'in'
scripts/email_nag.py:477:21: E128 continuation line under-indented for visual indent
scripts/email_nag.py:478:21: E128 continuation line under-indented for visual indent
scripts/email_nag.py:479:21: E128 continuation line under-indented for visual indent
scripts/email_nag.py:480:21: E128 continuation line under-indented for visual indent
bugzilla/__init__.py:1:20: E231 missing whitespace after ','
bugzilla/__init__.py:1:28: E231 missing whitespace after ','
bugzilla/agents.py:1:1: F403 'from bugzilla.models import *' used; unable to detect undefined names
bugzilla/agents.py:2:1: F403 'from bugzilla.utils import *' used; unable to detect undefined names
bugzilla/agents.py:4:1: E302 expected 2 blank lines, found 1
bugzilla/agents.py:10:1: E302 expected 2 blank lines, found 1
bugzilla/agents.py:12:1: W293 blank line contains whitespace
bugzilla/models.py:33:1: E302 expected 2 blank lines, found 1
bugzilla/models.py:233:1: W293 blank line contains whitespace
bugzilla/models.py:234:45: W292 no newline at end of file
bugzilla/utils.py:17:10: E231 missing whitespace after ','
bugzilla/utils.py:21:47: E231 missing whitespace after ','
bugzilla/utils.py:32:50: W291 trailing whitespace
bugzilla/utils.py:54:25: E221 multiple spaces before operator
Include bug title in the report (it could be capped at say 40 letters) so it breakdowns the monotony of the report and provides rich-data
bztools is too vague. We should rename it for something like relman-auto-nag
This will allow the script to run for the first time without bailing on the user
This line should be uncommented at some point to have the test back
https://github.com/mozilla/relman-auto-nag/pull/92/files#diff-bc5053c9bbeb5816bba9486bc6a9692bR40
The following functions are duplicated.
We should move that into a common place:
def createQuery(queries_dir, title, short_title, url, show_summary):
file_name = queries_dir + str(datetime.date.today()) + '_' + short_title
if not os.path.exists(queries_dir):
os.makedirs(queries_dir)
qf = open(file_name, 'w')
qf.write("query_name = \'" + title + "\'\n")
qf.write("query_url = \'" + url + "\'\n")
qf.write("show_summary = \'" + str(show_summary) + "\'\n")
return file_name
def createQueriesList(queries_dir, weekday, print_all):
queries = []
for url in urls:
if weekday >= 0 and weekday < 5 and url[0] == 5:
queries.append(createQuery(queries_dir, title=url[1][0], short_title=url[1][1], url=url[1][2], show_summary=url[1][3]))
if weekday == 0 and url[0] == 0:
queries.append(createQuery(queries_dir, title=url[1][0], short_title=url[1][1], url=url[1][2], show_summary=url[1][3]))
if weekday == 3 and url[0] == 3:
queries.append(createQuery(queries_dir, title=url[1][0], short_title=url[1][1], url=url[1][2], show_summary=url[1][3]))
print queries
return queries
def cleanUp(queries_dir):
try:
for file in os.listdir(queries_dir):
if file.startswith(str(datetime.date.today())):
os.remove(os.path.join(queries_dir, file))
return True
except Exception as error:
print "Error: ", str(error)
return False
Hi,
This bug has been filed because a code search has revealed that code in this repository is using "BzAPI", the Bugzilla API proxy server at https://api-dev.bugzilla.mozilla.org/.
This service is due to be decommissioned. Fortunately, Bugzilla has a new native REST API which works somewhat similarly that you could use instead:
https://wiki.mozilla.org/Bugzilla:REST_API
Moving to that new API would require some porting work. If that seems like too much effort, fear not, because there is also a BzAPI compatibility layer built on top of the new API. Because it's compatible, using that should simply require a change of API endpoint.
Change:
https://api-dev.bugzilla.mozilla.org/latest/
(or https://api-dev.bugzilla.mozilla.org/1.3/)
to
https://bugzilla.mozilla.org/bzapi/
You should not expect the api-dev server to continue to work after Friday 14th November. So please test and migrate your production instance(s) over before then, either to the BzAPI compatibility endpoint or to the new native REST API.
Thanks for using BzAPI. It's been a great ride :-)
Gerv
We mustn't catch "auto-regress/"
setup.py shouldn't depend on remoteobjects already being installed, but instead allow installing dependencies.
Traceback (most recent call last):
File "setup.py", line 12, in <module>
version=__import__('bugzilla').__version__,
File "/Users/kewisch/mozilla/github/bztools/bugzilla/__init__.py", line 4, in <module>
from remoteobjects import http
ImportError: No module named remoteobjects
In the "Fixed in 60, Affecting 59 or 58" email, for the bugs affected in beta but fixed in nightly, I'd love to exclude the bugs that already have an uplift request (approval-mozilla-beta:? or +) flag set. That would eliminate a lot of noise from the query, since we're going to see the uplift requests anyway.
Currently the logging and screen output for the scripts will show the username & password in the Bugzilla URL output which is not ideal for putting this (eventually) on a shared server.
As it is, that email basically needs to be processed in one go because it doesn't provide a way to get an updated view as bugs are updated.
Seems that it is pretty common:
https://mzl.la/2KN89EH
In all of the cases, the last value is 0, we should figure out if this is necessary or not.
urls = [
(5, ["Unlanded Beta " + beta_version + " Bugs", "unlanded_beta", unlanded_beta_url, 0]),
(5, ["Unlanded Aurora " + aurora_version + " Bugs", "unlanded_aurora", unlanded_aurora_url, 0]),
(5, ["Unlanded ESR38 Bugs", "unlanded_esr38", unlanded_esr38_url, 0]),
(5, ["Tracked or Nominated for Tracking with Need-Info? Beta " + beta_version + " Bugs", "needinfo_beta", needinfo_beta_url, 0]),
Yesterday, it was merged day.
So, today, the email "Fixed in 52, Affecting 51, 50 or 49" is sent with nightly=52
However, there is a bunch of bugs which are Fixed in 51 but not uplifted to 50. Maybe we want to list them too in the email.
To follow what is usually done in the Python world, we should create a new directory
auto-nag/
and move bugzilla/ & scripts/ into it.
It will probably require some changes in import and so.
Hopefully, this will also simplifies tests and code coverage executions.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.