A long running/installed pdagent installation can accrue large numbers of files in the /var/lib/pdagent/outqueue/* directories, especially /var/lib/pdagent/outqueue/suc/ (assuming all things are working correctly). If left unmanaged this can fill the disk up, use up all of the inodes, etc.

There doesn't seem to be any shipped cron job/systemd timer/... in the package to clean these up periodically, it might be an idea to add one.

I'd like to request Python 3.x support as a feature consideration. Since the pdagent Python library is installed with the OS package, it allows users to more easily write new/custom integrations while leveraging pdagent's queuing functionality (without a subprocess call). Currently, only scripts written for Python 2.7 can take advantage of this.

Following the install directions for Ubuntu the system ends up in a dirty state, and breaks apt.

 Reading package lists... Done
Building dependency tree
Reading state information... Done
pdagent is already the newest version (1.7.2).
pdagent-integrations is already the newest version (1.6.2).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
2 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n]
Setting up pdagent (1.7.2) ...
/var/lib/dpkg/info/pdagent.postinst: 61: pycompile: not found
dpkg: error processing package pdagent (--configure):
installed pdagent package post-installation script subprocess returned error exit status 127
dpkg: dependency problems prevent configuration of pdagent-integrations:
pdagent-integrations depends on pdagent; however:
Package pdagent is not configured yet.

dpkg: error processing package pdagent-integrations (--configure):
dependency problems - leaving unconfigured
No apport report written because the error message indicates its a followup error from a previous failure.
Errors were encountered while processing:
pdagent
pdagent-integrations
E: Sub-process /usr/bin/dpkg returned an error code (1)

I also tried downloading the the latest tar file and running pd-send and because of how Ubuntu 20.04LTS only comes with python3 installed at /usr/bin/python3 it fails as /usr/bin/python does not exist by design.

Thanks, and please let me know if you need any other information.

Tom

Hi. When I run commands from build-linux/howto.md, make ubuntu command fails:

$ unzip ~/Downloads/pdagent-master.zip
$ cd pdagent-master/
$ gpg --homedir=build-linux/gnupg --gen-key
$ make ubuntu
...
Successfully tagged pdagent-ubuntu:latest
docker run \
	-v `pwd`:/usr/share/pdagent \
	-it pdagent-ubuntu \
		/bin/sh -c "/bin/sh build-linux/make_deb.sh /usr/share/pdagent/build-linux/gpg-deb /usr/share/pdagent/target"
Usage: build-linux/make_deb.sh {path-to-gpg-home} {path-to-package-installation-root}
Makefile:34: recipe for target 'target/deb' failed
make: *** [target/deb] Error 2

I believe that this happens due to incorrect naming of gpg-home directory. While gpg setup offers creating of build-linux/gnupg directory, make ubuntu uses build-linux/gpg-deb.
So I presume that build-linux/howto.md should be edited.

I am running a CentOS 8.3.2011 system on which I am attempting to install the PagerDuty Agent. However, problems were encountered both during the installation and the removal process. During installation, the postinst script appears to fail at line 68 with the following message:

chmod: cannot access '/usr/lib/python3/site-packages/pdagent/root_certs': No such file or directory
warning: %post(pdagent-1.7.3-1.noarch) scriptlet failed, exit status 1

As a result of this failure, the startup script for the service fails to deploy, and the service is not started. The cause of this problem is the version detection logic in lines 58-65. As found in makepkg.sh:106-110, the directories bundled into the RPM are as follows:

• /usr/lib/python2.7/site-packages
• /usr/lib/python3.6/site-packages
• /usr/lib/python3.7/site-packages
  In contrast, the postinst script selects between the following directories:
• /usr/lib/python3/site-packages/pdagent
• /usr/lib/python2.7/site-packages/pdagent
• /usr/lib/python2.6/site-packages/pdagent
  Of the above directories, only hosts using python2.7 will successfully install.

During removal, the prerm script appears to fail at line 43 with the following message:

  File "<string>", line 1
    import platform; print platform.python_version()
                                  ^
SyntaxError: invalid syntax
error: %preun(pdagent-1.7.3-1.noarch) scriptlet failed, exit status 1

Because the preflight checks fail, it is not possible to gracefully uninstall the RPM using yum. The purpose behind this logic (specifically line 59) appears to be the removal of the compiled python libraries, performed at postinst:71. A fix for this would be to use the version detection logic from postinst:58-65 in place of the version present in prerm:43-48, once the version in postinst is corrected. However, this doesn't take into consider the possibility of the default version of python being switched post installation. As such, the 'correct' fix would be to remove all .pyc files from all three installation directories.

How does the following timeline

On April 24, 2018, at 10 pm PST, our v1 REST API is entering a decommissioning period. Starting on that date, you will no longer be able to generate v1 REST API keys, but the v1 REST API will remain operational.

On October 19, 2018, at 10 pm, the v1 REST API will be decommissioned and no longer operational.

Impact users of pdagent ? When will pdagent be upgraded to support v2?

This was previously reported as #127

There is currently no way to pass several fields present in the PagerDuty Common Event Format (PD-CEF), most notably Severity, Component, Group, and Class. These fields are potentially very useful in certain deployment strategies, so it would be ideal to see them supported.

I'm resubmitting because we still need this to be solved. We have tried the suggestion of switching to go-pdagent but it has other bugs and we gave up after many days of effort trying to make it work for our situation.

See my comment for more context related to PagerDuty support of the integration (which extends beyond the scope of this bug)

The default SysV initscript returns 0 regardless of whether pdagent is running or not. This breaks tooling such as Puppet which then thinks pdagent is always running when it isn't, (or in my case tries to stop it even though it's not running).

There are some specific return codes for certain scenarios but basically returning 0 for running and non-zero for not running is a good start.

There is currently no way to pass several fields present in the PagerDuty Common Event Format (PD-CEF), most notably Severity, Component, Group, and Class. These fields are potentially very useful in certain deployment strategies, so it would be ideal to see them supported.

All done, as asked in: https://community.pagerduty.com/t/certificateerror-hostname-proxy-doesnt-match-either-of-pagerduty-com-pagerduty-com/1002/5
proxy has name internet_proxy.pagerduty.com, curl (internet_proxy.pagerduty.com:3128 http and https) worker fine. For some time all alerts were going threw this proxy.
But
For few days, pdagent was disabled, and now, I`m trying to set it to work.

pd-queue shows Traceback AttributeError: 'int' object has no attribute 'get'.
Trying to send any alarm causes:

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pdagent/sendevent.py", line 90, in send_event
    response = self._urllib2.urlopen(request)
  File "/usr/lib/python2.7/site-packages/pdagent/thirdparty/httpswithverify.py", line 153, in urlopen
    return url_opener_cache[ca_certs].open(url, **kwargs)
  File "/usr/lib64/python2.7/urllib2.py", line 429, in open
    response = self._open(req, data)
  File "/usr/lib64/python2.7/urllib2.py", line 447, in _open
    '_open', req)
  File "/usr/lib64/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/site-packages/pdagent/thirdparty/httpswithverify.py", line 136, in https_open
    return self.do_open(self._proxyHTTPSConnection, req)
  File "/usr/lib64/python2.7/urllib2.py", line 1200, in do_open
    raise URLError(err)
URLError: <urlopen error timed out>

Double checked proxy - proxy works.
What possible ways to understand what happend?

Ubuntu 14.04.6 and Python 2.7.16

sudo apt-get install pdagent pdagent-integrations results in:

Starting: pdagentTraceback (most recent call last):
  File "/usr/share/pdagent/bin/pdagentd.py", line 81, in <module>
    import pdagent.config
ImportError: No module named pdagent.config
No apport report written because the error message indicates its a followup error from a previous failure.
                                                                                                          dpkg: error processing package pdagent (--configure):
 subprocess installed post-installation script returned error exit status 1
dpkg: dependency problems prevent configuration of pdagent-integrations:
 pdagent-integrations depends on pdagent; however:
  Package pdagent is not configured yet.

dpkg: error processing package pdagent-integrations (--configure):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 pdagent
 pdagent-integrations
E: Sub-process /usr/bin/dpkg returned an error code (1)

[root@mo-0bda67593 bin]# pd-send -k cf2b1fbcc57646fdb1b6f3cc6f0564d0 -t trigger -d "Test test" -i server.test
Traceback (most recent call last):
File "/usr/bin/pd-send.py", line 126, in
import pdagent.config
ImportError: No module named pdagent.config

Hello,

We are using nagios with pager duty and we used pdagent in order to send webhook to our pager duty instance. We followed this integration guide https://www.pagerduty.com/docs/guides/nagios-integration-guide/. All our event are going directly to the legacy event rules and we use this kind of rules to route the event to right service.

As you can see we use not CEF fields from the payload, which works fine with event rules, but the new event orchestration does not allow us this flexibility anymore.

I would like to know why "custom payload" are not supported anymore. And please 🙏 do not ask me to use integration key per service, it's not at all easy to automate compare to one integration key and multiples routing.

In Ubuntu 16.04 PDAgent won't install via apt-get because it depends on the python-support package. The package has been removed from this version.

16.04 still has python 2.7 support, so installing python 2.7 and git cloning works, it just doesn't have all the binaries in the right place and no init script.

Please list this on your README

http://www.freshports.org/net-mgmt/py-pdagent/

This code should run only on >= Python2.7, since socket.create_connection() can receive up to 2 arguments in 2.6.
https://github.com/PagerDuty/pdagent/blob/master/pdagent/thirdparty/httpswithverify.py#L103-L104

Is there an option to stream pdagent logs not to file, but to JournalD?

Allow for:

• run as root
• command line arg to not deamonize (eg. -f for foreground)

I'm running Amazon Linux 2023 that comes preinstalled with python3 (3.9.16). I have created a symlink from python to python3:

[root@ip-10-7-3-153 bin]# which python3
/usr/bin/python3
[root@ip-10-7-3-153 bin]# which python
/usr/bin/python
[root@ip-10-7-3-153 bin]# ls -la /usr/bin/python
lrwxrwxrwx. 1 root root 16 Nov 27 13:08 /usr/bin/python -> /usr/bin/python3
[root@ip-10-7-3-153 bin]# python -V
Python 3.9.16

I'm trying to start Pagerduty (pdagent-1.7.3-1.noarch, pdagent-integrations-1.6.2-1.noarch) but I'm getting the following errors:

Nov 27 13:12:33 server.local systemd[1]: Started pdagent.service - PagerDuty Agent.
Nov 27 13:12:33 server.local pdagentd[43558]: Traceback (most recent call last):
Nov 27 13:12:33 server.local pdagentd[43558]:   File "/usr/share/pdagent/bin/pdagentd.py", line 75, in <module>
Nov 27 13:12:33 server.local pdagentd[43558]:     import pdagent.config
Nov 27 13:12:33 server.local pdagentd[43558]: ModuleNotFoundError: No module named 'pdagent'
Nov 27 13:12:33 server.local pdagentd[43558]: During handling of the above exception, another exception occurred:
Nov 27 13:12:33 server.local pdagentd[43558]: Traceback (most recent call last):
Nov 27 13:12:33 server.local pdagentd[43558]:   File "/usr/share/pdagent/bin/pdagentd.py", line 81, in <module>
Nov 27 13:12:33 server.local pdagentd[43558]:     import pdagent.config
Nov 27 13:12:33 server.local pdagentd[43558]: ModuleNotFoundError: No module named 'pdagent'
Nov 27 13:12:33 server.local systemd[1]: pdagent.service: Main process exited, code=exited, status=1/FAILURE
Nov 27 13:12:33 server.local systemd[1]: pdagent.service: Failed with result 'exit-code'.

It works completely fine if I'm using its old python2.x libraries:
cp -a /usr/lib/python2.7/site-packages/pdagent/ /usr/share/pdagent/bin/pdagent/

Does it run with python3? Or are there any plans to make it work with it?

It would be handy if the pdagent package used the Systemd unit on CentOS 7. Two reasons:

• That's how 99.9% of services are managed on that platform.
• I have to set environment variables for pdagent to use a proxy to get access to the API. With the SysV initscript I have to edit the actual initscript to set these which means the changes would get trashed with a package update. It's easier with a Systemd unit file to override some settings like the environment separately and keep the unit file shipped with the package unchanged so a package update is safer to apply as it doesn't clobber any modifications.

I can see there's a /var/lib/pdagent/scripts/pdagent.service that's shipped in the package but not used so I could move that into place and disable the initscript however a package update will just add the old initscript back in again so it would be better if the package was changed.

Having to build locally is an annoyance.
An up-to-date and ready to pull Docker image would be way more convenient.

To test the Nagios integration I attempted to send a custom notification on a service via the Nagios GUI. Currently "CUSTOM" is not an allowed event-type.

pd-nagios: error: argument -t/--event-type: invalid choice: 'CUSTOM' (choose from 'PROBLEM', 'ACKNOWLEDGEMENT', 'RECOVERY')

Hi, I have installed the pdagent on ubuntu 18.04.04 LTS, integrated with Nagios core and checked that it can send events to pagerduty. Still, when the notify cmd is executed by nagios i am getting this errors:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pdagent/heartbeat.py", line 147, in _process_response
    result = json.loads(response_str)
  File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
    s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'

2020-07-31 15:02:59,728 WARNING HeartbeatTask        pdagent.heartbeat    Error reading heartbeat response data:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pdagent/heartbeat.py", line 147, in _process_response
    result = json.loads(response_str)
  File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
    s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'

2020-07-31 16:02:59,731 WARNING HeartbeatTask        pdagent.heartbeat    Error reading heartbeat response data:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pdagent/heartbeat.py", line 147, in _process_response
    result = json.loads(response_str)
  File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
    s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'

2020-07-31 16:55:50,559 INFO    SendEventTask        pdagent.pdqueue      Processing event 1596214541429434_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.txt

2020-07-31 16:55:51,144 WARNING SendEventTask        pdagent.sendevent    Error reading response data while sending event:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pdagent/sendevent.py", line 133, in send_event
    result = json.loads(result_str)
  File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
    s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'

2020-07-31 16:55:51,145 ERROR   SendEventTask        pdagent.sendevent    Error sending event 1596214541429434_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.txt; Error code: 200, Reason: b'{"status":"success","message":"Event processed","incident_key":"server.fire"}'

2020-07-31 16:55:51,145 INFO    SendEventTask        pdagent.pdqueue      Changing 1596214541429434_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.txt type: pdq -> suc...

2020-07-31 17:02:59,731 WARNING HeartbeatTask        pdagent.heartbeat    Error reading heartbeat response data:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pdagent/heartbeat.py", line 147, in _process_response
    result = json.loads(response_str)
  File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
    s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'

How can i fix this?

Thanks in advance

The code on master appears to be 1.7.2, while the RPM packages at least appear to have newer code with a version number of 1.7.3. It sure would be nice if master had the latest stuff.

RHEL: 6.8
Python: 2.6.6
pdagent: v1.4

Attempting to run pd-queue status returns the following trace:

pd-queue status
Service Key                           Pending   Success  In Error
=================================================================
key                                     8         0         0
Traceback (most recent call last):
  File "/usr/bin/pd-queue", line 147, in <module>
    main()
  File "/usr/bin/pd-queue", line 135, in main
    args.func(load_agent_config(), parser, args)
  File "/usr/bin/pd-queue", line 123, in _status
    state.get("pending_events", empty_dict).get("count", 0),
AttributeError: 'int' object has no attribute 'get'

It would appear these are not honored when defined...

Hello
When running pdagent behind and proxy and adding the relevant env variables to: /lib/systemd/system/pdagent.service and reloading the service and daemon, we get the following error in /var/log/pdagent/pdagentd.log and the send event doesn't occur:

CertificateError: hostname '192.168.41.85' doesn't match either of '*.pagerduty.com', 'pagerduty.com'

192.168.41.85 refers to a proxy server we are behind:

My setup:
openssl 1.1.0
pdagent: 1.4
python: 2.7.9

the full log below:

File "/usr/lib/python2.7/httplib.py", line 850, in _send_output
self.send(msg)
File "/usr/lib/python2.7/httplib.py", line 812, in send
self.connect()
File "/usr/lib/python2.7/dist-packages/pdagent/thirdparty/httpswithverify.py", line 120, in connect
match_hostname(self.sock.getpeercert(), self.host)
File "/usr/lib/python2.7/dist-packages/pdagent/thirdparty/ssl_match_hostname.py", line 149, in match_hostname
% (hostname, ', '.join(map(repr, dnsnames))))
CertificateError: hostname '192.168.41.85' doesn't match either of '*.pagerduty.com', 'pagerduty.com'

looks like this is something python is doing to validate the pagerduty site certificate?

use github code to start pdagent service meet error
[pd@localhost ~]$ sudo pdagent/bin/pdagentd.py
Agent should not be run as root. Use: sudo service pdagent
Agent will now quit
[pd@localhost ~]$ pdagent/bin/pdagentd.py
Traceback (most recent call last):
File "pdagent/bin/pdagentd.py", line 81, in
import pdagent.config
ImportError: No module named config

Env:Redhat6.10
Python 2.7

If there's an existing pid file, pdagentd.py just prints a message and quits, regardless whether there's an actual process running or not.

Instead, pdagentd.py when starting, if a pid file exists, it should check for the existence of that process. If there is no process running it should simply clean up the stale pid and move on.

Trying to install pdagent on Ubuntu 22.04:
W: GPG error: https://packages.pagerduty.com/pdagent deb/ Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY AE0396CFF8253540

Running apt-get update on old installations:
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: https://packages.pagerduty.com/pdagent deb/ Release: The following signatures were invalid: EXPKEYSIG AE0396CFF8253540 Package Maintainer (PagerDuty, Inc.) [email protected]

Is there any word on when the new Incident Priority option will be available for pdagent so we can;

pd-send -k "xxxxxxxxx" -t trigger -d "Server is on fire" -i server.fire -p "P1"

The old perl based
notify_pagerduty_nagios.pl automatically included all the NAGIOS_* environment variables by default.

pdagent is the recommended replacement for the old script but it does not provide this option.

After a patch run raising the version to 1.7.3, pdagent fails to start:

$ sudo -u pdagent python3 $(pwd)/1.7.3/usr/share/pdagent/bin/pdagentd.py -f
Fatal Python error: Py_Initialize: Unable to get the locale encoding
LookupError: no codec search functions registered: can't find encoding

Current thread 0x00007f3338f96740 (most recent call first):
Aborted

A change to the systemd unit file means that a shell script is now executed, which attempts to use python3:

$ diff -ruNp 1.17.2/var/lib/pdagent/scripts/pdagent.service 1.7.3/var/lib/pdagent/scripts/pdagent.service
--- 1.17.2/var/lib/pdagent/scripts/pdagent.service      2020-05-06 14:41:21.000000000 +0000
+++ 1.17.3/var/lib/pdagent/scripts/pdagent.service      2020-06-30 15:55:42.000000000 +0000
@@ -6,7 +6,7 @@ After=network.target
 Type=simple
 ExecStartPre=/bin/mkdir -p /var/run/pdagent
 ExecStartPre=/bin/chown -R pdagent:pdagent /var/run/pdagent
-ExecStart=/usr/share/pdagent/bin/pdagentd.py -f
+ExecStart=/usr/share/pdagent/bin/pdagentd -f
 KillMode=process
 TimeoutStopSec=15
 RestartSec=15

$ diff -ruNp 1.17.2/usr/share/pdagent/bin/pdagentd 1.7.3/usr/share/pdagent/bin/pdagentd
--- 1.17.2/usr/share/pdagent/bin/pdagentd       1970-01-01 00:00:00.000000000 +0000
+++ 1.17.3/usr/share/pdagent/bin/pdagentd       2020-06-30 15:55:42.000000000 +0000
@@ -0,0 +1,10 @@
+#!/bin/bash
+
+DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
+
+if [ -x "$(command -v python3)" ];
+then
+    python3 $DIR/pdagentd.py "$@"
+else
+    python $DIR/pdagentd.py "$@"
+fi

Package from main repo:

$ repoquery --location pdagent
https://packages.pagerduty.com/pdagent/rpm/pdagent-1.7.3-1.noarch.rpm

Additional info:

$ cat /etc/redhat-release 
CentOS Linux release 7.8.2003 (Core)

$ rpm -q --whatprovides /usr/bin/python3
python3-3.6.8-13.el7.x86_64

One odd thing is that I don't see a 1.7.3 tag here on this repo

tl;dr: If you have multiple version of Python installed and 2.7 isn't the default, you can have issues.

More specifically, we were happily chugging along on the older version of pdagent, upgraded, and saw that it was failing.

Upon looking at the docs, they said Python 2.7 was required, which we have.

After some amount of head scratching, I found that all the files are pointing at "python" instead of "python2.7" and thus assuming that the system defaults to 2.7.

It'd be nice if either /etc/pdagent.conf or some other place allowed you to override which Python was being called. Potentially this could also be done as part of the apt/yum install actions, to see what should be there and sed it in or some such.

CentOS 7 ships with Python 2.7 and our current RPM package is made for Python 2.6.

(Originally reported by customer against Fedora 20)