Comments (13)
I just upgraded to PHP 8.3.10 on my Matomo server (previously 8.3.8) and marketplace is working again. I double-checked by switching back to 8.3.8 to verify that it didn't work on that version.
I also noticed that the curl version has changed (8.7.1 with php 8.3.8 docker image and 8.9.0 with php 8.3.10 docker image).
Finally, there is a fix about curl in PHP 8.3.9 changelog.
So, the issue seems to be fixed in the latest php or curl version.
from matomo.
Calling the URI from CLI works fine as well (curl https://plugins.matomo.org/… gives the expected JSON reponse).
When calling the URI from a simple PHP testscript using Guzzle I also get a response from the Matomo endpoint.
When I pass a random URL to the method trying to fetch the data (Piwik\Plugins\Marketplace\Api\Service::fetch()
) then it will return a response as well.
When I mirror the JSON reponse of the marketplace from another instance, then the request is fine as well.
So curl is working ✔️, the server is able to handle requests (seems to be no firewall issue) ✔️ . Somehow the connection from Matomo to the Matomo marketplace URI is faulty.
from matomo.
The connection error lead me into Matomos HTTP class, which sets up custom curl request commands.
In there I found the error. The following line instructs curl to always request all encodings it supports:
Lines 677 to 680 in 4a5a877
(introduced in #17009 and afaik reasonable).
Outgoing requests show that my server sends the header accept-encoding: deflate, gzip, br
. When I remove brotli (br), then Matomo will work just fine and the marketplace is available again. When I use brotli only, the response is empty.
I was able to reproduce the issue with a simple script:
<?php
$requestUrl = 'https://plugins.matomo.org/api/2.0/plugins?keywords=&purchase_type=&query=&sort=lastupdated&release_channel=latest_stable&prefer_stable=1&piwik=5.0.3&php=8.1.28&mysql=10.6.17-MariaDB&num_users=9&num_websites=14';
//$requestUrl = 'https://httpbin.dev/brotli';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $requestUrl);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_ENCODING, 'br');
$data = curl_exec($ch);
print_r($data);
However: This script works fine on a local test machine.
Running the request via CLI works on both machines (curl --output -H 'Accept-encoding: br' "https://plugins.matomo.org/api/2.0/plugins?keywords=&purchase_type=&query=&sort=lastupdated&release_channel=latest_stable&prefer_stable=1&piwik=5.0.3&php=8.1.28&mysql=10.6.17-MariaDB&num_users=9&num_websites=14"
), so this seems to be a php-curl issue.
I have contacted my hosting provider, maybe php-curl was not compiled with brotli.
FYI: I wrote the a Guzzle script for the same request and this works. Guzzle decompresses the reponse using a stream wrapper.
from matomo.
I have contacted my hosting provider, maybe php-curl was not compiled with brotli.
The hosting uses Red Hat Enterprise Linux 7 with curl version 7.29. Since this is a long-term support OS, that's a rather old version, and the provider therefore installed an additional curl. PHP is provided in multiple versions with the popular Remi repo (https://rpms.remirepo.net/). Since these versions are supposed to run on RHEL 7, they are built against version 7.29. Brotli is supported in curl since version 7.57 (https://curl.se/changes.html).
Due to the mentioned empty CURLOPT_ENCODING
argument php-curl now sends the “newer” list of accepted values, which includes brotli, but is not able to work with the response eventually.
My hotfix: I hardcoded $curl_options[CURLOPT_ENCODING] = 'deflate, gzip'
in line
Line 680 in 4a5a877
Question: This issues did not pop up until this month. My provider said that they run this software setup for a long time already. Did the Matomo Marketplace API Server maybe activate brotli responses just now?
Suggestion: As written before, if Matomo would work with a stream wrapper - as Guzzle does - this would not trigger an error. Maybe add a stream wrapper or even replace the custom curl scripts with a library like Guzzle? Depends how many users are affected by the issue.
from matomo.
I can reproduce the exact same behaviour, including @pixelbrackets test script and the fix.
System is: Red Hat Enterprise Linux Server release 7.9
Curl version is 8.7.1, though
from matomo.
@patrickli @samjf Were there any changes done to our Marketplace API Server recently, which might cause the problem?
from matomo.
Not anything I'm aware of. @samjf did we upgrade PHP version for it? I just checked and it is default
.
from matomo.
@patrickli @sgiehl It is possible the servers could have been upgraded in that time -- from memory it was before 3 weeks ago though. I'll DM you more details.
from matomo.
This actually sounds more like a problem with php curl rather than an issue with Matomo or our server.
Could be a similar issue as reported here: CpanelInc/libcurl#3
from matomo.
I ran into this today setting up my first instance in Docker using the fpm-alpine image. From debugging, I get the same empty response. Curl from host and container CLI both work.
Workaround works for me on 5.0.3 with PHP 8.2, applied with sed from the host:
docker exec matomo_app \
sed -i 's/_ENCODING] = "/&deflate, gzip/' /var/www/html/core/Http.php
On the admin dash it manifests as 3 identical errors down the page, but all three linked to the FAQ about setting up the cron. (on the bright side, cron got configured much sooner...)
from matomo.
Hi,
I'm having the same problem here, the marketplace won't connect anymore.
PHPinfo confirms that curl is compiled with brotli. The hotfix suggested by @pixelbrackets works for me too.
I have access to a second server where the marketplace works fine and on which curl isn't compiled with brotli.
Interesting things : forcing matomo to use http instead of https by adding force_matomo_http_request = 1
in the general section of configuration also "solves" the problem too (temporarily).
from matomo.
Hey @MerleCedric
Thanks for the update. So it looks like a PHP / curl specific issue.
Fixing it on our end by setting specific encoding doesn't sound like a wise decision anyway.
And setting it to an empty string allows to use all available encodings.
If anyone still experiences that problem after updating PHP / curl, please let us know, so we can reconsider this.
from matomo.
FYI I sucessfully checked the solution described by @MerleCedric. My hoster installed an new curl version and PHP. My example script works again. And the Matomo Marketplace is available again when curl uses brotli compression. So it was indeed an issue with curl and/or PHP. All done. ✔️
from matomo.
Related Issues (20)
- Auth for tracking API HOT 1
- [Feature request] Add a "exclude traffic from referrer" option HOT 1
- [Bug] Matomo wizard and SSL connections HOT 1
- Admin for Matomo still says token_auth allowed for widgets but stopped working for me
- All Websites dashboard accepts invalid dates
- Conversions/Revenue columns are empty when using report search
- [Bug] DBStats: cacheDataByArchiveNameReports could take hours to complete
- Goals on/off slider
- [Bug] Unhandled exception processing >50,000 dimensions for custom reports HOT 1
- [Bug] Incorrect data for exit rates in transition overview with segments.
- Check-Engine-Light signal for the main interface HOT 2
- [Bug] For same idvisit, idvisitor is different in different tables.
- [Bug] Campaign Tracking in Dashboard - Incorrect total yearly value for campaign names
- Add a link icon to the report name tool tips
- [Bug] All Websites report slower in Matomo 5 than Matomo 3 when displaying date ranges HOT 1
- [Bug] Location resolution issue GeoiP database HOT 1
- In JS Tracking Code method setDocumentTitle, use location.hostname instead of document.domain property
- [Bug] Matomo API segmentation on action Actions.getPageUrls gives inconsistent results and device type switches from upper case to lower case HOT 9
- [Bug] Compare-to is broken for Evolution: Conversion HOT 1
- [Bug] HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from matomo.