Implement a driver that supports URL base purging
https://docs.incapsula.com/Content/API/sites-api.htm#Purge2

I'd love to see MaxCDN support as well:

https://www.maxcdn.com/features/api/

https://github.com/MaxCDN/php-maxcdn

Cloudflare has a 500 url limit for cache invalidation so it would make sense to split the array into smaller chunks and call the api with these chunks.

I'm happy to work on this and submit a PR but I don't want to change the interface to purgeUrls - ie: I'm not sure what it should return if I made this change.

If you let me know what you would like to do I'll work on it.

I think we just need to use getIsPreview instead: https://github.com/craftcms/cms/blob/develop/src/web/Request.php#L690

Seeing this in my logs alot:

{
    "message": "Unabled to find tag. Unknown Event 'craft\\events\\ElementEvent'.",
    "context": {
        "trace": [],
        "memory": 11372896,
        "category": "upper"
    },
    "level": 300,
    "level_name": "WARNING",
    "channel": "craftcms",
    "datetime": {
        "date": "2019-11-26 16:31:06.124181",
        "timezone_type": 3,
        "timezone": "America/Detroit"
    },
    "extra": []
}

I'm trying to install on both a local and remote instance, as hosted on Heroku. When using the web interface, a message is displayed saying "Couldn’t install plugin." The CLI approach claims to succeed, but the plugin remains disabled in the admin section. I checked the log files, but couldn't find anything of interest. Let me know if I can provide any further detail or if its simply a compatibility issue with the latest Craft version.

I’m trying to get Upper (1.4.2) working with Cloudflare.

Despite seeing cache-control: public, s-maxage=604800 and x-upper-cache headers I’m also getting cf-cache-status: MISS.

If I add ‘Edge Cache TTL’ in the Page Rule, alongside ‘Cache Level: Cache Everything’, pages start to cache on the CDN, however, it also means that cache-control: private, no-cache is ignored, which messes up the control panel and other dynamic pages.

What might I be missing here?

@ostark spent a bit of time tracking down an issue for @devinellis and @caseycoates in which if you increased the number of Element queries past a given point, it'd end up causing an immediate 502 error.

"Immediate" is important here, because initially the thought was that it could be timeout related or the like, but it ends up being that the error being thrown was:

2019/06/22 02:29:30 [error] 61#61: *13 upstream sent too big header while reading response header from upstream, client: 172.18.0.1, server: , request: "GET /help/whats-new HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php7.3-fpm.sock:", host: "localhost:8000"

After a bit of hunting around, it appeared that the Docker container for Nginx and php-fpm were fairly stock default; bumping up the the following as per Nginx-Craft fixed the issue:

        fastcgi_buffer_size 16k;
        fastcgi_buffers 4 16k;

It ends up being that the issue was Upper including an excessive number of X-Cache-Tag headers, to the point where it overflow Nginx's initial buffer for the request from the upstream php-fpm:

X-Cache-Tag: moz-inboel36635 moz-inbose95 moz-inbost18183 moz-inboseomaticGlobals moz-inboel33803 moz-inboel803 moz-inbose17 moz-inboel33223 moz-inboel44423 moz-inbose103 moz-inboel41433 moz-inboel40281 moz-inboel37485 moz-inboel36565 moz-inboel36563 moz-inboel36573 moz-inboel36575 moz-inboel36577 moz-inboel36579 moz-inboel36581 moz-inboel36583 moz-inboel36585 moz-inboel36587 moz-inboel36589 moz-inboel36593 moz-inboel36595 moz-inboel36597 moz-inboel36599 moz-inboel36601 moz-inboel36605 moz-inboel36607 moz-inboel36609 moz-inboel36611 moz-inboel36613 moz-inboel36615 moz-inboel36617 moz-inboel36619 moz-inboel36621 moz-inboel36623 moz-inboel36625 moz-inboel36603 moz-inboel36591 moz-inboel36627 moz-inboel36629 moz-inboel36631 moz-inboel36633 moz-inboel19275 moz-inboel19687 moz-inboel19847 moz-inboel25051 moz-inboel37903 moz-inboel40231 moz-inboel33641 moz-inboel33653 moz-inboel33661 moz-inboel33657 moz-inboel34519 moz-inboel42265 moz-inboel33643 moz-inboel33645 moz-inboel33813 moz-inboel33649 moz-inboel33829 moz-inboel33833 moz-inboel33665 moz-inboel33683 moz-inboel33667 moz-inboel33671 moz-inboel33675 moz-inboel33679 moz-inboel39811 moz-inboel33341 moz-inboel33367 moz-inboel33369 moz-inboel33847 moz-inboel33397 moz-inboel34061 moz-inboel34065 moz-inboel33387 moz-inboel33383 moz-inboel33373 moz-inboel33375 moz-inboel34557 moz-inboel33379 moz-inboel33343 moz-inboel33357 moz-inboel33353 moz-inboel33361 moz-inboel33865 moz-inboel33869 moz-inboel33873 moz-inboel33877 moz-inboel33881 moz-inboel33895 moz-inboel33903 moz-inboel33899 moz-inboel33905 moz-inboel33907 moz-inboel33911 moz-inboel33915 moz-inboel33923 moz-inboel33933 moz-inboel33937 moz-inboel33941 moz-inboel33945 moz-inboel33949 moz-inboel33953 moz-inboel34021 moz-inboel34023 moz-inboel34027 moz-inboel34031 moz-inboel34035 moz-inboel34039 moz-inboel34043 moz-inboel34047 moz-inboel34051 moz-inboel34055 moz-inboel34057 moz-inboel37973 moz-inboel33391 moz-inboel33393 moz-inboel42213 moz-inboel42217 moz-inboel33965 moz-inboel33967 moz-inboel33969 moz-inboel34011 moz-inboel33973 moz-inboel33975 moz-inboel33979 moz-inboel33983 moz-inboel33987 moz-inboel33991 moz-inboel33995 moz-inboel33999 moz-inboel34003 moz-inboel34007 moz-inboel34015 moz-inboel33413 moz-inboel33461 moz-inboel33415 moz-inboel34187 moz-inboel33483 moz-inboel34139 moz-inboel34153 moz-inboel34161 moz-inboel34169 moz-inboel34173 moz-inboel34189 moz-inboel34201 moz-inboel42193 moz-inboel42233 moz-inboel34205 moz-inboel34219 moz-inboel34215 moz-inboel34221 moz-inboel34229 moz-inboel34231 moz-inboel34235 moz-inboel34237 moz-inboel34245 moz-inboel34247 moz-inboel34241 moz-inboel34243 moz-inboel34249 moz-inboel34255 moz-inboel33539 moz-inboel33547 moz-inboel44675 moz-inboel44795 moz-inboel44695 moz-inboel44821 moz-inboel33557 moz-inboel33595 moz-inboel33957 moz-inboel44863 moz-inboel45063 moz-inboel45113 moz-inboel45137 moz-inboel44895 moz-inboel44757 moz-inboel44901 moz-inboel33565 moz-inboel33961 moz-inboel33579 moz-inboel33587 moz-inboel33599 moz-inboel33625 moz-inboel33633 moz-inboel33591 moz-inboel33583 moz-inboel34295 moz-inboel34297 moz-inboel34299 moz-inboel34305 moz-inboel34483 moz-inboel34485 moz-inboel34489 moz-inboel34513 moz-inboel34515 moz-inboel34371 moz-inboel37783 moz-inboel34415 moz-inboel34435 moz-inboel34431 moz-inboel34419 moz-inboel34427 moz-inboel34439 moz-inboel34475 moz-inboel34437 moz-inboel34443 moz-inboel34447 moz-inboel34451 moz-inboel34455 moz-inboel34459 moz-inboel34463 moz-inboel34467 moz-inboel34471 moz-inboel34479 moz-inboel33259 moz-inbose4 moz-inboel33263 moz-inboel33265 moz-inboel33267 moz-inboel41045 moz-inbost9691 moz-inboel41047 moz-inboel41049 moz-inboel41051 moz-inboel41053 moz-inboel41057 moz-inboel41059 moz-inboel41055 moz-inboel41061 moz-inboel10173 moz-inboel10759 moz-inboel11179 moz-inboel10755 moz-inboel19683 moz-inboel36567

(this is a truncated version of the headers)

Given the way upper works with CloudFlare, I'm not sure if you'd consider this to just be "how Upper works" or if there might be some issue that could be address here, so I filed it.

Thoughts?

We'd like a config value that lets us set one or more site-wide cache tags. This way we can tag everything with ['cms', 'production'], etc.

We're trying to switch from Varnish to Cloudflare for our HTML caching. However, we run multiple "apps" under our domain and we'd rather not clear all of the Cloudflare cache every time we deploy.

We can take a stab at submitting a PR if you let us know how you would prefer to handle this functionality. One simple solution would be to add the keyPrefix as a tag on its own.

Thanks @ostark !

Since bots often use HEADs to check for content updates or if the server is alive, it makes sense to cache HEADs via CDN.

With this same reasoning, I think the upper process should attach the cache headers to both GETs and HEADs.

We run two sites with each their own Craft installation on the same Varnish server. I think we run into cache invalidation problems because both Craft installations attempt to purge a tag with same name, for instance "el18".

Could this be solved by adding a namespace to x-cachetag so elements are tagged site1_el18 / site2_el18?

At first I thought this was an issue with the super table field plugin but when I disable the Upper plugin I no longer have this issue.
Attached the stack trace.

Craft edition & version | Craft Pro 3.7.36
Upper | 1.9.1
Super Table | 2.7.1

PHP version | 7.4.27

yii\base\InvalidConfigException: Invalid owner ID: 26 in /home/vagrant/code/webiste/vendor/verbb/super-table/src/elements/SuperTableBlockElement.php:311
Stack trace:
#0 /home/vagrant/code/webiste/vendor/craftcms/cms/src/helpers/ElementHelper.php(413): verbb\supertable\elements\SuperTableBlockElement->getOwner()
#1 /home/vagrant/code/webiste/vendor/craftcms/cms/src/helpers/ElementHelper.php(413): craft\helpers\ElementHelper::rootElement()
#2 /home/vagrant/code/webiste/vendor/craftcms/cms/src/helpers/ElementHelper.php(413): craft\helpers\ElementHelper::rootElement()
#3 /home/vagrant/code/webiste/vendor/craftcms/cms/src/helpers/ElementHelper.php(451): craft\helpers\ElementHelper::rootElement()
#4 /home/vagrant/code/webiste/vendor/ostark/upper/src/EventRegistrar.php(224): craft\helpers\ElementHelper::isDraftOrRevision()
#5 /home/vagrant/code/webiste/vendor/ostark/upper/src/EventRegistrar.php(41): ostark\upper\EventRegistrar::handleUpdateEvent()
#6 [internal function]: ostark\upper\EventRegistrar::ostark\upper\{closure}()
#7 /home/vagrant/code/webiste/vendor/yiisoft/yii2/base/Event.php(312): call_user_func()
#8 /home/vagrant/code/webiste/vendor/yiisoft/yii2/base/Component.php(642): yii\base\Event::trigger()
#9 /home/vagrant/code/webiste/vendor/craftcms/cms/src/services/Elements.php(1722): yii\base\Component->trigger()
#10 /home/vagrant/code/webiste/vendor/verbb/super-table/src/fields/SuperTableField.php(1110): craft\services\Elements->deleteElement()
#11 /home/vagrant/code/webiste/vendor/craftcms/cms/src/base/Element.php(4178): verbb\supertable\fields\SuperTableField->beforeElementDelete()
#12 /home/vagrant/code/webiste/vendor/craftcms/cms/src/elements/MatrixBlock.php(454): craft\base\Element->beforeDelete()
#13 /home/vagrant/code/webiste/vendor/craftcms/cms/src/services/Elements.php(1672): craft\elements\MatrixBlock->beforeDelete()
#14 /home/vagrant/code/webiste/vendor/craftcms/cms/src/fields/Matrix.php(1075): craft\services\Elements->deleteElement()
#15 /home/vagrant/code/webiste/vendor/craftcms/cms/src/base/Element.php(4178): craft\fields\Matrix->beforeElementDelete()
#16 /home/vagrant/code/webiste/vendor/verbb/super-table/src/elements/SuperTableBlockElement.php(399): craft\base\Element->beforeDelete()
#17 /home/vagrant/code/webiste/vendor/craftcms/cms/src/services/Elements.php(1672): verbb\supertable\elements\SuperTableBlockElement->beforeDelete()
#18 /home/vagrant/code/webiste/vendor/verbb/super-table/src/fields/SuperTableField.php(1110): craft\services\Elements->deleteElement()
#19 /home/vagrant/code/webiste/vendor/craftcms/cms/src/base/Element.php(4178): verbb\supertable\fields\SuperTableField->beforeElementDelete()
#20 /home/vagrant/code/webiste/vendor/craftcms/cms/src/elements/MatrixBlock.php(454): craft\base\Element->beforeDelete()
#21 /home/vagrant/code/webiste/vendor/craftcms/cms/src/services/Elements.php(1672): craft\elements\MatrixBlock->beforeDelete()
#22 /home/vagrant/code/webiste/vendor/craftcms/cms/src/services/Matrix.php(1197): craft\services\Elements->deleteElement()
#23 /home/vagrant/code/webiste/vendor/craftcms/cms/src/services/Matrix.php(734): craft\services\Matrix->_deleteOtherBlocks()
#24 /home/vagrant/code/webiste/vendor/craftcms/cms/src/fields/Matrix.php(1036): craft\services\Matrix->saveField()
#25 /home/vagrant/code/webiste/vendor/craftcms/cms/src/base/Element.php(4160): craft\fields\Matrix->afterElementPropagate()
#26 /home/vagrant/code/webiste/vendor/craftcms/cms/src/services/Elements.php(2719): craft\base\Element->afterPropagate()
#27 /home/vagrant/code/webiste/vendor/craftcms/cms/src/services/Elements.php(784): craft\services\Elements->_saveElementInternal()
#28 /home/vagrant/code/webiste/vendor/craftcms/cms/src/controllers/GlobalsController.php(273): craft\services\Elements->saveElement()
#29 [internal function]: craft\controllers\GlobalsController->actionSaveContent()
#30 /home/vagrant/code/webiste/vendor/yiisoft/yii2/base/InlineAction.php(57): call_user_func_array()
#31 /home/vagrant/code/webiste/vendor/yiisoft/yii2/base/Controller.php(178): yii\base\InlineAction->runWithParams()
#32 /home/vagrant/code/webiste/vendor/yiisoft/yii2/base/Module.php(552): yii\base\Controller->runAction()
#33 /home/vagrant/code/webiste/vendor/craftcms/cms/src/web/Application.php(293): yii\base\Module->runAction()
#34 /home/vagrant/code/webiste/vendor/craftcms/cms/src/web/Application.php(602): craft\web\Application->runAction()
#35 /home/vagrant/code/webiste/vendor/craftcms/cms/src/web/Application.php(272): craft\web\Application->_processActionRequest()
#36 /home/vagrant/code/webiste/vendor/yiisoft/yii2/base/Application.php(384): craft\web\Application->handleRequest()
#37 /home/vagrant/code/webiste/web/index.php(23): yii\base\Application->run()
#38 {main}

The SQL being executed was:

INSERT INTO `craft_upper_cache` (`url`, `tags`, `headers`, `siteId`, `dateCreated`, `dateUpdated`, `uid`)...

Following rfc2616:

The HTTP protocol does not place any a priori limit on the length of a URI. Servers MUST be able to handle the URI of any resource they serve, and SHOULD be able to handle URIs of unbounded length if they provide GET-based forms that could generate such URIs. A server SHOULD return 414 (Request-URI Too Long) status if a URI is longer than the server can handle (see section 10.4.15).

I suggest to update the datatype in the installation (table creation) here

On large pages, it is possible that the XKEY/Surrogate-Key/Cache-Tag exceeds the allowed length of a header by the web server.
IIRC, when this happens, it is generally not handled gracefully and you might get a fatal error from the web server.

I remember when I dealt with this, I had to go an bump the value in nginx and Varnish.

It would be nice if Upper had a config setting to automatically truncate the header, before it was output, to prevent this.

It could have a reasonable default set to whatever nginx or apache uses.

Our site has two domains that return different Craft Multi-sites - a .com domain and a .co.uk domain. We've set up a Cloudflare domain for each. How do we use Upper for both of them? From what I can see there is only room for a single CF domain?

Follow the discussion:
craftcms/cms#3482

Possible problem:
API rate limits

If you create an entry with initial state disabled, then later change it to enabled, cache isn't purged it seems.

Is this "as intended" or a bug?

Currently using Craft 3.6.18 and Upper 1.8.0 for Cloudflare enterprise. We use Servd as our host.

What I have been seeing is that when we do new deploys on Servd (a release), it will regenerate the cpresources files, but at times the files aren't ready yet when one of our content editors visit the site, and certain cpresources css and js files end up returning a 404.

After each deploy, we do do a clear-caches/cp-resources and clear-caches/upper-purge-all but it doesn't seem to fix the issue.

The results are seemingly random cases of control panel pages not having all their styles, or lacking functionality because of missing JS.

I took a look at on EventRegistrar::registerFrontEndEvents() and it seems to be checking if it's a CP request, but I wish it could also exclude the cpresources. I don't think there's a method on the request class that provides an easy result to this, but if there is I'd appreciate it.

Ideal outcome:
There's an upper setting in the upper.php settings file that allows me to exclude cpresources from the cache-control header, or maybe there's some direction of how I can work around this problem.

Is there a config option or a way to prevent query string parameters from creating a new entry in the upper_cache table please?

My table is getting full of duplicates and therefore takes much longer to clear.

If you get a lot of 404s then this can lead to further issues.

In EventRegistrar.php line 99 a check could be added, although you might want to call a method that checks for other status codes as well, or only insert a 200 code.

if ($response->hasCacheControlDirective('private') || $response->hasCacheControlDirective('no-cache') || $response->statusCode == 404) {

Forgive my ignorance - how do I set the max-age globally? I'm using Fastly if that matters

Should say that GitHub Flavored Markdown is supported.

I've noticed that only get generated and added to the response headers (in my case Cloudflare Cache-Tags) when the Response is a cold cache.

So on first load, the response has all necessary CacheTags headers, but it's possible that the cloudflare cache will be cleared for reasons.

Since Craft's template cache is caching content, any entries that were loaded within the {% cache %} tags no longer get added to the response headers.

I'm seeing this on CraftCMS 3.7.32 and Upper 1.9.1.

I'm not sure that this was always a problem, but perhaps it's a new problem from the 3.7 update?

Yet another driver ;-)

Hi! I'm having an issue with keys inexplicably disappearing after a few seconds—but only when using the Fastly driver.

I can't seem to reproduce it with the Dummy driver, so before I setup a Fastly install just to repro it locally + get more specific I wanted to open an issue here and see if something sounds familiar/similar to anything you've seen previously.

Generally, here's what's happening:

0. Given a Section with a few Entries displayed at /blog
1. Create/Publish/Delete an Entry/Draft within the Section
2. I briefly see the surrogate key for that Section in the Surrogate-Key header when I send a curl (or in-browser) request to /blog
3. However, after a few seconds, I can send that same curl command, and the key for that section is no longer there

Based on what's happening, I'd imagine it's related to one of the purge jobs, but it doesn't make sense that it'd be clearing removing keys from the page. Of course, if you need anything else just let me know! Also happy to record my screen + send that, if it helps.

Environment information:

• Craft 3.6.17
• PHP 7.3.31
• postgres 11.9
• Hosted on Heroku
• Using Fastly, of course

create table {{%upper_cache}} ...Exception: SQLSTATE[HY000]: General error: 3750 Unable to create or change a table without a primary key, when the system variable 'sql_require_primary_key' is set. Add a primary key to the table or unset this variable to avoid this message. Note that tables without a primary key can cause performance problems in row-based replication, so please consult your DBA before changing this setting.
The SQL being executed was: CREATE TABLE upper_cache (
uid varchar(40) NOT NULL UNIQUE,
url text NOT NULL,
urlHash varchar(32) NOT NULL,
headers text NULL DEFAULT NULL,
tags text NOT NULL,
siteId int(11),
dateCreated datetime(0) NOT NULL,
dateUpdated datetime(0) NULL DEFAULT NULL
) ENGINE = InnoDb DEFAULT CHARACTER SET = utf8 DEFAULT COLLATE = utf8_unicode_ci (/vendor/yiisoft/yii2/db/Schema.php:678)

2019-03-04 11:08:04 [-][-][01rarb4pmuca8mloccqut8eoth][error][yii\base\ErrorException:2] yii\base\ErrorException: explode(): Empty delimiter in .../versions/0.3.0/vendor/yiisoft/yii2/helpers/BaseInflector.php:480 Stack trace: #0 .../versions/0.3.0/vendor/craftcms/cms/src/web/ErrorHandler.php(76): yii\base\ErrorHandler->handleError(2, 'explode(): Empt...', '...', 480) #1 [internal function]: craft\web\ErrorHandler->handleError(2, 'explode(): Empt...', '...', 480, Array) #2 .../versions/0.3.0/vendor/yiisoft/yii2/helpers/BaseInflector.php(480): explode('', 'dm') #3 .../versions/0.3.0/vendor/ostark/upper/src/models/Settings.php(109): yii\helpers\BaseInflector::slug('dm', '') #4 .../versions/0.3.0/vendor/ostark/upper/src/Plugin.php(97): ostark\upper\models\Settings->getKeyPrefix()

Settings -> getKeyPrefix calls Inflector::slug with empty replacement, replacement is directly used in an explode call in Yii, this fails because its empty. Whether this is a Upper problem or Yii is debatable.

Does Upper support multiple varnish servers?

Can we do VARNISH_URL=http://1.1.1.1/,http://2.2.2.2 or something?

For particular cases like our cache settings, the Client request need from us to set the proper HOST attribute to the headers in order to specifies the correct domain name of the server.

The CloudFlare driver currently requires the user to use their account level API key, a key which if compromised gives an attacker complete control over their CloudFlare account. Needless to say this is somewhat less than ideal.

As of August 2019 CloudFlare allows the creation of scoped tokens than just have the privileges required to clear caches in a zone / domain:

This is then just passed in a Authorization: Bearer <token> header on the CF request in place of the X-Auth-Key / X-Auth-Email headers.

I appreciate this would be a breaking change for CF users, but it's much more secure, so still seems like a good idea. Perhaps we could support both for now with a deprecation warning for users of X-Auth-Key?

Happy to take a stab at a PR if it would be welcome and/or you don't have time @ostark

Ill try and append more information to this ticket, but we keep seeing the option to clear our Fastly cache disappear as a checkbox option in the control panel. When we uninstall and reinstall, the option returns, but after a day or two, disappears again.

https://www.dropbox.com/s/ny7onmj94bgo3yw/Screenshot%202018-05-10%2014.29.34.png?dl=0&raw=1

Is it possible to clear the cache for multiple Cloudflare zones/domains? I’m using Upper on a Craft install where each site has a different domain. Is there a way to configure multiple zoneIds or domains (such as an array with nested config options under cloudflare in config/upper.php or arrays for zoneId and domain)?

Requires #2 to be solved for Non-Enterprise plans.
For Enterprise user just another PurgeDriver is required > see CloudFlare API

Obviously, the CP needs to be uncacheable.

This happens somewhat by default, as PHP will set nocache headers whenever a session is started, backed on your session.cache_limiter (ini value)[https://www.php.net/manual/en/session.configuration.php#ini.session.cache-limiter].

However, the CP login page does not start a session, which can result in a cacheable login page, preventing the login from working (due to a cached CSRF).

So, how I'm getting around this is attaching my own behavior to craft\web\Response:

<?php
namespace modules\appmodule\behaviors;

use Craft;
use craft\web\Response;
use yii\base\Behavior;

class ResponseBehavior extends Behavior
{
    const UNCACHABLE_VALUE = 'private, no-cache';

    public function events()
    {
        return [
            Response::EVENT_BEFORE_SEND => 'makeCpUncacheable',
        ];
    }

    public function makeCpUncacheable()
    {
        if (Craft::$app->getRequest()->getIsCpRequest()) {
            $this->makeUncacheable();
        }
    }

    public function makeUncacheable()
    {
        $this->owner->getHeaders()->set('Cache-Control', self::UNCACHABLE_VALUE);
    }
}

This is also nice because in my twig templates I can call {% do craft.app.response.makeUncacheable() %}, and have the actual header values in one place. That is convenient, as depending on your cache service configuration, the required value of the cache-control header can be very specific (e.g. for Fastly it must be private to be uncacheable.

Anyway, I thought this could be a nice addition to CacheControlBehavior. Let me know and I'll send a PR.

Having the same issue with MySQL 5.7.35 and Craft 3.7.30.1. Always have to delete the table by hand to install it.

> create table {{%upper_cache}} ...Exception: SQLSTATE[42S01]: Base table or view already exists: 1050 Table 'pcms_upper_cache' already exists
The SQL being executed was: CREATE TABLE `pcms_upper_cache` (
	`uid` varchar(40) NOT NULL UNIQUE,
	`url` text NOT NULL,
	`urlHash` varchar(32) NOT NULL,
	`headers` text NULL DEFAULT NULL,
	`tags` text NOT NULL,
	`siteId` int(11),
	`dateCreated` datetime(0) NOT NULL,
	`dateUpdated` datetime(0) NULL DEFAULT NULL
) ENGINE = InnoDb DEFAULT CHARACTER SET = utf8 (/src/cms/vendor/yiisoft/yii2/db/Schema.php:676)
#0 /src/cms/vendor/yiisoft/yii2/db/Command.php(1307): yii\db\Schema->convertException(Object(PDOException), 'CREATE TABLE `p...')
#1 /src/cms/vendor/yiisoft/yii2/db/Command.php(1102): yii\db\Command->internalExecute('CREATE TABLE `p...')
#2 /src/cms/vendor/yiisoft/yii2/db/Migration.php(323): yii\db\Command->execute()
#3 /src/cms/vendor/ostark/upper/src/migrations/Install.php(29): yii\db\Migration->createTable('{{%upper_cache}...', Array)
#4 /src/cms/vendor/craftcms/cms/src/db/Migration.php(52): ostark\upper\migrations\Install->safeUp()
#5 /src/cms/vendor/craftcms/cms/src/db/MigrationManager.php(232): craft\db\Migration->up(true)
#6 /src/cms/vendor/craftcms/cms/src/base/Plugin.php(140): craft\db\MigrationManager->migrateUp(Object(ostark\upper\migrations\Install))
#7 /src/cms/vendor/craftcms/cms/src/services/Plugins.php(535): craft\base\Plugin->install()
#8 /src/cms/vendor/craftcms/cms/src/console/controllers/PluginController.php(80): craft\services\Plugins->installPlugin('upper')
#9 [internal function]: craft\console\controllers\PluginController->actionInstall('upper')
#10 /src/cms/vendor/yiisoft/yii2/base/InlineAction.php(57): call_user_func_array(Array, Array)
#11 /src/cms/vendor/yiisoft/yii2/base/Controller.php(178): yii\base\InlineAction->runWithParams(Array)
#12 /src/cms/vendor/yiisoft/yii2/console/Controller.php(182): yii\base\Controller->runAction('install', Array)
#13 /src/cms/vendor/craftcms/cms/src/console/Controller.php(221): yii\console\Controller->runAction('install', Array)
#14 /src/cms/vendor/yiisoft/yii2/base/Module.php(552): craft\console\Controller->runAction('install', Array)
#15 /src/cms/vendor/yiisoft/yii2/console/Application.php(180): yii\base\Module->runAction('plugin/install', Array)
#16 /src/cms/vendor/craftcms/cms/src/console/Application.php(89): yii\console\Application->runAction('plugin/install', Array)
#17 /src/cms/vendor/yiisoft/yii2/console/Application.php(147): craft\console\Application->runAction('plugin/install', Array)
#18 /src/cms/vendor/yiisoft/yii2/base/Application.php(384): yii\console\Application->handleRequest(Object(craft\console\Request))
#19 /src/cms/craft(22): yii\base\Application->run()
#20 {main}

Originally posted by @elfacht in #55 (comment)

Hi.

After installing Upper, and trying to create some new categories, I get the following error message:

Getting unknown property: craft\elements\Category::sectionId

Full exception/stack:

2018-03-16 13:12:11 [192.168.10.1][93][f0i82gim7haivglpl7qcn5j8q4][error][yii\base\UnknownPropertyException] yii\base\UnknownPropertyException: Getting unknown property: craft\elements\Category::sectionId in /home/vagrant/code/vendor/yiisoft/yii2/base/Component.php:154
Stack trace:
#0 /home/vagrant/code/vendor/craftcms/cms/src/base/Element.php(753): yii\base\Component->__get('sectionId')
#1 /home/vagrant/code/vendor/ostark/upper/src/EventRegistrar.php(199): craft\base\Element->__get('sectionId')
#2 /home/vagrant/code/vendor/ostark/upper/src/EventRegistrar.php(32): ostark\upper\EventRegistrar::handleUpdateEvent(Object(craft\events\ElementEvent))
#3 [internal function]: ostark\upper\EventRegistrar::ostark\upper\{closure}(Object(craft\events\ElementEvent))
#4 /home/vagrant/code/vendor/yiisoft/yii2/base/Event.php(310): call_user_func(Object(Closure), Object(craft\events\ElementEvent))
#5 /home/vagrant/code/vendor/yiisoft/yii2/base/Component.php(636): yii\base\Event::trigger('craft\\services\\...', 'afterSaveElemen...', Object(craft\events\ElementEvent))
#6 /home/vagrant/code/vendor/craftcms/cms/src/services/Elements.php(513): yii\base\Component->trigger('afterSaveElemen...', Object(craft\events\ElementEvent))
#7 /home/vagrant/code/vendor/craftcms/cms/src/controllers/CategoriesController.php(487): craft\services\Elements->saveElement(Object(craft\elements\Category))
#8 [internal function]: craft\controllers\CategoriesController->actionSaveCategory()
#9 /home/vagrant/code/vendor/yiisoft/yii2/base/InlineAction.php(57): call_user_func_array(Array, Array)
#10 /home/vagrant/code/vendor/yiisoft/yii2/base/Controller.php(157): yii\base\InlineAction->runWithParams(Array)
#11 /home/vagrant/code/vendor/craftcms/cms/src/web/Controller.php(74): yii\base\Controller->runAction('save-category', Array)
#12 /home/vagrant/code/vendor/yiisoft/yii2/base/Module.php(528): craft\web\Controller->runAction('save-category', Array)
#13 /home/vagrant/code/vendor/craftcms/cms/src/web/Application.php(237): yii\base\Module->runAction('categories/save...', Array)
#14 /home/vagrant/code/vendor/craftcms/cms/src/web/Application.php(445): craft\web\Application->runAction('categories/save...', Array)
#15 /home/vagrant/code/vendor/craftcms/cms/src/web/Application.php(221): craft\web\Application->_processActionRequest(Object(craft\web\Request))
#16 /home/vagrant/code/vendor/yiisoft/yii2/base/Application.php(386): craft\web\Application->handleRequest(Object(craft\web\Request))
#17 /home/vagrant/code/web/index.php(28): yii\base\Application->run()
#18 {main}

Disabling the Upper-plugin make the exception go away.

As far as I can see, Upper doesn't handle category objects?
There's at least nothing related to this in ELEMENT_PROPERTY_MAP or TAG_PREFIX_

Is there anything I can do to prevent this error?
Thanks!

Versions:
PHP 7.1.12
Craft Pro 3.0.0-RC15
Upper 1.3.0

Not every CacheProxy/CDN does support Surrogate Keys. By storing the relations between the tag and the URL in an internal map, this could be emulated. So, instead of purging the tags, we would purge (multiple) URLs - the result should be the same.

Just wanted to see what your plans were for this plugin and Craft 4?

I have a few pages on my site that are very dynamically heavy therefore I would like to disable caching for those pages. I have tried including the follow cache header as stated in the docs:

{% header "Cache-Control: private, no-cache" %}

But CloudFlare seems to be ignoring the header, or it is being overwritten by something as the pages with that header are still getting cached and when viewing in the network tab have the following header present: cache-control: public, max-age=43200.

Setup/Config:

• Craft Pro 3.0.33
• PHP 7.2.10
• Upper 1.4.2
• Cloudflare (NOT enterprise) so useLocalTags => true
• Cache Level: Cache Everything Page Rule set up to cache everything (https://websiteurl.com/*)

Im not sure if its my ignorance on the subject to wether this is something that is possible, if I have setup something incorrectly or wether this is an issue with the plugin. Any help would be greatly appreciated. Thanks.

Getting this error whenever I try to install the plugin via php craft plugin/install upper.

./Docker/bin/craft plugin/install upper
*** installing upper
    > create table {{%upper_cache}} ... done (time: 0.015s)
  > Create index: urlhash_idx
    > create unique index urlhash_idx on {{%upper_cache}} (urlHash) ... done (time: 0.014s)
    > execute SQL: ALTER TABLE {{%upper_cache}} ADD FULLTEXT INDEX tags_fulltext (tags ASC) ...Exception: SQLSTATE[HY000]: General error: 1221 Incorrect usage of spatial/fulltext/hash index and explicit index order
The SQL being executed was: ALTER TABLE `upper_cache` ADD FULLTEXT INDEX tags_fulltext (tags ASC) (/var/www/html/vendor/yiisoft/yii2/db/Schema.php:678)
#0 /var/www/html/vendor/yiisoft/yii2/db/Command.php(1304): yii\db\Schema->convertException(Object(PDOException), 'ALTER TABLE `up...')
#1 /var/www/html/vendor/yiisoft/yii2/db/Command.php(1099): yii\db\Command->internalExecute('ALTER TABLE `up...')
#2 /var/www/html/vendor/yiisoft/yii2/db/Migration.php(219): yii\db\Command->execute()
#3 /var/www/html/vendor/ostark/upper/src/migrations/Install.php(35): yii\db\Migration->execute('ALTER TABLE {{%...')
#4 /var/www/html/vendor/craftcms/cms/src/db/Migration.php(52): ostark\upper\migrations\Install->safeUp()
#5 /var/www/html/vendor/craftcms/cms/src/db/MigrationManager.php(232): craft\db\Migration->up(true)
#6 /var/www/html/vendor/craftcms/cms/src/base/Plugin.php(140): craft\db\MigrationManager->migrateUp(Object(ostark\upper\migrations\Install))
#7 /var/www/html/vendor/craftcms/cms/src/services/Plugins.php(527): craft\base\Plugin->install()
#8 /var/www/html/vendor/craftcms/cms/src/console/controllers/PluginController.php(44): craft\services\Plugins->installPlugin('upper')
#9 [internal function]: craft\console\controllers\PluginController->actionInstall('upper')
#10 /var/www/html/vendor/yiisoft/yii2/base/InlineAction.php(57): call_user_func_array(Array, Array)
#11 /var/www/html/vendor/yiisoft/yii2/base/Controller.php(181): yii\base\InlineAction->runWithParams(Array)
#12 /var/www/html/vendor/yiisoft/yii2/console/Controller.php(184): yii\base\Controller->runAction('install', Array)
#13 /var/www/html/vendor/craftcms/cms/src/console/Controller.php(193): yii\console\Controller->runAction('install', Array)
#14 /var/www/html/vendor/yiisoft/yii2/base/Module.php(534): craft\console\Controller->runAction('install', Array)
#15 /var/www/html/vendor/yiisoft/yii2/console/Application.php(181): yii\base\Module->runAction('plugin/install', Array)
#16 /var/www/html/vendor/craftcms/cms/src/console/Application.php(89): yii\console\Application->runAction('plugin/install', Array)
#17 /var/www/html/vendor/yiisoft/yii2/console/Application.php(148): craft\console\Application->runAction('plugin/install', Array)
#18 /var/www/html/vendor/yiisoft/yii2/base/Application.php(392): yii\console\Application->handleRequest(Object(craft\console\Request))
#19 /var/www/html/craft(23): yii\base\Application->run()
#20 {main}
*** installed upper successfully (time: 0.104s)

I'm running:
mysql: 8.0.23
php: 8.0.1
craft: 3.6.2

Tags need to be LONGTEXT.

Problem: Scheduled Entries do not purge the cache automatically.

Idea

A console command keeps an eye on entries.postDate and entries. expiryDate. This command must be scheduled via cron to invalidated the cache without human interaction.

guzzlehttp/guzzle ^6.2 | ^7.0

... not sure if the interface changed...