Short description

sendCacheControlHeader in .htaccess is different from "<!-- expires on: dd-mm-yy hh:mm -->" comment in cached html files

Steps to reproduce the problem

2. set typo3 cache-period to e.g. 14 days
3. Place an element on a page, which has a start-stop time in near future, e.g. in 2 hours.

What is the expected behaviour?

.htaccess "ExpiresByType text/html M..." to match <!-- expires on: ... -->

What is observed behaviour?

ExpiresByType text/html M... is always set to the deault-cacheperiod of typo3.

Hello,

I am using the TYPO3 8.7.3 with this extension.
I have setup a page with force generation of static cache, but I am getting this error in the backend module that is 'User or group is set'.

May I know the cause of this error?
So I can check my setup and try to solve it.

Thanks!

TYPO3 clears a page cache by using the tags pageId_{UID}
Now staticfilecache does this correctly IF the entry hasn't expired.

However, if the entry has expired then retrieving the cache identifiers by tag name doesn't work as the method looks like this:

   /**
     * Finds and returns all cache entries which are tagged by the specified tag.
     *
     * @param string $tag The tag to search for
     * @return array An array with identifiers of all matching entries. An empty array if no entries matched
     */
    public function findIdentifiersByTag($tag)
    {
        $this->throwExceptionIfFrontendDoesNotExist();
        $cacheEntryIdentifiers = [];
        $cacheEntryIdentifierRows = $GLOBALS['TYPO3_DB']->exec_SELECTgetRows(
            $this->identifierField,
            $this->tableList,
            $this->tagsTable . '.tag = ' . $GLOBALS['TYPO3_DB']->fullQuoteStr($tag, $this->tagsTable) . ' AND ' . $this->tableJoin . ' AND ' . $this->notExpiredStatement,
            $this->identifierField
        );
        foreach ($cacheEntryIdentifierRows as $cacheEntryIdentifierRow) {
            $cacheEntryIdentifiers[$cacheEntryIdentifierRow['identifier']] = $cacheEntryIdentifierRow['identifier'];
        }
        return $cacheEntryIdentifiers;
    }

does not find entries that are expired.
However, as the files have to be removed even in that case - especially if they don't expire - the notExpiredStatement has to be ignored in this case.
WHICH means ONLY when we are exeucting this method during flushing of the cache.

This is especially relevant if one doesn't use sendCacheControlHeaderRedirectAfterCacheTimeout (which is the default setting) because then the file will still be used instead of the new content.

Hi,

getUri https://github.com/lochmueller/staticfilecache/blob/master/Classes/StaticFileCache.php#L169
here might produce a wrong URL, e.g. if I set recreateURI to true but don't use an absRefPrefix.

In which case the URL might look like this:
https://domain.orgPath
instead of https://domain.org/Path

as without absRefPrefix no prepnding "/" will be added.
This can be easily fixed by using ltrim and adding the slash manually.

Also another scenario: absRefPrefix can include the full domain, in which case the resulting URL might look like this:
https://domain.orghttps://domain.org/Path

This one requires the new uri to be parsed instead. It would probably be correct to use the end-hostname then.

So far there were no new releases at https://extensions.typo3.org/extension/nc_staticfilecache/

Where can we find the current extension in the TER?

e.g. https://git.typo3.org/Packages/TYPO3.CMS.git/commitdiff/5bfc3e65ca0ee879ae3410aedf7105f0aa40c4bf

e.g. Implement backend 'context menu' item to 'cache this branch now'. or EXT:crawler?

The footer on my page says:

Does that mean the statically cached file will only be kept for 10 minutes, or do I get that wrong?
If so, where can I modify that expiry?

hi @lochmueller, thank you for the great boost mode method!
Unfortunately i cannot setup the boost mode. This command should recreate the static files after expire, right?

/typo3/sysext/core/bin/typo3 staticfilecache:cache:runcacheboostqueue

But it does not work and i can not find anything about it in the documentation.
Just an info: in 'tx_staticfilecache_queue' table every page_uid is 0.

After saving content the folloing error get thrown.

Noticed that the error occurs when clearing the cache.

Class '\SFC\Staticfilecache\Cache\StaticFileBackend' not found

Error thrown in file
/html/typo3/typo3_src-8.7.5/typo3/sysext/core/Classes/Cache/CacheManager.php in line 305.

15 TYPO3\CMS\Core\Cache\CacheManager::createCache("staticfilecache")


/html/typo3/typo3_src-8.7.5/typo3/sysext/core/Classes/Cache/CacheManager.php:

00256:         foreach ($this->cacheConfigurations as $identifier => $_) {
00257:             if (!isset($this->caches[$identifier])) {

00258:                 $this->createCache($identifier);

00259:             }
00260:         }


14 TYPO3\CMS\Core\Cache\CacheManager::createAllCaches()


/html/typo3/typo3_src-8.7.5/typo3/sysext/core/Classes/Cache/CacheManager.php:

00209:             return;
00210:         }

00211:         $this->createAllCaches();

00212:         if (!isset($this->cacheGroups[$groupIdentifier])) {
00213:             throw new NoSuchCacheGroupException('No cache in the specified group \'' . $groupIdentifier . '\'', 1390337130);


13 TYPO3\CMS\Core\Cache\CacheManager::flushCachesInGroupByTags("pages", array)


/html/typo3/typo3_src-8.7.5/typo3/sysext/core/Classes/DataHandling/DataHandler.php:

08440:         /** @var CacheManager $cacheManager */
08441:         $cacheManager = $this->getCacheManager();

08442:         $cacheManager->flushCachesInGroupByTags('pages', array_keys($tagsToClear));

08443: 
08444:         // Execute collected clear cache commands from page TSConfig


12 TYPO3\CMS\Core\DataHandling\DataHandler::processClearCacheQueue()


/html/typo3/typo3_src-8.7.5/typo3/sysext/core/Classes/DataHandling/DataHandler.php:

01341:         }
01342:         if ($this->isOuterMostInstance()) {

01343:             $this->processClearCacheQueue();

01344:             $this->resetElementsToBeDeleted();
01345:         }


11 TYPO3\CMS\Core\DataHandling\DataHandler::process_datamap()


/html/typo3/typo3_src-8.7.5/typo3/sysext/backend/Classes/Controller/EditDocumentController.php:

00575:             // Perform the saving operation with DataHandler:
00576:             $tce->process_uploads($_FILES);

00577:             $tce->process_datamap();

00578:             $tce->process_cmdmap();
00579:             // If pages are being edited, we set an instruction about updating the page tree after this operation.


10 TYPO3\CMS\Backend\Controller\EditDocumentController::processData()


/html/typo3/typo3_src-8.7.5/typo3/sysext/backend/Classes/Controller/EditDocumentController.php:

01961:         // Checks, if a save button has been clicked (or the doSave variable is sent)
01962:         if ($this->doProcessData()) {

01963:             $this->processData();

01964:         }
01965: 


9 TYPO3\CMS\Backend\Controller\EditDocumentController::mainAction(TYPO3\CMS\Core\Http\ServerRequest, TYPO3\CMS\Core\Http\Response)

8 call_user_func_array(array, array)


/html/typo3/typo3_src-8.7.5/typo3/sysext/backend/Classes/Http/RouteDispatcher.php:

00052:         $targetIdentifier = $route->getOption('target');
00053:         $target = $this->getCallableFromTarget($targetIdentifier);

00054:         return call_user_func_array($target, [$request, $response]);

00055:     }
00056: 


7 TYPO3\CMS\Backend\Http\RouteDispatcher::dispatch(TYPO3\CMS\Core\Http\ServerRequest, TYPO3\CMS\Core\Http\Response)


/html/typo3/typo3_src-8.7.5/typo3/sysext/backend/Classes/Http/RequestHandler.php:

00137:         /** @var RouteDispatcher $dispatcher */
00138:         $dispatcher = GeneralUtility::makeInstance(RouteDispatcher::class);

00139:         return $dispatcher->dispatch($request, $response);

00140:     }
00141: }


6 TYPO3\CMS\Backend\Http\RequestHandler::dispatch(TYPO3\CMS\Core\Http\ServerRequest)


/html/typo3/typo3_src-8.7.5/typo3/sysext/backend/Classes/Http/RequestHandler.php:

00070:         // Check if the router has the available route and dispatch.
00071:         try {

00072:             return $this->dispatch($request);

00073:         } catch (InvalidRequestTokenException $e) {
00074:             // When token was invalid redirect to login


5 TYPO3\CMS\Backend\Http\RequestHandler::handleRequest(TYPO3\CMS\Core\Http\ServerRequest)


/html/typo3/typo3_src-8.7.5/typo3/sysext/core/Classes/Core/Bootstrap.php:

00313: 
00314:         // Execute the command which returns a Response object or NULL

00315:         $this->response = $requestHandler->handleRequest($request);

00316:         return $this;
00317:     }


4 TYPO3\CMS\Core\Core\Bootstrap::handleRequest(TYPO3\CMS\Core\Http\ServerRequest)


/html/typo3/typo3_src-8.7.5/typo3/sysext/backend/Classes/Http/Application.php:

00090:         }
00091: 

00092:         $this->bootstrap->handleRequest($this->request);

00093: 
00094:         if ($execute !== null) {


3 TYPO3\CMS\Backend\Http\Application::run()


/html/typo3/typo3_src-8.7.5/typo3/sysext/backend/Resources/Private/Php/backend.php:

00022:     $classLoader = require __DIR__ . '/../../../../../../vendor/autoload.php';
00023: 

00024:     (new \TYPO3\CMS\Backend\Http\Application($classLoader))->run();

00025: });


2 {closure}()


/html/typo3/typo3_src-8.7.5/typo3/sysext/backend/Resources/Private/Php/backend.php:

00023: 
00024:     (new \TYPO3\CMS\Backend\Http\Application($classLoader))->run();

00025: });


1 require("/html/typo3/typo3_src-8.7.5/typo3/sysext/backend/Resources/Private/Php/backend.php")


/html/typo3/typo3_src-8.7.5/typo3/index.php:

00001: <?php
00002: 

00003: require __DIR__ . '/sysext/backend/Resources/Private/Php/backend.php';

Check scheme in UriFrontend isValidEntryIdentifier

The guzzle client set the staticfilecache cookie with 30 sec expire time.
After 30 sec is the cookie expired and the file cache response with 200 code.
Result: The command do not recreate the file cache after 30 sec run-time.

The path to the htaccess fluid template should be configurable.

Use-case:
Currently, all header values get escaped by the default fluid behaviour.
This breaks headers like CSP or http/2 server push.
When using this headers, the template needs to be customized.

Implement crawler extension just like Kaspers staticpub extension has. The big question here is how to support crawling multiple domains. If you have any suggestions, please contact me.

Hi,
support buttons should open with target _top or _blank ;-)
Otherwise it opens in TYPO3 Backend frame and this seems not working.

Greetings
Markus

Bug Report

Current Behavior
I don't have boost mode enabled and when I click "Clear cache for this page", static files for the page are not flushed.

Expected behavior/output
Click on "Clear cache for this page" should flush static files for the page.

Environment

• TYPO3 version(s): 8.7.17
• staticfilecache version: 6.0.0
• Is your TYPO3 installation set up with Composer (Composer Mode): no
• OS: Linux

Possible Solution
Sorry, no idea :-)

Additional context
Flushing front-end caches flushes static files for the page (but also for all other pages).

If I access the info module in the typo3 backend, i get a fluid error.

Feature Request

Is your feature request related to a problem? Please describe.
SFC should send cache cache tags via "X-Cache-Tags" header in the request and also in the static file.

http://foshttpcache.readthedocs.io/en/stable/response-tagging.html#tags

Describe the solution you'd like
There should be a service class that geht the cache tags from the TSFE. This service should used in the current request and also in the static file htaccess file.

Hi Tim,

on t3cb I asked you for a possible reason, why the .htaccess rule set is not working on certain hosting providers, which do not expose server variables like %{DOCUMENT_ROOT}. (e.g. DomainFactory, 1&1)
In your talk you mentioned that you have successful created a configuration which works also there.
Could you please provide an example? It would be very appreciated. (I got to get it running there with older versions already... but not with the new version of the extension).

Alongside two other questions:

1. Is there a good way to test/debug the variables that are used in the .htaccess config?
2. In the documentation you only mention the location where to put the staticfilecache block for the
   "oldschool .htaccess rewrite rules". But which location is the the correct one, for the .htaccess rules which come with recent TYPO3 releases starting 7 LTS up to 9?

Best regards,
Uwe

I have made a dynamic manifest.json file as a typeNum and unfortunately staticfilcache adds comments to my file, which makes .json invalid..... :-(

I would like to keep the functionality to add comments to the genetaed .html files, but disable caching for a list of filenames.... is that possible?

I falsely submitted this bug report to the TYPO3 Core tracker, but moved it to the old static file cache bug tracker on forge.typo3.org already, but actually it belongs here.... (found this out later, after I found out that staticfilecache is the casue for this). There's the forge ticket:

https://forge.typo3.org/issues/84527

Folgende scenario:

• ich bin im typo3 backend angemeldet
• leere ich das frontendcache
• ich logge aus
• lade die seite neu

Ergebniss:

• typo3 frontend cache wird generiert aber file cache nicht!
• bis zum nächste cache ablauf wird der filecache garnicht verwendet

Wenn ich in browser aber im privat modus das erste mal die seite aufrufe dann ist alles ok, beide cache wird generiert. Ich denke das ist darum weil nach abmeldung aktuell wird die cookie be_typo_user nicht gelsöcht. Kann sein dass es ein typo3 fehler ist weil das typo3 fronteend cache wird aufgebaut, nur ein bestimmte hook lauft nicht ab welche staticfilecache benutzt?

If I cache a page as static file and that page sends a HTTP header, e.g. TypoScript additionalHeaders those headers are lost.
It would be nice if they are added to the .htaccess to be set when the file is retrieved instead.

Hey there, I'm not quiet sure if this really is an issue in the extension or a misconfiguration in my system. I'm hoping you can help me to solve this. I don't use any frontend users, but I get the message "User or group are set" in info module on every page. I tried it in a separate browser without active backend session. I digged a little deeper and found this:

In NoUserOrGroupSet.php the following method "TYPO3\CMS\Frontend\Controller\TypoScriptFrontendController->isUserorGroupSet()" checks if $GLOBALS['TSFE']->fe_user->user is an array. When I debug it, I get an Array with a few "ses_.." keys e.q. "ses_anonymous" an "ses_data".

Any idea what causes this issue?

Short description

If I enter the code in the .htaccess, I get an output like
�‹��í[krÛH’þ-Eô�ª¹±-¹[ À—�É${dùÙkÙ�[îé...

Steps to reproduce the problem

1. TYPO3 8.7.12 and staticfilecache 5.3.3
2. Enter the lines from the .htaccess code in your documentation to my .htaccess

What is the expected behaviour?

Get the usual HTML page compressed.

More Details

• Which browsers/versions does it happen on?
  chrome / firefox
• Did this work before?
  tested it on muliple projects

Thanks a lot in advance.

Hej there,

unfortunately, the original extension in the TER is not yet 8.7 compatible. I was hoping your fork was, but the backend module does not.

Are you planning on supporting 8.7 fully?

Short description

When I try to open page and I choose "Force the static cache generation (!!!)" I have this error

Core: Exception handler (WEB): Uncaught TYPO3 Exception: #1476107295: PHP Warning: array_diff_assoc(): Argument #1 is not an array in /home/users/website/public_html/doamin.com/vendor/typo3/cms/typo3/sysext/frontend/Classes/Controller/TypoScriptFrontendController.php line 3489 | TYPO3\CMS\Core\Error\Exception thrown in file /home/users/website/public_html/doamin.coml/vendor/typo3/cms/typo3/sysext/core/Classes/Error/ErrorHandler.php in line 107. Requested URL: https://domain.com/test
...

Steps to reproduce the problem

Edit page and choose "Force the static cache generation (!!!)" open the page and I see "Oops, an error occurred!"

More Details

TYPO3 Version : 8.7.9 - 8.7.12
PHP Versin : 7.1.8

Hi @lochmueller,
I upgraded to TYPO3 8 LTS and switched from nc_staticfilecache to staticfilecache.
It all works good except the fact that when an editor edits something in the BE the static file is still serving the old page in the FE.
The editor has to clear frontend caches to see the edit.
I'm still investigating this but maybe you can point me in the right direction faster.
Thanks anyway for this beautiful extension!

Boost mode is on
Latest realurl (2.3.2)
linkVars = L(0-1)
'FE' => [
'disableNoCacheParameter' => '1',
'loginSecurityLevel' => 'rsa',
],

The question is, how can i debug this?

Currently the backend display the children of the selected page. This mode has several drawbacks (e.g. editor could not select the root node to display the entries of the homepages).

So this should be configuable:
Mode "Current" (default): Display the entries of the current page
Mode "Child": Display the entries of all childs
Mode "Both": Display the entries of the current page and all children.

Take care of the description in the backend module (more general or custom for each mode).
Furthermore the "Page ID" should be part of the table in "Both" mode.

TYPO3 version: 8.7.10
Ext. version: 5.2.0

When the .htaccess file got generated, all HTTP headers get escaped by Fluid.
This can break the some Headers, e.g. http/2 push or CSP headers.

Probably not that important as currently there is only a use statement inside "Configuration/TCA/Overrides" which shouldn't be a problem as those files are required / included per file.
However, it is probably best to still avoid doing so and only use those for classes in case this might change in the future.
For ext_localconf.php / ext_tables.php it is required to not have a "use" statement there as those files are concatenated and so one might run into fatal errors if multiple extensions use them.

Would there be an official ter release?
package not available via composer.typo3.org

Bug Report

Current Behavior
Not generating static files and telling "static cache disabled on domain" for every single page

Expected behavior/output
After installation the Extension should be able to cache all pages which do not have login forms.

Environment

• TYPO3 version(s): 9.4.0
• PHP: 7.2.9
• Webserver Setup: nGinx v1.11.10 (proxy) + Apache v2.4.10
• staticfilecache version: 6.0.2
• Composer Mode: No
• OS: Linux (Debian 8.11)

Feature Request

Is your feature request related to a problem? Please describe.
Add possibility to enable/disable static file cache for each domain record.

Describe the solution you'd like
Introduce the same checkbox like in the page properties (enable by default) to disable the generation of static file caches. Add a new Rule to check, if the current domain record allow static file cache generation.

See title and screenshot. Applies to all pages. Cache entries of root pages are displayed on ID 0.

LLL:EXT:staticfilecache should be LLL:EXT:nc_staticfilecache in pages.php

$tmp = [
    'tx_staticfilecache_cache' => [
        'exclude' => 0,
        **'label' => 'LLL:EXT:staticfilecache/Resources/Private/Language/locallang.xlf:staticfilecache.tx_staticfilecache_cache',**
        'config' => [
            'type' => 'check',
            'default' => '1',
        ],
    ],
    'tx_staticfilecache_cache_force' => [
        'exclude' => 0,
        **'label' => 'LLL:EXT:staticfilecache/Resources/Private/Language/locallang.xlf:staticfilecache.tx_staticfilecache_cache_force',**
        'config' => [
            'type' => 'check',
            'default' => '0',
        ],
    ],
];

So I tried to figure out boost mode and am curious. What is the status on boost mode? It doesn't look completely developed. The lack of documentation also seems to indicate that.

So I figured I have to schedule runcacheboostqueue and cleanupcacheboostqueue. That will automatically update pages that are changed though the backend. That works great. 👍

But than I looked at the StaticFileBackend. So remove will schedule an update of the page... So far so good. But calling flush will schedule only expired entries for an update and calling collectGarbage will actually remove all outdated without scheduling anything. I would expect flush to schedule everything and collectGarbage to schedule only expired entries.

And there is more which I haven't tested but just figure though the source: If a page gets unavailable, the update will have a != 200 result and try to remove the entry from the cache ~ which should schedule it again. So that should result in an endless retry cycle if i'm not overlooking something.

So I figure that it can be used but doesn't work as advertised since collecting cache garbage (which should definitely be done) will make the cache cold without scheduling a warmup.

Did I figure that out correctly? Do you need help or do you want to continue develop the boost mode?

Can this officially be used? / Or is it really used by composer?

Hi,

if one has a page reachable under multiple URLs - for SEO the corresponding canonical tag exists - then only the first call to that page will be cached.
The reason: TYPO3 has the page in the cache and doesn't call insertPageIncache again.
Maybe one should check at the end of content processing, if the page was cachable:
meaning not INT script and not set as no_cache and if so, create the entry if it doesn't exist.

On a site that's not in "composer mode": can I download the code, run composer locally and thus add all the required dependencies to the extension?

Or is there another way to make it work? Can you point me in the right direction?

It appears that [FE][lockIP] must be 0 in order to avoid

The page is not static chachable via TypoScriptFrontend
User or group are set

Don't know if that's a requirement, something that should be documented, a coincidence.

If you run into problems, it is possible that you need to flush fe_sessions and fe_session_data, as in my case.

Currently, the RAW URI is used as cache identifier in \SFC\Staticfilecache\StaticFileCache. This works well as long as the whole URI is alphanumerical + a couple of extra chars. If however the URI contains special characters, TYPO3\CMS\Core\Cache\Frontend\AbstractFrontend throws an exception when validating the cache key against the regex in TYPO3\CMS\Core\Cache\Frontend\FrontendInterface::PATTERN_ENTRYIDENTIFIER.

In our case there is a korean subpage at http://domain.com/kr/뉴스/프레스-릴리즈/ which contains special characters.

A proposed fix for this issue would be easy: Punycode-escape the whole URL before returning it from in \SFC\Staticfilecache\StaticFileCache::getUri

unserializing configuration fails while installing

Due to PHP strict_mode unserializing extension configuration fails while installing extension.

Steps to reproduce the problem

1. Take a TYPO3 instance where staticfileacache has never been installed
2. Activate staticfilecache in Extension Manager

What is the expected behaviour?

Activated extension.

What is observed behaviour?

Oops, an error occurred!
unserialize() expects parameter 1 to be string, null given

unserialize is called twice with (yet) missing Ext-Configuration (\SFC\Staticfilecache\Configuration::registerHooks(), \SFC\Staticfilecache\Configuration::registerCachingFramework()).
Thus, $GLOBALS['TYPO3_CONF_VARS']['EXT']['extConf']['staticfilecache'] is NULL, but strict_mode forces unserialize to accept only a string.

HTTP/2 Push is a really nice function, and also possible with some Ext. for TYPO3. But these extensions not working, if staticfilecache enabled. The reason is clear. But it would be nice, if you can find a way to implementing sending HTTP/2 Push headers too.

Maybe on generating the static files, a headers file will be created. And on an request a real simple PHP Script is called to read these header file and send it to the browser?

Maybe thats a possible way to keep the overhead (like calling TYPO3 API, DB, etc.) low.

Bug Report

Current Behavior
The error
Core: Error handler (FE): PHP Warning: count(): Parameter must be an array or an object that implements Countable in /var/www/vhosts/[REMOVED]/web/typo3conf/ext/staticfilecache/Classes/StaticFileCache.php line 260
is shown in backend log.

Expected behavior/output
no warning in backend log

Environment

• TYPO3 version(s): 8.7.18
• PHP version: 7.2.8
• staticfilecache version: 6.0.1
• Is your TYPO3 installation set up with Composer (Composer Mode): yes
• OS: Linux

Possible Solution
http://php.net/manual/de/migration72.incompatible.php#migration72.incompatible.warn-on-non-countable-types

Additional context
none

Add a configuration for the different hooks in the caching process.
In same cases the 'insertPageIncache' is to early and hooks like 'contentPostProc-output' or even 'hook_eofe' are better. Configure this by the extension configuration for each TYPO3 instace.

The expire time in writeHtAccessFile uses time() + lifetime
instead of EXEC_TIME as is used when setting the cache entry information.

However, it might make sense to have it a few seconds later so that the page cache expires before the htaccess rule expires. However, probably still better to use EXEC_TIME and manually add a second or so.

... to prevent that .htaccess settings effect other cache entries except the related cache entry.

What about removing the version configuration within composer.json? It's not necessary if you tag each each version in git directly