Giter Club home page Giter Club logo

sourceopt's Introduction

EXT:sourceopt

Build Status Scrutinizer Code Quality Code Coverage

  • SourceOpt : reformatting the (x)HTML output & removal of new-lines, comments and generator-info
  • RegExRep : search and replace strings using your regular expressions ; embrace regex and migrate now
  • SVGstore : combines all SVG selected within elements into one <symbol> file and replaces <img> by <use>

Installation

sourceopt/composer.json

Lines 19 to 20 in 2346673

"php": "^7.4||^8.0",
"typo3/cms-core": "^9.5||^10.4||^11.5||^12.0"

composer require lochmueller/sourceopt
  • via TypoScript

    [constants]

    @import 'EXT:sourceopt/Configuration/TypoScript/constants'
    

    [setup]

    @import 'EXT:sourceopt/Configuration/TypoScript/setup'
    
  • via input device

    • add [EXT:sourceopt/Configuration/TypoScript] into Include static at Includes in Template

Configuration

Include the extension and go to the Constant Editor of a template where you find all options under PLUGIN

Performance

The PHP process need server performance, because there are several search/replace operations in the logic

Reference

Note: The following features are executed in reverse order

SourceOpt

TypoScript [constants] || prepend config. at [setup]

Property Type Description Default
sourceopt.enabled boolean Is the optimization enabled for this template 1
sourceopt.removeGenerator boolean Remove <meta name="generator" content="TYPO3 CMS"> 1
sourceopt.removeComments boolean Remove HTML-Comments 1
sourceopt.removeComments.keep array Spare these listed comments: Regular expressions that match comments that should not be removed. Very useful e.g. to keep the TYPO3SEARCH-Comments so indexed_search can work properly .10
sourceopt.removeComments.keep.10 string Spare TYPO3SEARCH-Comments from removal /^TYPO3SEARCH_/usi
sourceopt.headerComment string Your additional (appended) header comment [empty]
sourceopt.formatHtml integer Formats the code beautiful and easy readable. New lines and tabs are used in the usual way of illustrating the structure of an XML code.
Options
* 0 => off
* 1 => no line break at all (code in one line)
* 2 => minimalistic line breaks (structure defining box-elements)
* 3 => aesthetic line breaks (important box-elements)
* 4 => logic line breaks (all box-elements)
* 5 => max line breaks (all elements).
4
sourceopt.formatHtml.tabSize integer Defines the size of the tabs used for formating. If blank one tab is used. If a number is specified the specified number of blank spaces is used instead. This will only work together with formatHtml [empty]
sourceopt.formatHtml.debugComment boolean Includes a comment at the end of the html source code that points the numbers of open html tags. This will only work together with formatHtml [empty]

RegEx Replace

TypoScript [setup] incl. stdWrap

config.replacer {
  search {
    1 = /(?<="|')\/?(fileadmin|typo3temp|uploads)/

    give-me-cherries = /fruit/

    wrapBoldly < tmp.find
    wrapBoldly.wrap = /|/
  }
  replace {
    1 = //cdn.tld/$1

    give-me-cherries = cherry

    wrapBoldly < tmp.repl
    wrapBoldly.wrap = <b>|</b>
  }
}

Note: both arrays will be ksort'ed in "natural order"

HowTo migrate

from jweiland/replacer : 1.4 - 2.x
  • regex replace config\.tx_(?:\w*replace\w*) to config.replacer
  • remove config.tx_replacer.enable_regex = 1
from maxserv/replacecontent 2013 - 2017
  • regex replace config\.tx_(?:\w*replace\w*) to config.replacer
from typo3-ter/ja_replacer 2009 - 2013 || phorax/ja-replacer 2016
composer install jweiland/replacer

& replace config.tx_ja_replacer to config.tx_replacer

  • regex replace config\.tx_(?:\w*replace\w*) to config.replacer
composer install jweiland/replacer

& replace plugin.fereplace.pairs to config.tx_replacer .. ah .. and one regex

  • search (\n\s*)(\d+)(?:{\s+|\.)old\s*=\s*([^\n]+).+?new\s*=\s*([^\n]+)
  • replace $1search.$2 = $3\n$1replace.$2 = $4
from typo3-ter/regex 2009
  • regex replace config\.regex to config.replacer
  • regex replace (?:\s)(\w+)\s*=\s* to search.$1 =
  • regex replace (\w+)\.replacement\s*=\s* to replace.$1 =
from any other tool or just for regex
  • regex replace (?:config|plugin)\.tx_any_other_tool to config.replacer
  • inside block search
    • regex replace \/ to \\/ (carefully)
    • regex replace \s*=\s*(.+) to = /$1/
  • inside block replace
    • regex replace \s*=\s*(.+) to = $1
    • consider a PR for conversion specifics

SVGstore

TypoScript [constants] || prepend config. at [setup]

Property Type Description Default
svgstore.enabled boolean Is the SVG extract & merge enabled for this template 1
svgstore.fileSize integer Maximum file size of a SVG to include (in [byte]) 50000

ToDo:

sourceopt's People

Contributors

7elix avatar agentivcreatur avatar bmoex avatar can0199 avatar ceremony64 avatar erhaweb avatar fharzendorf avatar futureminds-it avatar geldmacher avatar holgerkraemer avatar josefglatz avatar kanti avatar lochmueller avatar lolli42 avatar netcoop avatar runepiper avatar velletti avatar web-it-solutions avatar xerc avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

sourceopt's Issues

Line-breaks get removed in the value-attribute of input fields

Multi-line content in the value-attribute of input-tags is converted to single line.
E.g. powermail uses hidden input fields to pass along multi-line content of textarea-inputs.

Is it possible to keep line-breaks untouched in the value-attribute of input-tags?

TYPO3 9.5.x TER

Hey there and thanks for your great extension.

When will you upload the newest version for TYPO3 9.5.x to the TER?

Thanks in advance :-)

include SVG parsing

there is a indentation issue with inline SVG

<path></path>
<path></path>
<path/>
  <path/>

<path></path>
<g>
  <path></path>
</g> </svg>
<path/><g>
    <path/></g> </svg>

sourceopt.formatHtml = 4
sourceopt.formatHtml.tabSize = 2

Documentation should include warning that huge html content is slow

I was just looking at a site that had huge server side rendering times. It turned out that nearly 4 seconds were eaten by the main hook "cleanUncachedContent" handling a 130kB HTML source file. With disabling the extension parse time dropped to 150 milliseconds.

Imho, the documentation should at least contain a big fat warning that parsing is very slow for parsing "bigger" HTML content.

PHP Warning: preg_match(): Empty regular expression

Hi! This message keeps appearing in TYPO3 error log:
Core: Error handler (FE): PHP Warning: preg_match(): Empty regular expression in /var/www/clients/client1/web1/dev32aug/typo3conf/ext/sourceopt/Classes/Manipulation/RemoveComments.php line 70
Also - comment "This website is powered by TYPO3 - inspiring people to share!" isn't removed.
I'm using TYPO3 v7.6.22 and Sourceopt 1.0.0

Thank you.

Query in SvgFileRepository not compatible with sql_mode=only_full_group_by

'sql_mode=only_full_group_by' is a default setting in the latest versions of MariaDB and MySQL.

The query in SvgFileRepository.php doesn't meet this requirement. Easiest way to fix it without rewriting the query is adding the select fields to the group-by:

->groupBy('sys_file.uid', 'sys_file.storage', 'sys_file.identifier', 'sys_file.sha1')

Provide support for PHP 7.3

In the constraints in ext_emconf.php the PHP version is limited to 7.2 only. Could you please increase the version constraint?

Classes/User/FrontendHook.php missing in latest *.t3x

Hi,

today I have updated to the latest version from the TYPO3 repository in EM. I saw the change, that Classes/User/PageRendererHook.php was removed and wondered, why there is no new Hook to replace the old one.

To get sure, I redownload the *.t3x file from http://typo3.org/extensions/repository/view/sourceopt and installed it.

Again, no hook found in the folders and the extension doesn't work in the system.

I looked at this github repository and saw: sourceopt/Classes/User/FrontendHook.php.

I downloaded and installed the latest master from github and now the extension work as expected.

It may be true that that this class is missing in the latest release in the TYPO3 repository?

Best regards

sourcopt and 7.6.8 disable not working

strange behavior - trying sourceopt with 7.6.8. after installation sourcopt is packing the html correct with config 4.

but disableing it within constants or direct in files (constants.txt or setup.txt) isn't working.

[Bug] PHP 7.4 & sourceopt 2.0.0 does have a bug

I'm using:

TYPO3 v9.5.13
PHP v7.4.1
sourceopt v2.0.0

My typoscript contains this:

config {
  ##########################################
  ### Generator entfernen
  ########################################## 
  sourceopt{
    enabled = 1
    removeBlurScript = 1
    removeGenerator = 1
    removeComments = 1
    formatHtml = 1
    dropEmptySpaceChar = 1
  }
}

But when I render the frontend it just keeps the part <meta name="generator" content="TYPO3 CMS" />

Output in one line impossible

$htmlArray[$z] = ' ';

I was wondering, what this whitespace is good for? If the user wants the output to be in one line and "super optimizied", this whitespace leeds to "> <" everywhere.

Does the code really needs it here? Maybe we can add a config.notEvenOneWhitespace to the constants to allow the user to change between whitespaces between tags and no whitespaces.

I used this config:

sourceopt.enabled = 1
sourceopt.removeGenerator = 1
sourceopt.removeBlurScript = 1
sourceopt.removeComments = 1
sourceopt.formatHtml = 1

TypeError with 4.0 due to wrong array access

With 4.0 I get this on every page (cleared all caches):

substr() expects parameter 1 to be string, null given | TypeError thrown in file sourceopt/Classes/Service/CleanHtmlService.php in line 220

On this position in code: It accesses on first run ($x=0) an invalid offset $htmlArray[-1]

for ($x = 0; $x < \count($htmlArray); ++$x) {
            // check if the element should stand in a new line
            $newline = false;
            if ('<?xml' == substr($htmlArray[$x - 1], 0, 5)) {

(which is a consequence of strict_types=1)

Error in CleanHtmlService.php

I can not reproduce, but sometimes I have this error in my administrationprotocol.

Core: Error handler (FE): PHP Warning: preg_match(): Empty regular expression in /var/www/httpdocs/typo3conf/ext/sourceopt/Classes/Service/CleanHtmlService.php line 525

Should not be possible due to (!empty), but...

[BUG] new line inside tag replacement - no white space kept

Hi,

if I have something like this:

<div>foo
bar</div>

With no whitespace between "foo" and "bar" except one new line. Then that whitespace gets completely lost and in the end the display will be "foobar" instead.
Most likely there will be a whitespace somewhere after or before a line break, so it will very likely not be an issue but it should still be fixed.

Tried with formatHtml = 1 and formatHtml = 4

cleanUncachedContent hook should not be registered in FE only

ext_localconf contains:

if (TYPO3_MODE == 'FE') {
... register main parsing hook
}

This "if" should vanish and the hook should be registerde always, otherwise this hook registration can not be seen in the BE "Configuration" module.

Took me some time to find this going-amok hook when I did a performance analysis ...

TER Version still requires TYPO3 9.5.99

I don't know if this version if going to TER, but there is a version 2.0, it just has different settings in the config like the one mentioned in the title.

Will that be changed?

New release

The extension can't currently be required with composer, since 1218f59 is not part of any tag. Could you release a patch version so we can switch from dev-master to a tag again?

PHP version constraint in ext_emconf.php doesn't match version constraint in composer.json

Bug Report
Current Behavior
PHP version constraint in ext_emconf.php doesn't match version constraint in composer.json
ext_emconf.php: 7.4.0-8.0.99
composer.json: ^7.4||^8.0
The later constaint (composer.json) allows installation under PHP 8.0 and 8.1 (up all version < 9). The constraint in ext_emconf.php only up to 8.0.99.
Expected behavior/output
Both constaints should be the same.
Possible Solution
Change contraint in ext_emconf.php to '7.4.0-8.99.99' or at least to '7.4.0-8.1.99'

SVG and self-closing tags break indentation

Is seems sourceopt cannot deal with svg tags (that is also path, group, etc) or maybe even with self-closing tags (path is always self-closing, e.g.).
When using formatHtml = 4, the source keeps indenting like this:

<path .../>
    <path .../>
        <path .../>

But is should be like this:

<path .../>
<path .../>
<path .../>

Even though formatHtml=4 seems like sugar on the cake and might everything but not recommended for real world scenarios, it would be nice to have it working properly, sind well-formated HTML is also a form of debugging. Things like double p-tags (which happens, if a f:format.html is wrapped inside a p-tag ... well, yes, it happens ...) would be easily recognizable.

Also, source formatting is a very weak topic in TYPO3 and only 2 ext (this one and "min") provide such a feature. And since TYPO3 is mainly a configuration monster, we need way more configuration than currently available!!! :-)
(seriously: Why not provide a "tag selector", so the user can define, what should be treated as block level element and what not - very much like phpstorm does)

Thank you!

[FEATURE] add javascript / css minifier option

TYPO3 by default does only a very simplistic javascript / css compression when activiating the corresponding compressJS / compressCSS options.
It would be nice to optionally add some existing third party libraries for this and use:
$GLOBALS['TYPO3_CONF_VARS'][TYPO3_MODE]['jsCompressHandler']
and
$GLOBALS['TYPO3_CONF_VARS'][TYPO3_MODE]['cssCompressHandler']

Warning: If this one is overriden, it might also be necessary to fix some "paths" in the CSS files (if it is excluded from concatination), see method \TYPO3\CMS\Core\Resource\ResourceCompressor::compressCssFile
especially:

if (!file_exists((PATH_site . $targetFile)) || $this->createGzipped && !file_exists((PATH_site . $targetFile . '.gzip'))) {
            $contents = $this->compressCssString(GeneralUtility::getUrl($filenameAbsolute));
            if (strpos($filename, $this->targetDirectory) === false) {
                $contents = $this->cssFixRelativeUrlPaths($contents, PathUtility::dirname($filename) . '/');
            }
            $this->writeFileAndCompressed($targetFile, $contents);
        }

Of course, the best option is probably to use a corresponding FE workflow but it would still be nice to have the option to compress those files without building a FE workflow.

New changes

Hi,
please push a new version to Packagist & TER with all new changes.

Update 9.5 in TER?

Hi there,

Is a new version for TYPO3 9.5 soon available in TER?

Thanks!!

[BUG] PHP 8.1 Undefined array key

problems with the use of sourceopt under
Typo3 11.5.16
PHP 8.1
sourceopt: 4.0.5

Errors:

PHP Warning: Undefined array key "svgstore." in /app/web/typo3conf/ext/sourceopt/Classes/Middleware/SvgStoreMiddleware.php line 30
PHP Warning: Undefined array key "formatHtml." in /app/web/typo3conf/ext/sourceopt/Classes/Service/CleanHtmlService.php line 72

Fix: check isset before use

Installation via composer and packagist is not working under TYPO3 8.7.0

Get:

Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - lochmueller/sourceopt dev-master requires typo3/cms-core ~6.2.0||~7.6.0||~8.4.0||~8.6.0||dev-master -> no matching package found.
    - lochmueller/sourceopt dev-master requires typo3/cms-core ~6.2.0||~7.6.0||~8.4.0||~8.6.0||dev-master -> no matching package found.
    - Installation request for lochmueller/sourceopt dev-master as 0.9.3 -> satisfiable by lochmueller/sourceopt[dev-master].

PHP Runtime Deprecation Notice if svgstore is enabled

If i enable svgstore (svgstore.enabled = 1) this exception pops up:

PHP Runtime Deprecation Notice: Automatic conversion of false to array is deprecated in /Users/***/typo3conf/ext/sourceopt/Classes/Service/SvgStoreService.php line 158

TYPO3 11.5.16
sourceopt 4.0.5

Whitespaces

When whitespaces get removed from a text, there's still one space before and one space after the text left:

<a> Label </a>

Expected behaviour: All whitespaces are removed around the text.

EXT:sourceopt v2.0.0
TYPO3 v10.4.4

Exception: ... SvgStoreService::addFileToSpriteArr() must be of the type array or null ...

After updating of sourceopt to version to 4.0.2 i got this error (and WEB Site is broken):

Uncaught TYPO3 Exception: Return value of HTML\Sourceopt\Service\SvgStoreService::addFileToSpriteArr() must be of the type array or null, bool returned | TypeError thrown in file /html/typo3/typo3conf/ext/sourceopt/Classes/Service/SvgStoreService.php in line 132. ...

Line-breaks are removed from ical Export

Hi,

I'm using TYPO3 9.5.7, sourceopt in the newest version, also tx_news in the newest version and we offer in the frontend ical files from tx_news for downloading. If I use sourceopt (tried formatHtml width config from 1-4) it removes the linebreaks from the ical files, and for this reason they are not working anymore. Is it possible to exclude some kind of special filetypes like ical?

Best regards and many thanks,
Michaela

SVGStore must not apply to <picture><img src="pic.svg"></picture>

The picture html construct allows the browser to select the most suitable picture based on the device specifications (e.g. orientation, resolution, etc.). It does require a single img element within it to properly function and to allow for backward compatibility as well (e.g. Internet Explorer and other trash)

However when the new svgstore feature replaces said <img> with a <svg>, it essentially breaks the element and the alternative images do not work.

More on <picture>: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/picture

[BUG] Split html into it's elements

Hello,
For some reason some pages returns this error:

Oops, an error occurred!
PHP Warning: count(): Parameter must be an array or an object that implements Countable in /var/www/html/web/typo3conf/ext/sourceopt/Classes/Service/CleanHtmlService.php line 186

It looks like preg_split (here) split html into it's elements doesn't returns array and it returns FALSE. This issue occures only on some pages.

Problem with special Pagetype

Hello,
since Version 4 i get problems with xml or json pages (special typeNums), they are not getting rendered.
I tried to deactivate sourceopt in typoscript for this pagetypes, but still they are not getting rendered.
Any ideas what I need to do?
Best regards and thank you,
Michaela

Php 7.3 support

As far as see, you have skipped support for PHP 7.3 in version 4.0.3 in composer.json, but not in ext_emconf.php.

I have a composer installation using PHP 7.3 and had to downgrade to version 4.0.1 now due to a bug in 4.0.2.

I think PHP 7.3 should be supported as it is the minimum required version for TYPO3 10 LTS.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.