dciccale / node-htmlprocessor Goto Github PK

View Code? Open in Web Editor NEW

47.0 7.0 22.0 288 KB

Process html files using special comments

License: MIT License

JavaScript 82.11% HTML 17.89%

node-htmlprocessor's Introduction

htmlprocessor

npm install -g htmlprocessor

Grunt/Gulp task

This module is the processor behind grunt-processhtml, gulp-processhtml tasks.

For plenty of examples visit the documentation.

CLI

Outputs help

$ htmlprocessor -h
Usage: htmlprocessor file-to-process.html [options]

  -h, --help             display this help message
  -v, --version          display the version number
  -l, --list             file to output list of replaced files
  -o, --output           file to output processed HTML to
  -d, --data             pass a JSON file to processor
  -e, --env              specify an environment
  -r, --recursive        recursive processing
  -c, --comment-marker   change the comment marker
  -i, --include-base     set the directory to include files from
  -s, --strip            strip blocks matched by other environments
  --custom-block-type    specify custom block type

Outputs version number

$ htmlprocessor -v

Outputs to file-to-process.processed.html.

$ htmlprocessor file-to-process.html

Outputs to processed/file.html.

$ htmlprocessor file-to-process.html -o processed/file.html

Pass some data

$ htmlprocessor file-to-process.html -o processed/file.html -d data.json

Specify an environment

$ htmlprocessor file-to-process.html -o processed/file.html -e dev

Allow recursive processing

$ htmlprocessor file-to-process.html -o processed/file.html -r

Change the comment marker to 

$ htmlprocessor file-to-process.html -o processed/file.html --comment-marker process

List option

Create a list of files that were replaced and use that list to streamline the build process.

Note: This new option does not affect in any way the previous existing functionality (i.e. it's backward compatible).

$ htmlprocessor file-to-process.html -o processed/file.html --list wrk/replacement.list

Assumning you have this code in an HTML (or JSP)

  .
  .
  .
  <!-- build:css content/myApplication.min.css -->
  <link rel="stylesheet" href="js/bower_components/bootstrap/dist/css/bootstrap.css" />
  <link rel="stylesheet" href="content/bootstrap-responsive.min.css" needed />
  <link rel="stylesheet" href="content/styles.css" />
  <link rel="stylesheet" href="content/myApplicationStyles.css" />
  <!--/build-->
  .
  .
  .
  <!-- build:js js/myApplication.min.js -->
  <script src="js/bower_components/jquery/dist/jquery.js"></script>
  <script src="js/bower_components/angular/angular.js"></script>
  <script src="js/bower_components/angular-route/angular-route.js"></script>

  <!-- App libs -->
  <script src="app/app.js"></script>
  <script src="app/filters/filters.js"></script>
  <script src="app/controllers/applications.js"></script>
  <!--/build-->
  .
  .
  .

The file "wrk/replacement.list" will contain something like this:

file-to-process.html:js/bower_components/bootstrap/dist/css/bootstrap.css
file-to-process.html:content/bootstrap-responsive.min.css
file-to-process.html:content/styles.css
file-to-process.html:content/myApplicationStyles.css
file-to-process.html:js/bower_components/jquery/dist/jquery.js
file-to-process.html:js/bower_components/angular/angular.js
file-to-process.html:js/bower_components/angular-route/angular-route.js
file-to-process.html:app/app.js
file-to-process.html:app/filters/filters.js
file-to-process.html:app/controllers/applications.js

And you can use these commands to concatenate and eventually minify without having to update the build to tell it where it should pickup each files. Also, in this way it orders the global file content in the same manner as your individual includes originally were.

sh -c "cat `cat wrk/replacement.list | grep '\.js$' | cut -d: -f2` > dist/js/myApplication.js"
sh -c "cat `cat wrk/replacement.list | grep '\.css$' | cut -d: -f2` > dist/css/myApplication.css"

If you processed more than a single "html" file, you can change the grep like this:

... | grep 'file-to-process.html:.*\.js$' | ... > dist/js/myApplication.js
... | grep 'other-file-to-process.html:.*\.js$' | ... > dist/js/myApplicationOther.js

The originating file name is included in the list file for that very purpose.

License

See LICENSE.txt

node-htmlprocessor's People

Contributors

Stargazers

Watchers

Forkers

robertjlooby nesk makovich tarzan cpu999 ulpian joeldenning satyajitsinh raibaz zhangciwu jayeb sake92 davidzoli jacquerie presleyhank gotcha9999 advancedcsg danzimm nci-ccr-oit

node-htmlprocessor's Issues

-d paths should resolve relative to the current directory

Hi, thanks for writing this. Really useful and flexible. Running into a glitch when passing data: it seems to resolve relative to the executable. Prefixing path with '.' did not help.

$ htmlprocessor -v
0.1.5 

~/repos/rrocks on tag_display*
$ ls *.json
index_opts_latest.json package.json
~/repos/rrocks on tag_display*

$ htmlprocessor index.html -d './index_opts_latest.json' 

module.js:340
    throw err;
          ^
Error: Cannot find module '/usr/local/lib/node_modules/htmlprocessor/bin/index_opts_latest.json'
    at Function.Module._resolveFilename (module.js:338:15)
    at Function.Module._load (module.js:280:25)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at Object.<anonymous> (/usr/local/lib/node_modules/htmlprocessor/bin/htmlprocessor:26:22)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:497:10)
~/repos/rrocks on tag_display*

Thanks!

Unfriendly error message for undefined variables

Getting this when running via the CLI or gulp-htmlprocessor after a fresh install on Windows.

C:\Code\FullScreenPokemon>htmlprocessor src\index.html -o lib\index.html
undefined:8
((__t = ( version )) == null ? '' : __t) +
          ^

ReferenceError: version is not defined
    at eval (eval at template (C:\Code\FullScreenPokemon\node_modules\htmlprocessor\node_modules\lodash\dist\lodash.js:6306:22), <anonymous>:8:11)
    at Function.template (C:\Code\FullScreenPokemon\node_modules\htmlprocessor\node_modules\lodash\dist\lodash.js:6312:16)
    at HTMLProcessor.template (C:\Code\FullScreenPokemon\node_modules\htmlprocessor\lib\blocktypes\template.js:8:30)
    at HTMLProcessor._replace (C:\Code\FullScreenPokemon\node_modules\htmlprocessor\lib\htmlprocessor.js:68:45)
    at HTMLProcessor.<anonymous> (C:\Code\FullScreenPokemon\node_modules\htmlprocessor\lib\htmlprocessor.js:91:21)
    at Array.forEach (native)
    at HTMLProcessor._replaceBlocks (C:\Code\FullScreenPokemon\node_modules\htmlprocessor\lib\htmlprocessor.js:86:10)
    at HTMLProcessor.processContent (C:\Code\FullScreenPokemon\node_modules\htmlprocessor\lib\htmlprocessor.js:115:18)
    at HTMLProcessor.process (C:\Code\FullScreenPokemon\node_modules\htmlprocessor\lib\htmlprocessor.js:102:15)
    at C:\Code\FullScreenPokemon\node_modules\htmlprocessor\index.js:43:24

Edit 8/12: The cause is that index.html contains a version variable that isn't being provided. There should be a friendlier error message about this, yes?

<html>
    <body>
        <% version %>
    </body>
</html>

--list option resulting in TypeError('Unknown encoding: ' + encoding)

process-html version: 0.1.10 release
node version: 4.2.0

As title says, I'm seeing an issue when using the --list option. See detailed error trace below:

_stream_writable.js:239
    throw new TypeError('Unknown encoding: ' + encoding);
    ^

TypeError: Unknown encoding: encoding
    at WriteStream.setDefaultEncoding (_stream_writable.js:239:11)
    at new WriteStream (fs.js:1830:10)
    at Object.fs.createWriteStream (fs.js:1789:10)
    at HTMLProcessor.processContent (/Users/fredrik/node_modules/gulp-processhtml/node_modules/htmlprocessor/lib/htmlprocessor.js:177:19)
    at DestroyableTransform.processContent [as _transform] (/Users/fredrik/node_modules/gulp-processhtml/index.js:30:33)

Seems the problem is the encoding property of the config object passed to createWriteStream. Looking at line #177

177 listFile = fs.createWriteStream(this.options.list, {
178       flags:    "a",
179       encoding: "encoding",
180       mode:    parseInt('744', 8)
181     });

I couldn't find and documentation for passing "encoding" as the value for that property. Changing it to e.g utf8 fixes the issue for me. I assumet the value "encoding" does work (or at least has worked) for users of the module so I'm guessing it's a versioning thing (perhaps something's changed with node). Any ideas?

Nesting template/if/include not working

Hi- love this tool, not sure if using it correctly. I'm doing conditionals with properties to include images--

<!-- build:template -->
<% if (WantImages) { %>

    <div style='display:none'><img src='/images/converted/react-native-todo.jpg'/></div>

<% } %>
<!-- /build -->

Processing with ./node_modules/htmlprocessor/bin/htmlprocessor index.html -d './index_opts_latest.json' -o build/index.html -r gives the right output:

<div style='display:none'><img src='/images/converted/react-native-todo.jpg'/></div>

But: Nesting an include within it:

<!-- build:template -->
<% if (WantImages) { %>

    <!-- build:include image_tags.txt -->
    <!-- /build -->

<% } %>
<!-- /build -->

Results in un-processed rules in the output

<!-- build:template -->
<% if (WantImages) { %>

    <div style='display:none'><img src='/images/converted/react-native-todo.jpg'/></div>

<% } %>
<!-- /build -->

Is this expected/correct? Can solve it by calling htmlprocessor again, but was hoping the -r would do it. Thanks.

Call to _getMatch not passing in 'line'

The Parser prototype method, _getMatch(), has a signature with one parameter, line, being passed in but in _writeToList(), 'line' is not getting passed in.

Parser.prototype._writeToList = function (line, filePath) { console.log('_writeToList line: ', line); var match = this._getMatch(); <== if you pass 'line' in here things work again. console.log('match'); if (match) { this.listFile.write(filePath + ':' + match + '\n'); } }

Cannot include some files

Hi! I'm using gulp plugin (https://github.com/Wildhoney/gulp-processhtml) in my project (https://github.com/mcmimik/Test-for-processhtml-bug).

There is one .html file in my project, with multiple included templates (.htm) that include other files (.svg).
The result is a bug: https://monosnap.com/file/3yokG0sxmPkevKudXMTsZkbvTmaWdN

I've spent plenty hours to figure out what's wrong, but it's not clear for me.

--list option is outputting unexpected content

when running htmlprocessor www/index.html -o dist/index.html --list replacement.list the replacement.list file has the following output rather than the src attribute value only:

build: <link rel="stylesheet" href="vendor/bootstrap/dist/css/bootstrap.css"/>,vendor/bootstrap/dist/css/bootstrap.css

I would expect the following:

build: vendor/bootstrap/dist/css/bootstrap.css

Lodash version used is vulnerable

Hi,
npm audit for module htmlprocessor(0.2.6) points to a vulnerable dependency as shown below

`High Prototype Pollution

Package lodash

Patched in >=4.17.12

Dependency of htmlprocessor [dev]

Path htmlprocessor > lodash

More info https://npmjs.com/advisories/1065`

May I know how to resolve this or any update on it? It will be very helpful.

The remove-feature stopped working in v0.1.10(?)

I'm not sure why, but somewhere in between v0.1.9 and v0.1.10, the remove-functionality stopped working.

Take the following example:

<!-- build:remove:testa -->
 test A
<!-- /build -->
<!-- build:remove:testb -->
 test B
<!-- /build -->

When using grunt-processhtml like so:

testa: {
    options: {
        strip: true
    },
    files: [...]
}

The result is (surprisingly):

 test A
<!-- build:remove:testa -->
 test B
<!-- /build -->

whereas v0.1.9 yielded (correctly):

 test A

So, even though strip worked, remove somehow did not.

Keep charset attribute after process

When I use the processhtml, it doesn't keep the charset attribute.

<!-- build:js lib/lib.min.js -->
    <script charset="UTF-8" src="lib/some_strange_encoding.js"></script>
<!-- /build -->

Results in:

    <script src="lib/lib.min.js"></script>

Order of attributes matters

dciccale/grunt-processhtml#27

In version 0.3.3 this is still an issue.

<!-- build:[src] /images -->
<img alt="some image" src="/src/cat.png">
<!-- /build -->

<!-- changed to -->

<img alt="some image" src="/images/cat.png">


<!-- build:[src] /images -->
<img src="/src/cat.png" alt="some image">
<!-- /build -->

<!-- changed to -->

<img src="/src/cat.png">

Any suggestions and fixes would be great.

Another thing: what is the best way to replace all image URLs in the document with the same URL?

Is it possible to add a query string (cache buster) to the URLs?

Hi, for forcing browsers to reload JS/CSS assets if the html is rebuilt, I'd like to append a "random" (Math.random() or Date.now() or the like) query string to the URLs, like:

<!-- build:js app.min.js -->
<script src="my/lib/path/lib.js"></script>
<script src="my/deep/development/path/script.js"></script>
<!-- /build -->

<!-- changed to -->
<script src="app.min.js?v=12351345"></script>

Is that possible somehow? Thanks

<link> become self-closing during build processing

<link> elements are not required to be self-closed with HTML, just XHTML, but build:css outputs self-closing elements when un-closed elements are originally defined.

Input:

<!-- build:css ./vendor/angular/angular-csp.min.css -->
<link rel="stylesheet" href="../node_modules/angular/angular-csp.css">
<!-- /build -->

Output:

<link rel="stylesheet" href="./vendor/angular/angular-csp.min.css"/>

Expected Output (un-closed element):

<link rel="stylesheet" href="./vendor/angular/angular-csp.min.css">

Environment:

Windows 8.1
Node.js 6.9.4
npm 3.10.10
htmlprocessor 0.2.4

Problems with onload attribute in a block template

I have this code in my index.html to prevent css blocking:

<!-- build:template
        <link rel="stylesheet" href="css/style.css" media="none" onload="if(media != 'all')media = 'all'">
 /build -->

But, when i saw the result file, i found this code:

<link rel="stylesheet" href="css/style.css" media="none" onload="&quot;all&quot;!=media&&(media=&quot;all&quot;)">

Look is a problem with the slashes character and i can't found a soulution.
I found a 'cheat' using data functionality, to replace the 'all' value, and solve my problem, but i think this is a bug.

Support custom attributes for inline styles like the scope attr

As seen in e0f25fd, you introduced the scope attribute. I want to use grunt-processhtml to get inlined css under the style element with the attribute amp-custom which is for Google AMP-Sites. It would be nice, if I could set a custom string like so:

<!-- build:css inline amp-custom ../assets/styles/main.min.css -->
	foobar
<!-- /build -->

Results in:

<style amp-custom>
	foobar
</style>

If this is not possible, I'd like to have the option to suppress the output of <style> overall.

missing support for boolean attributes

A boolean attribute value is set by its presence or absence. Moreover, "true" and "false" are not allowed by the html spec 2.4.2. Boolean attributes.