Giter Club home page Giter Club logo

preprocess's Introduction

preprocess

NPM

Linux Build Status Windows Build Status Coverage Status dependencies dev-dependencies

Preprocess HTML, JavaScript, and other files with directives based off custom or ENV configuration

Configuration

Install via npm:

$ npm install --save preprocess

What does it look like?

<head>
  <title>Your App</title>

  <!-- @if NODE_ENV='production' -->
  <script src="some/production/lib/like/analytics.js"></script>
  <!-- @endif -->

</head>
<body>
  <!-- @ifdef DEBUG -->
  <h1>Debugging mode - <!-- @echo RELEASE_TAG --> </h1>
  <!-- @endif -->
  <p>
  <!-- @include welcome_message.txt -->
  </p>
</body>
var configValue = '/* @echo FOO */' || 'default value';

// @ifdef DEBUG
someDebuggingCall()
// @endif

Directive syntax

Basic example

The most basic usage is for files that only have two states, non-processed and processed. In this case, your @exclude directives are removed after preprocessing

<body>
    <!-- @exclude -->
    <header>You're on dev!</header>
    <!-- @endexclude -->
</body>

After build

<body>
</body>

All directives

  • @if VAR='value' / @endif This will include the enclosed block if your test passes
  • @ifdef VAR / @endif This will include the enclosed block if VAR is defined (typeof !== 'undefined')
  • @ifndef VAR / @endif This will include the enclosed block if VAR is not defined (typeof === 'undefined')
  • @include This will include the source from an external file. If the included source ends with a newline then the following line will be space indented to the level the @include was found.
  • @include-static Works the same way as @include but doesn't process the included file recursively. Is useful if a large file has to be included and the recursive processing is not necessary or would otherwise take too long.
  • @extend file.html / @endextend This will use the source from the external file indicated with the @extend tag to wrap the enclosed block.
  • @extendable This tag is used to indicate the location in a file referenced using @extend where the block enclosed by @extend will be populated.
  • @exclude / @endexclude This will remove the enclosed block upon processing
  • @echo VAR This will include the environment variable VAR into your source
  • @foreach $VAR in ARR / @endfor This will repeat the enclosed block for each value in the Array or Object in ARR. Each value in ARR can be interpolated into the resulting content with $VAR.
  • @exec FUNCTION([param1, param2...]) This will execute the environment FUNCTION with its parameters and echo the result into your source. The parameter could be a string or a reference to another environment variable.

Extended html Syntax

This is useful for more fine grained control of your files over multiple environment configurations. You have access to simple tests of any variable within the context (or ENV, if not supplied)

<body>
    <!-- @if NODE_ENV!='production' -->
    <header>You're on dev!</header>
    <!-- @endif -->

    <!-- @if NODE_ENV='production' -->
    <script src="some/production/javascript.js"></script>
    <!-- @endif -->

    <script>
    var fingerprint = '<!-- @echo COMMIT_HASH -->' || 'DEFAULT';
    </script>

    <script src="<!-- @exec static_path('another/production/javascript.js') -->"></script>
</body>

With a NODE_ENV set to production and 0xDEADBEEF in COMMIT_HASH this will be built to look like

<body>
    <script src="some/production/javascript.js"></script>

    <script>
    var fingerprint = '0xDEADBEEF' || 'DEFAULT';
    </script>

    <script src="http://cdn2.my.domain.com/another/javascript.js"></script>
</body>

With NODE_ENV not set or set to dev and nothing in COMMIT_HASH, the built file will be

<body>
    <header>You're on dev!</header>

    <script>
    var fingerprint = '' || 'DEFAULT';
    </script>

    <script src="http://localhost/myapp/statics/another/javascript.js"></script>
</body>

You can also have conditional blocks that are hidden by default by using the fictional !> end tag instead of --> after your condition:

<!-- @if true !>
<p>Process was run!</p>
<!-- @endif -->

JavaScript, CSS, C, Java Syntax

Extended syntax below, but will work without specifying a test

normalFunction();
//@exclude
superExpensiveDebugFunction()
//@endexclude

anotherFunction('/* @echo USERNAME */');

Built with a NODE_ENV of production :

normalFunction();

anotherFunction('jsoverson');

Like HTML, you can have conditional blocks that are hidden by default by ending the directive with a ** instead of */

angular.module('myModule', ['dep1'
    , 'dep2'
    /* @if NODE_ENV='production' **
    , 'prod_dep'
    /* @endif */
    /* @exclude **
    , 'debug_dep'
    /* @endexclude */
]);

Note: Hidden by default blocks only work with block comments (/* */) but not with line comments (//).

CSS example

body {
/* @if NODE_ENV=='development' */
  background-color: red;
/* @endif */

}
// @include util.css

(CSS preprocessing supports single line comment style directives)

Shell, PHP

#!/bin/bash

# @include util.sh

API

preprocess(source[, context[, options]]) -> preprocessedSource

Preprocesses a source provided as a string and returns the preprocessed source.

source

Type: String (mandatory)

The source to preprocess.

context

Type: Object Default: process.env

The context that contains the variables that are used in the source. For @extend variants and @include the additional context property src is available inside of files to be included that contains the current file name. This property is also available in the context of the source file if one of the preprocessFile*() API variants are used.

options

Type: Object

The options object allows to pass additional options to preprocess. Available options are:

options.fileNotFoundSilentFail

Type: Boolean Default: false

When using @include variants and @extend, preprocess will by default throw an exception in case an included file can't be found. Set this option to true to instruct preprocess to fail silently and instead of throwing to write a message inside of the preprocessed file that an included file could not be found.

options.srcDir

Type: String Default: process.cwd()

The directory where to look for files included via @include variants and @extend.

options.srcEol

Type: String Default: EOL of source string or os.EOL if source string contains multiple different or no EOLs.

The end of line (EOL) character to use for the preprocessed result. May be one of:

  • \r\n - Windows
  • \n - Linux/OSX/Unix
  • \r - legacy Mac
options.type

Type: String Default: html

The syntax type of source string to preprocess. There are 3 main syntax variants:

  • html, aliases: xml
  • js, aliases: javascript, jsx, json, c, cc, cpp, cs, csharp, java, less, sass, scss, css, php, ts, tsx, peg, pegjs, jade, styl
  • coffee, aliases: bash, shell, sh

preprocessFile(srcFile, destFile[, context[, callback[, options]]])

Preprocesses a sourceFile and saves the result to destFile. Simple wrapper around fs.readFile() and fs.writeFile().

srcFile

Type: String (mandatory)

The path to the source file to preprocess.

destFile

Type: String (mandatory)

The path to the destination file where the preprocessed result shall be saved.

context

See context attribute description of preprocess() function.

callback

Type: function(err)

The callback function that is called upon error or completion. Receives an error if something goes wrong as first parameter.

options

See options attribute description of preprocess() function. Differs only in that the default srcDir value is set to the path of the provided source file instead of process.cwd() and the default type is derived from source file extension.

preprocessFileSync(srcFile, destFile[, context[, options]])

Preprocesses a sourceFile and saves the result to destFile. Simple wrapper around fs.readFileSync() and fs.writeFileSync().

srcFile

Type: String (mandatory)

The path to the source file to preprocess.

destFile

Type: String (mandatory)

The path to the destination file where the preprocessed result shall be saved.

context

See context attribute description of preprocess() function.

options

See options attribute description of preprocess() function. Differs only in that the default srcDir value is set to the path of the provided source file instead of process.cwd() and the default type is derived from source file extension.

Usage Examples

var pp = require('preprocess');

var text = 'Hi, I am <!-- @echo USERNAME -->';

pp.preprocess(text);
// -> Hi, I am jsoverson

pp.preprocess(text, {USERNAME : "Bob"});
// -> Hi, I am Bob

// specify the format to use for the directives as the third parameter
pp.preprocess(text, {USERNAME : "Bob"}, {type: 'html'});
// -> Hi, I am Bob

// Preprocess files asynchronously
pp.preprocessFile(src, dest, context, callback, options);

// Preprocess files synchronously
pp.preprocessFileSync(src, dest, context, options);

Contributing

In lieu of a formal styleguide, take care to maintain the existing coding style. Add unit tests for any new or changed functionality. Lint and test your code using jshint

Release History

  • 3.2.0

    • Fix npm incompatibility on CI (@pioug)
    • Run CI with LTS and latest Node versions (@pioug)
    • Support if-else statement (@hu9o, #121)
    • Add JSON extension as alias for js (@jirikrepl, #111)
  • 3.1.0

    • Added .jsx file extension as an alias for js (@BendingBender, #79)
    • Added .tsx file extension as an alias for js (@rosendi, #100)
    • Bumped XRegExp to v3
  • 3.0.1/2 Fixes for backward compatibility and regex cleanups (thanks to @anseki for suggestions, #77)

  • 3.0.0

    Breaking changes:

    • If a file requested by @include or @extend can not be found, preprocess will now throw by default with a possibility to opt in to the legacy behavior via the fileNotFoundSilentFail option (@BendingBender, #35).
    • Fixed multiple issues with newlines (@BendingBender, #8), this may result in output that differs from earlier versions.
    • The srcDir option was moved to the options object and now defaults to process.cwd instead of throwing by default (@BendingBender, #68)

    New functionality:

    • All block directives (ones that have a start and an end token, like @if/@endif) are now processed recursively (@Frizi, #61)
    • Added hidden by default configuration blocks for js (@mallowigi, #40) and html (@Frizi, #66)

    Fixes:

    • fixed @exec in files included via @include and @extend (@BendingBender, #58)
    • changed @extend and @exclude html regex so that directives may appear more than once in one line (@BendingBender, #36)
    • fixed multiple issues with coffescript syntax (@BendingBender, #39)
    • fixed @if and @foreach to not require trailing whitespace (@BendingBender, #74)
  • 2.3.1 Fixed @echo and @exec directives to allow - and * characters, fixed @exec with multiple params (@BendingBender, #21, #45, #51, #54).

  • 2.3.0 Added support for @include-static (@BendingBender)

  • 2.2.0 Added support for @foreach and @extend (@orionstein)

  • 2.1.1 Added support for .styl files via js regex (@nsonnad)

  • 2.1.0 Added automatic support for numerous formats, merged @exec, hidden by default html tags, added simple directives

  • 2.0.0 Added ability to echo strings, added conditional comments, removed lodash, merged 17, 13, 15, 16

  • 1.2.0 Added processing for hash-style comments (@marsch). Added more file aliases.

  • 1.1.0 Added deep inclusion, fixed sequential ifs

  • 1.0.1 Fixed multiple inline echo statements

  • 1.0.0 Pulled from grunt-preprocess to stand alone

License

Copyright Jarrod Overson

Written by Jarrod Overson

Licensed under the Apache 2.0 license.

preprocess's People

Contributors

amitport avatar armandabric avatar bendingbender avatar cmtegner avatar edpelesh avatar flaise avatar frizi avatar froddd avatar hu9o avatar jojanaho avatar jsoverson avatar justmoon avatar kirly-af avatar mallowigi avatar mrydengren avatar netwjx avatar nsonnad avatar orionstein avatar patyaikasha avatar pioug avatar qix- avatar xax avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

preprocess's Issues

Measure performance

After using this tool on project with reasonable size, my build times skyrocketed from ~20s to ~1m 20s, which means it adds a lot more processing than all the other steps combined. And there are a lot of them (pretty much every single file gets analysed at least once, most of them are compiled). And there is almost no processing currently done, so most of its overhead is dedicated to pattern matching. We already have unit tests here, but every suite has only a minimial snippet of code.

New test suite should be done in order to test performance on real sized files and to be able to compare between versions. After that, performance optimizations can be done with more certainty.

What about a simple loop?

I'm currently working on a project which requires dynamic assets to be added to the main HTML file. A loop would have been great for this kind of purpose. Today I'm going to use Lodash and its minimal templating but if in the future you add loops I will come back to your project because the syntax is great for preprocessing purposes.

Tests failing on Windows due to new EOL handling

@include directive is not properly tested or behaves incorrectly. restoreEol is causing a lot of \r\n to be introduced.
I tried to link this issue with git's autocrlf setting, but it failed in each case.

Running "jshint:lib" (jshint) task
>> 2 files lint free.

Running "nodeunit:preprocess" (nodeunit) task
Testing preprocess_test.js..........FF............
>> preprocess - @include files
>> Message: Should include files and indent if ending with a newline
>> Error: 'a!foobar!\r\n c' == 'a!foobar!\n c'
>> at Object.exports.preprocess.@include files (test\preprocess_test.js:573:10)
>> at Object.exports.preprocess.setUp (test\preprocess_test.js:37:5)

>> preprocess - @include files
>> Message: Should include files (js, block)
>> Error: 'a!foobar!Hello js!\r\n !bazqux!c' == 'a!foobar!Hello js!\n !bazqux!c'
>> at Object.exports.preprocess.@include files (test\preprocess_test.js:577:10)
>> at Object.exports.preprocess.setUp (test\preprocess_test.js:37:5)

>> preprocess - @include files
>> Message: Should include files and indent if ending with a newline (js, block)
>> Error: 'a!foobar!\r\n c' == 'a!foobar!\n c'
>> at Object.exports.preprocess.@include files (test\preprocess_test.js:581:10)
>> at Object.exports.preprocess.setUp (test\preprocess_test.js:37:5)

>> preprocess - @include-static files
>> Message: Should include-static files and indent if ending with a newline, just like @include
>> Error: 'a!foobar!\r\n c' == 'a!foobar!\n c'
>> at Object.exports.preprocess.@include-static files (test\preprocess_test.js:623:10)
>> at Object.exports.preprocess.setUp (test\preprocess_test.js:37:5)

>> preprocess - @include-static files
>> Message: Should include files (js, block), but not recursively
>> Error: 'a!foobar!/* @exec hello(\'js\') /\r\n / @include static.txt /c' == 'a!foobar!/ @exec hello(\'js\') /\n / @include static.txt */c'
>> at Object.exports.preprocess.@include-static files (test\preprocess_test.js:627:10)
>> at Object.exports.preprocess.setUp (test\preprocess_test.js:37:5)

>> preprocess - @include-static files
>> Message: Should include-static files and indent if ending with a newline (js, block), just like @include
>> Error: 'a!foobar!\r\n c' == 'a!foobar!\n c'
>> at Object.exports.preprocess.@include-static files (test\preprocess_test.js:631:10)
>> at Object.exports.preprocess.setUp (test\preprocess_test.js:37:5)

Warning: 6/213 assertions failed (763ms) Use --force to continue.

Aborted due to warnings.

Include once

Is there currently a system which will include a file exactly once? So if this include appears in a file further along the parsing route (either after or inside the current file) it will be ignored.

If there is not such a directive, would you accept a PR to add one?

Multiple conditions in one line and/or nested conditions

Hi,

Here's an amended ifdef test that fails with multiple conditions on one line, and with nested conditions. HTH.

  'ifdef': function(test) {
    test.expect(6);

    var input,expected,settings;

    input = "a<!-- @ifdef NONEXISTANT -->b<!-- @endif -->c";
    expected = "ac";
    test.equal(pp.preprocess(input, { }), expected, 'Fail case, should not be included');

    input = "a<!-- @ifdef NODE_ENV -->b<!-- @endif -->c";
    expected = "abc";
    test.equal(pp.preprocess(input, { NODE_ENV: 'dev'}), expected, 'Success case, should be included');

    input = "a/* @ifdef NONEXISTANT */b/* @endif */c";
    expected = "ac";
    test.equal(pp.preprocess(input, { },'js'), expected, 'Fail case, should not be included');

    input = "a/* @ifdef DEFA && DEFB */b/* @endif */c";
    expected = "abc";
    test.equal(pp.preprocess(input, { DEFA: true, DEFB: true },'js'), expected, 'Fail case, should not be included');

    input = "a\n// @ifdef DEFA \nb\n// @ifdef DEFB\nc\n// @endif\n// @endif\nd";
    expected = "abcd";
    test.equal(pp.preprocess(input, { DEFA: true, DEFB: true },'js'), expected, 'Fail case, should not be included');

    input = "a/* @ifdef NODE_ENV */b/* @endif */c";
    expected = "abc";
    test.equal(pp.preprocess(input, { NODE_ENV: 'dev'},'js'), expected, 'Success case, should be included');

    test.done();
  },

Include with a variable?

Is there an option to put a variable in an include?

  <!-- @if BLAH -->
    <!-- @include BLAH/file.html -->
  <!-- @endif -->

Tried also

<!-- @include $BLAH/file.html -->

But it doesn't replace the variable, so it doesn't find the file.

Can't echo strings with minus (-) sign

Hello guys,

Have a look at the following code:

var pp = require('preprocess');
var text = 'Hello <!-- @echo "minus-world" -->';
var out = pp.preprocess(text, {}, 'html');
console.log(out);

I expect it to print

Hello minus-world

But I get back an unprocessed value

Hello <!-- @echo "minus-world" -->

If you remove the minus sign inside of an echo value everything's working fine.

Indentation

It can be interesting to handle indentation when using include directive.
I make a small POC on my fork diorcety@60bf208
It will better to provide a way to enable/disable the feature.

Try to improve performance

After integrating a performance measuring system, try to improve performance while verifying the effects of the efforts.

An obvious optimization would be to cache created RegExp objects so that they don't have to be recompiled on each recursive invocation.

Missing Support for Source Maps

Couldn't find anyone complaining about this, so I had to open this Issue eventually, though I guess I'm not the only one struggling with this.

When preprocessing scripts and styles with gulp (or grunt - doesn't matter), I do also set up source maps. However preprocess does not support them and therefore wrong line numbers (and even wrong files) are returned.

screen shot 2015-06-05 at 10 13 47 pm

My current workaround is initialising source maps *after * preprocessing files. This does at least show me the code that is executed, however I can't connect my local files in the Chrome Dev Tools sources and have to search for the executed code by hand.

Regex bug in @exec

Hi,

I'm using the following string:

"route": "^androidapps",
    "files": [
        "<!-- @exec resolveAsset('styles/android-apps.css') -->",
        "<!-- @exec resolveAsset('scripts/android-apps.js') -->",
        "<!-- @exec resolveAsset('_feeds/androidApps.config.js') -->"
    ],

But the regex doesn't seem to correctly be picking this up, I can see that the function works and is correctly exposed. When I run this, the output is:

"route": "^androidapps",
    "files": [
        "<!-- @exec resolveAsset('styles/android-apps.css') -->",
        "<!-- @exec resolveAsset('scripts/android-apps.js') -->",
        "_feeds/androidApps.config.js"
    ],

As you can see the last one is replaced, but the first two aren't. I did some digging and it appears some of the regex in https://github.com/jsoverson/preprocess/blob/master/lib/regexrules.js#L10 appears to be incorrect, it's only matching the last rule. I'm not sure if this is something I'm doing wrong though?

@exec not working in included files

Not sure if this is a bug or intended, but it seems it isn't possible to use @exec in included HTML files. The comment is removed, but the function data isn't added. Other functions such as @echo seem to work fine though.
The exact same @exec works fine in the parent html file, but doesn't seem to work at all when included.

Pass variables to includes?

There have been times where I'd like to tell an include something based on parent template … Maybe there's a better way that I'm not aware of, but it would be nice to have the ability to pass variables to the preprocessed include files.

For a Grunt example, grunt-include-replace:

@@include('/path/to/include/message.html', {"name": "Joe Bloggs"})

Would it makes sense for Preprocess to allow for variables, defined at the template level (i.e., at the time of the include call) to be passed to the included files?

context attribute srcDir always required, but undocumented

The program

var fs = require('fs');
var pp = require('preprocess');

pp.preprocess(
    fs.readFileSync('example.js', { encoding : 'UTF-8' }),
    {},
    'js'
);

throws the exception

TypeError: Arguments to path.join must be strings
    at path.js:360:15
    at Array.filter (native)
    at Object.exports.join (path.js:358:36)
    at ./node_modules/preprocess/lib/preprocess.js:63:32
    at String.replace (native)
    at Object.preprocess (./node_modules/preprocess/lib/preprocess.js:59:11)
    at Object.<anonymous> (./bug.js:4:4)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)

The problem is that the second parameter to preprocess, the context object, is expected to have a key srcDir and a path part value, e.g. { srcDir: '.' } works. This is documented nowhere.

Error when parsing the `U+FEFF` character

I'm downloading a 3rd party file that has a funky character in it (I think it's the U+FEFF character - it is in the js file from this project) and then running preprocess on it. My preprocessor script looks something like this:

define(['jquery'], function ($) {
    /* @include path/to/my/file.js */
});

The resulting file ends up with an "unsafe character" which then causes jshint (and buster-coverage) to puke :P However, the original file does not show the "unsafe character" error.

Would be great if preprocessor could recognize the character and know what to do with it.

(I actually know next to nothing about this stuff but came across this as I was debugging some errors in the application I'm working on. See http://en.wikipedia.org/wiki/Zero-width_no-break_space)

Thanks!

Allow objects to be passed to `@echo` directive

Currently passing an object to the preprocess function gives toString result of the passing object ie.

pp.preprocess('var foo = "/* @echo foo */" ', { foo : { bar : 42 }}, 'js');
//  result is 'var foo = "[object Object]"'

What about provide json value of an object ?

CSS?

Hi,

Would I be able to use this on CSS? If not, do you think that would be a useful feature to add to preprocess?

Btw, I love this code! Thanks!

Not found prints to file

Just found that when running through gulp-preprocess (which I think is just a wrapper) it will print

/path/to/file not found

Directly into the concatenated script. Is there intended to be a catchable error here, e.g. that could output to a build tool like gulp? I've altered my version to add /* */ around this error so that at least it doesn't break your script entirely; if you think that's reasonable I'll send a pull request.

Problems with preprocessing php in html files

When I have included php in an html file, I run into several obstacles. It is not clear how to comment items out. For example:

    <?php echo 'hi'; ?>
    <div>Hi again</div>
    <!-- @include hi_3.php -->

That fails sometimes, and likewise, sometimes it fails when I use // style comments for inclusion.

Also, another (perhaps too specific to me) issue is that if I don't include <?php ... ?> tags in Sublime Text, syntax highlighting doesn't work properly, and if I do include them (since the included files sometimes come between existing <?php tags), the end file breaks; so I cleverly (or so I thought) excluded the <?php tags like so:

   // @exclude
      <?php
    // @endexclude

And It works! But it seems hackish, and I'm not sure that it is the best solution. Shouldn't the tag prior to the <?php have to use the other commenting style?

I've been using this with gulp and livereload for awhile now, and I love it, but I'm developing in wordpress currently, and the processing seems to get done about one in three times; the other times a non-processed file missing the included files (and missing the @include snippet) is passed through. This is strange behavior. Any thoughts?

Maybe this is specific to gulp-preprocess?

CoffeeScript preprocessing deletes statements

There's something fatally awry with the end-termination of CoffeeScript preprocessing. The typical effect is that the statement after a @if and after an @endif are both omitted, even when there are blank lines between the condition and the statement. The regexes appear wrong -- they look for JS-style comment terminators after the comments, which never exist in CS. But I admit, I haven't yet worked through a precise identification of where the patterns are broken. My test issue is:

    ## @if NODE_ENV == 'development'
    host = 'localhost'
    protocol = 'http'
    port = 3001
    ## @endif

    console.log "Socket info", protocol, host, port

After preprocessing, both the console.log and the host assignment are both omitted.

Add performance measuring and comparison

Add performance measuring for the test suite along with some special tests that process larger files to produce more real-world performance characteristics (mostly the same as #64).

Change the srcDir context attribute to be optional and default it to cwd

At the moment, the srcDir context attribute has to be specified in order for the @include/@include-once/@extend to be able to find requested files. This can be however conveniently made optional by defaulting this attribute to current working directory while still letting the user to specify another directory if needed.

This would impose a breaking change.

@exec seems to not support multiple params correctly?

I'm not 100% if this is a true bug or me breaking something, but when I try and use @exec it crams all of the parameters into arguments[0]

Looking at the preprocess/test/preprocess_test.js tests, it looks as though they wouldn't pick this up as they split then join the array anyway.

function multiParams() { return arguments.length; }

/* @exec multiParams("one", "two", "three") */

Expect: 3
Actual: 1

Hacky workaround
function multiParams() { String(arguments[0]).split(',').length; }

Yet it seems my workaround breaks the ability to call with other env vars as params

'/* @exec valueOrDefault(FOO_PORT, '8080') */' -> 'FOO_PORT',

@define directive

To add properties to the context dynamically. Would be very useful...

You've published in your .idea directory in NPM

Looks like it's ignored in .gitignore, but showing up when I install via NPM. Not a big issue, just something to be aware of.

curl http://registry.npmjs.org/preprocess/-/preprocess-1.2.0.tgz | tar -ztvf - | grep .idea

-rw-r--r-- 502/20           10 2013-04-07 07:40 package/.idea/.name
-rw-r--r-- 502/20          166 2013-04-07 07:40 package/.idea/encodings.xml
-rw-r--r-- 502/20         1119 2013-04-07 07:40 package/.idea/misc.xml
-rw-r--r-- 502/20          274 2013-04-07 07:40 package/.idea/modules.xml
-rw-r--r-- 502/20          283 2013-04-07 07:40 package/.idea/preprocess.iml
-rw-r--r-- 502/20          139 2013-04-07 07:40 package/.idea/scopes/scope_settings.xml
-rw-r--r-- 502/20          182 2013-04-07 07:40 package/.idea/vcs.xml
-rw-r--r-- 502/20        15073 2013-04-07 07:49 package/.idea/workspace.xml

N00b question: How to set process.env?

Hi,

Sorry if I missed this in your docs (or the preprocess docs).

What's the recomended way to set the states (DEBUG/DEV)?

Found this tip on stack:

export NODE_ENV=development

But when echo NODE_ENV nothing's set.

Optimally, I'd like to set my ENV from my Gruntfile.js. Is this possible? I didn't see an example in your repo's Gruntfile.

Sorry if this is a dumb question. :(

Thanks!

Nested !> does not work

For example

  <!-- @if true !>
  <title>
  <!-- @echo TITLE !>
  </title>
  <!-- @endif -->

This will compile to:

  <title>
  <!-- @echo TITLE !>
  </title>

If I change

  <!-- @echo TITLE !>

to

  <!-- @echo TITLE -->

.... the interpolation will work but it will result in malformed HTML.

html/js regex might not work with mac/windows eol

Example:

 // @ifdef DEBUG
 someDebuggingCall()
 // @endif

is using regex "(?://|/_)[ \t]@ifdef[ \t]([^\n_])[ \t](?:*/)?" which assume eol is "\n"

if your end of line is "\r\n", then "DEBUG\r" is searched in context instead of "DEBUG"

until fixed, a quick workaround is to use the /* */ syntax:

 /*@ifdef DEBUG*/
 (...)
 /*@endif*/

Allow Strings to be Passed to `@echo`

Currently, @echo can only print vars. Storing those values in an external file doesn't always make sense for future maintenance. So instead of this:

<!-- @exclude -->
<script src="path/to/dev.js"></script>
<!-- @endexclude -->
<!-- @echo BUILDSCRIPTTAG -->

It would be nice to be able to do this:

<!-- @exclude -->
<script src="path/to/dev.js"></script>
<!-- @endexclude -->
<!-- @echo '<script src="path/to/build.js"></script>' -->

This would allow for keeping the build.js tag commented out during development. As well as making sure any (non-NodeJS) maintainers down the line can inspect the src to figure out the intended process.

Great package though! You've saved me so much tedium in writing my own regex for replacing things for each new occasion.

User Script headers cause problems

User scripts have headers containing @-prefixed directives just like this does, including one which is named @include — this (obviously) causes trouble when attempting to use this for user scripts (which is one of the most applicable uses of this)

Not sure how to fix, except maybe ignore things within the UserScript header block.

Allow no-op without preprocess

I may be missing something, but it seems that preprocess code will screw your app in the eventuality it's executed without preprocess.

A close analogy for preprocess is available in Microsoft's conditional compilation for JScript and conditional comments for HTML. An essential aspect of these features is that the conditional code is read as comments and thus has no runtime effect if the environment doesn't know how to handle these.

Compare this to preprocess – where, in the absence of preprocess, all the code intended for execution by preprocess will get executed.

Consider this example, where I have my default code written in a simple and unambiguous way, with preprocess code added at the end:

var services = {
    urls : {
        resources : 'http://example1.com/resources/',
        search    : 'http://example2.com/search/'
    }
};

/* @if NODE_ENV=='development' */
services.urls.resources = 'http://dev.example1.com/resources/';
/* @endif */

Without preprocess, the comments mean nothing, so services.urls.resources is re-assigned regardless. This is especially crap if you've got non-trivial logic, and preprocess-less environments execute a bunch of contradictory statements. So I effectively have to re-structure my code (and probably my environment variables) to be preprocess-safe:

var nodeEnvDevelopment = /* @echo NODE_ENV=='development' */ || false;

var services = {
    urls : {
        resources : 'http://example1.com/resources/',
        search    : 'http://example2.com/search/'
    }
};

if( nodeEnvDevelopment ) services.urls.resources = 'http://dev.example1.com/resources/';

This adds the development overhead of forcing my code to be primarily geared towards preprocess, rather than preprocess code being a drop-in conditional extra, and even then the code would still end up with meaningless runtime overhead.

I would suggest an extension to the current comment start/end and @TOKEN delimiters to allow 'safe' preprocess code:

var services = {
    urls : {
        resources : 'http://example1.com/resources/',
        search    : 'http://example2.com/search/'
    }
};

/*
@preprocess
@if NODE_ENV=='development'
services.urls.resources = 'http://dev.example1.com/resources/';
@endif
@end
*/

With really powerful preprocessing tools like Browserify, you can understand the notion that code written with Browserify in mind will break if used without preprocessing, but considering preprocess is so simple, it seems a shame to force dependency.

Is this possible, inline echo

Just including inline echo so things don't always have to be defined as a var or in a file?
I'm basically asking if this is possible rather than asking if it can be. If it can't I'll consider a pull request if you're interested.

<!-- @if NODE_ENV='production' -->
<!-- @echo <script>var myPoductionSpecificThing = {foo: 'bar'};</script> -->
<!-- @echo <script>var myPoductionSpecificThing2 = {bar: 'baz'};</script> -->
<!-- @endif -->

output:

<script>var myPoductionSpecificThing = {foo: 'bar'};</script>
<script>var myPoductionSpecificThing2 = {bar: 'baz'};</script>

Next release 2.4.0 or 3.0.0

@Frizi and @orionstein, I'd like to release the next version soon. However, I'm looking at the changelog and I have doubts as which version number I should give this release.

There are a number of changes that might subtly break existing setups (like hidden blocks and coffescript extensions). All in all I'm in doubt of whether to call it 2.4.0 or 3.0.0 .

What do you guys think?

In getTestTemplate() empty template is evaluated as true

In preprocess.js the following method gets test = "%0D" (newline)

function getTestTemplate(test) {
  /*jshint evil:true*/
  test = test || 'true==true';

  // force single equals replacement
  test = test.replace(/([^=])=([^=])/g, '$1==$2');

  return new Function("context", "with (context||{}){ return ( " + test + " ) }");
}

this is causing test to be evaluated as true because it's a newline, and the

new Function("context", "with (context||{}){ return ( " + test + " ) }");

breaks with "SyntaxError: Unexpected token )".
This happens only under Windows (8.1 x64), on MAC it runs fine.

Full stack

Running "preprocess:ironclad" (preprocess) task
Warning: Unexpected token ) Use --force to continue.
SyntaxError: Unexpected token )
    at Object.Function (<anonymous>)
    at getTestTemplate (D:\ironclad\node_modules\grunt-preprocess\node_modules\preprocess\lib\preprocess.js:123:10)
    at testPasses (D:\ironclad\node_modules\grunt-preprocess\node_modules\preprocess\lib\preprocess.js:127:16)
    at D:\ironclad\node_modules\grunt-preprocess\node_modules\preprocess\lib\preprocess.js:78:12
    at String.replace (native)
    at preprocess (D:\ironclad\node_modules\grunt-preprocess\node_modules\preprocess\lib\preprocess.js:77:11)
    at D:\ironclad\node_modules\grunt-preprocess\node_modules\preprocess\lib\preprocess.js:68:22
    at String.replace (native)
    at Object.preprocess (D:\ironclad\node_modules\grunt-preprocess\node_modules\preprocess\lib\preprocess.js:58:11)
    at D:\ironclad\node_modules\grunt-preprocess\tasks\preprocess.js:58:36

Also, the following test is failing:

Running "nodeunit:preprocess" (nodeunit) task
Testing preprocess_test.js..........F......
>> preprocess - include files
>> Message: Should include files and indent if ending with a newline
>> Error: 'a!foobar!\r\n c' == 'a!foobar!\n c'
>> at Object.exports.preprocess.include files (test\preprocess_test.js:443:10)
>> at Object.exports.preprocess.setUp (test\preprocess_test.js:35:5)

Warning: 1/76 assertions failed (26ms) Use --force to continue.

Aborted due to warnings.

getTestTemplate() can still fail due to empty/newline string

I'm running into the same problem that was brought up in issue #46, but for an exclude test. I commented on that issue (#46 (comment)) but haven't seen any responses, so maybe no one is getting notified since the issue is closed?

I'm running a project that appears to end up passing a test string containing only a new line. I'm on Windows, so I think this is coming through as "\r\n". I'm not sure if something outside of preprocess.js is passing something it really shouldn't be, or if it's a result of the regex replacement that happens earlier in the script.

Either way, this blows up in getTestTemplate(). I was able to prevent it by adding a line at the top of testPasses() that contains: if( test && test.replace( /[\n\r\s]*/gi, '' ) == '' ) { test = '' }. (Obviously, this is just saying "after replacing all spaces, cr, and lf, if we're left with an empty string, then set test to an empty string").

I don't understand everything going on in preprocess.js well enough to know if this is a truly viable fix, but for now it's what I'm using to stop it from failing.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.