broccolijs / broccoli Goto Github PK

Since windows is currently officially unsupported this isn't really a bug, but I still want to bring it up. This is what you currently get if you use broccoli on windows:

EPERM, operation not permitted 'C:\Users\Josef\Desktop\broccoli-sample-app\tmp\merged_tree-tmp_dest_dir-4gkuvKyb.tmp\index.html'

The first build runs through without problems. This problem occurs at rebuilding. More information is here:

nodejs/node-v0.x-archive#6599 (comment)

If the build has an error, the watcher should try building again after the delay.

https://github.com/joliss/broccoli/blob/master/lib/watcher.js#L18
It remembers the new hash even if the build for it failed. That's no good. I think it should retry building it. Maybe just some files were locked.

Relevant for https://github.com/joliss/broccoli/issues/72 if watcher gets refactored.

Broccoli looks very promising where it actually fixes many of the pain points in other build tools. So thanks for making this!

A big architecture mistake Grunt made was passing an instance of itself around to the Gruntfile and especially to grunt plugins:

module.exports = function(grunt) {};

As opposed to:

var grunt = require('grunt');
module.exports = function() {};

With passing an instance of grunt, a peer dependency is created. With that peer dependency, only one instance of grunt can be used at a time. Which means every plugin we use must be compatible with that single version.

Fortunately grunt doesn't update very often and this pain is mostly avoided. But if development on Broccoli is active, it might become a huge pain for Broccoli users when they discover their installed plugins are not always compatible with each other.

Using an instance will also make it more difficult to run Broccoli programmatically. Currently Grunt has many tools that augment it (lineman, yeoman, bbb, assemble) and each has an awkward time doing it as it requires manipulating the single passed instance of grunt. As opposed to simply var grunt = require('grunt'); and consuming as a dependency.

Hey,

from time to time I come by and take a look at broccoli. Here are a few questions:

• Naming conventions:

  • Transformer: Either a Preprocessor or a Compiler
  • Preprocesser: Has one input and one output file
  • Compiler: Has many input and one output file

  Is that correct?

• Why is it called a Preprocesser and not just Processor? Is there some limitation that forbids me to use it on a compiled file?

• In the ember app kit stirfiry file are the lines

  ignoredModules: [
    'resolver'
  ],

Why are they needed? What do they do exactly? Why isn't it called "ember-resolver" like the bower component?

Could we add loadBrocfile(function)? Or in other words the ability to pass in a function programmatically without using an actual file for that.

If I delete routes/index.js and there's still, say, a leftover unit test that imports 'appkit/routes/index', I get an error like

Error: ENOENT, no such file or directory 'tmp/merged_tree-tmp_dest_dir-GlEGjZuR.tmp/appkit/routes/index.js'

It didn't take long to realize what was up, but the error makes it look like some internal broccoli bug rather than something you did wrong in your code. Perhaps this issue belongs in another repo but my hunch is that this is central enough to broccoli to make sense here?

I implemented a simple Sass compiler which uses promises like this:

SassCompiler.prototype.transform = function (srcDir, destDir) {
  var files = broccoli.helpers.multiGlob(["**/*.scss"], { cwd: srcDir });
  var compiledFiles = files.map(function(file) {
    if(!file.match(/(\/|^)_[^\/]+$/)) {
      return new RSVP.Promise(function(resolve, reject) {
        sass.render(extend(this.options, {
          file: file,
          success: resolve,
          error: reject,
          includePaths: [ srcDir ]
        }));
      }).then(function(css) {
        var outputFile = path.join(destDir, file.replace(/scss$/, "css"));
        return RSVP.denodeify(fs.writeFile)(outputFile, css);
      });
    }
  }, this);
  return RSVP.all(compiledFiles);
}

This works nicely, but if the sass file contains syntax errors, I get output like this:

undefined

Build failed

I've verified that the promise has failed and resolves to a compilation error which describes what's gone wrong, but this error is not shown by broccoli it seems.

http://localhost:4200/some/dir

Would be nice to get a little file browser or something.

I'm trying to use a test runner but it needs some files to include into the browser and watch for changes, but broccoli serve doesn't fit this model.

One knee-jerk thought is to have a symlink to the latest merged tree that you could point the test runner to with something like broccoli serve <link>.

I have some questions about the plans for architecture that the 0.1.0 blog post doesn't cover (or I missed it):

• It looks like the existing plugins are all for transforming trees. Are there plans for validating trees, ala jshint et al? Those would take a tree as input, but don't have an output tree (just logs)
• Also related, any plans for making plugins reusable or sharable between various tools? Specifically the node-task spec should be interesting here: https://github.com/node-task/spec/wiki - though based on the discussion in the blog post, I suspect that the proposed Record format will not be sufficient. Not super urgent, since the plans for node-task have been pushed to a later release.
• Related to the above, what are plans for logging? Grunt does a reasonable job in this area, while Gulp, last I checked, had no logging. This is especially importing for plugins/tasks that validate their input like jshint. Currently each grunt task has to implement that logging from scratch, so there's definitely potential for abstracting.

Currently the server and the file watcher are intertwined. I think the server could be extracted into a plugin. Doing so would lead to a cleaner and more extensible design and a leaner core.

Currently, Broccoli's description reads:

Browser compilation library – a framework for building applications that run in the browser

This description seems wrong and misleading, and updating it would be great.

I'm playing with broccoli and it's awesome.
I did a broccoli build dist and changed something and did a new broccoli build dist. Now broccoli-cli is complaining:

Error: Directory "dist" already exists. Refusing to overwrite files.

It seems to be the intended behavior. So broccoli recommends to build releases like broccoli build dist/v1-0-0 , broccoli build dist/v1-0-1. Is that correct?

It would be nice if broccoli put all the temporary folders into one tmp folder.

Btw tested it on windows, it doesn't crash, I messed around a bit, and it appears to work. It's quite fast :) almost instant. The app included in EAK I tested it with isn't that complex though.

I'm trying to switch from Grunt to Broccoli, but I'm finding the lack of a broccoli watch command troublesome.

Here's the scenario:

I have a Vagrant instance which is configured to serve both the backend (a JSON API implemented in node.js) and the frontend (an Ember app) from the same host. This avoids having to deal with CORS.

The static files are currently served by the node.js backend, so I'd have to change that while developing.

Also, I'd have to call broccoli serve from inside Vagrant, instead of from the local host, which is inconvenient.

So, unless there's some other config option that I'm missing, I suggest adding a broccoli watch command, or a --watch flag to broccoli build, which only handles rebuilding, leaving the server part out.

I installed all the required components for the Broccoli Sample App but I am getting the following error.

fs.js:654
  return binding.readdir(pathModule._makeLong(path));
                 ^
Error: ENOENT, no such file or directory 'bower_components'
    at Object.fs.readdirSync (fs.js:654:18)
    at Object.bowerTrees (/www/node/geddybroc/node_modules/broccoli/lib/tree.js:106:20)
    at module.exports (/www/node/geddybroc/Brocfile.js:40:45)
    at Object.loadBrocfile (/www/node/geddybroc/node_modules/broccoli/lib/helpers.js:202:31)
    at getBuilder (/www/node/geddybroc/node_modules/broccoli/lib/cli.js:62:22)
    at Object.broccoliCLI [as cli] (/www/node/geddybroc/node_modules/broccoli/lib/cli.js:28:27)
    at Object.<anonymous> (/usr/local/share/npm/lib/node_modules/broccoli-cli/bin/broccoli:11:10)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)

... Reload LiveReload files ... is logged 4 times on each change.

Are there unit tests for broccoli? (thanks)

context on Hacker News

brocollifile.js is too long for a standard build file name

How about "Brocfile.js"?

I wanted them to be called `Brocfile`s

I also prefer `Brocfile` to `Broccolifile`.

Broccoli is just one of those words that never looks spelt correctly, even when it is.

The person who started the thread misspelled the word 3 times.

I want to use broccoli as middleware instead of running broccoli serve. This isn't currently possible but I hacked to together an example by requiring the Watcher and middleware directly.

var broccoli = require('broccoli');
var Watcher =  require('./node_modules/broccoli/lib/watcher');
var middleware =  require('./node_modules/broccoli/lib/middleware');

var app = require('express')();

var tree = broccoli.loadBrocfile();
var builder = new broccoli.Builder(tree);
var watcher = new Watcher(builder);
var assets = middleware(watcher);

if ('development' === app.get('env')) {
  app.use(assets);
}

app.listen(3000);
// if Brocfile.js had configuration to serve app.js 
// http://localhost:3000/app.js would be served

From https://github.com/joliss/broccoli/issues/7 it seems like you are considering adding this in future. If this feature is built in then there's no need for exposing Watcher and middleware but if should exist as a module then they would need to be exposed.

Any thoughts? Thanks for the great work on this project.

I've been noticing that the Broccoli Watcher task is taking up a constant 4% CPU on a medium-sized project. I think Watcher should not poll by default, but instead preform a check every time its then() method is called.

If someone wants the Watcher to poll, then they could easily setup a setInterval(watcher.check, 100). This would also ensure that when the middleware/server receives a request for a file, it will always be built and up to date.

@joliss Thoughts? If this seems good to you, I'd be happy to issue a PR for this change.

The "eak" and "es6" branches are still around and 3 months w/o changes.

It would be handy for broccoli serve to support history API URLs instead of hashes.

I'm not too familiar with hapi but it looks like you can rewrite the URL with request.setUrl(url) in the onRequest handler. Maybe this can check whether the request matches an asset and if not then rewrite as /?

Is this something that broccoli serve should (optionally) know about and handle, or is this the job of some other package which would use broccoli programatically?

I can easily get a flat example working no problem

var sourceTree = broccoli.makeTree('css');
var allCSS = concat(sourceTree, {
  inputFiles: [
    '*.css'
  ],
  outputFile: '/dist/site.min.css'
});

but the moment I put that "css" dir inside another more "root" level directory I run into trouble with getting the output to show up (in fact no output gets generated at all when I do a serve or a build) assets/css instead css in the below example

var sourceTree = broccoli.makeTree('assets');
var allCSS = concat(sourceTree, {
  inputFiles: [
    'css/*.css'
  ],
  outputFile: '/dist/site.min.css'
});

I assume broccoli works with a nested directory structure, but what am I missing to get the ouput working? Thanks again for the blazing fast build tool !

It seems that broccoli is very fragmented right now. I realize it's common practice in node-land to split off small project into separate packages, but we could still consolidate all those package into one repository, and it would be easier to browse the source. IMO, at least the core parts, like broccoli-transform should live together with the main project.

See joliss/broccoli-sass#4 (comment).

I tried to add this myself and provide a pull request, but I can't figure out how to make Hapi do this. The idea is that I can have a backend application running locally on another port (or wherever), and have Broccoli's server proxy everything that it doesn't handle to this other server. Lineman has a similar feature built in.

Hapi does have a proxy handler built in, and it also has the directory handler that Broccoli uses, but I can't figure out how to use the two together. Maybe someone who has more experience with Hapi has an idea how to make this work?

I know something similar was already shot down but considering the frequency of having to type of "broccoli" it would be nice if there was an alias of "broc" instead. The same goes for aliasing "Broccolifile.js" to "Brocfile.js"

Hi pals,

Today i release broccoli-replace (like grunt-replace) for text pattern replacements.

Would be great to hear your opinion and tell me if I'm doing things in the right way ...

Example:

var publicFiles = broccoli.makeTree('public');

publicFiles = replace(publicFiles, {
  files: [
    '**/*.html'
  ],
  patterns: [
    {
      match: 'user',
      replacement: 'foo'
    }
  ]
});

Plugin (already at npm):

https://github.com/outaTiME/broccoli-replace

Fork of sample app:

https://github.com/outaTiME/broccoli-sample-app

My idea is align the replacements grunt / gulp / broccoli in the same way ...

Thks and awesome wooooork !

Great library! Love the simplicity of thinking in trees rather than files.

Not sure where to post this, but thought it might be useful to others...

A note for those using the LiveReload app: Be sure to quit that app before running "brocolli server", or you're going to get: Error: listen EADDRINUSE (i.e. port in use).

Or alternatively, if you want to keep both running side-by-side (different projects etc.) just remember to change the livereload port number in lib/server.js (line 75).

D.

I propose something like:

// Brocfile.js
var broccoli = new (require('broccoli')),
    serve = require('broccoli-serve'),
    store = require('broccoli-store')

var tree = broccoli.makeTree(...)
...
serve(tree, 4200)
store(tree, './build')

module.exports = broccoli

Instead of returning multiple trees, the Brocfile just returns a builder instance which emits events to which the CLI listens too.

This makes it possible to have multiple output folders just like in grunt.

plz

I couldn't really see a way to concat files without using the ES6 transpiler plugin, so I wrote broccoli-concat - https://github.com/zaius/broccoli-concat - is this something that you would want as an official plugin?

I tried to keep consistent with your style on the ES6 transpiler, but let me know if there is anything I should change.

I noticed a while ago that my projects get more and more top level files. Gruntfile.js, .jshintrc, .gitignore, bower.json, package.json, ... . All those files have their purpose. I would like Broccoli to work without a top level file. And here's how:

// server.js

var app = require('express')();
app.use(app.router);
app.use(require('broccoli')(/* Stuff from Broccolifile.js */));

app.listen(8000);

The clou: Broccoli as an express middleware (i.e. function(req, res, next) {...}). It handles all the requests not handled by the router. If it's currently rebuilding it simply waits until it's done.

This solution would be super lightweight. What do you think @joliss?

I'm building broccoli-pages plugin that will convert files with md extension to HTML files. I want to be able to control wether a file exists in the output depending on file's metadata. If metadata doesn't have template property, I want the file to not be created in output and be removed if it already exists.

Can I return null value to remove an item from the tree(or exclude it from being rendered)?

I couldn't help but notice this tool sounds a lot like Brunch. While I'm not intending to diminish the purpose and usefulness of your vision, I'm curious to know what major differences there will be to make contributing a worthwhile effort.

I've got some bower dependencies which have duplicate names in their repositories which then clobber each other when they are concatenated (eg with broccoli-es6-concatenator in legacyFilesToAppend)

An example of this is https://github.com/pid/speakingurl, which has an index.js file (not its main) and if you install ember-canary via a URL dependency that also gets named index.js

{
  "name": "app",
  "dependencies": {
    "ember": "http://builds.emberjs.com/canary/ember.js",
    "speakingurl": "~0.9.0"
  }
}

When I then try and concatenate these together like so:

var applicationJS = compileES6(appAndDependencies, {
    ...
    legacyFilesToAppend: [
      'index.js', // ember
      'speakingurl.min.js'
    ],
    ...
});

I then end up with the wrong index.js being used.

Should the bower tree only include the main files defined in bower.json?
Although that wouldn't help if there were two libs setup with file URLs as they would both legitimately be called index.js.

Maybe the bower tree should (optionally?) include the parent directory in the paths? Then I could uniquely reference them as so:

var applicationJS = compileES6(appAndDependencies, {
    ...
    legacyFilesToAppend: [
      'ember/index.js', 
      'speakingurl/speakingurl.min.js'
    ],
    ...
});

You should create a central organisation for all Brocolli related repositories, maybe something like brocollijs. Both Grunt and Gulp do this.

When creating a new plugin based on the broccoli-filter, I noticed that a lot of dependencies are downloaded that are unnecessary. If the helpers were moved out of this project then they could be required here and in broccoli-filter separately which would reduce the dependencies a new plugin has.

since: joliss@3e4150b

I ran the broccoli-sample-app, and this is looking very promising.

It looks like the sample app doesn't have anything ready for CSS or live reloading, though it appears that has already been built into Broccoli.

This isn't really an "issue," but I just wanted to make a note of something regarding this line: https://github.com/joliss/broccoli/blob/master/lib/server.js#L102

That probably works fine for auto-refreshing the page, but I am concerned if you don't specifically give it the files that have changed, it won't inject CSS without a full reload. Injecting CSS as opposed to reloading the page entirely is much faster, and I believe in order for LiveReload to work that way, it needs to know the CSS file that has changed (the compiled one that is being served on the page).

I'd also like to introduce you to BrowserSync by @shakyShane if you were already not aware of it. It works similarly to LiveReload, but also synchronizes link clicks, form input, and scroll position across any browser without the use of an extension.

https://github.com/joliss/ember-app-kit/blob/broccoli/package.json#L62

I don't send a PR because it's likely that you've changed many things since.

Here is what I want:

• lib/generator.js
• lib/preprocessors/handlebars.js
• lib/compilers/es6.js
• lib/compilers/static-file.js
• index.js <- register calls

Hi Jo,

it possible to make the build over the project root folder ? im trying to generate some files there ...

here my code:

i want to took README.md from docs and put in project root

module.exports = function (broccoli) {

  'use strict';

  var fs = require('fs');
  var filename = './node_modules/pattern-replace/README.md';
  var readme = fs.readFileSync(filename).toString();
  // initialize section
  var sections = {};
  // http://regex101.com/r/wJ2wW8
  var pattern = /(\n#{3}\s)(.*)([\s\S]*?)(?=\1|$)/ig;
  var match;
  while ((match = pattern.exec(readme)) !== null) {
    var section = match[2];
    var contents = match[3];
    sections[section] = contents;
  }

  // took contents from readme section

  var getSectionContents = function (name) {
    return sections[name] || '_(Coming soon)_'; // empty
  };

  // broccoli

  var replace = require('broccoli-replace');
  var srcFiles = broccoli.makeTree('docs');
  srcFiles = replace(srcFiles, {
    files: [
      'README.md'
    ],
    variables: {
      'options': function () {
        var source = getSectionContents('Replacer Options');
        return source;
      }
    }
  });
  return [srcFiles];

};

$ broccoli build .
Error: Directory "." already exists. Refusing to overwrite files.

thks !!

I noticed the README says there's no windows support yet. Wanted to open the discussion here while broccoli is still young. What's the hold up?

I have a broccoli (broccoli serve) on port 4200. If I try running a second instance with broccoli server -p 4700, it works like it should, starting a second instance and binding to port 4700, but it has the following output

╰─$ broccoli serve --port 4700                                                                  1 ↵
Serving on http://0.0.0.0:4700


... Uhoh. Got error listen EADDRINUSE ...
Error: listen EADDRINUSE
    at errnoException (net.js:901:11)
    at Server._listen2 (net.js:1039:14)
    at listen (net.js:1061:10)
    at Server.listen (net.js:1127:5)
    at Server.listen (/Users/dan/Projects/highways/frontend/node_modules/broccoli/node_modules/tiny-l
r/lib/server.js:138:15)

(Stacktrace truncated)

As soon as I stop the instance on 4200, starting the instance on 4700 works without errors. My guess is that it always checks port 4200, regardless of the port option passed in. If I wasn't on the train, I would dig into the code 😄

https://github.com/joliss/broccoli/blob/master/lib/builder.js#L15

If a build is scheduled, then the promise for the scheduled build should be returned. This would make it safe to call .then() on the output.

Also: If a build is currently in progress, it could schedule another build that runs after the current one has finished and build() returns the promise for that build then.

Just a suggestion: If you made the transpiler a filter, using other module systems or more evolved es6 preprocessing would be possible in the future (like shiming the class system). Anyway, having a file concatenator would be nice.