brettz9 / asyouwish Goto Github PK

Up until FF18, AsYouWish worked ok, but thereafter there have been problems particularly concerning local file i/o.

There are 2 workarounds for this problem. Please note I have tested these solutions only using AYW v0.5.2. But I'm fairly sure that the fixes would apply also to Brett's latest versions. Also, my comments regarding AYW only pertain to file i/o: that is, opening and closing files from a filepicker or directly from a filepath string, reading and writing files, creating new files, renaming files, etc. For testing purposes I'm assuming you will have the necessary code to test these capabilities.

The 2 solutions:

1. Use FF18 protected from upgrades by giving it its own profile. Load in AYW from its .xpi file. Now you have a standalone version of FF which is unrestricted by FF's privelege restrictions.

Shortcomings to this: you miss out on the enhancements of later versions of FF
Advantages: Put FF18 as a shortcut on desktop and you have a standalone app

2. Here is the better solution: use portableFirefox which is downloadable from portableapps.com. (Note: I've tested this with v34.05 of portableFF - but confident later versions would also work.)

You download portableFF, and place it in its own folder on your C: drive (or on your USB drive if you prefer).  There is a slight difference in functionality between these 2 scenarios - more of which below.

You load in AYW from its xpi file (Note: I haven't tested by loading in from FF add-on tool. I prefer to save the xpi to disk somewhere, then load in from there.)

Voila! FF+AYW should now work as you would expect it to - unrestricted access to the filesystem using XPCOM API's nsifile etc ---- that is to say after you do a couple of minor tweaks which I'll now explain.

Tweak 1: If you have any code which uses XPCOM's create(0,0) it will produce an empty read-only file. Then all your write calls will fail. To avoid this, make sure you write: create(0,0777)

Note: this problem only occurs when you have portableFF activated from the C:drive. From a usb drive, or from FF18, create(0,0) is ok.

Tweak 2: If you are outputting your file content to a div using multiple columns, -moz-column-fill:auto will not work as expected. The columns will become balanced at the end of long file content, and pagination will go screwy. To fix this, you need to place an extra

element with padding-top set by css to a large amount (eg 1000px) as the last

element inside the div. So make your content div :

In your css you have .pad {padding-top:1000px;}

Hope that helps.

🚨 You need to enable Continuous Integration on all branches of this repository. 🚨

To enable Greenkeeper, you need to make sure that a commit status is reported on all branches. This is required by Greenkeeper because it uses your CI build statuses to figure out when to notify you about breaking changes.

Since we didn’t receive a CI status on the greenkeeper/initial branch, it’s possible that you don’t have CI set up yet. We recommend using Travis CI, but Greenkeeper will work with every other CI service as well.

If you have already set up a CI for this repository, you might need to check how it’s configured. Make sure it is set to run on all new branches. If you don’t want it to run on absolutely every branch, you can whitelist branches starting with greenkeeper/.

Once you have installed and configured CI on this repository correctly, you’ll need to re-trigger Greenkeeper’s initial pull request. To do this, please delete the greenkeeper/initial branch in this repository, and then remove and re-add this repository to the Greenkeeper App’s white list on Github. You'll find this list on your repo or organization’s settings page, under Installed GitHub Apps.

I'm having trouble implementing your extension. Here is a very simplified test caseof a file i/o using enablePrivilege which works flawlessly on firefox 12 and 13 but not at all on 17.

I'm trying to get it to work on ff17, using your asyouwish extension. I've installed the extension correctly, I think.

If you look at the short self-contained html file at:
www.users.tpg.com.au/epbyrne/asyouwish

you can see the xpcom calls from javascript function. You can copy and run it in, as I say, earlier versions of ffox.

From reading the entries on bugzilla, and your docs on github, I'm unclear as to whether to embed any of your code in this html file, or whether running the extension alone is sufficient to force ff17 to allow the xpcom calls in my htm file.

This is really important to me, as I have developed a full-featured text processor which is for the desktop and which therefore mandates full file i/o through a filepicker.

Any clarification of implementing your extension into a simple html would be really appreciated.

Thanks, and congrats on an obviously fine project.

I am quite convinced at this point that the only real answer to this is to make a fork of Firefox that combines 19 and 14. This isn't as hard as it sounds... Firefox's code is pretty well documented and contrary to what Moz may imply, it doesn't change that often. The core is still held over from Netscape. The changes between the versions are so minor it would not be hard to restore the functionality of enablePrivilege. Mozilla as we know it is gone. The noble vision of Blake Ross.... corrupted beyond restoration. Like you said, this is a feudal fife governed by group-think. It's like Netscape all over again, and it will invariably suffer a similar fate. But in these times, people are afraid. Unless we wage the revolt... no one will. Only INTPs like us are so obsessed with abstracted responsibility that we'd go this far, am I right? ;)

I do think people inside Moz will help us if we ask. There are still a lot of rebels there, and even some arrogant enough to give us what we want if it'll shut us up. It might seem scary to not have Moz's security fixers at your back but these people really don't test their stuff to any meaningful extent. The security flaws are a result of their carelessness, and besides, it takes time to capitalize on them. There are other builds that aren't updated as often, and you don't hear complaints about them. It's not the end of the world.

But if we fight them here, it'll be a major coup for freedom. It'll change the whole web.

Could you consider working on xqib(xQuery in the broswer)?

Xqib( XQuery in the browser), furthering it as a programming language:

http://www.xqib.org/

http://archive.xmlprague.cz/2011/presentations/xqib.pdf

https://websci.informatik.uni-freiburg.de/teaching/ws201112/xmldb/sheets/xqib-documentation.pdf

http://cs.brown.edu/~kraskat/pub/www09-xqib.pdf

Hi Brett, question for you.

Something I used to do before Mozilla removed the ability to obtain elevated privileges was to let the user browse and select a file using an <input type="file"> element on an HTML form. Then I would get the full path name of the file using the value property of the input element and read it in using javascript and XPCOM components.

Your addon solves the reading in part but unless I am missing something it does not allow one to get of the full path name from the file input element. This requires "UniversalFileRead" privilege, see mozilla bug https://bugzilla.mozilla.org/show_bug.cgi?id=143220 and in particular the patch at https://bug143220.bugzilla.mozilla.org/attachment.cgi?id=288696.

Is there a way I could be able to read the full file name from a file control using the AsYouWish add-on, or if not could the facility be added ?

Thanks

I've had enough drama. Palemoon has grown and developed, and I frankly find its design superior. So I'm switching.

AYW does not work with Palemoon because Palemoon doesn't support Australis (so says the project lead). So, if we removed the Australis mods, that might be something. Another idea is to provide multiple editions for multiple purposes. For example, we could have one that allows only local pages access, and another that allows both websites and local pages access.

Hi Brett,

When I load, or save, a series of files in a batch mode, I'm noticing a significant lag. For example, a 20 file batch of even fairly short text files is taking 3-4 seconds to load. Similarly when saving. It should really be less than 1 second.

It's possible this is due to my screwy code, as a lot goes on with the i/o besides the actual read/writes. But I was wondering if you did any timings on low-level file i/o which might show up when you do a lot of files in a loop.

For what it's worth, here are my lowest level read/write functions:

/* ================ LOW LEVEL FILE READ/WRITE====================== */

// file to be read encoded utf-8
function file_read_fileio(Components, file) // file must be an nsifile - returns a string
{
var charset = "UTF-8";
const replacementChar = '0x0020'; // a space
var data = "";
var fstream = Components.classes["@mozilla.org/network/file-input-stream;1"].createInstance(Components.interfaces.nsIFileInputStream);
var cstream = Components.classes["@mozilla.org/intl/converter-input-stream;1"].createInstance(Components.interfaces.nsIConverterInputStream);

fstream.init(file, -1, 0, 0);
cstream.init(fstream, charset, 1024, replacementChar); // in 1024 byte chunks
var out = {};
var read = 0;
do {
read = cstream.readString(0xffffffff, out); // read as much as we can and put it in out.value
data += out.value;
} while(read != 0);
cstream.close();
fstream.close();
return data;
}

function file_write_fileio(Components, file, data) // file must be an nsifile, data is a string
{
var foStream = Components.classes["@mozilla.org/network/file-output-stream;1"].createInstance(Components.interfaces.nsIFileOutputStream);

// use 0x02 | 0x10 to open file for appending.
foStream.init(file, 0x20 |0x02, 00004, null);

var converter = Components.classes["@mozilla.org/intl/converter-output-stream;1"].createInstance(Components.interfaces.nsIConverterOutputStream);
converter.init(foStream, "UTF-8", 0, 0);
converter.writeString(data);
converter.flush();
converter.close(); // this closes foStream
}

/* ================================================== */

Thanks and regards,

Eamon

Hi Brett and thank you for writing the AsYouWish addon which I hope very much will solve the problem I and others have, namely how to access the local file system from javascript in Firefox.

My question is about where I can best find information on how to convert certain pieces of code, specifically.

    var file = Components.classes["@mozilla.org/file/local;1"]
            .createInstance(Components.interfaces.nsILocalFile);
    try
    {
        file.initWithPath( FLFileName.replace(/\//g, "\\\\") );
    }
    catch(e)
    {
        if (e.result!=Components.results.NS_ERROR_FILE_UNRECOGNIZED_PATH) throw e;
        reportWindow.showAlert("File '" + FLFileName + "' cannot be loaded: relative paths are not allowed. Please provide an absolute path to this file.");
        rslt.value=null;
        return;
    }

I tried the code below, assuming the "file" parameter would replace the object I was creating with "Component.classes[....]". But I'm guessing and the code below fails with a permissions error when referencing Components.results.NS_ERROR_FILE_UNRECOGNIZED_PATH. I guess some other privilege is needed or that code needs to be re-factored - where can I look to find information on such a conversion ?

AsYouWish.requestPrivs('file', function (file) {
try {
//var file = Components.classes["@mozilla.org/file/local;1"]
//.createInstance(Components.interfaces.nsILocalFile);
try
{
file.initWithPath( FLFileName.replace(///g, "\") );
}
catch(e)
{
if (e.result!=Components.results.NS_ERROR_FILE_UNRECOGNIZED_PATH) throw e;
reportWindow.showAlert("File '" + FLFileName + "' cannot be loaded: relative paths are not allowed. Please provide an absolute path to this file.");
rslt.value=null;
return;
}
}
catch(e)
{
alert("EXCEPTION1: " + e);
}
}, errBack);

I'm having an issue with AYW activation. Here's my test case:

I load my app from a desktop shortcut.

Target: "C:\Program Files (x86)\Mozilla Firefox18\firefox.exe" file:///C:/xthdraft/v5/v5-3/index.html
Start in: "C:\Program Files (x86)\Mozilla Firefox18"

When I load the app like this, it opens in firefox OK but the AsYouWish object is unrecognised. So my file i/o functions won't work.

UNTIL I refresh the browser. Then the AYW object loads and I'm good to go.

However, if I start the app without a shortcut, by opening firefox normally, then loading the app from FF's file menu, the AYW object loads and all it good.

To make this a bit clearer: if I load FF the first time, AYW is not recognised. I need to refresh the browser. Note that firebug does not exhibit this shortcoming. Therefore my app has nothing to do with this behaviour.

My question is: how do I make AYW active IMMEDIATELY on FF loading?

Once active, AYW is bulletproof.

Many thanks.

Hi Brett me again.

Do you know of any reason why AsYouWish would not work with the firefox nightlies. I'm using version 20.0a1 (2013-01-05) and window.AsYouWish returns undefined. Fine on 17.0.1 and 18 beta.

My test code

$ cat /cygdrive/C/Temp/test.html
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<script>
function test()
{
alert("window.AsYouWish=" + window.AsYouWish);
}
</script>

<body>

<input type="button" value="Click Me" onclick="test()" />
</body>

</html>
$

Will attach screenshots of the resullts.

Hi Brett

I believe I can show that there is a significant memory leak either in the AsYouWish addon itself or in the way Firefox 17 interfaces with it. To reproduce

(1) Download the test case attached to mozilla bug 497732 (https://bugzilla.mozilla.org/show_bug.cgi?id=497732). The attachment is a .jar file which you have to download and unpack.
(2) Open file test.html in the browser WITHOUT having AsYouWish installed.
(3) The test case allows a number of repetitions to be specified - default 100. It opens and closes popups in quick succession the requested number of times. Make sure your popup blocker settings will allow it to do this.
(4) Run a few tests using the default 100 repetitions. Monitor the memory usage - I am on a windows 7 platform using the windows task manager to do this. For me the memory usage goes up during the tests but falls back more or less to what it was shortly afterwards.
(5) Repeat step (4) with AsYouWish installed. For me the memory usage rapidly climbs and does NOT fall back after the tests. After 3 tests with 100 repetitions each my firefox process was approaching 1G in size. On the fourth attempt the process exit and the crash reporter appeared.

I hope this can be reproduced and tackled because the addon is otherwise a perfect solution for the problem I have with wanting to access the local file system from javascript.

Thanks,

Duncan.

Hello Brett,

First of all thanks and congratulations for your great add-on that takes the monkey off our side.

Before enablePrivilege was removed I had this code :

netscape.security.PrivilegeManager.enablePrivilege('UniversalXPConnect');
var rdf = Components.classes['@mozilla.org/rdf/rdf-service;1'].getService(Components.interfaces.nsIRDFService);
var datasource = rdf.GetDataSource(uri);
// ...

If there any way to have a replacement for this code with your addon ?

Thanks again,
Kind regards,
Jérémy

Hello Brett,

I received your email about my precedent issue and again thank you very much for helping.
I tried your solution but didn't manage to have it working. I get 2 errors : one is quite much a warning telling that components are deprecated, will be soon removed and another that prevent my code from working.

Here is my code :

Datasource.build = function(id, url, params, callback) {
    AsYouWish.requestPrivs(['chrome'], function (chrome) {
        var elem = $(id);
        var uri = new Datasource(url, params).render();
        var rdf = chrome.Cc['@mozilla.org/rdf/rdf-service;1'].getService(Components.interfaces.nsIRDFService);
        var datasource = rdf.GetDataSource(uri);

        Datasource.createManagers(id, elem);
        Datasource.registerCallback(id, callback);

        Datasource.clear(id);

        datasource.QueryInterface(Components.interfaces.nsIRDFRemoteDataSource);
        datasource.QueryInterface(Components.interfaces.nsIRDFXMLSink);

        DatasourceManager.push();

        elem.database.AddDataSource(datasource);

        datasource.addXMLSinkObserver(Datasource.observers[id]);
    });
}

The warning is triggered by
var rdf = chrome.Cc['@mozilla.org/rdf/rdf-service;1'].getService(Components.interfaces.nsIRDFService);
and the real error is triggered by
elem.database.AddDataSource(datasource); telling me the code isn't allowed to wrap ...

Do you have an idea about this ?

Thanks again,
Kind regards
Jérémy

Hi Brett,

I've been working with AYW for the past week or so, and have all my file i/o functions for my program working fine under ff17

But today, they suddenly stopped working, and I noticed that the options panel is blank!

There doesn't seem to be any obvious way of repopulating this panel, or altering it for that matter, from either your GUI interface or the other one using ff addon options button.

Without searching through your code base, is there a quick fix to this?

Maybe I have to reinstall the extension?

Eamon (aka xthdraft)