odatnurd / overrideaudit Goto Github PK
View Code? Open in Web Editor NEWEasily Manage Your Sublime Text Package Overrides
Home Page: https://odatnurd.net/overrideaudit/
License: MIT License
Easily Manage Your Sublime Text Package Overrides
Home Page: https://odatnurd.net/overrideaudit/
License: MIT License
Currently only files in the unpacked package directory that also exist inside of the sublime-package
file that is currently being used by Sublime text are considered to be overrides and everything else is ignored.
It's conceivable that you might create an override for a file in a package, followed by the package author releasing a new version that deletes or renames the file that you were overriding. Now the file is no longer considered an override but could possibly cause problems.
As an (admittedly contrived) example, consider if a package had a python plugin that implemented one of its commands which you overrode to tweak something, and the author of the package renamed that file and added new features.
Now on package update your file is no longer an override, so you get no notification. In the meantime depending on the names of the two versions of the file in question your version of the file may be loaded after the "official" one, clobbering the command away in favor of yours, which is making it act like an override even though it doesn't outwardly appear to.
If this is a concern, we could do something like:
[?]
in front of them to indicate we don't know what they areSome package source files are stored using tabs as the method of indentation. If using such a file for the basis of your override (perhaps by using PackageResourceViewer, for example) while you have the option set to convert tabs to spaces, you end up with an override that is functionally identical to the original (apart from your changes) but which is indented using a different mechanism.
When performing a diff of such a file you get informed that every line in the file is different which obscures the actual changes that you've made and makes it harder to detect what's going on.
It would thus be quite helpful if the diff was able to ignore leading and trailing white space when performing a diff of files to make it easier to isolate your changes. I would imagine that this would be configurable based on file type, as some files (e.g. Python source) is sensitive to the indentation level and should not be compared in such a way.
As far as I have been able to determine, there is no good way to achieve this via difflib
short of creating a hand rolled solution that takes it into account or using an existing library (possibly as a dependency) although my searching hasn't turned up any likely candidates thus far.
When using Sublime Text with OverrideAudit on multiple computers, the package overrides need to be recreated manually on each machine. The overrides can potentially be synchronized with user settings. It can be done by storing the actual package overrides in the subfolder inside the User/ directory (i.e. User/package overrides/) and creating a symlink in the Packages/ directory.
Files in subfolders of the User/ are ignored by Sublime Text. This feature, along with symlinks technique suggested for synchronization using Dropbox can be used to synchronize the overrides across multiple computers.
The suggested files and folders structure:
Packages/
*symlink to example package 1 override*/
*symlink to example package 2 override*/
User/
package overrides/
*example package 1 override*/
overriden files
*example package 2 override*/
overriden files
I have tested such configuration and it seems to work without any issues.
Implement a command that can perform a diff operation between an unpacked override and the packed version of the file that Sublime is currently using as a base (either the shipped version or the file from the completely overriden version, depending on context).
For any operation that modifies the contents of a package (delete override, cleanup, cleanup of case issues, etc) the package or packages in question should probably be inserted into the list of ignored packages prior to doing the operation, and then removed once they're done.
Some work would have to be done to make sure that this doesn't trigger an automated expiration check when the packages are removed from the list while still making sure that if other operations are happening in the meantime (e.g. something being upgraded or the user un-ignoring something manually) still would.
Possibly this could be done by extending AutoReportTrigger
with some static methods you could call to give it a list of packages, and it would add them to the ignored list and take them out, tracking which they were so that it would know it doesn't need to trigger a check.
This would need to happen if a package directory or override was renamed due to case issues, because the only way to do that is to move the file away to a different name and back (otherwise the underlying OS assumes you're trying to rename a file to itself).
In what other cases would this be needed?
Implement a command that provides a list of all packages which currently have some form of override enabled. This requires verifying if a sublime-package
is installed for a package that is shipped (complete override) as well as comparing the contents of an unpacked Package directory to a sublime-package
file to see if any files have the same names, taking OS specific case differences into account.
I am not a fan of using context menu. A keyboard friendly layout would make navigation much easier. Maybe something like git: graph
of GitSavvy?
While working from a report that shows overrides (e.g. Override Report or a Bulk Diff), there may be several overrides that you need to verify. Depending on the situation you may decide to keep some overrides as is, edit some, and delete others.
In these cases it would be useful if it was possible to mark the overrides as actions are taken, so that as you come back to the report you can keep track of your progress.
To this end, context menu items should be added to override items to allow for such marking.
[-]
to indicate that this override has been deletedIn this case, "accepting" an override means that you have decided after viewing a diff (possibly inline) and/or editing that you have handled this override and it is in an acceptable state.
[+]
as an indication that this override has been handledThese actions (particularly marking the override as deleted) would not take effect in other reports that might show the same overrides, is that an issue?
It may be possible to flag an edit or diff view of a file with an extra setting that indicates the report view that was used to launch it (and the position inside) so that these options would be available from within the edit/diff as well. Is that handy?
Would it be handy if, in the case of a bulk diff the action of deleting or accepting an override in this manner figured out what portion of the report buffer represents the current override and automatically folds it out of your way? If so, should that always happen or be configurable?
When the archive and the override files are identical, I don't need the report to show that to me. It would make more sense to diff package files and automatically freshen them when there are no differences.
If not by default, then at least through a manual command.
This occurs frequently when linking packages from the repo into your Data folder and then ST updates them to master
.
Currently OverrideAudit checks every time you open or save a file to see if it's possible that the file is an override (i.e. that it is in the Packages
folder and part of a package that could conceivably have an override) and if so it enables the override integration that allows you to diff the contents of your override.
Where this is an issue is that (for performance reasons) this does not exhaustively check that the file actually represents an override, which means that errors can occur when you try to diff and it is finally discovered that this is not really an override.
Potential fixes for this include deferring a more deep check to see if a file is actually an override at the time the context menu is being presented (possibly still too slow) or detecting the error at diff time and removing the context menu.
This may be something that slips into the 1.0 milestone during final testing prior to release, but in case it is not this is one of the first things that will be done for the next version.
There should be a command which would perform a check similar to a Bulk Diff of all overrides in a single or all packages (possibly implemented via those commands or having common code factored out) to find all overrides that don't have any differences from the underlying file so that they can be cleaned up.
Optionally via a configuration option, this would be able to remove any folders in the Packages
directory that become empty as a result of removing files from them. Any folders that were already empty would be left alone under the theory that we don't know why they're there and they might be important to the package in question.
While determining what files should be cleaned up, this should skip over any unpacked package folders that are symlinks or contain a control file/directory that indicates that they're under source control (e.g. .svn
, .hg
, .git
) as in those cases these folders may be part of something external being used for overrides such as someone working on changes to the default packages.
What gets skipped over should be configurable so that it can be tweaked to the user's preferences. It may also be interesting/useful to have a counter option that allows you to include such a folder even if it would otherwise be ignored.
Should this use a diff to determine if there are any differences, or calculate a CRC32 and compare it to the version in the sublime-package
file to ensure that the file is fully identical, or possibly be configured one way or the other?
It may or may not be possible for some plugin file to be overridden and be e.g. sensitive to the line endings used, which would not be compared via a simple diff.
For something that is going to have such repercussions (removing multiple files and folders) it doesn't seem right to just go ahead and remove the files without confirming with the user. Additionally the list could easily be so large as to make a confirmation dialog unreadably large if it contained all of the files.
To this end the command could potentially generate a new style of report that would show you in context exactly what would be removed (much like the OverrideList currently does) and either offer a context sensitive command via the menu/context menu/command palette to "commit" to the action or trigger a Yes/No/Cancel dialog when you close it (note: I'm not sure if you can cancel a close that's in progress so that may or may not be doable).
I added a Default.sublime-keymap
file to my unpacked ShellScript package, intending eventually to make a PR. At some point much later, I realized one of the rules was bugged and went to change it, but I didn't remember what Default.sublime-keymap
was called.
OverrideAudit to the rescue! Bulk Diff the ShellScript package and it'll tell me which file I tinkered with! Except not quite: Since there is no Default.sublime-keymap
in the original package, it doesn't appear in the diff at all.
New files in a package's folder are part of the diff.
Only existing packages [files] are diffed.
Fails to load; lots of greyed-out menu items in the context menu due to commands not being available.
reloading plugin OverrideAudit.override_audit
Traceback (most recent call last):
File "/opt/sublime_text/sublime_plugin.py", line 109, in reload_plugin
m = importlib.import_module(modulename)
File "./python3.3/importlib/__init__.py", line 90, in import_module
File "<frozen importlib._bootstrap>", line 1584, in _gcd_import
File "<frozen importlib._bootstrap>", line 1565, in _find_and_load
File "<frozen importlib._bootstrap>", line 1532, in _find_and_load_unlocked
File "/home/fichte/.config/sublime-text-3/Installed Packages/OverrideAudit.sublime-package/override_audit.py", line 24, in <module>
ImportError: No module named 'OverrideAudit.lib'
A specialized Override Report that only shows files that are different from the base OR a way to skip unmodified files in the Bulk Diff Report (and I'd code fold to level 2).
I have full package overrides for lots of Shipped Packages, but most of those files are unmodified. There are dozens of snippets I've never even looked at. They all show up in OA reports. When I do an Audit, what I really care about are the files I've added, deleted, or changed.
The default OA behaviour works really well if you're using OA to tweak a single file for personal use, but not as well when you're developing or integrating changes to sublimehq/Packages
.
This package rocks.
When diffing an override that is contained within a subfolder of a package, the diff header mixes path separators.
--- Installed Packages\OverrideAudit\messages/install.txt 2017-04-04 08:20:16
+++ Packages\OverrideAudit\messages/install.txt 2017-04-04 09:07:01
@@ -1,5 +1,7 @@
OverrideAudit
=============
+
+© OdatNurd 2017
OverrideAudit helps you keep track of all of your package overrides, easily
allowing you to see what overrides are in place, how they're different from the
PackageResourceViewer: Open Resource
OverrideAudit
messages
install.txt
OverrideAudit: Diff Override
/
and \
style path separatorsThe diff would show only a single style of path separator. It might make sense to use the path separator that is used by the current OS. Although, saying that, I use Windows and would be quite happy to see Unix style path separators used exclusively, but probably I'm the odd one out there. ;)
Sublime Text build: 3125
OverrideAudit: v1.0.0
Operating System: Windows
Case Sensitive Filesystem: No
ST's mini diff has started displaying totally wrong markers on certain files. It looks like the reference document used to create the diff, is heavily outdated.
I am able to reproduce this issue for sure by enabling/disabling OverrideAudit.
When opening a git tracked Packages/Perl/Perl.sublime-syntax file, the mini diff should only show markers on lines which have not been committed yet.
The mini diff shows markers on many more lines even if the whole file is commited to git.
Note: The Syntax Package should contain changes compared to the shipped builtin syntax packages to trigger the bug.
Currently, when a diff indicates that there are no changes between an override and its source file, the only text displayed is some placeholder text to tell you that there are no differences.
For some use cases/some users, it would be useful if such a diff still contained the diff context header that indicates the two files that participated in the diff and their relative time stamps.
Since this is not the usual behaviour anyone expects from a diff, if implemented this would be a configurable item that defaults to being disabled so that users that want/need this information can enable it if desired.
Such a setting could be allowed to control if this happens for only single file diffs or also for bulk diffs as well. In the case of a bulk diff the utility of the header would be to tell you the time stamps of the two files, since the location of the files can be determined by looking at the package header.
This implements the ability from a diff view (either a single diff or from the bulk diff report) to move an unpacked override file to the trash or to open the override file. This allows for the ability to quickly review and discard any overrides which are no longer providing anything of value or to make quick modifications.
Additionally, for a file which has been opened from a diff view by any of the above outlined options, a command should be available from the context menu or from the command palette to re-execute the diff.
Implement a command that provides a comprehensive report of all currently installed packages, delineating which are dependencies and which are currently being ignored via settings.
The report should contain an indication on whether a package is a package that ships with sublime or installed by the user, and in the case of a user installed package if the package contains sublime-package
and/or some unpacked components.
Monokai
color schemeThis would add a content menu/command palette entry that is only available in a single file diff that would refresh the diff in the current view in the same way that reports can be.
Currently the only way to accomplish this is to go back to the place you originally initiated the diff and perform the same action again, or use existing commands to swap between an edit view of the override file and its diff.
Before the initial release of the package, verification needs to be done that all commands do something sensible when invoked when there are no overrides of any kind present.
More specifically this needs to make sure that some visual indication is made when the command is not doing anything, such as a popup or a notification in the status bar. Some of the commands currently do that (where some equals "at least one") but for best user experience it should not come as a surprise when you try something and nothing happens.
The detection in place for packages that are dependencies assumes that the dependency has been installed by package control, and so it does not appropriately detect a dependency that has been locally installed (e.g. is currently under development).
Such a dependency package would be properly detected and handled as a dependency.
These dependency packages are not properly detected and so they appear as regular packages in the Package Report.
Using OverrideAudit itself as an example, installing a sublime-package
version of the package alongside an unpacked version does not show the contents of the syntax
or lib
folders as possible overrides, even if the content of the local file is different than the packed version.
This manifests as not showing such files as overrides in the Override Report, as potential candidates for a single file diff or from showing up in a bulk diff (single or multiple package).
I really like the context menu on the tab that allows one to diff or delete the current override.
I would find it really useful to also have an option to revert an override.
Open an override, right click the tab and see a menu item to revert the override. This would unpack the original file and replace the contents of the current buffer with that.
Have an override open, maybe even perform a diff, and see that you want to start fresh from the original file. Right click the tab and choose Delete Override
. Close the tab, and then go through the whole process of finding and opening the file again from PackageResourceViewer.
On systems with a file system that is not sensitive to case (e.g. Windows and MacOS), Sublime Text internally treats files as identical as long as the only difference is their case. Thus for example, DEFAULT/exec.py
, Default/EXEC.PY
and Default/exec.py
are all treated as identical. This also follows for sublime-package
file names, wherein you can create a complete override of Lua.sublime.package
with a LUA.sublime-package
.
OverrideAudit works with these files the same way that Sublime appears to in this situation, and uses the names and contents of the shipped and installed sublime-package
files as "gospel" when determining what file names should be represented as.
It is conceivable that someone going from e.g. Windows to Linux and transferring their configuration over could find themselves having issues, where suddenly their overrides no longer function the same way as they used to.
This could manifest as whole new competing packages, duplicated extensions (e.g. menu item additions) or in the case of python plugin files and load order commands that don't operate as expected.
My initial thoughts for this would be to treat it similarly to #15 as a potential new report that shows you issues that may crop up, along with a confirmation to repair them, but there may be a better way.
Parts of this may be covered by #16 for overrides files that are in a correctly cased package directory but have the wrong case themselves. Additionally this may suffer from the same potential issue as #15 if implemented as a report and the contents change.
Implement a command which will find all simple overrides for all packages and bulk diff them all at once, producing output to a buffer similar to that used in Find in Files
. This would provide the ability to see at a glance all existing overrides and what changes they may be making.
Since package expiration is based solely on the time stamps of the related files, it is probably very common that a package updates and many or all overrides that exist for it are expired even though there are no functional changes made.
In order to stop being notified that the override is expired, it is currently necessary for the user to open the override and save it to update the last modification time. For relatively few overrides this can be streamlined by setting diff_unchanged
to "open"
so that as you manually diff to see, unchanged files open and you can immediately save and close.
For the use case of many overrides it would be handiest to use a bulk diff of the package to see which overrides you need to focus on and then easily mark one or many overrides as being "OK" without having to take extra steps
To that end, there should be something like a context menu item available on overrides that are expired that allows you to "freshen" them without having to open them. Additionally any package that has at least one expired override should offer a context menu to allow you to similarly freshen all overrides in one shot.
Since this could conceivably mask problems, I'm envisioning that this would prompt you to verify that you meant to do it the same way that deletion does. This would probably be configurable in the same way and possibly allow you to specify that you only want to confirm on a "bulk freshen" but not on an individual one.
How objectionable would it be if, when doing a "Bulk Freshen" every override was touched instead of just those that are currently expired?
Clearly that is a lot easier to implement as a context item on the package because the list of all overrides is easier to come by than those that are expired once the report is finished generating.
My current thoughts are that this could be considered to be a benefit in that it would mean that all overrides for a package were all "reviewed" (for lack of a better term) at the same time.
I think this is really only an issue for packages installed via PackageControl, because it seems like when PackageControl updates a package it creates an entirely new sublime-package
file, in which case the entire contents of the package would always have the current date and so every override would be expired all at once.
By contrast the packages that ship with sublime (at least in Linux 3126) contain files with different time stamps.
As currently implemented, the code base collects the list of packages for every command that is executed. Additionally, doings things like collecting override information and/or diffing them is something that does not take zero time.
Failing a complicated operation (likely per platform) to watch the contents of all package folders to know when files are being added or removed so that this information could be calculated once and cached, at the very least the more time prohibitive operations such as diffing should be done in a background thread.
Perform an inspection to check to see if any overrides in use are "out of date" due to their being older than the files that they are overriding. This would work not only for a complete override but also single unpacked override files.
The report should call out overrides which may need to be rethought/recreated/removed so as not to block incoming changes.
ignored_packages
to notice when a package is probably being installed or updated, and generate a report if anything is out of date
The last item in the list has been swapped for listening to changes in the standard preferences changing so that we can see when a file is being added and removed from the `ignored_packages` setting. This is something that PackageControl does when it installs or updates packages, so this seems like a good time to generate a potential report.
Additionally, the item for generating a report once per day for an extended session has been removed. The idea currently is that with reports happening every time a package upgrades and whenever the sublime version changes, there is no need for an ongoing automated report; any changes that happen during a session are most likely something you're doing outside the auspices of Package Control, so you probably know what you're doing.
Currently the only thing done with complete overrides is to tell you when they exist, check them to see if they're expired or not and to ensure that their source package is used as the base when diffing and determining overrides. #15 would add the ability to accept them as still valid if desired.
There could be something similar to the existing reports and diffs that would let you work just between the two packed versions of the file and ignoring the unpacked file, if any. Doing so would allow you to see things like how your complete override differs and if the composition of the package has changed.
I'm not sure exactly how useful that would be because I'm not entirely sure how popular such a complete override is. The initial version of OA has the required code to collect that information but it's not realized in any commands.
If it's not a particularly common practice then in all likelyhood just having an indication that there is a complete override and it may need to be reconsidered could be enough in the short term.
Unable to open diff for package with a *
in its name.
From the override overview, which looks as follows:
WARNING: Showing only expired overrides!
WARNING: Non-expired overrides may exist!
Report Generated: 2019-02-20 13:31:49
[ IU] Swift for F*ing Sublime
`- [X] Swift.sublime-syntax
When I right-click on Swift.sublime-syntax
and request a diff for the file, the exception below occurs.
Traceback (most recent call last):
File "/home/fichte/.config/sublime-text-3/Installed Packages/OverrideAudit.sublime-package/lib/threads.py", line 96, in <lambda>
File "/home/fichte/.config/sublime-text-3/Installed Packages/OverrideAudit.sublime-package/override_audit.py", line 1371, in <lambda>
File "/home/fichte/.config/sublime-text-3/Installed Packages/OverrideAudit.sublime-package/override_audit.py", line 1381, in _pkg_loaded
File "/home/fichte/.config/sublime-text-3/Installed Packages/OverrideAudit.sublime-package/lib/packages.py", line 715, in __getitem__
KeyError: 'ing Sublime'
I have a package installed that mimics the name of another package. I do not see overrides between the original (which is effectively being partly overridden) and this camouflaged package.
// Package Control.sublime-settings
"package_name_map":
{
"real package name": "Package disguises itself as this"
},
Overrides are listed and diffable.
No overrides noticed
While in a Bulk Diff report (either single or multiple package), any override that starts with a leading period (e.g. .gitignore
) is not correctly recognized as an override filename. It lacks the proper syntax scope, which causes it to not syntax highlight correctly and not support the context menu items for overrides.
A with this name should still be detected as an override, even if it is unlikely that it would be used. For example, PackageResourceViewer
seems to hide such files from the list of overrides.
The filename in question (.gitignore
as well as .github/ISSUE_TEMPLATE.md
in this case) was not correctly syntax highlighted.
[Steps to follow to reproduce the issue. Feature Requests can omit this if desired.]
Preferences > Browse Packages...
from the menuOverrideAudit
.gitignore
OverrideAudit
packageA declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.