Plugin API about powerline HOT 33 OPEN

A nice touch here is that the user can allow or disallow plugins to take control of the statusline. Another solution could be to have a config directive like "disallow_plugin_themes": ["nerdtree", "..."] for disabling single plugins instead.

In what I suggested plugins can only add subthemes (i.e. buffer-local themes)*. Thus instead of separate config option "theme": {"nerdtree": null} should be enough.

* We will have nasty default theme modifying API if it was otherwise. I see no way how this API can make good-looking resulting theme with every possible combination of plugins.

Now that segments are written in Python, we would have to require plugin authors to potentially learn Python to write segments, instead of just using vimscript.

Making vim+json only segments is doable though for me it is unnecessary: I am already using python for speeding up my plugins and even the solution not allowing to invoke vim at all is fine.

Python modules work quite differently from vimscript and autoloading. How would we resolve the path and module issues for all the different vim configurations and bundle setups out there? Lookup plugins with a powerline.json file in their main folder and cache the paths, and later load them into Powerline? How would we do segment imports?

I propose one of the following:

1. sys.path.append is called by the plugin itself, json holds information about module name and name of the function in this module. Modules are then imported by powerline. It is assumed that sys.path.append is required for plugin to work properly so it will always be used. Alternatively json also holds path that is fed to sys.path.append, path is always relative to the JSON file (absolute paths should be an exception, don’t encourage authors writing code that is incompatible with most of the addon managers).

2. Segments are added by some API. It is non of our business how it will be called as far as it is called before showing statusline.

3. Powerline says that all segments data written in python should be located in rtp/powerline/segments/__init__.py, powerline imports it on its own locally using something like

   sys.path.insert('rtp/powerline')
   try:
       import segments
       # call some API functions on data in segments.{attribute(s)}.
   finally:
       sys.path.pop(0)
   

   (in case python allows this and won’t cache modules). Does not work in case somebody needs to install plugin to ~/.vim, so should probably be skipped.

All of the three are segments definition. Subthemes definition is done always by json files, likely in rtp/powerline/subthemes/*.json, syntax is equivalent of user_config['vim']['theme'] from the example. Matchers are added just like segments.

Vim+JSON segment definition can be added to any of the above at will if required, though for 2. and 3. it will be a separate mechanism and for 1. just an extension to already existing.

We need to have a solid and tested API from the beginning, because once plugin authors start to use it it would be a lot of work updating it. We could include a version tag in the JSON files to be able to update the API more easily.

I don’t have any objections, it looks like a good idea.

from powerline.

Not sure if I understand you correctly regarding the plugin buffer-local themes. Do you mean that we shouldn't automatically enable support for stuff like nerdtree, but rather let the user explicitly enable support for each plugin Powerline supports?

I think I agree that we should just say that all segments must be written in Python. The syntax is super simple anyways, and most plugin authors probably won't have to write more than a couple of lines to create their segments.

Regarding the path handling, I think I like suggestion 2 the best. I don't know how vim does namespacing when using the :python command, so it would require some experimenting. Since Python has first class functions we could provide a function for plugin authors that does something like:

def my_plugin_segment(some_argument=True):
    pass

from powerline import register_segments
register_segments(my_plugin_segment, ...)

In order for this to work, Powerline would have to make its path publicly available inside vim, do you have any suggestions on how to do that? I'm thinking that we could assign g:Powerline_path from inside the Python script. We would also have to force users to load Powerline before any other plugins by sourcing it from their vimrc, similarly to how plugin loaders does it. What do you think?

from powerline.

Not sure if I understand you correctly regarding the plugin buffer-local themes. Do you mean that we shouldn't automatically enable support for stuff like nerdtree, but rather let the user explicitly enable support for each plugin Powerline supports?

Not exactly. Support is enabled automatically for certain buffers, but plugin cannot modify any existing subthemes. Thus my aurum can’t add, for example, current revision number/branch name to NERDTree buffer. But neither it can add branch name to default subtheme (that is used for most files) without user interaction. Another case are buffer own files: aurum will add subtheme displayed for aurum://commit buffer unless user explicitely disabled it to do so.

Regarding the path handling, I think I like suggestion 2 the best. I don't know how vim does namespacing when using the :python command, so it would require some experimenting. Since Python has first class functions we could provide a function for plugin authors that does something like:

No experimenting, I’ve seen the C code. Vim executes any code passed to this commands in the namespace of the module it creates when python is first used.

In order for this to work, Powerline would have to make its path publicly available inside vim, do you have any suggestions on how to do that? I'm thinking that we could assign g:Powerline_path from inside the Python script. We would also have to force users to load Powerline before any other plugins by sourcing it from their vimrc, similarly to how plugin loaders does it. What do you think?

I always do such job by

python import vim, sys
python sys.path.append(vim.eval("'/path/to/powerline'"))
python import powerline

(in case you have seen my plugins steps 1 and 2 are masked in frawor, which also protects from double-appending the same path). I don’t see any need in assigning g:Powerline_path. Just add a file like

" File rtp/autoload/powerline.vim
" All other files are assumed to be in rtp/python/**/*.py
scriptencoding utf-8
if exists('g:powerline_loaded')
    finish
endif
let g:powerline_loaded=1
function powerline#Dummy()
endfunction
function powerline#InitializePython()
    python import vim, sys, os
    python sys.path.append(os.path.join(vim.eval("expand('<sfile>:h:h')"), "python"))
    let g:powerline#init=function('powerline#Dummy')
endfunction
let g:powerline#init=function('powerline#InitializePython')

. Actual import is done by using import powerline somewhere after call g:powerline#init(). Maybe it makes sense to switch to dictionary functions in order to force users never use powerline#InitializePython:

" File rtp/autoload/powerline.vim
" All other files are assumed to be in rtp/python/**/*.py
scriptencoding utf-8
if exists('g:powerline_loaded')
    finish
endif
let g:powerline_loaded=1
function s:.dummy()
endfunction
function s:.init()
    python import vim, sys, os
    python sys.path.append(os.path.join(vim.eval("expand('<sfile>:h:h')"), "python"))
    let g:powerline#init=s:dummy
    unlet s:init
endfunction
let g:powerline#init=s:init

. This way only the first caller has a chance to call it twice, and only if it intentionally saved old function reference. But actual call now has to be call call(g:powerline#init, [], {}).

from powerline.

Regarding the path handling, I think I like suggestion 2 the best.

(From #2) How will this work in tmux and powerline? I guess we need the fourth:

1. powerline.path variable similar to sys.path and defaulting to [os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), 'config', 'segments')], it can be changed by plugins prior to displaying powerline. May be a temporary addition (with higher priority) to sys.path or a temporary replacement.
2. Either python modules or just JSON files describing segments in directories mentioned in powerline.path. I would rather prefer python modules and description of segments in theme file: filestatus segment uses segment attribute from filestatus module or* filestatus attribute from powerline.segments.{vim|tmux|…} module, aurum.status uses status attribute from aurum module.
3. Some user JSON configuration that adds directories to powerline.path (purely for tmux/zsh users, vim ones can expect plugin writers use powerline.path.append directly).

* Alternatives, not simultaneous behaviors used whichever succeeds first.

from powerline.

See new commit. Not running a PR as I will probably come up with a better ideas and definitely have to do some polishing: like allowing ~ in path. Currently it allows having

# config.json
{
    "common": {
        "path": ["/home/zyx/.vim/dev/aurum/powerline"]
    },
    …
}
# themes/vim/default.json
{
    "left": [
        …
        {
            "module": "aurum",
            "name": "status",
        },
        …
    ],
    …
}

and have segment function defined in /home/zyx/.vim/dev/aurum/powerline/aurum.py in .status attribute. Untested.

Just something to start with that can be discussed.

Note: "module" meaning has changed, it now defaults not to core but to powerline.ext.(ext).segments.core. It is assumed that foreign extensions will throw something meaningful (i.e. NotImplementedError) if called from improper extension: at least tmux and terminal segments have too much in common (currently one is copy of the other) to separate them.

from powerline.

This issue is one of the last ones we have to find a good solution for before we can do an initial release of Powerline.

How about this: We could use runtime! to search all plugins for something like powerline/register_plugin.vim, which could contain some code that makes the plugin's segments and matchers available for usage in themes (possibly automatically injecting segments or matchers so it will be enabled without user interaction?) Not entirely sure how this could be implemented in a clean way, any ideas?

from powerline.

Adding segments is not much problem: there is one thing that is both necessary and enough: adding appropriate path to sys.path and you will get access to needed segments; I already implemented that part in aurum (but not currently pushed).

Buffer-local themes is a bit trickier. Currently (untested) you need to add to sys.path and add thing like "aurum.matchers.commit": "aurumcommit" to config.json: ext/vim/local_themes and copy aurumcommit.json (pretending it exists) to ~/.config/powerline/themes/vim/aurumcommit.json. It would be better if both or, at least, the second part could be avoided. I may suggest using add_local_theme method of powerline object to achieve this. Not sure whether or not runtime! powerline/register/*.vim is worth supporting (note: not runtime! powerline/register_plugin.vim in order to support old-style “every bit of junk in ~/.vim” installations), but it will be convenient (otherwise I need to add initialization of plugins supporting powerline just before powerline initialization).

from powerline.

How did you implement it in aurum? I've been thinking more about it and I was thinking that Powerline itself could walk through &rtp on startup, and check if each path has a powerline directory in it, with themes and stuff. In aurum's case you would have this directory structure in the root of the repo:

powerline/
    aurum/
        matchers.py
        segments.py
        themes/
            commit.json

Powerline then does the following:

• Walks through &rtp and checks if there is a powerline directory in each path
• Discovers powerline dir in aurum plugin, and adds this path to sys.path
• Uses your add_local_theme method to add the matchers and the themes
• The themes would have "module": "powerline.aurum.segments" for each segment since the segments now are available in sys.path [edit: could possibly add a default_module option for themes to avoid specifying the module for each segment?]

What do you think about this way of doing it?

from powerline.

How did you implement it in aurum?

Just adding call FraworLoad('@%powerline') before initializing powerline. autoload/powerline.vim loaded by this call “contains” the code that adds python path to sys.path (I can put the file here, but without experience with python support in frawor or using a debugger you won’t ever notice where path is added). Everything else but the addition to sys.path is a maintainence stuff necessary only to support querying status and branch names in a separate process.

How did you implement it in aurum? I've been thinking more about it and I was thinking that Powerline itself could walk through &rtp on startup, and check if each path has a powerline directory in it, with themes and stuff. In aurum's case you would have this directory structure in the root of the repo:

There is no need to add this to code, putting the job of modifying sys.path on the plugin writers is better. Simple runtime! macros/powerline/*.vim is enough, macros in will do all the necessary stuff (especially concerning the fact that just sys.path is not enough).

In fact I would be against this behavior and won’t use it. All python modules are going to python/ dir and nowhere else. All python3 modules are going to python3 and nowhere else. This is the convention that has support in frawor.

The themes would have "module": "powerline.aurum.segments" for each segment since the segments now are available in sys.path [edit: could possibly add a default_module option for themes to avoid specifying the module for each segment?]

default_module idea is good.

from powerline.

I see. Well the directory structure was just a suggestion, the modules could go in a python dir if that's the convention. I feel that you have much more experience with the vim/python combination than me, so it would be awesome if you could create a demo of how you feel this should work.

from powerline.

I see. Well the directory structure was just a suggestion, the modules could go in a python dir if that's the convention.

It is only frawor convention. Somebody puts them directly in plugin or some subdirectory there. I saw more different variants like putting into pylibs or a subdirectory into autoload. Main idea is that you should allow plugin writers follow whatever convention they are already following, not produce unnecessary additions to sys.path and spread files across filesystem. Using runtime! macros/powerline/*.vim or runtime! powerline/register/*.vim in powerline/ext/vim/powerline.vim allows everybody keep their conventions and still not require user add more lines to vimrc like aurum does now. Or you can even do nothing, just set up the vim variable g:powerline_loaded and defer initialization of powerline until VimEnter: this way plugins can put powerline-specific initialization into if exists('g:powerline_loaded') block to some file inside plugin/ directory, not sure what is the best.

Isn’t there some sites where we can make vim plugin writers vote on the issue? I see no big advantages in either of runtime! and variable approaches. Deferred initialization should probably happen in either case, for the same reason you added it to vim-powerline (to make plugins initialize themselves in a natural way in place of pushing them with :runtime and, probably, observing weird behavior when vim repeats sourcing when it starts walking on &rtp).

from powerline.

I agree that it would be best if we could get some feedback from other plugin authors, but I don't know where we should ask for that. My first thought is on Reddit, but I'm not sure if any of the "big" plugin authors even read /r/vim.

from powerline.

I did some minor changes to your default module theme key as the code you submitted didn't work with vim. It's tested now and it works well with the new common segments module.

from powerline.

I did some minor changes to your default module theme key as the code you submitted didn't work with vim. It's tested now and it works well with the new common segments module.

You have successfully defeated the purpose: theme_config.get('default_module') there is getting default_module value out of only the default theme. You need to either recreate Segments() instance every time you process theme or pass this value as an argument to get. You do neither.

What was the problem with vim?

from powerline.

Ah, I see: forgot additional arguments to get_filler and get_string.

from powerline.

Note: I get errors with either old and new code, but they are related to highlighting.

from powerline.

Crap, I didn't see that at all. The problems were with the arguments. I'll revert to your version. We should probably create some tests soon to catch errors like this.

from powerline.

See pull request #75, it fixes highlighting issues.

from powerline.

Regarding your way of handling default_module, the reason why I decided to try to rewrite it was that it makes no sense to pass a default_module to string and filler segments, and I wanted to avoid that. Also it doesn't look ideal to me to pass this around as a method argument, and I think it would be better if it was a property of the theme that the segments could access. What are your thoughts on this?

from powerline.

It did not make sense to pass self to both of them and segment to filler segment. @staticmethod purges out the self argument from the function definition and denies usage of it by the method, but it did not disappear completely.

You can always resort to the other idea: create Segments instance in Theme.__init__, not in Powerline.__init__.

from powerline.

You can always resort to the other idea: create Segments instance in Theme.init, not in Powerline.init.

See last commit to #76.

from powerline.

I like that much better, nicely done.

from powerline.

I'm trying to make a demo plugin and make the required changes to Powerline in order to complete this issue, and I've run into a problem. I've declared powerline to be a namespace package, and I'm thinking that plugin authors should distribute segments in the powerline.contrib.segments.<plugin> namespace. The problem arises with the way paths must be added for this to work: Plugin paths appear to be required to be added to sys.path before powerline is imported for the first time, or Python will stop searching for other namespace packages:

>>> import sys
>>> sys.path.append('/path/to/plugin/python') # this path contains a powerline.contrib module
>>> import powerline.segments # imports powerline.segments from the main powerline package
>>> import powerline.contrib # imports powerline.contrib from the appended path above

Now this fails if powerline has been imported before the plugin has been added to sys.path:

>>> import powerline.segments
>>> import sys
>>> sys.path.append('/home/kim/powerline-plugin/python')
>>> import powerline.contrib
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named 'powerline.contrib'

For this to work properly, Powerline plugin paths would have to be added to sys.path before powerline is imported for the first time (in bindings/vim/powerline.vim).

@ZyX-I Do you have any ideas on how this could be resolved?

from powerline.

@ZyX-I Do you have any ideas on how this could be resolved?

Don’t do the

I've declared powerline to be a namespace package, and I'm thinking that plugin authors should distribute segments in the powerline.contrib.segments. namespace.

? Why do you need namespace package? Mercurial is fine with absolute import of its extensions (though there are two more options: importing extension specified by exact path and (default) import hgext.module_name). Absolute import is currently known to work perfectly. For vim we need nothing more then we currently have regarding this issue. What would be fine is configurable default configuration (args merge and some options like "before") for the segments so that I don’t need to supply u"before": u"symbol" for branch segment in each buffer-local theme I create using add_local_theme. For other we seem to need nothing more (i.e. I currently can’t imagine the use-case when what we already have is not sufficient).

What are you trying to achieve? The only thing I know about a namespace package is this rejected PEP with example code how these packages can currently be implemented. It also may be interesting to see this two mercurial functions that are used to load mercurial extensions. Mercurial is not a namespace package, though it loads hgext.{extname} by default (if path is not supplied and trying to load this did not raise ImportError, in which case just {extname} is tried).

from powerline.

Well, I'm trying to make it possible to have a powerline namespace, instead of having to have contrib packages in their own namespace. I think the ideal solution would be to be able to use the existing Python import functionality, which the example code in the rejected PEP shows how to do (and I thought it seemed like a decent solution - if it worked, of course). I didn't know about imp or how mercurial loads packages, but it looks simple enough, I'll take a look at it.

from powerline.

I have to be honest, I really don't know exactly how to resolve this issue. I think I'll give it a rest for now, but feel free to either implement the required changes, or provide a list of specific changes that are required for proper third-party plugin support so I can begin working on it. The issue is blocking the initial release, so hopefully we'll be able to resolve it soon.

from powerline.

I vote for the mercurial strategy. While the namespaced idea is nice, it just won't work. I think trying to hack in dynamic namespace resolution into python is un-pythonic, and we should just use a top-level extension namespace (or no namespace whatsoever powerline_nerdtree, or any combination thereof).

from powerline.

Hi, I see that this issue has been opened for a long time, also a lot of work seems to have gone into it.
However I don't see any theme for plugins like ctrlp. I would really like to see it like we did with vim-powerline.

from powerline.

There is one thing not covered by any APIs: default segment color. Suggestion: add generic highlight groups to colorschemes. Names are to be discussed, my suggestion is much likely not the best one (some names feel bad to me, but I do not know how to fix them). Suggested names:

• background (same as fileformat/fileencoding/filetype)
• information:optional (same highlight as branch)
• information:regular (same as file name)
• information:problem (something red) (to be used in segments like #376)
• information:change (same as VCS status M or unsaved changes indicator)
• information:critical (like PASTE)
• gradient (same as line_percent_gradient)

Make lint checker require all of these standard highlight groups.

from powerline.

We really need to implement this asap. I've made an attempt on a basic external plugin example for plugin authors. @kovidgoyal @ZyX-I can you guys check out Lokaltog/powerline-contrib@5c77c2750dacebfa784587a6f47d71a1c54ee2e6 and give me your feedback on that commit? Thanks.

from powerline.

Unfortunately, I am totally swamped at the moment, so dont wait for feedback from me.

from powerline.

Revised names:

• information:unimportant (fileformat/fileencoding/filetype)
• information:additional (branch)
• information:regular (file)
• information:prioritized (unsaved changes)
• information:critical (paste)
• issue:regular (syntastic code style issues)
• issue:important (intended for errors in tests)
• issue:critical (intended for errors during build process)
• information:additional:gradient
• information:regular:gradient
• information:prioritized:gradient (all gradients may only have difference in foreground with groups without :gradient suffix; may be all information: groups should have their gradient counterparts)

from powerline.

Marking this as “configuration” since these names are the only thing left that is a part of this issue.

from powerline.