chrieke / mkdocs-exclude-search Goto Github PK

View Code? Open in Web Editor NEW

27.0 3.0 2.0 104 KB

🔎 A mkdocs plugin that excludes selected chapters from the docs search index.

License: MIT License

Python 98.28% Makefile 1.72%

mkdocs-plugin mkdocs mkdocs-material plugin documentation search

mkdocs-exclude-search's Introduction

mkdocs-exclude-search

A mkdocs plugin that excludes selected chapters from the docs search index.

If you only need to exclude a few pages or sections, mkdocs-material now introduced built-in search exclusion! The mkdocs-exclude-search plugin complements this with more configuration options (wildcard exclusions, ignoring excluded subsections). It also provides search-exclusion functionality to regular mkdocs users.

Setup

Install the plugin using pip:

pip install mkdocs-exclude-search

Activate the search and exclude-search plugins in mkdocs.yml. search is required, otherwise exclude-search has no effect!

plugins:
  - search
  - exclude-search

More information about plugins in the MkDocs documentation.

Configuration

List the markdown files to be excluded under exclude using the format <path>/<to>/filename.md in the docs folder.
Exclude specific heading subsections using the format <path>/<to>/filename.md#some-heading. Chapter names are all lowercase, - as separator, no spaces.
Exclude all markdown files within a directory (and its children) with dirname/*.
Exclude all markdown files with a specific name within all subdirectories with dirname/*/filename.md or /*/filename.md.
To still include a subsection of an excluded file, list the subsection heading under ignore using the format <path>/<to>/filename.md#some-heading.
To exclude all unreferenced files (markdown files not listed in mkdocs.yml nav section), use exclude_unreferenced: true. Default false.

plugins:
  - search
  - exclude-search:
      exclude:
        - first.md
        - dir/second.md
        - third.md#some-heading
        - dir2/*
        - /*/fifth.md
      ignore:
        - dir/second.md#some-heading
      exclude_unreferenced: true

nav:
    - Home: index.md
    - First chapter: first.md
    - Second chapter: dir/second.md
    - Third chapter: third.md
    - Fourth chapter: dir2/fourth.md
    - Fifth chapter: subdir/fifth.md

This example would exclude:

the first chapter.
the second chapter (but still include its some-heading section).
the some-heading section of the third chapter.
all markdown files within dir2 (and its children directories).
all markdown files named fifth.md within all subdirectories.
all unreferenced files

mkdocs-exclude-search's People

Contributors

Stargazers

Watchers

Forkers

seppi91 tyris

mkdocs-exclude-search's Issues

Build errors in 0.6.1

Hi, I'm getting errors when building docs while using mkdocs_exclude_search==0.6.1

Stack trace below:

...
Successfully installed mkdocs-exclude-search-0.6.1

$ mkdocs build
INFO     -  Cleaning site directory
INFO     -  Building documentation to directory:
            /Users/michael/github/SeleniumBase/site
Traceback (most recent call last):
  File "/Users/michael/.virtualenvs/sbase9/bin/mkdocs", line 8, in <module>
    sys.exit(cli())
  File "/Users/michael/.virtualenvs/sbase9/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/Users/michael/.virtualenvs/sbase9/lib/python3.9/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/Users/michael/.virtualenvs/sbase9/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/michael/.virtualenvs/sbase9/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/michael/.virtualenvs/sbase9/lib/python3.9/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/Users/michael/.virtualenvs/sbase9/lib/python3.9/site-packages/mkdocs/__main__.py", line 187, in build_command
    build.build(config.load_config(**kwargs), dirty=not clean)
  File "/Users/michael/.virtualenvs/sbase9/lib/python3.9/site-packages/mkdocs/commands/build.py", line 317, in build
    config['plugins'].run_event('post_build', config=config)
  File "/Users/michael/.virtualenvs/sbase9/lib/python3.9/site-packages/mkdocs/plugins.py", line 104, in run_event
    result = method(**kwargs)
  File "/Users/michael/.virtualenvs/sbase9/lib/python3.9/site-packages/mkdocs_exclude_search/plugin.py", line 258, in on_post_build
    included_records = self.select_included_records(
  File "/Users/michael/.virtualenvs/sbase9/lib/python3.9/site-packages/mkdocs_exclude_search/plugin.py", line 228, in select_included_records
    elif self.is_ignored_record(rec_file_name, rec_header_name, to_ignore):
  File "/Users/michael/.virtualenvs/sbase9/lib/python3.9/site-packages/mkdocs_exclude_search/plugin.py", line 148, in is_ignored_record
    (
TypeError: 'NoneType' object is not iterable

Plugin does nothing if exclude_tags is not defined

Hi @chrieke,
In your most recent version 0.3.0 the absence of exclude_tags in the configuration file makes the plugin unuseable as it doesn't exclude anything. I think there is a mistake in the boolean expression at https://github.com/chrieke/mkdocs-exclude-search/blob/main/mkdocs_exclude_search/plugin.py#L49 as it should be an "and" expression.

Building problem: tests directory is still there

#18 (comment)

Maybe if you build by hand it would be better to use a github workflow.

Here is one of mine:

https://github.com/SFDO-Tooling/Snowfakery/blob/main/.github/workflows/release.yml

Exclude directory

We collect meeting notes in a directory and want to exclude all of them. It is a long list of files and every week a new exclude line would need to be added. Therefore it would be nice to be able to just exclude the whole directory.

Error when building

Hey, I just installed the plugins, but when I tried to build it, it returns this error

  File "..\Python\Python 3.9\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "..\Python\Python 3.9\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "..\Python\Python 3.9\Scripts\mkdocs.exe\__main__.py", line 7, in <module>
  File "..\Python\Python 3.9\lib\site-packages\click\core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "..\Python\Python 3.9\lib\site-packages\click\core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "..\Python\Python 3.9\lib\site-packages\click\core.py", line 1668, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "..\Python\Python 3.9\lib\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "..\Python\Python 3.9\lib\site-packages\click\core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "..\Python\Python 3.9\lib\site-packages\mkdocs\__main__.py", line 187, in build_command
    build.build(config.load_config(**kwargs), dirty=not clean)
  File "..\Python\Python 3.9\lib\site-packages\mkdocs\commands\build.py", line 317, in build
    config['plugins'].run_event('post_build', config=config)
  File "..\Python\Python 3.9\lib\site-packages\mkdocs\plugins.py", line 104, in run_event
    result = method(**kwargs)
  File "..\Python\Python 3.9\lib\site-packages\mkdocs_exclude_search\plugin.py", line 256, in on_post_build
    navigation_items = [
  File "..\Python\Python 3.9\lib\site-packages\mkdocs_exclude_search\plugin.py", line 257, in <listcomp>
    list(nav_chapter.values())[0].replace(".md", "/")
AttributeError: 'list' object has no attribute 'replace'

This is the config that I used for the build

  - exclude-search:
      exclude:
        - contribute.md
      exclude_tags: true
      exclude-unreferenced: true

Build error in 0.6.2 when "nav:" is not used

With version 0.6.2 I get the errors below when running mkdocs build.
Because I do not use any "navigation section" aka nav: in file mkdocs.yml, this might be my issue here.

Reverting to 0.5.4 does work.

❯ mkdocs build
INFO     -  Cleaning site directory
INFO     -  Building documentation to directory: /Users/hbokh/Projects/Spindle/infra/infra-docs/site
Traceback (most recent call last):
  File "/Users/hbokh/.virtualenvs/mkdocs/bin/mkdocs", line 8, in <module>
    sys.exit(cli())
  File "/Users/hbokh/.virtualenvs/mkdocs/lib/python3.9/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Users/hbokh/.virtualenvs/mkdocs/lib/python3.9/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Users/hbokh/.virtualenvs/mkdocs/lib/python3.9/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/hbokh/.virtualenvs/mkdocs/lib/python3.9/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/hbokh/.virtualenvs/mkdocs/lib/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/hbokh/.virtualenvs/mkdocs/lib/python3.9/site-packages/mkdocs/__main__.py", line 187, in build_command
    build.build(config.load_config(**kwargs), dirty=not clean)
  File "/Users/hbokh/.virtualenvs/mkdocs/lib/python3.9/site-packages/mkdocs/commands/build.py", line 317, in build
    config['plugins'].run_event('post_build', config=config)
  File "/Users/hbokh/.virtualenvs/mkdocs/lib/python3.9/site-packages/mkdocs/plugins.py", line 104, in run_event
    result = method(**kwargs)
  File "/Users/hbokh/.virtualenvs/mkdocs/lib/python3.9/site-packages/mkdocs_exclude_search/plugin.py", line 258, in on_post_build
    navigation_items = explode_navigation(navigation=config.data["nav"])
  File "/Users/hbokh/.virtualenvs/mkdocs/lib/python3.9/site-packages/mkdocs_exclude_search/utils.py", line 27, in explode_navigation
    for chapter in navigation:
TypeError: 'NoneType' object is not iterable

Mention that Material for MkDocs 9 has search exclusion built in

It may be worth mentioning in the README that Material for Mkdocs (free tier) now supports search exclusion out of the box

Thanks so much for creating this plugin while we waited the past couple of years for the upstream library to publish support.

search plugin not being detected

I implemented mkdocs-exclude-search locally and all seemed to work well. Unfortunately, upon deploying to our server I noticed the plugin wasn't taking effect. When looking at the build logs I see the following:

DEBUG    -  mkdocs-exclude-search plugin is activated but has no effect as search plugin is deactivated!

However, I can confirm the - search plugin is included directly above - exclude-search in our mkdocs.yml config.

Exclusion of a single file does not work anymore

Hi Christoph,
I've just tried your official 0.4.0 release and used it within our documentation repository. It looks like something got changed during refactoring/merge to master. I've tried to exclude a changelog.md which exists in various subfolders

/changelog.md
 - subdir/
   - changelog.md
 - another_dir/
   - changelog.md

I tried several exclusion rules like */changelog.md or changelog.md but none of them worked. When I looked at the code I found this

mkdocs-exclude-search/mkdocs_exclude_search/plugin.py

Line 129 in 9df45cf

fnmatch(rec_file_name, f"*{file_name.replace('.md', '')}?")

To my understanding we are cutting the file extension and try to match afterwards the filenames. e.g.
fnmatch('subdir/changelog.html', *'changelog'?)

this will be evaluated to False as the question mark at the end only matches a single character but not .html
So either we are replacing the question mark with a .html so it will always use this extension when matching or we ignore the file extension and use .*

like to hear your thoughts on it :)

edit: also the asterisk at the beginning will unintentionally match some other files like do-not-match-changelog.html. Right now this is no issue for me but I don't know if you intended this behavior.

Not excluding.

Hi, we have a pretty minimal docs/ directory and most markdowns are "rendered" dinamically on build with some plugins. Can we exclude things that are not on docs/ directory but on site/? We're trying to exclude some directories from site/ but we still see them on searchindex.json

We have a size problem with our index and we need to reduce it so excluding some directories seems like our best option.

Thanks!

Error during execution

Hey there, just tried this for the first time and got the following error:

INFO    -  exclude-search: generated/tags/ 
INFO    -  exclude-search: generated/tags/#contents-grouped-by-tag 
INFO    -  exclude-search: generated/tags/#api 
INFO    -  exclude-search: generated/tags/#misc 
INFO    -  exclude-search: generated/tags/#perks 
INFO    -  exclude-search: generated/tags/#reference 
INFO    -  exclude-search: generated/tags/#tutorial 
Traceback (most recent call last):
  File "/home/stan/.local/bin/mkdocs", line 8, in <module>
    sys.exit(cli())
  File "/usr/lib/python3/dist-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3/dist-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3/dist-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/stan/.local/lib/python3.8/site-packages/mkdocs/__main__.py", line 133, in serve_command
    serve.serve(
  File "/home/stan/.local/lib/python3.8/site-packages/mkdocs/commands/serve.py", line 141, in serve
    config = builder()
  File "/home/stan/.local/lib/python3.8/site-packages/mkdocs/commands/serve.py", line 136, in builder
    build(config, live_server=live_server, dirty=dirty)
  File "/home/stan/.local/lib/python3.8/site-packages/mkdocs/commands/build.py", line 295, in build
    config['plugins'].run_event('post_build', config=config)
  File "/home/stan/.local/lib/python3.8/site-packages/mkdocs/plugins.py", line 96, in run_event
    result = method(**kwargs)
  File "/home/stan/.local/lib/python3.8/site-packages/mkdocs_exclude_search/plugin.py", line 79, in on_post_build
    rec_main_name, rec_subchapter = rec["location"].split("/")[-2:]
ValueError: not enough values to unpack (expected 2, got 1)

my config:

  - exclude-search:
      exclude:
        - generated/tags.md
        - tags.md

Recursive include directory?

It would be nice if we could include entire directories (that contain other directories). This could be done by using fnmatch to do the pattern matching instead so something like exclude: directory/ or exclude: directory/* works as expected -- rather than just the files in a particular directory.

Should not install a package called "tests"

This package accidentally (I assume) installs a package called "tests".

E.g. in python3.10/site-packages/tests/

mkdocs-exclude-search/setup.py

Line 32 in f89c9fd

packages=find_packages(),

This will cause problems with other packages called "tests".

Thank you: otherwise it works well.

Error during build

Incredibly useful Python package by the way. Solved my exact problem.

Getting this error when running the example in the readme.md:

[I 210302 14:09:09 watcher:104] Running task: builder (delay: None)
INFO    -  Running task: builder (delay: None)
INFO    -  Building documentation... 
ERROR   -  Config value: 'plugins'. Error: Invalid Plugins configuration 
[E 210302 14:09:09 ioloop:909] Exception in callback <bound method LiveReloadHandler.poll_tasks of <class 'livereload.handlers.LiveReloadHandler'>>
    Traceback (most recent call last):
      File "/usr/local/lib/python3.7/site-packages/tornado/ioloop.py", line 907, in _run
        return self.callback()
      File "/usr/local/lib/python3.7/site-packages/livereload/handlers.py", line 69, in poll_tasks
        filepath, delay = cls.watcher.examine()
      File "/usr/local/lib/python3.7/site-packages/livereload/watcher.py", line 105, in examine
        func()
      File "/usr/local/lib/python3.7/site-packages/mkdocs/commands/serve.py", line 129, in builder
        **kwargs
      File "/usr/local/lib/python3.7/site-packages/mkdocs/config/base.py", line 210, in load_config
        "Aborted with {} Configuration Errors!".format(len(errors))
    mkdocs.exceptions.ConfigurationError: Aborted with 1 Configuration Errors!
ERROR   -  Exception in callback <bound method LiveReloadHandler.poll_tasks of <class 'livereload.handlers.LiveReloadHandler'>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/tornado/ioloop.py", line 907, in _run
    return self.callback()
  File "/usr/local/lib/python3.7/site-packages/livereload/handlers.py", line 69, in poll_tasks
    filepath, delay = cls.watcher.examine()
  File "/usr/local/lib/python3.7/site-packages/livereload/watcher.py", line 105, in examine
    func()
  File "/usr/local/lib/python3.7/site-packages/mkdocs/commands/serve.py", line 129, in builder
    **kwargs
  File "/usr/local/lib/python3.7/site-packages/mkdocs/config/base.py", line 210, in load_config
    "Aborted with {} Configuration Errors!".format(len(errors))
mkdocs.exceptions.ConfigurationError: Aborted with 1 Configuration Errors!

I'll try using a previous version. This is version 4.0.0.

Is it possible to include the glob or regex support to exclude the files from search.

We are planing to generate document website using mkdocs plugin. our projects repos contains list of md files and jupyter notebooks (ipynb) so due to that seach-index.json file size (150mb) increasing rapidly. Search is freezing so that we decided to not include the notebooks from all over repos and sub directories.

We didn't find the best options to exclude all the jupyter notebooks using file extension from search using mkdocs-exclude-search.

We have to provide the full file path to exclude the notebook that is so difficult since there are many jupyter notebooks and we are generating from multi repos.

exclude-search:
exclude:
- projects/A//.ipynb
- projects/A////.ipynb
- projects/B////.ipynb
- projects/C////.ipynb
- etc

Please suggest if you have any alternatives to resolve the problem. If we support file extensions to exclude from search it will be more helpful.

If we have similar functionality like below in mkdocs-exclude-search it will make our life easier.

exclude:
glob:
- exclude/this/path/*
- ".tmp"
- ".pdf"
- ".gz"
regex:
- '..(tmp|bin|tar)$'

Thanks for the help!!

Exclude unreferenced files from search

Would it be possible/feasible to implement an option to exclude all unreferenced files (i.e. those that can only be accessed by directly go to its URL)?

Build error after upgrade to mkdocs-exclude-search 0.6.0-0.6.3

After updating mkdocs-exclude-search from 0.5.3 to 0.6.3 I get the following build error:

mkdocs build
INFO     -  [macros] - Macros arguments: {'module_name': 'main', 'modules': [], 'include_dir': '', 'include_yaml': [], 'j2_block_start_string': '', 'j2_block_end_string': '', 'j2_variable_start_string': '', 'j2_variable_end_string': '', 'verbose': False}
INFO     -  [macros] - Extra variables (config file): ['social', 'generator', 'version']
INFO     -  [macros] - Extra filters (module): ['pretty']
INFO     -  Cleaning site directory
INFO     -  Building documentation to directory: site
Traceback (most recent call last):
  File ".tox/docs/bin/mkdocs", line 8, in <module>
    sys.exit(cli())
  File ".tox/docs/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File ".tox/docs/lib/python3.9/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File ".tox/docs/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ".tox/docs/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ".tox/docs/lib/python3.9/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File ".tox/docs/lib/python3.9/site-packages/mkdocs/__main__.py", line 187, in build_command
    build.build(config.load_config(**kwargs), dirty=not clean)
  File ".tox/docs/lib/python3.9/site-packages/mkdocs/commands/build.py", line 317, in build
    config['plugins'].run_event('post_build', config=config)
  File ".tox/docs/lib/python3.9/site-packages/mkdocs/plugins.py", line 104, in run_event
    result = method(**kwargs)
  File ".tox/docs/lib/python3.9/site-packages/mkdocs_exclude_search/plugin.py", line 278, in on_post_build
    navigation_items = explode_navigation(navigation=nav)
  File ".tox/docs/lib/python3.9/site-packages/mkdocs_exclude_search/utils.py", line 31, in explode_navigation
    chapter_paths = list(chapter.values())[0]
AttributeError: 'str' object has no attribute 'values'
ERROR: InvocationError for command .tox/docs/bin/mkdocs build (exited with code 1)

Relevant configuration from mkdocs.yml:

plugins:
  - search
  - macros
  - exclude-search:
      exclude:
        - licenses/*

Environment

Python 3.9.8

pip list
Package                    Version
-------------------------- ---------
beautifulsoup4             4.10.0
certifi                    2021.10.8
charset-normalizer         2.0.9
click                      8.0.3
distlib                    0.3.4
filelock                   3.4.2
ghp-import                 2.0.2
idna                       3.3
importlib-metadata         4.10.0
Jinja2                     3.0.3
lxml                       4.7.1
Markdown                   3.3.6
MarkupSafe                 2.0.1
mergedeep                  1.3.4
mkdocs                     1.2.3
mkdocs-exclude-search      0.6.3
mkdocs-htmlproofer-plugin  0.8.0
mkdocs-macros-plugin       0.6.3
mkdocs-material            8.1.4
mkdocs-material-extensions 1.0.3
packaging                  21.3
pep517                     0.12.0
pip                        21.3.1
pip-tools                  6.4.0
platformdirs               2.4.1
pluggy                     1.0.0
py                         1.11.0
Pygments                   2.11.1
pymdown-extensions         9.1
pyparsing                  3.0.6
python-dateutil            2.8.2
PyYAML                     6.0
pyyaml_env_tag             0.1
requests                   2.26.0
setuptools                 58.5.3
six                        1.16.0
soupsieve                  2.3.1
termcolor                  1.1.0
toml                       0.10.2
tomli                      1.2.2
tox                        3.24.5
urllib3                    1.26.7
virtualenv                 20.13.0
watchdog                   2.1.6
wheel                      0.37.1
zipp                       3.7.0