pawelsikora / mkdocs-with-confluence Goto Github PK
View Code? Open in Web Editor NEWMkDocs plugin for uploading markdown documentation to Confluence via Confluence REST API
License: MIT License
MkDocs plugin for uploading markdown documentation to Confluence via Confluence REST API
License: MIT License
Hello,
my upload to confluence API fails with error for a certain page in my set of markdown files, so I suspect there is a certain md content that is not properly converted. Is there a way to increase verbosity of the mkdocs run in order to get more details?
Here is what I have so far:
...
[2023-06-01T10:48:14.399Z] ERROR - Error reading page 'outbound_pipeline.md': 400 Client Error: for url: https://XXXXXXXXX.net/confluence/rest/api/content/
...
[2023-06-01T10:48:14.400Z] INFO - * Mkdocs With Confluence: Using the Pipeline - *UPDATE*
[2023-06-01T10:48:14.400Z] INFO - * Mkdocs With Confluence: Using the Pipeline for DDP - *NEW PAGE*
[2023-06-01T10:48:14.400Z] Trying to ADD page 'Using the Pipeline for DDP' to parent0(FTP Links Automaton) ID: 2999877739
[2023-06-01T10:48:14.400Z] INFO - * Mkdocs With Confluence: The MFT Pipeline - *NEW PAGE*
[2023-06-01T10:48:14.400Z] Traceback (most recent call last):
[2023-06-01T10:48:14.400Z] File "/usr/local/bin/mkdocs", line 8, in <module>
[2023-06-01T10:48:14.400Z] sys.exit(cli())
[2023-06-01T10:48:14.400Z] ^^^^^
[2023-06-01T10:48:14.400Z] File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
[2023-06-01T10:48:14.400Z] return self.main(*args, **kwargs)
[2023-06-01T10:48:14.400Z] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[2023-06-01T10:48:14.400Z] File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1055, in main
[2023-06-01T10:48:14.400Z] rv = self.invoke(ctx)
[2023-06-01T10:48:14.400Z] ^^^^^^^^^^^^^^^^
[2023-06-01T10:48:14.400Z] File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
[2023-06-01T10:48:14.400Z] return _process_result(sub_ctx.command.invoke(sub_ctx))
[2023-06-01T10:48:14.400Z] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2023-06-01T10:48:14.400Z] File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
[2023-06-01T10:48:14.400Z] return ctx.invoke(self.callback, **ctx.params)
[2023-06-01T10:48:14.400Z] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2023-06-01T10:48:14.400Z] File "/usr/local/lib/python3.11/site-packages/click/core.py", line 760, in invoke
[2023-06-01T10:48:14.400Z] return __callback(*args, **kwargs)
[2023-06-01T10:48:14.400Z] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2023-06-01T10:48:14.400Z] File "/usr/local/lib/python3.11/site-packages/mkdocs/__main__.py", line 250, in build_command
[2023-06-01T10:48:14.400Z] build.build(cfg, dirty=not clean)
[2023-06-01T10:48:14.400Z] File "/usr/local/lib/python3.11/site-packages/mkdocs/commands/build.py", line 308, in build
[2023-06-01T10:48:14.400Z] _populate_page(file.page, config, files, dirty)
[2023-06-01T10:48:14.400Z] File "/usr/local/lib/python3.11/site-packages/mkdocs/commands/build.py", line 177, in _populate_page
[2023-06-01T10:48:14.400Z] page.markdown = config.plugins.run_event(
[2023-06-01T10:48:14.400Z] ^^^^^^^^^^^^^^^^^^^^^^^^^
[2023-06-01T10:48:14.400Z] File "/usr/local/lib/python3.11/site-packages/mkdocs/plugins.py", line 520, in run_event
[2023-06-01T10:48:14.400Z] result = method(item, **kwargs)
[2023-06-01T10:48:14.400Z] ^^^^^^^^^^^^^^^^^^^^^^
[2023-06-01T10:48:14.400Z] File "/usr/local/lib/python3.11/site-packages/mkdocs_with_confluence/plugin.py", line 334, in on_page_markdown
[2023-06-01T10:48:14.400Z] self.add_page(page.title, parent_id, confluence_body)
[2023-06-01T10:48:14.400Z] File "/usr/local/lib/python3.11/site-packages/mkdocs_with_confluence/plugin.py", line 541, in add_page
[2023-06-01T10:48:14.400Z] r.raise_for_status()
[2023-06-01T10:48:14.400Z] File "/usr/local/lib/python3.11/site-packages/requests/models.py", line 1021, in raise_for_status
[2023-06-01T10:48:14.400Z] raise HTTPError(http_error_msg, response=self)
[2023-06-01T10:48:14.400Z] requests.exceptions.HTTPError: 400 Client Error: for url: https://XXXXXXXX.net/confluence/rest/api/content/
I think the issue might be related to the content inside the md page I am submitting, but how can I know if I cannot see the error message coming with the HTTP 400 from the Confluence space.
Hence the first request would be: Is there a way to increase verbosity of the logs?
Second request: Is there a document that described the 'sensitive' characaters that should not appear on the .md
page when attempting to convert to HTML with mkdocs?
Thanks for your consideration!
Warm regards
Carsten
First, thanks a lot for publishing this plugin, it does work with confluence quickly and with no trouble at all.
My challenge is how to get it to interact with other mkdocs plugins for some more advanced rendering. When generating the confluence pages, it seems that the underlying renderer (md2cf/mistune) does not take them into account.
I am using this in the mkdocs.yml
file:
[...]
# See https://squidfunk.github.io/mkdocs-material/reference/code-blocks/
theme:
name: material
features:
- content.code.annotate
markdown_extensions:
- tables
- def_list
- admonition
- toc
- markdown_inline_graphviz
- plantuml_markdown
- mdx_truly_sane_lists
# See https://squidfunk.github.io/mkdocs-material/reference/code-blocks/
- pymdownx.highlight:
anchor_linenums: true
line_spans: __span
pygments_lang_class: true
- pymdownx.inlinehilite
- pymdownx.superfences
- pymdownx.smartsymbols
- pymdownx.arithmatex:
generic: true
- pymdownx.snippets:
check_paths: true
plugins:
- monorepo
- search
# https://github.com/pawelsikora/mkdocs-with-confluence
- mkdocs-with-confluence:
host_url: https://flexport.atlassian.net/wiki/rest/api/content
space: !ENV CONFLUENCE_SPACE
parent_page_name: !ENV CONFLUENCE_PARENT_PAGE
username: !ENV CONFLUENCE_USER
password: !ENV CONFLUENCE_TOKEN
enabled_if_env: MKDOCS_TO_CONFLUENCE
verbose: false
debug: false
dryrun: false
And the rendered pages don't render correctly at least:
Which are all rendered O.K. when generating an html site.
Any ideas would be very appreciated.
First off, thank you for writing this plugin. This is going to help me greatly.
Unfortunately when trying to get this plugin running I am encountering the following error:
ERROR - Error reading page 'index.md': 'MkdocsWithConfluence' object has no attribute 'dryrun'
Traceback (most recent call last):
File "/Users/theD1360/Library/Caches/pypoetry/virtualenvs/docs-project-A5zrhKJ8-py3.9/bin/mkdocs", line 8, in <module>
sys.exit(cli())
File "/Users/theD1360/Library/Caches/pypoetry/virtualenvs/docs-project-A5zrhKJ8-py3.9/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/Users/theD1360/Library/Caches/pypoetry/virtualenvs/docs-project-A5zrhKJ8-py3.9/lib/python3.9/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/Users/theD1360/Library/Caches/pypoetry/virtualenvs/docs-project-A5zrhKJ8-py3.9/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/theD1360/Library/Caches/pypoetry/virtualenvs/docs-project-A5zrhKJ8-py3.9/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/theD1360/Library/Caches/pypoetry/virtualenvs/docs-project-A5zrhKJ8-py3.9/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/Users/theD1360/Library/Caches/pypoetry/virtualenvs/docs-project-A5zrhKJ8-py3.9/lib/python3.9/site-packages/mkdocs/__main__.py", line 250, in build_command
build.build(cfg, dirty=not clean)
File "/Users/theD1360/Library/Caches/pypoetry/virtualenvs/docs-project-A5zrhKJ8-py3.9/lib/python3.9/site-packages/mkdocs/commands/build.py", line 308, in build
_populate_page(file.page, config, files, dirty)
File "/Users/theD1360/Library/Caches/pypoetry/virtualenvs/docs-project-A5zrhKJ8-py3.9/lib/python3.9/site-packages/mkdocs/commands/build.py", line 177, in _populate_page
page.markdown = config.plugins.run_event(
File "/Users/theD1360/Library/Caches/pypoetry/virtualenvs/docs-project-A5zrhKJ8-py3.9/lib/python3.9/site-packages/mkdocs/plugins.py", line 520, in run_event
result = method(item, **kwargs)
File "/Users/theD1360/Library/Caches/pypoetry/virtualenvs/docs-project-A5zrhKJ8-py3.9/lib/python3.9/site-packages/mkdocs_with_confluence/plugin.py", line 274, in on_page_markdown
self.update_page(page.title, confluence_body)
File "/Users/theD1360/Library/Caches/pypoetry/virtualenvs/docs-project-A5zrhKJ8-py3.9/lib/python3.9/site-packages/mkdocs_with_confluence/plugin.py", line 571, in update_page
if not self.dryrun:
AttributeError: 'MkdocsWithConfluence' object has no attribute 'dryrun'
Software | Version |
---|---|
python | 3.9 |
poetry | 1.4.0 |
mkdocs | ^1.4.2 |
mkdocs-material | ^9.1.5 |
mkdocs-with-confluence | ^0.2.7 |
plugins:
- mkdocs-with-confluence:
host_url: https://<confluence domain>/wiki/rest/api/content
space: ""
parent_page_name: Qualification Engine (Qual-E)
username: [email protected]
password: mypassword
dryrun: true
verbose: true
debug: true
One interesting thing to note is that the error seems to be limited to just that config key.
All other config items seem to be loaded properly. It seems the self.dryrun
property is not getting set properly in
plugin.py
.
if self.config["dryrun"]:
print("WARNING - Mkdocs With Confluence - DRYRUN MODE turned ON")
self.dryrun = True
else:
self.dryrun = False
I have tried to publish the minimal example provided by mkdocs new .
to confluence using mkdocs-with-confluence
without success:
$ WITH_CONFLUENCE=1 mkdocs build
INFO - Mkdocs With Confluence: Exporting MKDOCS pages to Confluence turned ON by var {env_name}==1!
INFO - Cleaning site directory
INFO - Building documentation to directory: /Users/mkj/Developer/playgrounds/mkdocs/mkdocs-confluence-examples/site
Number of Files: 1
Traceback (most recent call last):
File "/Users/mkj/.pyenv/versions/mkdocs-with-confluence/bin/mkdocs", line 8, in <module>
sys.exit(cli())
File "/Users/mkj/.pyenv/versions/3.9.7/envs/mkdocs-with-confluence/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/Users/mkj/.pyenv/versions/3.9.7/envs/mkdocs-with-confluence/lib/python3.9/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/Users/mkj/.pyenv/versions/3.9.7/envs/mkdocs-with-confluence/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/mkj/.pyenv/versions/3.9.7/envs/mkdocs-with-confluence/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/mkj/.pyenv/versions/3.9.7/envs/mkdocs-with-confluence/lib/python3.9/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/Users/mkj/.pyenv/versions/3.9.7/envs/mkdocs-with-confluence/lib/python3.9/site-packages/mkdocs/__main__.py", line 187, in build_command
build.build(config.load_config(**kwargs), dirty=not clean)
File "/Users/mkj/.pyenv/versions/3.9.7/envs/mkdocs-with-confluence/lib/python3.9/site-packages/mkdocs/commands/build.py", line 287, in build
nav = config['plugins'].run_event('nav', nav, config=config, files=files)
File "/Users/mkj/.pyenv/versions/3.9.7/envs/mkdocs-with-confluence/lib/python3.9/site-packages/mkdocs/plugins.py", line 102, in run_event
result = method(item, **kwargs)
File "/Users/mkj/.pyenv/versions/3.9.7/envs/mkdocs-with-confluence/lib/python3.9/site-packages/mkdocs_with_confluence/plugin.py", line 45, in on_nav
p = spaces + self.__get_page_title(n)
File "/Users/mkj/.pyenv/versions/3.9.7/envs/mkdocs-with-confluence/lib/python3.9/site-packages/mkdocs_with_confluence/plugin.py", line 274, in __get_page_title
return re.search("\\s*Page\\(title='(.*)',", section).group(1)
AttributeError: 'NoneType' object has no attribute 'group'
mkdocs.yml
:
site_name: Publish from MkDocs to Confluence
plugins:
- search
- mkdocs-with-confluence:
host_url: !ENV [HOST, "n/a"]
space: !ENV [SPACE, "foo"]
parent_page_name: !ENV [PARENT, "foo"]
username: !ENV [USER, "foo"]
password: !ENV [PW, "foo"]
enabled_if_env: WITH_CONFLUENCE
Environment variables have been tested and work properly (using confluence api directly).
This happend with the following setup:
$ pip list
Package Version
---------------------- ---------
certifi 2021.10.8
charset-normalizer 2.0.7
click 8.0.3
colorama 0.4.4
formats 0.1.1
ghp-import 2.0.2
httpretty 1.1.4
idna 3.3
importlib-metadata 4.8.1
Jinja2 3.0.2
Markdown 3.3.4
MarkupSafe 2.0.1
md2cf 1.0.3
mergedeep 1.3.4
mistune 0.8.4
mkdocs 1.2.3
mkdocs-with-confluence 0.2.2
packaging 21.0
pip 21.3.1
pyparsing 3.0.1
python-dateutil 2.8.2
PyYAML 5.3.1
pyyaml_env_tag 0.1
requests 2.26.0
setuptools 57.4.0
six 1.16.0
tortilla 0.5.0
urllib3 1.26.7
watchdog 2.1.6
zipp 3.6.0
Hi, I have generated a documentation with the structure presented in the image below:
When I try to uplopad the documentation to Confluence, only Home
, Architecture
, and Installation
are getting published. Nothing under the Stacks
subpage gets uploaded to Confluence as seen in the image below:
This is my mkdocs.yml configuration file:
site_name: Monitoring
theme:
name: windmill
plugins:
- search
- mkdocs-with-confluence:
host_url: https://confluencebt.host/rest/api/content
space: CONFLUENCE_SPACE
parent_page_name: Monitorizare
username: confluence_user
password: confluence_password
enabled_if_env: MKDOCS_TO_CONFLUENCE
verbose: true
debug: true
dryrun: false
nav:
- Home: "index.md"
# - Introduction: "index.md"
- Architecture: "architecture.md"
- Installation: "installation.md"
- Stacks:
- Metrics:
- Architecture: "stacks/metrics/architecture.md"
- Installation and Operation: "playbooks/metrics/README.md"
- Logs:
- Architecture: "stacks/logs/architecture.md"
- Installation and Operation: "playbooks/logs/README.md"
- Traces:
- Architecture: "stacks/traces/architecture.md"
- Installation and Operation: "playbooks/traces/README.md"
INFO - Mkdocs With Confluence: Exporting MKDOCS pages to Confluence turned ON by var MKDOCS_TO_CONFLUENCE==1!
Number of Files in directory tree: 10
DEBUG - SECTION title: Section(title='Stacks')
DEBUG - SECTION title: Section(title='Metrics')
DEBUG - SECTION title: Section(title='Logs')
DEBUG - SECTION title: Section(title='Traces')
DEBUG - Handling Page 'Home' (And Parent Nav Pages if necessary):
DEBUG - Get section first parent title...:
DEBUG - WRN(list index out of range): No first parent! Assuming DEBUG - Monitorizare...
DEBUG - None
DEBUG - Get section second parent title...:
DEBUG - ERR(list index out of range) No second parent! Assuming second parent is main parent: Monitorizare...
Monitorizare
DEBUG - ONLY ONE PARENT FOUND. ASSUMING AS A FIRST NODE after main parent config Monitorizare
DEBUG - PARENT0: Monitorizare, PARENT1: Monitorizare, MAIN PARENT: Monitorizare
#################################################
INFO - * Mkdocs With Confluence: Find Page ID: PAGE NAME: Monitorizare
URL: https://confluencebt.host/rest/api/content?title=Monitorizare&spaceKey=CONFLUENCE_SPACE&expand=history
ID: 129549192
DEBUG - JUST ONE STEP FROM UPDATE OF PAGE 'Monitorizare'
DEBUG - CHECKING IF PARENT PAGE ON CONFLUENCE IS THE SAME AS HERE
INFO - * Mkdocs With Confluence: Find PARENT OF PAGE, PAGE NAME: Monitorizare
INFO - * Mkdocs With Confluence: Find Page ID: PAGE NAME: Monitorizare
URL: https://confluencebt.host/rest/api/content?title=Monitorizare&spaceKey=CONFLUENCE_SPACE&expand=history
ID: 129549192
PARENT NAME: Documentation
DEBUG - ERR, Parents does not match: 'Monitorizare' =/= 'Documentation' Aborting...
DEBUG - Handling Page 'Installation and Operation' (And Parent Nav Pages if necessary):
DEBUG - Get section first parent title...:
DEBUG - SECTION title: Section(title='Logs')
DEBUG - Logs
DEBUG - Get section second parent title...:
DEBUG - SECTION title: Section(title='Stacks')
Logs
DEBUG - PARENT0: Logs, PARENT1: Stacks, MAIN PARENT: Monitorizare
How can I change the configuration so the plugin will create the required subpages?
This seems like a dream come true, but there are problems here with usability and password security. I may not store my password in clear-text in the mkdocs.yml
file. Since a piece of code is not my personal possession but something I wish to share with a team, I would also not want to place my username in the mkdocs.yml
file.
Hello, what about using md2cf APIs also for uploading to confluence? It's an alive project, it would eliminate lot of duplicated efforts and expose some neat feature like the on_changed
machinery which only upload changed stuff.
I think that would be a great addition to your super cool plugin! If you think we could discuss that I could definitely work on that, so what do you think?
Thank you, regards
setup.py should declare the packages you depend on so that pip will install these.
Dear @pawelsikora
First : thanks for your plugin.
I'm trying to use it with Gitlab CI/CD.
I've got some questions (some basic usage).
But, before, let me present my configuration
When running, I've got the following pipeline error message
ModuleNotFoundError: No module named 'requests'
I've put my configuration for the plugin into the mkdocs.yml
plugins:
- search:
lang:
- fr
- en
- macros
- mkdocs-confluence: #For conversion of mkdocs into confluence
host_url: https://my_confluence.fr/rest/api/content?type=page&spaceKey=bac&expand=ancestros&limit=100
space: bac
parent_page_name: Test_Florent
username: myuser_name
password: "MYPASSWORD!*ù$^4_with_special_character"
verbose: true
dryrun: true
AND LOT : thanks for your support.
It should be possible to manage whether the index markdown file becomes an about page or populates the content of the parent page.
It seems desirable at least that some part of the index.md file become part of the parent page, but I understand why the default should be to create an About page. I think what I'm asking for should be considered an enhancement - some people would prefer the parent page to have TOC via the Children display macro.
Hi,
as far as I can see, this plugin only supports username/password authentication.
For instance for ci-jobs I don't want to expose my password. Instead I want to use a Personal Access Token.
Is it possible to support Access Tokens too?
Installed plugin and plugin block in mkdocs.yml . I keep getting this error:
INFO - Your documentation should shortly be available at: https://symplr-software.github.io/innovation-documentation-pipeline/
WARNING - Mkdocs With Confluence: Exporting MKDOCS pages to Confluence turned OFF: (set environment variable MKDOCS_TO_CONFLUENCE to 1 to enable)
Number of Files in directory tree: 12
Please advise
Hi,
Thank you for your package, it looks awesome !
We use a private confluence website using SSO (with 2FA) to access it. It seems quite difficult to upload directly using the provided username / password arguments so we would like to generate the confluence pages and upload them manually. We didn't find in the doc any way to save the pages before upload. Is that something currently available ?
Thank you for your help
Dear,
I've got an SSL error message when I try to use your excellent code from a gitlab pipeline.
It seems that I need to pass cacert as parameter.
Does this feature exist ?
Without your library and directly with CURL
I can reach my confluence. So I think that my issue root cause is the absence of SSL/CACERT in your example.
For instance :
curl --cacert curl-ca-bundle.crt --url "https://myconfluence.com/rest/api/content/251548690" -H "Authorization: Bearer fjhs_my_tokenskjd"
This curl request send my well informations. Whereas, if I suppress the --cacert
option, I've got the same SSL error message.
requests.exceptions.SSLError: HTTPSConnectionPool(host='myconfluence.com', port=443): Max retries exceeded with url: /rest/api/content?title=Home&spaceKey=TDF&expand=history (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1131)')))
A lot of thanks for your support.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.