I tried to install org-transclusion via Doom Emacs' packages.el, like i've done for a couple of other packages that can't be installed via Melpa.

Unfortunately, when I run bin/doom sync, I get the following error:

  > Installing packages...
    > Updating recipe repos...
    → Cloning org-transclusion...smirror-mirror...
    x There was an unexpected package error
      Cloning into '/Users/quirin/.emacs.d/.local/straight/repos/org-transclusion'...
      fatal: Remote branch master not found in upstream origin

This is what I'm using in packages.el:

(package! org-transclusion
  :recipe (:host github
           :repo "nobiot/org-transclusion"
           :files ("org-transclusion.el"))
  )

Am I doing something wrong here?

When a paragraph is transcluded via a dedicated target, the region seems to include the next paragraph; too much. Need to review the logic.

Thank you for sharing your work! I think the secondary-selection face, currently used on line 142, does not always work well for areas that can/should be edited.

An alternative would be to define a face and use that instead. The benefit is that it makes it possible for other themes (or users in their configs) to change styles. This diff shows an example. It defines only a background colour (one for light, another for dark), which means that it should not change the text colours used in the block.

 org-transclusion.el | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/org-transclusion.el b/org-transclusion.el
index 576ee9c..970ff05 100644
--- a/org-transclusion.el
+++ b/org-transclusion.el
@@ -34,6 +34,15 @@
 (require 'org-element)
 (require 'org-id)
 
+;;-----------------------------------------------------------------------------
+;; Faces
+(defface org-transclusion-block
+  '((((class color) (min-colors 88) (background light))
+     :background "#f0f0f0" :extend t)
+    (((class color) (min-colors 88) (background dark))
+     :background "#181a20" :extend t))
+  "Face for transcluded block.")
+
 ;;-----------------------------------------------------------------------------
 ;; Variables
 ;; Most of these should be defcustom
@@ -139,7 +148,7 @@ Assume the RAW-LINK is a valid tranclusion link."
       ;; ensures that the location of overlay is not shifted for the remove
       ;; function.when a new line is added back to allow space for the original
       ;; tranclusion link for the remove function.
-      (overlay-put ov 'face 'secondary-selection)
+      (overlay-put ov 'face 'org-transclusion-block)
       (overlay-put ov 'path path)
       (overlay-put ov 'tc-src-buf buf)
       (overlay-put ov 'tc-src-marker marker)

Feel free to adapt this to your particular needs. It is just an idea.

I want to transclude a heading the same way I refile headings: (de)indenting the transcluded heading also would be great.
This has been mentioned in the forum as the problem of headings levels
A very good use case of transclusion would be refiling a heading while keeping its contents on the original file via transclusion.
To make this, the destination indendation level and the source relative indentation levels (with respect to the transcluded heading) would have to be added

As an example:

Destination.org

#+TITLE: Destination
* Hi all
** [[otc:id:0fa7f97e-eb9c-4dad-b67f-0bb6c766d4b7][I want to transclude in this destination file]]

Source.org

#+TITLE: Source

* I want to transclude
:PROPERTIES:
:ID:       0fa7f97e-eb9c-4dad-b67f-0bb6c766d4b7
:END:
I will be editing this text soon
** Note what happens if I make a subheading
The problem is the indentation level
** This note is level2 in Source but it should be level3 in Destination
The link is indented 2 lines in Source file + 1 line in Destination file

What Destination.org currently looks like after transclusion

#+TITLE: Destination
* Hi all
* I want to transclude
:PROPERTIES:
:ID:       0fa7f97e-eb9c-4dad-b67f-0bb6c766d4b7
:END:
I will be editing this text soon
** Note what happens if I make a subheading
The problem is the indentation level
** This note is level2 in Source but it should be level3 in Destination
The link is indented 2 lines in Source file + 1 line in Destination file

What I would like Destination.org to look like after transclusion:

#+TITLE: Destination
* Hi all
** I want to transclude
:PROPERTIES:
:ID:       0fa7f97e-eb9c-4dad-b67f-0bb6c766d4b7
:END:
I will be editing this text soon
*** Note what happens if I make a subheading
The problem is the indentation level
*** This note is level2 in Source but it should be level3 in Destination
The link is indented 2 lines in Source file + 1 line in Destination file

I am having issues with uuid transclusion. In the following, the first one is expanded fine, but not the second one. I can click and open the link in emacs though. ??

[[file:steps_with_ffmpeg.org][Setting up ffMPEG and Mediainfo]]
# [[id:560ca036-101c-448d-99ff-8a3026e23cfc][windows environment variable setting]]

I am having trouble transcluding with ID links in org-roam files. org-transclude works fine when using an id-link to a header in one of my "regular" org files that belong to org-agenda-files.

I tried two types of ID links in org-roam files, and neither transclude:

1. File-level ID. I have a PROPERTIES drawer at the top of an org-roam file with an ID.
2. Header-level ID. This is the usual PROPERTIES drawer within a headline.

All IDs are generated using org-id.
I did update org-id-locations using (org-id-update-id-locations '("path/to/my-org-roam/*.org")).

Any suggestions?

EDIT:

• even with org-agenda-files, the performance is fickle. The transclusion renders sometimes, and sometimes it doesn't.

Hi,

I've got pretty large Org files and as far as I understood, things like overlays do cause severe performance degradation in large Org files.

It would be nice to read about performance implications in the readme so that people using large Org files know that this is a show-stopper for this nice Org extension or not.

Thanks!

In light of issue 1, notice an incomplete color value introduced by commit f876e4f:

@@ -2,6 +2,6 @@
   '((((class color) (min-colors 88) (background light))
      :background "#efefef" :extend t)
     (((class color) (min-colors 88) (background dark))
-     :foreground "#bfc0c" :background "#1e1e1e" :extend t))
+     :foreground "#bfc0c4" :background "#1e1e1e" :extend t))
   "Face for transcluded block."
   :group 'org-transclusion)

Completed it for you.

And thanks for your contributions!

Thanks for writing this critical implementation of the Zettelkasten
workflow within the scope of org-roam. I believe this package would
lead to achieving the core idea of Zettelkasten.

I am reporting an issue that is somehow not expected to me.

If a user is in a working note. The user would like to link a note to
this working note. The expected procedure would be:

1. org-roam-insert
2. find and insert the corresponding note
3. activate org-transclusion-mode

After this, the user would be able to see all the notes in the
original notes transcluded in the current working notes. I've attached
two images to show. The contents of the notes are attached at the end.

I observed an issue after step-3, i.e., activating org-transclusion-mode.

The underscore link in Figure 1. (testing example using file note), is
only partially included after org-transclusion-mode is activated.

The user would expect to see all the content in this note (testing
example using file note), which is a three paragraphs note. The
current implementation only shows the last paragraph. This can be seen
in Figure 2.. "This is the 3rd paragraph. I like emacs." is the only
content that is included.

The original notes are a three paragraphs note. The content is shown
like this:

This is the 1st paragraph. I like org-roam.

This is the 2nd paragraph. I like org-transclusion.

This is the 3rd paragraph. I like emacs.

I was wondering if this is the expected behavior?

Figure 1:

Figure 2:

testing_example_using_file.org:

#+TITLE: testing example using file
#+CREATED:       [2020-11-06 Fri 14:32]
#+LAST_MODIFIED: [2020-11-06 Fri 15:30]

This is the 1st paragraph. I like org-roam.

This is the 2nd paragraph. I like org-transclusion.

This is the 3rd paragraph. I like emacs.

testing_example_a.org::*headline 2:

#+TITLE: testing example A
#+CREATED:       [2020-11-06 Fri 14:30]
#+LAST_MODIFIED: [2020-11-06 Fri 15:31]

* headline 1
  :PROPERTIES:
  :ID:       UUID-A
  :END:
* headline 2
  :PROPERTIES:
  :ID:       UUID-B
  :END:

I wrote something in headline 2, and this is the 1st paragraph.

I wrote something in secondary paragraph.

I also wrote something in third paragraph.
* headline 3

I want to transclude all the content under a headline, excluding the headline itself though. what is the right way of doing that?

Glad to know there is more people interested on transclusion!
alphapapa is mantaining a list of transclusion resources if you didn't know about it (in fact I discovered this repo via alphapapa). Maybe you can get some ideas from here.

I've been using org transclusion for a couple of months and it's pretty good!

The most recurring issue I've had is that using doom emacs and evil mode, undo and redo do something strange and substitute without notice the transcluded element with its contents, effectively copying it

If I have a file main.org with transclusion and I undo and redo several times, I got from this:

#+transclude: t
[[file:whatever.org]]
#+transclude: t
[[file:ohno.org]]

to this:

* whatever
** a test headline
** these are the contents of the file whatever.org
*** these shouldn't be here
* ohno
** this is not meant to be here
** these are the contents of the file ohno.org

A simple workaround is to add the following lines to the config to disable transclusion when undoing/redoing:

(advice-add 'evil-undo :before (lambda (count) (if org-transclusion-mode (org-transclusion-deactivate))))
(advice-add 'evil-redo :before (lambda (count) (if org-transclusion-mode (org-transclusion-deactivate))))

so what is a workaround for a dark theme? I am using zenburn.

Hi org-transclusion developers,

thanks very much for an extremely useful mode. As it turns out, it addresses almost precisely a need I have for my own research work [1]. The almost part is that I need it to target regular source code files rather than org-mode files. What I need to do is to link an org-mode file to a region of a (say) C++ source code file. The link needs to be done, ideally, via some identifier which exists in the source code file (say a UUID for instance), and some arbitrary "begin" and "end" markers. For example:

// {{{ 507ab9d0-d1a5-4fa9-97a9-d78a48a08c3d
<some content>
// }}}

I'd like to use org-transclusion to "point to" this region and display it on the org buffer, as well as provide a link to the buffer (as org-transclusion already does, if I understood it right). The link would then be something like this:

#+transclude: t
[[file:my_file.cpp#507ab9d0-d1a5-4fa9-97a9-d78a48a08c3d][source code]]

For extra bonus points, if it could use the org-babel language source font-locking it would be even more ideal :-) but that is not all that important. Anyway, is this possible? I have been reading through the documentation but I haven't found any thing specific to non-org mode "targets", apologies if I missed it.

Many thanks for your time.

Marco

[1] https://github.com/MASD-Project/dogen

Having 2 consecutive transcluded elements by id transcludes only the first one:

#+transclude: t :hlevel 4
[[id:ae860430-49af-46b5-9007-686ec4ddbeee][Definitions and distinctions]]
#+transclude: t :hlevel 4
[[id:0bb28e25-b8af-4c43-afb0-c96400949742][Flows]]

A workaround is to insert three equal signs or any other sequence of characters (a single whitespace works ok for example, but it has to be some text in between):

#+transclude: t :hlevel 4
[[id:ae860430-49af-46b5-9007-686ec4ddbeee][Definitions and distinctions]]
===
#+transclude: t :hlevel 4
[[id:0bb28e25-b8af-4c43-afb0-c96400949742][Flows]]

I am getting the following error when I press to activate. any thoughts? Also does this work with the whole org file too, or just the headlines?

if: Symbol’s value as variable is void: window-selection-change-functions

I am using Emacs 26.3 and Arch Linux newest version.

If I have:

* Foo
#+transclude: t :hlevel 2
[[file:baz][baz]]
* Bar

then when transclusion is on I can't make any changes when point is on the first * of the Bar heading

I have a transclusion with :hlevel 3, but when org-transclusion-include-first-section is set to t everything shows up at the level in the original file.

I was browsing through the code and noticed that, when a transclusion link is activated, the link is removed from the buffer before the transclusion overlay is added.

Is that necessary? Could the overlay be simply overlaid on top of the link instead? If that worked, it would be simpler and safer.

I can transclude enter files, but nothing smaller than that. For example:

Source.org

#+TITLE: Source

* SourceHeading1

Heading 1 text

* SourceHeading2

Heading 2 text

Destination.org

#+TITLE: Destination

#+transclude: t 
[[file::Source.org]]

#+transclude: t 
[[file:Source.org::*SourceHeading2]]

The first transclusion works, but the second doesn't do anything. But I can click on the *SourceHeading2 link and go to the relevant headline, so the org link itself seems to work.

From a comment to Demo 3:

Thank you. Yes, just tried; changing the headline name breaks the link. It relies on Org Mode link, so if it breaks, transclusion breaks, too.

Nevertheless... As the title is also cloned and sync'ed, you can change the name of headline in transclusion and origin.... I might do something to keep the link even when the headline name is edited within transclusion.

Also... I guess it's a good case for adding Org-ID to the mix, if you know that the headline name is likely to change. otc:id: is supported, but not the normal Org-ID id: yet.

Launching doom without any user configuration, org-transclusion is jumping to the transcluded buffer, instead of transcluding the headline:

The lower headings are missing:

* This is my headline
* This is another headline

Any idea why is this happening? Does doom emacs do not support indirect buffers?

I'm using Doom Emacs with it's default dark theme and the highlighting of transcluded blocks is way too strong to be pleasant.

Since people use all kinds of themes, would it be possible to make the highlighting generally less obstructive?

PS: I'm posting a couple of things here at the same time, hope you know this is meant to be constructive feedback rather than criticism, I'm loving your package already!

Don't know if it's by-design or not, but when I move a paragraph level transclusion link under a heading and activate 'transclusion-mode' it gets deleted. Before and after images. In the 2nd screenshot you can see the link has diapered, and it will not show even if I deactivate 'transclusion-mode' (sorry, in Hebrew, if not clear I can use another example).

I'm trying to create a "dashboard" org-roam note that includes things like my active projects, my daily agenda, etc. Would it be possible, whether within org-transclusion itself or reusing some of the core mechanisms, to include an org agenda buffer in the transcluded view?

Review this implementation of "cloning" a region of text into a different buffer, and sync them.

https://emacs.stackexchange.com/questions/56201/is-there-an-emacs-package-which-can-mirror-a-region/56202#56202

Via alphapapa's repo that collects attemps at enabling transclusions in Emacs

Following #21, I also observed another unexpected behavior using a
link with id:UUID link.

I've included the illustration in Figure 1. in #21. Please notice that
the second underscore note, i.e. (testing example using headline), is
id:UUID type of link.

This three paragraph note have been successfully transcluded and shown
in Figure 2. This is what the user expected.

The issue is that there is some extra information, (i.e.
:PROPERTIES:,:ID:,:END:), that the user would not expected to be
included in the new working note.

I hope org-transclusion can support source code file syntax highlight,
And support file lines range like GitHub, [[file:code.py::123-130]] to only display 123 - 130 lines. or limit lines after #+transclude: t :line-range 123-130 ?

My use case is as follows, I would like to transclude links that I have built up in the past. I have links that I have placed either as headline text or plain lists. I would like to still transclude these links explicitly through transclusion at point. The current implementation requires #+transclude and doesn't work for links that aren't the first text in a line.

* [[link]]
:properties:
:end:

- [[link]]

I would like to explicitly transclude them as follows

* [[link]]
:properties:
:end:
stuff from [[link]]

- [[link]]
stuff from [[link]]

the link can also be in the middle of text and transclusion would still work. There is no ambiguity because it is explicit transclusion. If need to explicitly transclude the whole buffer, can either innumerate all links and transclude them adding appropriate space or translucde only the first link on a line. My use case doesn't fall here so I don't have great many things to comment.

* this is a [[link]] is it not
:properties:
:end:
stuff from [[link]]

- this is a [[link]] is it not
stuff from [[link]]

Follow on from #5 as a backup solution.

Consider further https://www.emacswiki.org/emacs/backup-each-save.el

Noted in Demo 3.

I tried to use (add-hook 'org-mode-hook #'org-transclusion-mode) to enable org-transclusion-mode on all Org buffers.
But it caused obviously delay on saving buffers.
Is it possible to optimize it?
Enable this minor mode on org-mode will don't need to manual toggle mode frequently.

What would be the main differences of this package vs the other two?

Back up

Conclusion: this solution is not assuring if multiple big changes are expected in a session.

Tried adding version-control variable (below) to create multiple backup versions.
This only keeps multiple backup files (filename.org ~1~, filename.org ~2~) but does not alter the backup timing -- only once in a user session, the first time when the buffer is saved to the visited file.

(setq-local version-control t))
;; Creates mutiple backups.
;; The number of backup versions kept is defined by
;; kept-new-versions. Default is 2.

Org-transclusion does not handle metaup/down well when multiple headlines exist within one transclusion.

See the image. As the source face is correctly closed, I believe this is a matter of my coding mistakes. Need to revisit this.

Ref: https://org-roam.discourse.group/t/prototype-transclusion-block-reference-with-emacs-org-mode/830/62

tested org-refile on a transcluded entry, to see how it would break, and yes it does:

1. the entry stays (transcluded) in the file
2. a copy is created in the refile target file
3. this copy is read only

I've noticed that when attempting to transclude a kanban board that is generated by [org-kanban(https://github.com/gizmomogwai/org-kanban), it does not show up. Interestingly, this is only a problem when org-kanban is loaded and when it is not, the generated table shows up just fine.

I have a feeling that this problem may be shared by other dynamic block types but since I don't use those myself, I have yet to test it.

I understand you have the org-transclusion-paste-subtree function. I think it will be useful to add an option to set (or promote/demote) the headline level of transcluded content.

Hi @nobiot :

The only word that popped out from my mouth after watching the fourth video (https://www.youtube.com/watch?v=nO_JEXUeGkI) was WOW, AWESOME! Your approach is so nifty that that the package is getting better every day! 👍 The look of the transcluded org document was extremely legible.

How did you get the heading numbers on and off in the org document itself? Did you post any emacs.d config?

How does it integrate with org-ref (to include the bibliographic references) and org-roam (to hyperlink the notes with tags)?

However, the headings were not exported properly when you exported test.org to plaintext (albeit alright when exported to pdf and docx).

Cheers, and stay safe,
/z

Update the content in the orphaned transclusion source, you get a backtrace error (I think it is debugger-on-error is on).

Evaluating (remove-overlays) in the source buffer can remove this.

This is a really significant mode. Thanks for the efforts! I am having some trouble with some org files (see following). The content does display when is transclusion mode is activated. However, when the transclusion mode is deactivated, the link disappears; and the content is not exported with org-mode export dispatch. It works with other files so far though.

The MWE is as follows:

the master org file:

#+title: testing
#+transclude: t
[[file:completion_the_org_manual.org][completion]]

The referenced org file:

#+title: Completion (The Org Manual)
#+Date:2020-11-29 Sun
#+roam_key: https://orgmode.org/manual/Completion.html#Completion
#+roam_alias: org completion
#+roam_tags: "orgmode" completion "Emacs"

[2020-11-29 Sun 20:26] Complete word at point.

~M-Tab~

 At the beginning of an empty headline, complete TODO keywords.

1. After ‘~\~’, complete TeX symbols supported by the exporter.

2. After ‘~:~’ in a headline, complete tags.  Org deduces the list of
  tags from the ‘TAGS’ in-buffer option (see Setting Tags), the
  variable org-tag-alist, or from all tags used in the current buffer.

If I transclude a file with include-first-section underneath some header and then try to collapse the header, the transcluded contents are still visible

I want to start of by saying that I really like the work you have done here.

Now on to the feature request, It would be more convenient if edits could be conducted from within transcluding buffer. It would make it easier to see and possibly switch other content within transcluding buffer. Not a high priority because it is possible to open the edit buffer and the transcluding buffer side by side but would nevertheless have a positive impact on workflow.

To guard against multiple transclusions editing at the same time, either make it so that only one can be edited at the same time or each change needs to reflect in each visible transclusion before another character can be inserted.

When attempting to transclude an org file that has no headlines, you get an error

let*: Wrong type argument: integer-or-marker-p, nil

The link disappears from the transcluding buffer, too.

Hey all, I can't this to work no matter how I try it :(. It just eats my transclusion header when I activate or deactivate the mode. I am probably doing something wrong, but I can't understand what's happening in any of the videos. It's hard to understand what's actually happening in the videos as when I try to recreate something off visuals of someone else's formated buffers I don't know if I am recreating the buffer properly due to visual changes. I think an example file would be wonderful.

Below are my files cv.org

#+TITLE: Curriculum Vitae
#+KEYWORDS: vita, CV

#+transclude: t
[[file:common.org][Common Headlines]]

hello

common.org

#+AUTHOR: Jonathan Pavlik
#+OPTIONS: toc:nil num:nil
#+STARTUP: entitiespretty
what

Should that not work? Why is my transclude section being eaten? Thanks for the help.

Not sure how to diagnose this issue, which is occurring right now with emacs git native-comp (2 different builds: pure gtk which has xwidget enabled, and vanilla; I know, that's a lot of modifications).

when I try to insert an otc link with trnasclusion mode active, emacs cpu usage goes to 100% and I am only able to stop by sending SIGUSR1 or SIGUSR2 signal otthe emacs process. This sometimes generates the following in backtrac (if xwidget enabled):

Debugger entered--entering a function:
* xwidget-webkit-adjust-size-in-frame(#<frame ~/Test-source.org 0x5640b2f44cb0>)
  redisplay_internal\ \(C\ function\)()
  sit-for(0)
  org-insert-link(nil)
  funcall-interactively(org-insert-link nil)
  command-execute(org-insert-link)

But since the hang still occurs when xwidget is disabled, I don't think this is the real cause of the problem. I'll keep trying different builds in the hopes that I can help find the problem!

The original intention using org-roam is to create atomic notes. This
is the ideal approach. The atomic note should not be further divided
and no extra information in there.

However, I find it is really hard to write my note into the atomic
type of notes. Often, there is some extra information like where this
ideas comes from, some immature paragraph, some code snippets. In many
cases, a user would like to partially transclude some contents of a
note.

#+TITLE: ideas about partially transclusion
I would like to use the following note as an example.

This is the 1st paragraph. I like org-roam.

This is the 2nd paragraph. I like org-transclusion.

#+begin_export transclude
This is the paragraph that I want to transclude to other places.
#+end_export

Some other notes.

If this note is being transcluded to other notes, only
This is the paragraph that I want to transclude to other places. is
shown in other places.

This would encourage the user to transclude more ideas without having
burden on crafting the original note without the partial transclusion
function.

When I try to transclude a heading in some other place in the same file, it doesn't work. It even seems to delete both the link and the #+transclusion: t line.

I guess that's just not supported at the moment, right? Are you planning to include this in the future?

It might seem like an obscure thing to do, but I would actually use it quite often. For example, I like doing what is sometimes called literate devops, where it's quite handy to transclude certain blocks which I need over and over again, such as small blocks to login to some remote server, without having to duplicate this information.