jgm / citeproc Goto Github PK

CSL citation processing library in Haskell

License: BSD 2-Clause "Simplified" License

Makefile 0.50% Haskell 99.46% Shell 0.03%

csl bibliography pandoc citeproc citations

citeproc's Introduction

citeproc

This library generates citations and bibliography formatted according to a CSL style. Currently version 1.0.2 of the CSL spec is targeted.

This library is a successor to pandoc-citeproc, which was a fork of Andrea Rossato's citeproc-hs. I always found it difficult to fix bugs in pandoc-citeproc and decided that implementing citeproc from scratch would give me a better basis for understanding. This library has a number of other advantages over pandoc-citeproc:

it is much faster (as a rough benchmark, running the CSL test suite takes less than 4 seconds with this library, compared to 12 seconds with pandoc-citeproc)
it interprets CSL more faithfully, passing more of the CSL tests
it has fewer dependencies (in particular, it does not depend on pandoc)
it is more flexible, not being tied to pandoc's types.

Unlike pandoc-citeproc, this library does not provide an executable. It will be used in pandoc itself to provide integrated citation support and bibliography format conversion (so the pandoc-citeproc filter will no longer be necessary).

How to use it

The main point of entry is the function citeproc from the module Citeproc. This takes as arguments:

a CiteprocOptions structure, which includes the following options:
- linkCitations controls whether citations are hyperlinked to the bibliography.
- linkBibliography automatically linkifies any identifiers (DOI, PMCID, PMID, or URL) appearing in a bibliography entry. When an entry has a DOI, PMCID, PMID, or URL available but none of these are rendered by the style, add a link to the title (or, if no title is present, the whole entry), using the URL for the DOI, PMCID, PMID, or URL (in that order of priority). See Appendix VI of the CSL v1.0.2 spec.
a Style, which you will want to produce by parsing a CSL style file using parseStyle from Citeproc.Style.
Optionally a Lang, which allows you to override a default locale,
a list of References, which you can produce from a CSL JSON bibliography using aeson's decode,
a list of Citations (each of which may have multiple CitationItems).

It yields a Result, which includes a list of formatted citations and a formatted bibliography, as well any warnings produced in evaluating the style.

The types are parameterized on a CiteprocOutput instance a, which represents formatted content in your bibliographic fields (e.g. the title). If you want a classic CSL processor, you can use CslJson Text. But you can also use another type, such as a pandoc Inlines. All you need to do is define an instance of CiteprocOutput for your type.

The signature of parseStyle may not be self-evident: the first argument is a function that takes a URL and retrieves the text from that URL. This is used to fetch the "indendent parent" of a dependent style. You can supply whatever function you like: it can search your local file system or fetch the content via HTTP. If you're not using dependent styles, you can get by with \_ -> return mempty.

The citeproc executable

If the package is compiled with the executable flag, an executable citeproc will be built. citeproc reads a JSON-encoded Inputs object from stdin (or from a file if a filename is provided) and writes a JSON-encoded Result object to stdout. This executable can be used to add citation processing to non-Haskell projects.

citeproc --help will summarize usage information. See the man page for more information.

Known bugs and limitations

Although this library is much more accurate in implementing the CSL spec than pandoc-citeproc was, it still fails some of the tests from the CSL test suite (62/818). However, most of the failures are on minor corner cases, and in many cases the expected behavior goes beyond what is required by the CSL spec. (For example, we intentionally refrain from capitalizing terms in initial position in note styles. It makes more sense for the calling program, e.g. pandoc, to do the capitalization when it puts the citations in notes, since some citations in note styles may already be in notes and in this case their rendering may not require capitalization. It is easy to capitalize reliably, hard to uncapitalize reliably.)

citeproc's People

Contributors

Stargazers

Watchers

Forkers

denismaier wmhorne felixonmars tarleb kdmsnr turion benrbray odomanov taylrj sthagen mayebejames igrep

citeproc's Issues

Ibid.

I've just gave it a shot with pandoc compiled from the citeproc branch.
This here still doesn't work as expected:

---
csl: chicago-note-bibliography.csl
references:
  - id: test
    title: Test
    author:
      family: Doe
      given: John
---

Test [@test, 12].

Test.^[asdfasdf]

Test [@test, 12].

The second citation still renders with position="ibid". I think it should be subsequent.

(Is this the correct place for this issues? Or should I re-open it at the pandoc issue tracker?)

How to compile with citeproc executable?

How to compile with citeproc executable that I can not find the flag in Makefile?

first-reference-number wrongly set

Every time a citation references its first occuring, it's shifted by 1. For example:

6 H. Lane, H. Hapke, C. Howard, Natural language processing in action: Understanding, analyzing, and generating text with python, Manning Publications, 2019.
7 H. Lane, H. Hapke, C. Howard, Natural language processing in action, op. cit. note 7, pp. 11-16.

As you can see, it should read op. cit. note 6. This is happening for every reference including first-reference-number, and regardless of csl file used (tried 3 different).

Pandoc version: 2.11.0.4
Command used: pandoc filename.md -o ../build/filename.pdf --citeproc

Citeproc doesn't support et-al-subsequent-use-first

I mean, I know for sure the "et-al" abbreviations criteria are nowhere to be found, but I would guess other exist.

To reproduce, just use the 6th ed. APA style and try to cite the same multi-author reference twice.
Then convert from markdown to latex and et-al-subsequent-use-first="1" should make it so the second time you only get "dude et. all".
Starting with pandoc 2.11 though, they are always and forever named in long form.

Why does punctuation move to inside quotes?

What I did:

echo 'Lorem "ipsum".' | pandoc -t html -o - --citeproc

Output:

<p>Lorem “ipsum.”</p>

Now let's make sure this is caused by citeproc:

echo 'Lorem "ipsum".' | pandoc -t html -o -

Output:

<p>Lorem “ipsum”.</p>

Why did the period move to inside the quotes? There are not citations at the end of the sentence, so why change anything?

My real question: How do I tell citeproc to not do this (i.e., move the punctuation)?

Font size for citations

I'm transitioning from LaTeX to R Markdown (and so Pandoc and so citeproc) to write Beamer slides. I'm the kind of academic who uses lots of citations, including as in-line author-date citations on my slides. To reduce visual clutter, I then set the font of the in-line citations as small as possible, eg, \tiny. I'm trying to figure out whether this is possible in my new workflow.

I've looked into whether a custom CSL style would work. However, in this Zotero thread from 2008, a CSL developer indicates that CSL doesn't support font size styling, arguing that it should be handled downstream so that it matches the surrounding document. I haven't been able to find any more recent discussions of font size and CSL.

My next idea was to check out how citations are represented in the TeX file; if they're getting turned into \cite commands or some such, then maybe I could put a wrapper around that command in the preamble. (This is basically what I did with the biblatex commands when I wrote the LaTeX directly.) But it appears the citations just become plain text strings.

So is there any way I can intervene between the CSL styling and the generated TeX file to get something whose font size I can control?

Regression using pandoc-zotxt.lua: CSL JSON in-field markup not recognized

A regression:

The new pandoc from the "master" branch, source downloaded and self-compiled earlier today, does not seem to recognize CSL JSON in-field markup (e.g., <i>...</i>) if biblio data are obtained from Zotero using the pandoc-zotxt.lua filter without having this filter write an intermediary CSL JSON file to disk. (Not writing an intermediary/cache file is this filter’s default behaviour.)

Note that for using pandoc-zotxt.lua, the Zotero add-ons zotxt and, IIRC, Better BibTeX (the latter for useable citation keys) need to be installed.

By contrast, after adding a line zotero-bibliography: test.json to the markdown document’s metadata (which leads to a an intermediary/cache CSL JSON file being written to disk), the in-field markup is recognized, just as it used to be when using the "old" pandoc, with or without the intermediary .json file.

My hunch is that the "new" pandoc might not be properly identifying the incoming data from the pandoc-zotxt.lua filter as CSL JSON and hence might not be trying to parse the html-like in-field markup.

MWE; prerequisite: Zotero plus add-ons listed above installed and running, a Zotero item containing <i>...</i> in its title field:

pandoc -L pandoc-zotxt.lua -C -t markdown-citations << EOT

Blah [@a-zotero-item].

# References {-}

---
# zotero-bibliography: test.json
...
EOT

Actual output:

Blah (Doe 2020).

References {#references .unnumbered}
==========

::: {#refs .references .csl-bib-body .hanging-indent}
::: {#ref-a-zotero-item .csl-entry}
Doe, John 2020. "A Title with \<i\>a Few Words
Italicized\</i\>."
:::
:::

Expected output = output from the "new" pandoc with the zotero-bibliography line uncommented = output from the "old" pandoc, regardless of the zotero-bibliography line:

Blah (Doe 2020).

References {#references .unnumbered}
==========

::: {#refs .references .csl-bib-body .hanging-indent}
::: {#ref-a-zotero-item .csl-entry}
Doe, John 2020. "A Title with *a Few Words
Italicized*."
:::
:::

"?" at end of title produces a "?`" ligature (upside-down question mark) in latex with "lang: de"

$ pandoc --version
pandoc 2.11.4
Compiled with pandoc-types 1.22, texmath 0.12.1, skylighting 0.10.2,
citeproc 0.3.0.5, ipynb 0.1.0.1

With the new citeproc, if I have a title ending with a question mark ("?") in the bibliography and I am setting lang to German in the YAML metadata block (I tested de, de-DE or de-DE-1996), this gets set as "?``" in latex together with the closing double quotes, which gets interpreted as a "?`" ligature (upside-down question mark "¿") plus closing single quotes in pdf.

Example:

---
lang: de

references:
- type: article
  id: test
  author: David Graeber
  title: "What's the Point If We Can't Have Fun?"

nocite: test

---

Command to compile to pdf: pandoc --output="temp.pdf" --citeproc "temp.md"

Output pdf: temp.pdf

Depending on the CSL used, the same happens in citations within the text. It does not happen with lang: en.

Background / non-working workaround

This issue is somewhat similar to jgm/pandoc#5407. Back then a viable workaround was for me to insert an invisible space ("word joiner", unicode U+2060) after the "?". This time (and on a different computer), however, this doesn't work for me. With pdflatex, it leads to an error:

$ pandoc --pdf-engine=pdflatex --output="temp.pdf" --citeproc "temp.md
Error producing PDF.
! Package inputenc Error: Unicode character ⁠ (U+2060)
(inputenc)                not set up for use with LaTeX.

See the inputenc package documentation for explanation.
Type  H <return>  for immediate help.
 ...                                              
                                                  
l.90 ...`What's the Point If We Can't Have Fun?⁠

Try running pandoc with --pdf-engine=xelatex.

With xelatex, it leads to a warning and faulty output because the font doesn't have the U+2060 character and replaces it by a fullstop ("."):

$ pandoc --pdf-engine=xelatex --output="temp.pdf" --citeproc "temp.md"
[WARNING] Missing character: There is no ⁠ (U+2060) in font [lmroman10-regular]:mapping=tex-text;!

Apart from the default font, I tried mainfont: Linux Libertine O and mainfont: Noto Serif. The latter got rid of the warning message, but still lead to the unwanted fullstop in the output.

inline citation not respecting style

Using pandoc with a json csl library (generated by Zotero) I use a citation style for entry-encyclopedia. It seems to use the fields in the style but not the formatting or the oder of the elements.

what would I expect to get (visual editor):

result with pandoc:

the order of the elements is not as expected and the formatting is also different (Stern should be italic when in author field)
with a bit older version of citeproc the second field included in addition brackets, this seems to be gone since the author field can be surpressed

used software versions (Win 10)

pandoc 2.11.4
Compiled with pandoc-types 1.22, texmath 0.12.1, skylighting 0.10.2, citeproc 0.3.0.5, ipynb 0.1.0.1

test_bib.json

[
  {"id":"sternSternBecker2019b","abstract":"Grundrechtseingriffe und sonstige Beeinträchtigungen","author":[{"family":"Stern","given":"Klaus"}],"edition":"3","editor":[{"family":"Stern","given":"Klaus"},{"family":"Becker","given":"Florian"}],"event-place":"Köln","issued":{"date-parts":[[2019]]},"publisher":"Carl Heymanns Verlag","publisher-place":"Köln","title":"Stern/Becker_Rn161_175 Grundrechte-Kommentar","title-short":"Stern/Becker","type":"entry-encyclopedia"}
]

source.md

Lorem ipsum dolor sit amet^[@sternSternBecker2019b], consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.^[@sternSternBecker2019b, Rn 27.]

stueber_UniMannheim.csl (sorry for the length tested also with a shorter version, but would not like to introduce possible additional problems)

<?xml version="1.0" encoding="utf-8"?>
<style class="note" version="1.0" demote-non-dropping-particle="sort-only" default-locale="de-DE" xmlns="http://purl.org/net/xbiblio/csl">
  <!-- This style was edited with the Visual CSL Editor (http://editor.citationstyles.org/visualEditor/) -->
  <info>
    <title>Juristische Zitierweise (Stüber) (German), Universität Mannheim</title>
    <id>http://www.zotero.org/styles/juristische-zitierweise-stuber-german-universitat-mannheim</id>
    <link href="http://www.zotero.org/styles/juristische-zitierweise-stuber-german-universitat-mannheim" rel="self"/>
    <link href="http://www.zotero.org/styles/juristische-zitierweise" rel="template"/>
    <link href="www.niederle-media.de/Zitieren.pdf" rel="documentation"/>
    <author>
      <name>Philipp Zumstein</name>
    </author>
    <category citation-format="note"/>
    <category field="law"/>
    <summary>Angepasster juristischer Stil für die Uni Mannheim, aufbauend auf Juristische Zitierweise nach Stüber</summary>
    <updated>2018-08-31T12:44:57+00:00</updated>
    <rights license="http://creativecommons.org/licenses/by-sa/3.0/">This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License</rights>
  </info>
  <locale xml:lang="de-DE">
    <terms>
      <term name="accessed">abgerufen am</term>
    </terms>
  </locale>
  <macro name="author">
    <names variable="author" font-style="italic">
      <name delimiter=" / " name-as-sort-order="all"/>
      <label form="short" prefix=" (" suffix=")"/>
      <substitute>
        <names variable="editor"/>
      </substitute>
    </names>
  </macro>
  <macro name="author-note">
    <names variable="author" font-style="italic">
      <name form="short" delimiter="/" et-al-min="3" et-al-use-first="1" name-as-sort-order="all"/>
    </names>
  </macro>
  <macro name="locator-with-label">
    <group delimiter="&#160;">
      <label variable="locator" form="symbol"/>
      <text variable="locator"/>
    </group>
  </macro>
  <macro name="autor-editor-note">
    <names variable="author" font-style="italic">
      <name form="short" delimiter="/" et-al-min="3" et-al-use-first="1" sort-separator=""/>
      <substitute>
        <names variable="editor"/>
      </substitute>
    </names>
  </macro>
  <macro name="inbook">
    <group delimiter=": ">
      <text term="in"/>
      <group delimiter=", ">
        <choose>
          <if match="any" variable="title-short">
            <text variable="title-short"/>
          </if>
          <else>
            <text variable="container-title" form="short"/>
          </else>
        </choose>
      </group>
    </group>
  </macro>
  <macro name="firstpage-locator">
    <group delimiter=" ">
      <text variable="page-first"/>
      <text variable="locator" prefix="(" suffix=")"/>
    </group>
  </macro>
  <macro name="in-commentary">
    <group delimiter=": ">
      <text value="in"/>
      <choose>
        <if match="any" variable="title-short">
          <text variable="title-short"/>
        </if>
        <else>
          <text variable="container-title" form="short"/>
        </else>
      </choose>
    </group>
  </macro>
  <macro name="edition">
    <choose>
      <if match="any" is-numeric="edition">
        <text variable="edition" suffix=". "/>
        <text term="edition"/>
      </if>
      <else>
        <text variable="edition"/>
      </else>
    </choose>
  </macro>
  <macro name="volume">
    <group delimiter=" " suffix=", ">
      <text term="volume" form="short"/>
      <text variable="volume"/>
    </group>
  </macro>
  <macro name="author-editor-sort">
    <choose>
      <if type="entry-encyclopedia" match="any">
        <names variable="editor">
          <name delimiter=" / "/>
        </names>
      </if>
      <else>
        <text macro="author"/>
      </else>
    </choose>
  </macro>
  <citation>
    <layout delimiter="; " suffix=".">
      <choose>
        <if type="article-journal">
          <group delimiter=", ">
            <text macro="author-note"/>
            <group delimiter=" ">
              <text variable="container-title" form="short"/>
              <date date-parts="year" form="text" variable="issued"/>
            </group>
            <group delimiter=" ">
              <text variable="page-first"/>
              <text variable="locator" prefix="(" suffix=")"/>
            </group>
          </group>
        </if>
        <else-if type="article-magazine" match="all" variable="volume">
          <group delimiter=", ">
            <text macro="author-note"/>
            <group delimiter=" ">
              <text variable="container-title" form="short"/>
              <text variable="volume"/>
              <date date-parts="year" form="text" variable="issued" prefix="(" suffix=")"/>
            </group>
            <group delimiter=" ">
              <text variable="page-first"/>
              <text variable="locator" prefix="(" suffix=")"/>
            </group>
          </group>
        </else-if>
        <else-if type="book">
          <group delimiter=", ">
            <text macro="autor-editor-note"/>
            <text variable="title" form="short"/>
            <text macro="locator-with-label"/>
          </group>
        </else-if>
        <else-if type="chapter">
          <group delimiter=", ">
            <text macro="author-note"/>
            <text macro="inbook"/>
            <group delimiter=" ">
              <group delimiter="&#160;">
                <text term="page" form="short"/>
                <text variable="page-first"/>
              </group>
              <text variable="locator" prefix="(" suffix=")"/>
            </group>
          </group>
        </else-if>
        <else-if type="entry-encyclopedia">
          <group delimiter=", ">
			<text macro="in-commentary"/>
            <names variable="author" font-style="italic">
              <name form="short" delimiter="/"/>
            </names>
            <text variable="locator" suffix="."/>
          </group>
        </else-if>
        <else-if type="legal_case" match="any">
          <group delimiter=", ">
            <choose>
              <if match="any" variable="title-short">
                <text variable="title-short"/>
              </if>
              <else>
                <text variable="authority"/>
                <group delimiter=" - ">
                  <group delimiter=" ">
                    <text variable="genre"/>
                    <date form="numeric" variable="issued" prefix="v. "/>
                  </group>
                  <text variable="number"/>
                </group>
              </else>
            </choose>
            <group delimiter=" ">
              <text variable="container-title"/>
              <text variable="volume"/>
            </group>
            <text macro="firstpage-locator"/>
          </group>
        </else-if>
        <else>
          <group delimiter=", ">
            <text macro="author-note"/>
            <text variable="title" form="short"/>
            <text macro="locator-with-label"/>
          </group>
        </else>
      </choose>
    </layout>
  </citation>
  <bibliography>
    <sort>
      <key macro="author-editor-sort"/>
      <key variable="issued"/>
    </sort>
    <layout>
      <choose>
        <if type="article-journal">
          <group delimiter=", ">
            <text macro="author"/>
            <text variable="title"/>
            <group delimiter=" ">
              <text variable="container-title" form="short" suffix=" "/>
              <date date-parts="year" form="text" variable="issued"/>
            </group>
            <text variable="page"/>
          </group>
        </if>
        <else-if type="article-magazine" match="all" variable="volume">
          <group delimiter=", ">
            <text macro="author-note"/>
            <group delimiter=" ">
              <text variable="container-title" form="short"/>
              <text variable="volume"/>
              <date date-parts="year" form="text" variable="issued" prefix="(" suffix=")"/>
            </group>
            <text variable="page"/>
          </group>
        </else-if>
        <else-if type="book">
          <group delimiter=", ">
            <text macro="author"/>
            <text variable="title"/>
            <text macro="volume"/>
            <text macro="edition"/>
            <group delimiter=" ">
              <text variable="publisher-place" suffix=" "/>
              <date date-parts="year" form="text" variable="issued"/>
            </group>
          </group>
        </else-if>
        <else-if type="chapter">
          <group delimiter=", ">
            <text macro="author"/>
            <text variable="title"/>
            <group delimiter=": ">
              <text term="in"/>
              <group delimiter=", ">
                <names variable="editor" font-style="italic">
                  <name delimiter=" / " name-as-sort-order="all"/>
                  <label form="short" prefix=" (" suffix=")"/>
                </names>
                <text variable="container-title"/>
                <text macro="volume"/>
              </group>
            </group>
            <text variable="edition"/>
            <group delimiter=" ">
              <text variable="publisher-place"/>
              <date date-parts="year" form="text" variable="issued"/>
            </group>
            <text variable="page"/>
          </group>
        </else-if>
        <else-if type="thesis">
          <group delimiter=", ">
            <text macro="author"/>
            <text variable="title"/>
            <text variable="genre"/>
            <group delimiter=" ">
              <text variable="publisher-place"/>
              <date date-parts="year" form="text" variable="issued"/>
            </group>
          </group>
        </else-if>
        <else-if type="entry-encyclopedia">
          <names variable="editor" font-style="italic" delimiter=" / ">
            <name delimiter=" / " name-as-sort-order="all"/>
            <label form="short" prefix=" (" suffix=")"/>
          </names>
          <text variable="container-title" prefix=", " suffix=", "/>
          <text macro="volume"/>
          <group delimiter=", ">
            <text macro="edition"/>
            <group delimiter=" ">
              <text variable="publisher-place"/>
              <date variable="issued">
                <date-part name="year" form="long"/>
              </date>
              <group delimiter=", " prefix="(zitiert: " suffix=")">
                <text value="Bearbeiter" font-style="italic"/>
                <text macro="in-commentary"/>
              </group>
            </group>
          </group>
        </else-if>
        <else-if type="webpage" match="any">
          <group delimiter=", ">
            <text macro="author"/>
            <text variable="title"/>
            <group delimiter=" ">
              <text variable="URL"/>
              <group delimiter=" " prefix="(" suffix=")">
                <text term="accessed"/>
                <date form="numeric" variable="accessed"/>
              </group>
            </group>
          </group>
        </else-if>
        <else>
          <text macro="author"/>
          <text variable="title" prefix=", " suffix=", "/>
          <text variable="container-title" suffix=" "/>
          <date date-parts="year" form="text" variable="issued" suffix=", "/>
          <text variable="page"/>
        </else>
      </choose>
    </layout>
  </bibliography>
</style>

[Enhancement]: Allow nocite references to be numbered after inline references.

Pandoc appears to always put nocite citations at the beginning of your list of citations.

If using a CSL style that numbers citations (e.g. vancouver-superscript), these will be numbered before any inline citations, resulting in a document that has numbered references before the inline citation autonumbering begins.

I'm currently working on (admittedly an edge-case of) a document that has a lot of nocite references (standards, etc), and only a few inline citations, resulting in the numbering for inline citations beginning at '23' and going on from there.

In contrast to this behaviour, on other documents (where all citations are inline), using a numbered citation style works as expected (i.e. citation 1 is the first inline citation, and numbering continues from there).

Would it be possible for pandoc to (as an option) process inline references (for the purpose of numbering) before the YAML nocite references? (If there's already a way to do this, I haven't found it poring over doco, or in a quick search of issues here.)

As this doesn't impact you if your references aren't autonumbered, I'm not sure whether or not inline references should be processed first by default, but having an option to do this would be appreciated.

I've observed this behaviour on pandoc versions: 2.11.0.1, 2.11.0.2, 2.11.1.

On that note, the citeproc functionality that's baked into pandoc as of 2.11 is excellent. Fantastic work!

Please find a standalone example of this behaviour below:

---
references:

- id: iso/iec11172-1
  author:
    - literal: ISO/IEC Joint Technical Committee 1
  issued:
    - year: 1993
      month: 8
  language: en
  number-of-pages: '6'
  publisher: International Organization for Standardization
  title: >-
    ISO/IEC 11172-1 Information technology — Coding of moving pictures and
    associated audio for digital storage media at up to about 1,5 Mbit/s — Part
    1: Systems
  title-short: 'ISO/IEC 11172-1:1993(E)'
  type: book
  URL: 'https://www.iso.org/obp/ui/#iso:std:iso-iec:11172:-1:ed-1:v1:en'

- id: iso/iec11172-2
  author:
    - literal: ISO/IEC Joint Technical Committee 1
  issued:
    - year: 1993
      month: 8
  language: en
  number-of-pages: '112'
  publisher: International Organization for Standardization
  title: >-
    ISO/IEC 11172-2 Information technology — Coding of moving pictures and
    associated audio for digital storage media at up to about 1,5 Mbit/s — Part
    2: Video
  title-short: 'ISO/IEC 11172-2:1993(E)'
  type: book
  URL: 'https://www.iso.org/obp/ui/#iso:std:iso-iec:11172:-2:ed-1:v1:en'

- id: iso/iec11172-3
  author:
    - literal: ISO/IEC Joint Technical Committee 1
  issued:
    - year: 1993
      month: 8
  language: en
  number-of-pages: '150'
  publisher: International Organization for Standardization
  title: >-
    ISO/IEC 11172-3 Information technology — Coding of moving pictures and
    associated audio for digital storage media at up to about 1,5 Mbit/s — Part
    3: Audio
  title-short: 'ISO/IEC 11172-3:1993(E)'
  type: book
  URL: 'https://www.iso.org/obp/ui/#iso:std:iso-iec:11172:-3:ed-1:v1:en'

- id: iso/iec13818-1
  author:
    - literal: ISO/IEC Joint Technical Committee 1
  edition: '7'
  issued:
    - year: 2019
      month: 7
  language: en
  number-of-pages: '288'
  publisher: International Organization for Standardization
  title: >-
    ISO/IEC 13818-1 Information technology — Generic coding of moving pictures
    and associated audio information — Part 1: Systems
  title-short: 'ISO/IEC 13818-1:2019(E)'
  type: book
  URL: 'https://www.iso.org/standard/75928.html'

- id: misbst1602
  author:
    - literal: Motion Imagery Standards Board
  edition: '1'
  issued:
    - year: 2018
      month: 11
  language: en
  number-of-pages: '19'
  publisher: Motion Imagery Standards Board
  title: MISB ST 1602 Composite Imaging Local Set
  title-short: MISB ST 1602
  type: book
  URL: 'https://gwg.nga.mil/misb/docs/standards/ST1602.1.pdf'

- id: msibst0603
  author:
    - literal: Motion Imagery Standards Board
  edition: '5'
  issued:
    - year: 2017
      month: 10
  language: en
  number-of-pages: '8'
  title: MISB ST 0603 MISP Time System and Timestamps
  title-short: MISB ST 0603
  type: book
  URL: 'https://www.gwg.nga.mil/misb/docs/standards/ST0603.5.pdf'

- id: tia232
  author:
    - literal: Telecommunications Industry Association
  edition: Revision F
  issued:
    - year: 1997
      month: 10
  language: en
  number-of-pages: '54'
  publisher: Telecommunications Industry Association
  title: >-
    TIA-232 Interface Between Data Terminal Equipment and Data Circuit-
    Terminating Equipment Employing Serial Binary Data Interchange
  title-short: TIA-232-F
  type: book
  URL: >-
    https://global.ihs.com/doc_detail.cfm?&rid=TIA&item_s_key=00125234&item_key_date=870024&input_doc_number=232&input_doc_title=&org_code=TIA

nocite: |
 @iso/iec11172-1, @iso/iec11172-2, @iso/iec11172-3, @iso/iec13818-1, @misbst1602, @misbst1603
csl: 'https://www.zotero.org/styles/vancouver-superscript'
...

This is an inline reference[@tia232].

Thanks!

Add an option --no-citationmark-relocation

First and foremost, thank you very much for all your work. Scholarly writing became much more fluid since I discovered pandoc.

The issue I experiment is similar to #6 in that it is related with the diversity of conventions about the localisation of footnote marks between languages. As a French, my problem is that I would need the mark to be before the punctuation marks. The strange thing is that I didn't have it with pandoc-citeproc, as the footnote marks stayed where I placed my citations in the md file.

Although we could write lua filters to move the footnote marks after citeproc processing to match the pattern of the targeted languages, the simplest way to handle this may be to add an option that inhibits the mark/punctuation moving mechanism. Do you think that it could be feasible? After all, this is how regular footnotes work in pandoc.

Regression: --csl= relative path is ignored when specifying --resource-path=

pandoc 2.11.4
Compiled with pandoc-types 1.22, texmath 0.12.1, skylighting 0.10.2,
citeproc 0.3.0.5, ipynb 0.1.0.1

The following used to work flawlessly with pandoc 2.5 (which used pandoc-citeproc):

$ pandoc --resource-path=figures --csl=styles/american-medical-association.csl …

In other words, I created two subdirectories under my current working directory ./figures/ and ./styles/. The figures directory is assigned to the resource-path and the --csl= option points to a relative file path. Nonetheless, with pandoc 2.11.4 (which uses now citeproc instead of pandoc-citeproc) this results in the following error:

File styles/american-medical-association.csl not found in resource path

This is not according to the manual, which states:

If FILE is not found relative to the working directory, it will be sought in the resource path (see --resource-path) and finally in the csl subdirectory of the pandoc user data directory.

Two pandoc citeproc problems

When using the new --citeproc option (having used the pandoc-citeproc previously) I find the following problems:

the first item in a Reference list is preceded by an unexplicable number ‘10‘.
the url field in the biblatex entry is not found in the item in the Reference list.

I will try to attach a document showing the difference between the output of the old pandoc-citeproc and the new --citeproc resulting in the ‘Reference list’, but also show it here:

@online{Lonardo2017Appunti,
	author       = {Lonardo, A.},
	title        = {{Appunti per visitare la basilica di San Marco
                       a Venezia, chiave per scoprire i segreti
                       medioevali della laguna}},
	address      = {\emph{Gli Scritto} [blog]},
	publisher    = {},
	year         = {2017},
	month        = {7},
	day          = {3},
	doi          = {},
	url          = {http://www.gliscritti.it/blog/entry/4205},
	urldate      = {},
	shorttitle   = {{Appunti}},
	keywords     = {@ART-Venice-San-Marco-Pentecost-Cupola},
}

---
nocite: |
  @Lonardo2017Appunti
---

output when using depecrated `pandoc-citeproc` filter

Lonardo, A. 2017. ‘Appunti per visitare la basilica di San Marco a Venezia, chiave per scoprire i segreti medioevali della laguna’. Gli Scritto [blog]. 3 juli 2017. http://www.gliscritti.it/blog/entry/4205.

Reference list

10 Lonardo, A. 2017. ‘Appunti per visitare la basilica di San Marco a Venezia, chiave per scoprire i segreti medioevali della laguna’. Gli Scritto [blog]. 3 juli 2017.

New_pandoc_citeproc_problems.pdf

Regression with no-name citations

Currently this fails. We get Newspaper 2012e, 2012f instead of a,b.

>>===== MODE =====>>
citation
<<===== MODE =====<<




>>===== RESULT =====>>
(<i>Magazine</i> 2012a; <i>Magazine</i> 2012b; <i>Magazine</i> 2012c; <i>Magazine</i> 2012d; <i>Newspaper</i> 2012a; <i>Newspaper</i> 2012b)
<<===== RESULT =====<<


>>===== CSL =====>>

<?xml version="1.0" encoding="utf-8"?>
<style xmlns="http://purl.org/net/xbiblio/csl" class="in-text" version="1.0" demote-non-dropping-particle="display-and-sort" page-range-format="chicago">
  <info>
    <title>Chicago Manual of Style 17th edition (author-date)</title>
    <id>http://www.zotero.org/styles/chicago-author-date</id>
    <link href="http://www.zotero.org/styles/chicago-author-date" rel="self"/>
    <link href="http://www.chicagomanualofstyle.org/tools_citationguide.html" rel="documentation"/>
    <author>
      <name>Julian Onions</name>
      <email>[email protected]</email>
    </author>
    <contributor>
      <name>Sebastian Karcher</name>
    </contributor>
    <contributor>
      <name>Richard Karnesky</name>
      <email>[email protected]</email>
      <uri>http://arc.nucapt.northwestern.edu/Richard_Karnesky</uri>
    </contributor>
    <contributor>
      <name>Andrew Dunning</name>
      <email>[email protected]</email>
      <uri>https://orcid.org/0000-0003-0464-5036</uri>
    </contributor>
    <contributor>
      <name>Matthew Roth</name>
      <email>[email protected]</email>
      <uri> https://orcid.org/0000-0001-7902-6331</uri>
    </contributor>
    <category citation-format="author-date"/>
    <category field="generic-base"/>
    <summary>The author-date variant of the Chicago style</summary>
    <updated>2018-01-24T12:00:00+00:00</updated>
    <rights license="http://creativecommons.org/licenses/by-sa/3.0/">This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License</rights>
  </info>
  <locale xml:lang="en">
    <terms>
      <term name="editor" form="verb-short">ed.</term>
      <term name="container-author" form="verb">by</term>
      <term name="translator" form="verb-short">trans.</term>
      <term name="editortranslator" form="verb">edited and translated by</term>
      <term name="translator" form="short">trans.</term>
    </terms>
  </locale>
  <macro name="secondary-contributors">
    <choose>
      <if type="chapter entry-dictionary entry-encyclopedia paper-conference" match="none">
        <group delimiter=". ">
          <names variable="editor translator" delimiter=". ">
            <label form="verb" text-case="capitalize-first" suffix=" "/>
            <name and="text" delimiter=", "/>
          </names>
          <names variable="director" delimiter=". ">
            <label form="verb" text-case="capitalize-first" suffix=" "/>
            <name and="text" delimiter=", "/>
          </names>
        </group>
      </if>
    </choose>
  </macro>
  <macro name="container-contributors">
    <choose>
      <if type="chapter entry-dictionary entry-encyclopedia paper-conference" match="any">
        <group prefix=", " delimiter=", ">
          <names variable="container-author" delimiter=", ">
            <label form="verb" suffix=" "/>
            <name and="text" delimiter=", "/>
          </names>
          <names variable="editor translator" delimiter=", ">
            <label form="verb" suffix=" "/>
            <name and="text" delimiter=", "/>
          </names>
        </group>
      </if>
    </choose>
  </macro>
  <macro name="editor">
    <names variable="editor">
      <name name-as-sort-order="first" and="text" sort-separator=", " delimiter=", " delimiter-precedes-last="always"/>
      <label form="short" prefix=", "/>
    </names>
  </macro>
  <macro name="translator">
    <names variable="translator">
      <name name-as-sort-order="first" and="text" sort-separator=", " delimiter=", " delimiter-precedes-last="always"/>
      <label form="short" prefix=", "/>
    </names>
  </macro>
  <macro name="recipient">
    <choose>
      <if type="personal_communication">
        <choose>
          <if variable="genre">
            <text variable="genre" text-case="capitalize-first"/>
          </if>
          <else>
            <text term="letter" text-case="capitalize-first"/>
          </else>
        </choose>
      </if>
    </choose>
    <names variable="recipient" delimiter=", ">
      <label form="verb" prefix=" " text-case="lowercase" suffix=" "/>
      <name and="text" delimiter=", "/>
    </names>
  </macro>
  <macro name="substitute-title">
    <choose>
      <if type="article-magazine article-newspaper review review-book" match="any">
        <text macro="container-title"/>
      </if>
    </choose>
  </macro>
  <macro name="contributors">
    <group delimiter=". ">
      <names variable="author">
        <name and="text" name-as-sort-order="first" sort-separator=", " delimiter=", " delimiter-precedes-last="always"/>
        <label form="short" prefix=", "/>
        <substitute>
          <names variable="editor"/>
          <names variable="translator"/>
          <names variable="director"/>
          <text macro="substitute-title"/>
          <text macro="title"/>
        </substitute>
      </names>
      <text macro="recipient"/>
    </group>
  </macro>
  <macro name="contributors-short">
    <names variable="author">
      <name form="short" and="text" delimiter=", " initialize-with=". "/>
      <substitute>
        <names variable="editor"/>
        <names variable="translator"/>
        <names variable="director"/>
        <text macro="substitute-title"/>
        <text macro="title"/>
      </substitute>
    </names>
  </macro>
  <macro name="interviewer">
    <names variable="interviewer" delimiter=", ">
      <label form="verb" prefix=" " text-case="capitalize-first" suffix=" "/>
      <name and="text" delimiter=", "/>
    </names>
  </macro>
  <macro name="archive">
    <group delimiter=". ">
      <text variable="archive_location" text-case="capitalize-first"/>
      <text variable="archive"/>
      <text variable="archive-place"/>
    </group>
  </macro>
  <macro name="access">
    <group delimiter=". ">
      <choose>
        <if type="graphic report" match="any">
          <text macro="archive"/>
        </if>
        <else-if type="article-journal bill book chapter legal_case legislation motion_picture paper-conference" match="none">
          <text macro="archive"/>
        </else-if>
      </choose>
      <choose>
        <if type="webpage post-weblog" match="any">
          <date variable="issued" form="text"/>
        </if>
      </choose>
      <choose>
        <if variable="issued" match="none">
          <group delimiter=" ">
            <text term="accessed" text-case="capitalize-first"/>
            <date variable="accessed" form="text"/>
          </group>
        </if>
      </choose>
      <choose>
        <if type="legal_case" match="none">
          <choose>
            <if variable="DOI">
              <text variable="DOI" prefix="https://doi.org/"/>
            </if>
            <else>
              <text variable="URL"/>
            </else>
          </choose>
        </if>
      </choose>
    </group>
  </macro>
  <macro name="title">
    <choose>
      <if variable="title" match="none">
        <choose>
          <if type="personal_communication" match="none">
            <text variable="genre" text-case="capitalize-first"/>
          </if>
        </choose>
      </if>
      <else-if type="bill book graphic legislation motion_picture song" match="any">
        <text variable="title" text-case="title" font-style="italic"/>
        <group prefix=" (" suffix=")" delimiter=" ">
          <text term="version"/>
          <text variable="version"/>
        </group>
      </else-if>
      <else-if variable="reviewed-author">
        <choose>
          <if variable="reviewed-title">
            <group delimiter=". ">
              <text variable="title" text-case="title" quotes="true"/>
              <group delimiter=", ">
                <text variable="reviewed-title" text-case="title" font-style="italic" prefix="Review of "/>
                <names variable="reviewed-author">
                  <label form="verb-short" text-case="lowercase" suffix=" "/>
                  <name and="text" delimiter=", "/>
                </names>
              </group>
            </group>
          </if>
          <else>
            <group delimiter=", ">
              <text variable="title" text-case="title" font-style="italic" prefix="Review of "/>
              <names variable="reviewed-author">
                <label form="verb-short" text-case="lowercase" suffix=" "/>
                <name and="text" delimiter=", "/>
              </names>
            </group>
          </else>
        </choose>
      </else-if>
      <else-if type="legal_case interview patent" match="any">
        <text variable="title"/>
      </else-if>
      <else>
        <text variable="title" text-case="title" quotes="true"/>
      </else>
    </choose>
  </macro>
  <macro name="edition">
    <choose>
      <if type="bill book graphic legal_case legislation motion_picture report song" match="any">
        <choose>
          <if is-numeric="edition">
            <group delimiter=" " prefix=". ">
              <number variable="edition" form="ordinal"/>
              <text term="edition" form="short" strip-periods="true"/>
            </group>
          </if>
          <else>
            <text variable="edition" text-case="capitalize-first" prefix=". "/>
          </else>
        </choose>
      </if>
      <else-if type="chapter entry-dictionary entry-encyclopedia paper-conference" match="any">
        <choose>
          <if is-numeric="edition">
            <group delimiter=" " prefix=", ">
              <number variable="edition" form="ordinal"/>
              <text term="edition" form="short"/>
            </group>
          </if>
          <else>
            <text variable="edition" prefix=", "/>
          </else>
        </choose>
      </else-if>
    </choose>
  </macro>
  <macro name="locators">
    <choose>
      <if type="article-journal">
        <choose>
          <if variable="volume">
            <text variable="volume" prefix=" "/>
            <group prefix=" (" suffix=")">
              <choose>
                <if variable="issue">
                  <text variable="issue"/>
                </if>
                <else>
                  <date variable="issued">
                    <date-part name="month"/>
                  </date>
                </else>
              </choose>
            </group>
          </if>
          <else-if variable="issue">
            <group delimiter=" " prefix=", ">
              <text term="issue" form="short"/>
              <text variable="issue"/>
              <date variable="issued" prefix="(" suffix=")">
                <date-part name="month"/>
              </date>
            </group>
          </else-if>
          <else>
            <date variable="issued" prefix=", ">
              <date-part name="month"/>
            </date>
          </else>
        </choose>
      </if>
      <else-if type="legal_case">
        <text variable="volume" prefix=", "/>
        <text variable="container-title" prefix=" "/>
        <text variable="page" prefix=" "/>
      </else-if>
      <else-if type="bill book graphic legal_case legislation motion_picture report song" match="any">
        <group prefix=". " delimiter=". ">
          <group>
            <text term="volume" form="short" text-case="capitalize-first" suffix=" "/>
            <number variable="volume" form="numeric"/>
          </group>
          <group>
            <number variable="number-of-volumes" form="numeric"/>
            <text term="volume" form="short" prefix=" " plural="true"/>
          </group>
        </group>
      </else-if>
      <else-if type="chapter entry-dictionary entry-encyclopedia paper-conference" match="any">
        <choose>
          <if variable="page" match="none">
            <group prefix=". ">
              <text term="volume" form="short" text-case="capitalize-first" suffix=" "/>
              <number variable="volume" form="numeric"/>
            </group>
          </if>
        </choose>
      </else-if>
    </choose>
  </macro>
  <macro name="locators-chapter">
    <choose>
      <if type="chapter entry-dictionary entry-encyclopedia paper-conference" match="any">
        <choose>
          <if variable="page">
            <group prefix=", ">
              <text variable="volume" suffix=":"/>
              <text variable="page"/>
            </group>
          </if>
        </choose>
      </if>
    </choose>
  </macro>
  <macro name="locators-article">
    <choose>
      <if type="article-newspaper">
        <group prefix=", " delimiter=", ">
          <group delimiter=" ">
            <text variable="edition"/>
            <text term="edition"/>
          </group>
          <group>
            <text term="section" form="short" suffix=" "/>
            <text variable="section"/>
          </group>
        </group>
      </if>
      <else-if type="article-journal">
        <choose>
          <if variable="volume issue" match="any">
            <text variable="page" prefix=": "/>
          </if>
          <else>
            <text variable="page" prefix=", "/>
          </else>
        </choose>
      </else-if>
    </choose>
  </macro>
  <macro name="point-locators">
    <choose>
      <if variable="locator">
        <choose>
          <if locator="page" match="none">
            <choose>
              <if type="bill book graphic legal_case legislation motion_picture report song" match="any">
                <choose>
                  <if variable="volume">
                    <group>
                      <text term="volume" form="short" suffix=" "/>
                      <number variable="volume" form="numeric"/>
                      <label variable="locator" form="short" prefix=", " suffix=" "/>
                    </group>
                  </if>
                  <else>
                    <label variable="locator" form="short" suffix=" "/>
                  </else>
                </choose>
              </if>
              <else>
                <label variable="locator" form="short" suffix=" "/>
              </else>
            </choose>
          </if>
          <else-if type="bill book graphic legal_case legislation motion_picture report song" match="any">
            <number variable="volume" form="numeric" suffix=":"/>
          </else-if>
        </choose>
        <text variable="locator"/>
      </if>
    </choose>
  </macro>
  <macro name="container-prefix">
    <text term="in" text-case="capitalize-first"/>
  </macro>
  <macro name="container-title">
    <choose>
      <if type="chapter entry-dictionary entry-encyclopedia paper-conference" match="any">
        <text macro="container-prefix" suffix=" "/>
      </if>
    </choose>
    <choose>
      <if type="webpage">
        <text variable="container-title" text-case="title"/>
      </if>
      <else-if type="legal_case" match="none">
        <group delimiter=" ">
          <text variable="container-title" text-case="title" font-style="italic"/>
          <choose>
            <if type="post-weblog">
              <text value="(blog)"/>
            </if>
          </choose>
        </group>
      </else-if>
    </choose>
  </macro>
  <macro name="publisher">
    <group delimiter=": ">
      <text variable="publisher-place"/>
      <text variable="publisher"/>
    </group>
  </macro>
  <macro name="date">
    <choose>
      <if variable="issued">
        <group delimiter=" ">
          <date variable="original-date" form="text" date-parts="year" prefix="(" suffix=")"/>
          <date variable="issued">
            <date-part name="year"/>
          </date>
        </group>
      </if>
      <else-if variable="status">
        <text variable="status" text-case="capitalize-first"/>
      </else-if>
      <else>
        <text term="no date" form="short"/>
      </else>
    </choose>
  </macro>
  <macro name="date-in-text">
    <choose>
      <if variable="issued">
        <group delimiter=" ">
          <date variable="original-date" form="text" date-parts="year" prefix="[" suffix="]"/>
          <date variable="issued">
            <date-part name="year"/>
          </date>
        </group>
      </if>
      <else-if variable="status">
        <text variable="status"/>
      </else-if>
      <else>
        <text term="no date" form="short"/>
      </else>
    </choose>
  </macro>
  <macro name="day-month">
    <date variable="issued">
      <date-part name="month"/>
      <date-part name="day" prefix=" "/>
    </date>
  </macro>
  <macro name="collection-title">
    <choose>
      <if match="none" type="article-journal">
        <choose>
          <if match="none" is-numeric="collection-number">
            <group delimiter=", ">
              <text variable="collection-title" text-case="title"/>
              <text variable="collection-number"/>
            </group>
          </if>
          <else>
            <group delimiter=" ">
              <text variable="collection-title" text-case="title"/>
              <text variable="collection-number"/>
            </group>
          </else>
        </choose>
      </if>
    </choose>
  </macro>
  <macro name="collection-title-journal">
    <choose>
      <if type="article-journal">
        <group delimiter=" ">
          <text variable="collection-title"/>
          <text variable="collection-number"/>
        </group>
      </if>
    </choose>
  </macro>
  <macro name="event">
    <group>
      <text term="presented at" suffix=" "/>
      <text variable="event"/>
    </group>
  </macro>
  <macro name="description">
    <choose>
      <if type="interview">
        <group delimiter=". ">
          <text macro="interviewer"/>
          <text variable="medium" text-case="capitalize-first"/>
        </group>
      </if>
      <else-if type="patent">
        <group delimiter=" " prefix=". ">
          <text variable="authority"/>
          <text variable="number"/>
        </group>
      </else-if>
      <else>
        <text variable="medium" text-case="capitalize-first" prefix=". "/>
      </else>
    </choose>
    <choose>
      <if variable="title" match="none"/>
      <else-if type="thesis personal_communication speech" match="any"/>
      <else>
        <group delimiter=" " prefix=". ">
          <text variable="genre" text-case="capitalize-first"/>
          <choose>
            <if type="report">
              <text variable="number"/>
            </if>
          </choose>
        </group>
      </else>
    </choose>
  </macro>
  <macro name="issue">
    <choose>
      <if type="legal_case">
        <text variable="authority" prefix=". "/>
      </if>
      <else-if type="speech">
        <group prefix=". " delimiter=", ">
          <group delimiter=" ">
            <text variable="genre" text-case="capitalize-first"/>
            <text macro="event"/>
          </group>
          <text variable="event-place"/>
          <text macro="day-month"/>
        </group>
      </else-if>
      <else-if type="article-newspaper article-magazine personal_communication" match="any">
        <date variable="issued" form="text" prefix=", "/>
      </else-if>
      <else-if type="patent">
        <group delimiter=", " prefix=", ">
          <group delimiter=" ">
            <!--Needs Localization-->
            <text value="filed"/>
            <date variable="submitted" form="text"/>
          </group>
          <group delimiter=" ">
            <choose>
              <if variable="issued submitted" match="all">
                <text term="and"/>
              </if>
            </choose>
            <!--Needs Localization-->
            <text value="issued"/>
            <date variable="issued" form="text"/>
          </group>
        </group>
      </else-if>
      <else-if type="article-journal" match="any"/>
      <else>
        <group prefix=". " delimiter=", ">
          <choose>
            <if type="thesis">
              <text variable="genre" text-case="capitalize-first"/>
            </if>
          </choose>
          <text macro="publisher"/>
        </group>
      </else>
    </choose>
  </macro>
  <citation et-al-min="4" et-al-use-first="1" disambiguate-add-year-suffix="true" disambiguate-add-names="true" disambiguate-add-givenname="true" givenname-disambiguation-rule="primary-name" collapse="year" after-collapse-delimiter="; ">
    <layout prefix="(" suffix=")" delimiter="; ">
      <group delimiter=", ">
        <choose>
          <if variable="issued accessed" match="any">
            <group delimiter=" ">
              <text macro="contributors-short"/>
              <text macro="date-in-text"/>
            </group>
          </if>
          <!---comma before forthcoming and n.d.-->
          <else>
            <group delimiter=", ">
              <text macro="contributors-short"/>
              <text macro="date-in-text"/>
            </group>
          </else>
        </choose>
        <text macro="point-locators"/>
      </group>
    </layout>
  </citation>
  <bibliography hanging-indent="true" et-al-min="11" et-al-use-first="7" subsequent-author-substitute="&#8212;&#8212;&#8212;" entry-spacing="0">
    <sort>
      <key macro="contributors"/>
      <key variable="issued"/>
      <key variable="title"/>
    </sort>
    <layout suffix=".">
      <group delimiter=". ">
        <text macro="contributors"/>
        <text macro="date"/>
        <text macro="title"/>
      </group>
      <text macro="description"/>
      <text macro="secondary-contributors" prefix=". "/>
      <text macro="container-title" prefix=". "/>
      <text macro="container-contributors"/>
      <text macro="edition"/>
      <text macro="locators-chapter"/>
      <text macro="collection-title-journal" prefix=", " suffix=", "/>
      <text macro="locators"/>
      <text macro="collection-title" prefix=". "/>
      <text macro="issue"/>
      <text macro="locators-article"/>
      <text macro="access" prefix=". "/>
    </layout>
  </bibliography>
</style>
<<===== CSL =====<<




>>===== INPUT =====>>
[
  {
    "container-title": "Magazine",
    "id": "item1",
    "issued": {
      "date-parts": [
        [
          2012
        ]
      ]
    },
    "title": "Title A",
    "type": "article-magazine"
  },
  {
    "container-title": "Magazine",
    "id": "item2",
    "issued": {
      "date-parts": [
        [
          2012
        ]
      ]
    },
    "title": "Title B",
    "type": "article-magazine"
  },
  {
    "container-title": "Magazine",
    "id": "item3",
    "issued": {
      "date-parts": [
        [
          2012
        ]
      ]
    },
    "title": "Title C",
    "type": "article-magazine"
  },
  {
    "container-title": "Magazine",
    "id": "item4",
    "issued": {
      "date-parts": [
        [
          2012
        ]
      ]
    },
    "title": "Title D",
    "type": "article-magazine"
  },
  {
    "container-title": "Newspaper",
    "id": "item5",
    "issued": {
      "date-parts": [
        [
          2012
        ]
      ]
    },
    "title": "Title E",
    "type": "article-magazine"
  },
  {
    "container-title": "Newspaper",
    "id": "item6",
    "issued": {
      "date-parts": [
        [
          2012
        ]
      ]
    },
    "title": "Title F",
    "type": "article-magazine"
  }
]

<<===== INPUT =====<<


>>===== VERSION =====>>
1.0
<<===== VERSION =====<<

Citeproc: Loss of whitespace in Bibliographies

For styles where there is no default suffix between some fields (e.g. the popular style for Cell https://www.zotero.org/styles/cell), in Pandoc V2.10 a space is placed between the items (DOCX output, look between the journal name and volume number, or between page number and DOI):

However, in Pandoc V2.11 the whitespace is missing and so journal-name>volume and pages>DOI are stuck together:

I also see this for LaTex output:

In the online visual CSL editor: https://editor.citationstyles.org/codeEditor/ — the references are rendered with spaces even when there is no explicit suffix.

--citation-abbreviations=FILE crashes on Zotero abbreviations.json

pandoc 2.11.4
Compiled with pandoc-types 1.22, texmath 0.12.1, skylighting 0.10.2,
citeproc 0.3.0.5, ipynb 0.1.0.1

The --citation-abbreviations=FILE option in Pandoc is incompatible in two ways with the following recent journal title abbreviation JSON file in the Zotero repository:
https://raw.githubusercontent.com/zotero/zotero/master/resource/schema/abbreviations.json

An "info" block at the beginning crashes citeproc.

{
	"info": {
		"URI": "http://www.zotero.org/abbreviations/default.json",
		"name": "MEDLINE + Title Word Abbreviations"
	},
	"default": {
		"container-title": {
			"academia peruana de cirugia": "Acad. Peru. Cir.",

CiteprocParseError: Could not parse abbreviations file https://raw.githubusercontent.com/zotero/zotero/master/resource/schema/abbreviations.json
Error in $.info.URI: parsing Map failed, expected Object, but encountered String

Title word fallback with "container-title-word" is ignored.
The word substitution rules are never applied.
An example list can be found in the second part of the above mentioned JSON file.

		"container-title-word": {
			"aachener": "Aachen.",
			"aakkos-": "aakkos.",

Citations in parentheses have a period before the close parentheses

I find that when I have a citation in the text of a footnote, it is wrapped in parentheses (a good improvement over pandoc-citeproc), but there is a period before the closing parenthesis (like this.).

Test case (test.md) – see the Zotero Style Repository for the CSL style used:

Lorem ipsum dolor sit amet^[Consectetur adipisicing elit: «sed do eiusmod tempor incididunt» [@doe_1989, 15].].

---
csl: universita-pontificia-salesiana.csl
suppress-bibliography: true
references:
- id: doe_1989
  author:
    - family: Doe
      given: John
  issued:
    - year: 1989
  publisher: ABC
  publisher-place: New York
  title: Tempor
  type: book
...

command: pandoc test.md --citeproc -t markdown-citations

result with pandoc 2.11.0.2:

Lorem ipsum dolor sit amet[^1].

[^1]: Consectetur adipisicing elit: «sed do eiusmod tempor incididunt»
    (J. [Doe]{.smallcaps}, *Tempor*, New York, ABC, 1989, 15.).

Translator being appended to author in author-in-text citation

With universita-pontificia-salesiana.csl, an author-in-text citation in a footnote has the contents of the translator macro appended to the author name, e.g., "[John]{.smallcaps}translated by Ferdinand, Works, …". The contents of the translator macro also appear later in the proper place. See below for a case that reproduces this.

Using a normal citation in the body text instead of a footnote, this does not happen—I'm using that as a workaround. I cannot get it to happen in the visual CSL editor with any of the locator options it offers.

The author macro does not use the translator as a substitute, and in this case, no substitute is needed. In the citation layout, the author macro is followed immediately by the recipient macro, but this doesn't use the translator variable or macro either.

Test case in shelltestrunner format (input, $ command, > expected output):

«Lorem ipsum»^[The translations are taken from @john_works_2001].

---
suppress-bibliography: true
notes-after-punctuation: false
references:
- id: john_works_2001
  author:
    - literal: John
  issued:
    - year: 2001
  publisher: ABC
  publisher-place: New York
  title: Works
  translator:
    - literal: Ferdinand
  type: book
...
$ pandoc -f markdown --csl universita-pontificia-salesiana.csl --citeproc -t markdown-citations
>
«Lorem ipsum»[^1].

[^1]: The translations are taken from [John]{.smallcaps}, *Works*,
    translated by Ferdinand, New York, ABC, 2001.

Result with pandoc 2.11.3.1:

Expected stdout: 
«Lorem ipsum»[^1].

[^1]: The translations are taken from [John]{.smallcaps}, *Works*,
    translated by Ferdinand, New York, ABC, 2001.

Got stdout:      
«Lorem ipsum»[^1].

[^1]: The translations are taken from [John]{.smallcaps}translated by
    Ferdinand, *Works*, translated by Ferdinand, New York, ABC, 2001.

CSL JSON rendering and entities

Currently in rendering CSL JSON, we escape <, >, and & using numerical entities.
This is correct for the HTML output expected by the test suite.
But it isn't correct if we're generating CSL JSON (e.g. in pandoc -t csljson). For that purpose &, <, and > should be unescaped.
How to handle this? (We currently use renderCslJson for both purposes.)

Case-insensitive citation abbreviations

Hey,
this is actually an old issue in pandoc-citeproc.
I checked yesterday and it is indeed still present. Basically, if the abbreviations.json file has "Modern Pathology": "Mod Pathol", but your bibtex entry is journal={Modern pathology} (lowercase P), it won't get abbreviated.

I'm no haskell expert, but if you point me towards the right place in the code, I might try drafting a PR.

Thanks in advance!

Missing delimiter between year and disambiguation character

I'm finishing up my thesis and I find the generated citation format rather odd.

Rendered citation

Rendered bibliography

When the reference date is missing and there's multiple bibliography entries with the same Author, the "no date" is rendered as "no datea"/"no dateb" and so on. The character used for disambiguation should be delimited somehow. With years it makes sense but witout the year, it looks like a typo.

Is there any workaround I could use to put a space or a semicolon there? To get (SAP, nedatováno;a) instead?

Integration into other Programs?

I was wondering if a longer term goal was to allow your citeproc library to be used from other applications? Current citeproc variants are not so easy to embed into other apps as they depend on a JS engine or Java etc. Sorry for my absolute ignorance, but can Haskell libraries be interfaced to programs written in other languages, and what would be required to do so?

Thank you!

Title case and parentheses

% citeproc --style ../styles/chicago-author-date.csl        
{
  "references":
[
  {
    "author": [
      {
        "family": "Doe",
        "given": "John"
      }
    ],
    "id": "doeA",
    "title": "Fruits (apples, pears, oranges) and veggies",
    "type": "book"
  }
]
}

output:

{
  "citations": [],
  "bibliography": [
    [
      "doeA",
      "Doe, John. n.d. <i>Fruits (apples, Pears, Oranges) and Veggies</i>."
    ]
  ],
  "warnings": []
}

We should have "(Apples" not "(apples".

[citeproc] Regression with respect to duplicate references

Following up on jgm/pandoc#6951 and 7d799bfcda749e4a3ad6fcae59ac5ccb80b77ffd

Pandoc 2.11

foo.md:

[@foo]

---
references:
  - id: foo
    title: inline
...

index.yaml:

references:
  - id: foo
    title: external

With Pandoc 2.9:

pandoc foo.md --bibliography index.yaml
<p><span class="citation" data-cites="foo">(“Inline,” n.d.)</span></p>
<div id="refs" class="references hanging-indent" role="doc-bibliography">
<div id="ref-foo">
<p>“Inline.” n.d.</p>
</div>
</div>

At 7d799bfcda749e4a3ad6fcae59ac5ccb80b77ffd, it seems to use "external", although it also produces multiple entries:

pandoc foo.md --citeproc --bibliography index.yaml
<p><span class="citation" data-cites="foo">(<span>“External,”</span> n.d.)</span></p>
<div id="refs" class="references csl-bib-body hanging-indent" role="doc-bibliography">
<div id="ref-foo" class="csl-entry" role="doc-biblioentry">
<p><span>“External.”</span> n.d.</p>
</div>
<div id="ref-foo" class="csl-entry" role="doc-biblioentry">
<p>———. n.d.</p>
</div>
</div>

Similarly, with multiple --bibliography arguments.
index2.yaml:

references:
  - id: foo
    title: external2

With Pandoc 2.9:

pandoc foo.md --bibliography index.yaml --bibliography index2.yaml
<p><span class="citation" data-cites="foo">(“External,” n.d.)</span></p>
<div id="refs" class="references hanging-indent" role="doc-bibliography">
<div id="ref-foo">
<p>“External.” n.d.</p>
</div>
</div>

At 7d799bfcda749e4a3ad6fcae59ac5ccb80b77ffd:

pandoc foo.md --citeproc --bibliography index.yaml --bibliography index2.yaml
<p><span class="citation" data-cites="foo">(<span>“External2,”</span> n.d.)</span></p>
<div id="refs" class="references csl-bib-body hanging-indent" role="doc-bibliography">
<div id="ref-foo" class="csl-entry" role="doc-biblioentry">
<p><span>“External2.”</span> n.d.</p>
</div>
<div id="ref-foo" class="csl-entry" role="doc-biblioentry">
<p>———. n.d.</p>
</div>
</div>

Punctuation merge at the end of citation suffix doesn’t work with unicode last characters

Hi, out of gratitude for this great piece of software I have tried a little bit to understand how Haskell works , but still I’m not so much familiar with it … I think this is the piece of code telling if a final dot should be added to a citation or not, isn’t it? Is it possible that it doesn’t catch cases where the final character isn’t an ASCII but a Unicode character?

https://github.com/jgm/pandoc/blob/68b298ed9aee405033da9a2b44ae86f2241a123d/src/Text/Pandoc/Citeproc.hs#L394-L405

I think, the merging of punctuation at the end of a citation suffix doesn’t work with unicode last characters, however, it works with ASCII characters. Maybe that (d:c:_) doesn’t allow _ to be a unicode character? At least in the LuaLaTeX/BibLaTeX/Biber pipeline I don’t get extra dots after footnote citations ending with a curly quote character.

It is difficult to provide a small MWE, because it depends on the CSL – the standard output is in parentheses, without a dot suffix. For the MWE, I will use the CSL provided at https://www.zotero.org/styles?q=id%3Auniversitat-freiburg-geschichte and call the file footnote_style.csl.

For testing purposes, I am running the following LaTeX code through the following command, using yesterdays’s nightly build, as in the current version, 2.11.2, there would even be brackets around the suffix as resolved in 9a40976 and 7db2cf5.

This doesn’t have an additional dot.\cite[Vgl.][3, and this is important!]{introduction}
But this has.\cite[Vgl.][3: „some words from the citation.“]{introduction}

pandoc -f latex -C --bibliography pandoc_test.bib --csl footnote_style.csl -t native

I am getting this output …

[Para [Str "This",Space,Str "doesn\8217t",Space,Str "have",Space,Str "an",Space,Str "additional",Space,Str "dot.",Cite [Citation {citationId = "introduction", citationPrefix = [Str "Vgl."], citationSuffix = [Str "3,",Space,Str "and",Space,Str "this",Space,Str "is",Space,Str "important!"], citationMode = NormalCitation, citationNoteNum = 0, citationHash = 0}] [Note [Para [Str "Vgl.",Space,Strong [Str "Editor,",Space,Str "Emil"],Str ":",Space,Str "Introduction",Space,Str "to",Space,Str "the",Space,Str "Essays,",Space,Str "in:",Space,Emph [Str "Herausgeber,",Space,Str "Herbert/Editor,",Space,Str "Emil",Space,Str "(Hgg.)"],Str ":",Space,Str "The",Space,Str "Ultimate",Space,Str "TeXnic",Space,Str "Bibliographer,",Space,Str "Edinburgh",Space,Str "2000,",Space,Str "S.",Space,Str "3\8211&17,",Str "",Space,Str "S.",Space,Str "3,",Space,Str "and",Space,Str "this",Space,Str "is",Space,Str "important",Str "!"]]],Space,Str "But",Space,Str "this",Space,Str "has.",Cite [Citation {citationId = "introduction", citationPrefix = [Str "Vgl."], citationSuffix = [Str "3:",Space,Str "\8222some",Space,Str "words",Space,Str "from",Space,Str "the",Space,Str "citation.\8220"], citationMode = NormalCitation, citationNoteNum = 0, citationHash = 0}] [Note [Para [Str "Vgl.",Space,Str "ebd.,",Space,Str "S.",Space,Str "3:",Space,Str "\8222",Str "some",Space,Str "words",Space,Str "from",Space,Str "the",Space,Str "citation",Str ".",Str "\8220."]]]]
,Div ("refs",["references","csl-bib-body"],[])
[Div ("ref-introduction",["csl-entry"],[])
[Para [Strong [Str "Editor,",Space,Str "Emil"],Str ":",Space,Str "Introduction",Space,Str "to",Space,Str "the",Space,Str "Essays,",Space,Str "in:",Space,Emph [Str "Herausgeber,",Space,Str "Herbert/Editor,",Space,Str "Emil",Space,Str "(Hgg.)"],Str ":",Space,Str "The",Space,Str "Ultimate",Space,Str "TeXnic",Space,Str "Bibliographer,",Space,Str "Edinburgh",Space,Str "2000,",Space,Str "S.",Space,Str "3\8211&17."]]]]

The important aspect is that the second suffix is taken as ,Str "\8222",Str "some",Space,Str "words",Space,Str "from",Space,Str "the",Space,Str "citation",Str ".",Str "\8220."
I would expect there not to be a dot after \8220, parallel to the way BibLaTeX treats the situation: ,Str "\8222",Str "some",Space,Str "words",Space,Str "from",Space,Str "the",Space,Str "citation",Str ".",Str "\8220"

pandoc_test.bib

@incollection{introduction,
	author = {Emil Editor},
	crossref = {collection},
	pages = {3–17},
	title = {Introduction to the Essays}}

@collection{collection,
	address = {Edinburgh},
	editor = {Herbert Herausgeber and Emil Editor},
	title = {The Ultimate \TeX nic Bibliographer},
	year = {2000}}

Suppress author name

Before the new citeproc protocol one was able to suppress the author name of a citation by adding a dash before the bibliography entry, e.g., -@Smith04. With the new protocol this ends up in LaTeX as: -Smith (2004). Is this a bug?

Can't find CSL in ~/.csl directory or follow relative paths to CSL files.

hi there! So this very old issue for the previous pandoc-citeproc filter suggests that that earlier iteration looked in ~/.csl for style files.

I currently have a ~/.csl directory sitting around containing all 2000-some CSL styles from the central CSL repository. Including, importantly, chicago-author-date.csl. But I can't for the life of me figure out how to actually get citeproc to find that repository.

I started with a defaults file in ~/.pandoc/defaults/research.yaml that contains, in relevant part,

citeproc: true
csl: chicago-author-date

but running pandoc -o testres.pdf --defaults=research markdown.md yields the error File chicago-author-date.csl not found in resource path.

Then I changed it to a full relative pathname: csl: ~/.csl/chicago-author-date.csl

But, same error.

Finally, I gave it a full absolute path, i.e., csl: /Users/myhome/.csl/chicago-author-date.csl and that worked.

Similar errors happen passing relative paths directly from the commandline rather than in a defaults file.

I'm using Pandoc 2.11.4 (installed via homebrew on on Macos catalina, if it matters).

Regression: chicago-fullnote-bibliography.csl: First note is not "full"

Formatting with chicago-fullnote-bibliography.csl, the first (foot-)note does not include the "full" details of the citation, just the short form.

Note that with a single citation per note this works as expected, but with two or more citations in a note, the "full" details are no longer rendered in the first note, as shown below. Subsequent notes include a year, too, which is unexpected as well.

pandoc -C -t markdown-citations <<EOT

Blah [@doe1; @doe2].

More blah [@doe1; @doe2].


# References {-}

---
csl: chicago-fullnote-bibliography.csl
references:
- id: doe1
  type: article
  author:
  - family: Doe
    given: John
  DOI: 10/abcde
  issued:
    - year: 2020
  title: An article
- id: doe2
  type: article
  author:
  - family: Doe
    given: John
  DOI: 10.1109/5.771073
  issued:
    - year: 2020
  title: Another article
...
EOT

Actual output:

Blah.[^1]

More blah.[^2]

References {#references .unnumbered}
==========

::: {#refs .references .csl-bib-body .hanging-indent}
::: {#ref-doe1 .csl-entry}
Doe, John. "An Article," 2020.
https://doi.org/[10/abcde](https://doi.org/10/abcde).
:::

::: {#ref-doe2 .csl-entry}
---------. "Another Article," 2020.
https://doi.org/[10.1109/5.771073](https://doi.org/10.1109/5.771073).
:::
:::

[^1]: Doe, "An Article," 2020; Doe, "Another Article," 2020.

[^2]: Doe, "An Article," 2020; Doe, "Another Article," 2020.

Expected = "old" pandoc (using pandoc -F pandoc-citeproc -t markdown-citations):

Blah.[^1]

More blah.[^2]

References {#references .unnumbered}
==========

::: {#refs .references .hanging-indent}
::: {#ref-doe1}
Doe, John. "An Article," 2020. <https://doi.org/10/abcde>.
:::

::: {#ref-doe2}
---------. "Another Article," 2020. <https://doi.org/10.1109/5.771073>.
:::
:::

[^1]: John Doe, "An Article," 2020, <https://doi.org/10/abcde>; John
    Doe, "Another Article," 2020, <https://doi.org/10.1109/5.771073>.

[^2]: Doe, "An Article"; Doe, "Another Article."

CiteprocLocaleNotFound: C.UTF-8 when running pandoc --from=biblatex --to=csljson on GitHub Actions

Using Pandoc 2.11.1.1 on a GitHub Actions ubuntu-18.04 image, I get an error CiteprocLocaleNotFound: C.UTF-8 when running the following command (CI log):

pandoc --from=biblatex --to=csljson /home/runner/work/manubot/manubot/manubot/pandoc/tests/bibliographies/bibliography.bib

The same command on a GitHub Actions build on macos-10.15 completes successfully.

This issue is reminiscent of jgm/pandoc-citeproc#420

Handling of extra spaces in initialize-with attribute

As for now, CSL doesn't provide a way to set the separator between the given name and the family name in normal order similar to sort-separator. Alas, some journals require a non-breakable space between the (abbreviated) given name and the family name.

With Zotero, as a workaround one can insert this non-breakable space in the initialize-with attribute like this:

<?xml version="1.0" encoding="utf-8"?>

<style xmlns="http://purl.org/net/xbiblio/csl" class="in-text" version="1.0.1" name-form="long" initialize-with=". "> <!-- with a non-breakable space after the dot in initialize-with value -->
    <citation>
    <layout>
      <names variable="author">
        <name>
          <name-part name="given" form="short"/>
        </name>
      </names>
    </layout>
    </citation>
    <bibliography/>
</style>

As there already is a (non-breakable) space in the initialize-with attribute, citeproc-js doesn't output the regular space that it would otherwise insert between the two name parts. Citeproc does the opposite, as shows the following output in LaTeX format using the CSL file above and a dummy bibliographic item containing only an author field:

pandoc --citeproc -t latex << EOF 

@minimal

--- 
csl: minimal.csl 
references: 
- id: minimal
  type: book
  author: 
    - family: Family
      given: Given
suppress-bibliography: true
... 
EOF

Produced output:

[WARNING] Citeproc: Bibliography entry with no printed form: minimal
G. Family

Desired output:

[WARNING] Citeproc: Bibliography entry with no printed form: minimal
G.~Family

Would it be possible to implement this behaviour? Or alternatively, to implement something like a --csl-strict or --normalize-spaces=false option, which would cause to output G.~ Family, allowing for easy post-processing?

Properly implement by-cite disambiguation

See discussion at citation-style-language/test-suite#41

Currently we're doing this too "locally."
"'by-cite' refers to using names for flip-to-the-back disambiguation just like the other methods. It means "don't do global name disambiguation before starting flip-to-the-back"."

Citeproc hangs when handling a BibTeX item with an attribute ending in a space

This issue was first reported as part of issue jgm/pandoc#7060, but deemed unrelated.

When I run the following command:

pandoc --from markdown --to html --citeproc --bibliography bib.bib --csl apa.csl -o extra_space.html extra_space.md

with bib.bib:

@InProceedings{Cole2014a,
  Title                    = {{Toward an argumentative grammar for socio-cultural/cultural-historical activity approaches to design research}},
  Author                   = {Cole, Michael and Engestr{\"o}m, Yrj{\"o} and Sannino, Annalisa and Guti{\'e}rrez, Kris and Jurow, Susan and Packer, Martin and Penuel, William R. and Johnson, Raymond and Severance, Samuel and Leary, Heather and Miller, Susan},
  Booktitle                = {{ICLS 2014 Proceedings }},
  Year                     = {2014},
  Pages                    = {1254--1263}
}

and extra_space.md:

BibTeX / Citeproc hangs when converting the reference: [@Cole2014a]

and apa.csl as attached, pandoc hangs. This happens with pandoc 2.11.2 and 2.11.4.

Workaround: The issue is caused by the Booktitle attribute ending with a space. Removing this space resolves the issue.

textsc in BibLaTeX author field ignored

In the new citeproc protocol pandoc will pick up text in a LaTeX textsc environment in a number of fields in a BibLaTeX entry, but not in the author field — unlike the old citeproc. Maybe I have missed something, maybe it is a bug. See the example below.

@online{Councileu2021Minute,
    author       = {{Council \textsc{eu}}},
    title        = {{Council \textsc{eu} Draft Minutes 22 February 2021}},
    year         = {2021},
    url          = {https://data.consilium.europa.eu/doc/document/ST-6435-2021-INIT/en/pdf},
}

Earlier

Council [eu]{.smallcaps}. 2021. 'Council [eu]{.smallcaps} Draft Minutes 22 February 2021'. 2021. https://data.consilium.europa.eu/doc/document/ST-6435-2021-INIT/en/pdf.

Recently

Council eu. 2021. 'Council [eu]{.smallcaps} Draft Minutes 22 February 2021'. 2021. https://data.consilium.europa.eu/doc/document/ST-6435-2021-INIT/en/pdf.

Entity handling in CSL Json

In the CslJson module, make sure we handle entities properly in parsing and rendering.
(Check citeproc-js behavior.)

Restrict automatic handling of quotation marks to ASCII " and ' when the "language" CSL field doesn't match the document's "lang" metadata

As Pandoc doesn't seem to format the quotation marks according to the lang variable, I have been used to type the french quotation marks (outer « and », inner “ and ”). The only problem with that is that citeproc does format the quotations marks according to the locale, so that my inner quotation marks (“ and ”) were changed to outer quotation marks (« and ») in citations. This occurred in a nested quotation, but it can also alter the rendering of foreign titles whose original typography one could wish or have to retain (like: Noah Yuval-Hacham, “You Shall Not Make for Yourself Any Graven Image...”: On Jewish Iconoclasm in Late Antiquity, Ars Judaica 6).

Would it be possible to restrict marks reformatting to ASCII characters " and ', like citeproc.js does? As far as I know, citeproc does also reformat the “” and ‘’ pairs. If so, with a lang variable set to fr-FR, the string “You Shall Not Make for Yourself Any Graven Image...” would be rendered as is, whereas "You Shall Not Make for Yourself Any Graven Image..." would be rendered as « You Shall Not Make for Yourself Any Graven Image... ».

How to use multiple citations as a unified subjective of a sentence?

Thanks for your amazing work!

The Problem

I have a sentence where 2 citekeys with the same author were inserted as the subjective, the expected output should be:

Coser (1956, 1957) argues that conflict can serve a positive role for the social relationship and social structure…

And the BibTeX file contents in ref.bib:

@book{coser1956,
  title = {The {{Functions}} of {{Social Conflict}}},
  author = {Coser, Lewis A.},
  year = {1956},
  pages = {188},
  publisher = {{The Free Press}},
  address = {{New York}},
  series = {The Functions of Social Conflict}
}

@article{coser1957,
  title = {Social {{Conflict}} and the {{Theory}} of {{Social Change}}},
  author = {Coser, Lewis A.},
  year = {1957},
  volume = {8},
  pages = {197--207},
  publisher = {{Wiley, London School of Economics and Political Science}},
  doi = {10.2307/586859},
  journal = {The British Journal of Sociology},
  number = {3}
}

I have tried to write in the Markdown file input.md:

---
title: This is the Title
bibliography: ref.bib
csl: gbt7714-en.csl
---

@coser1956 -@coser1957 argues that conflict can serve a positive role for the social relationship and social structure…

Run Pandoc command in terminal:

$ pandoc -C input.md -o output.docx

But it rendered with 2 separate parentheses wrapped the year in output.docx:

Coser (1956) (1957) argues that conflict can serve a positive role for the social relationship and social structure…

Not the expected:

Coser (1956, 1957) argues that conflict can serve a positive role for the social relationship and social structure…

Questions

Whether it's possible to write raw LaTeX command in Markdown to make a workaround like:

\citep{coser1956, coser1957} argues that conflict can serve a positive role for the social relationship and social structure…

And then convert the Markdown file via Pandoc to .docx, it should be like the ideal format:

Coser (1956, 1957) argues that conflict can serve a positive role for the social relationship and social structure…

I'm not sure if it’s possible. Or could you please provide alternative solutions for this issue? Thank you 😊

subsequent-author-substitute ("———") wrongly appears inside bibliography item

Relevant conditions: chapter item where author and editor are identical; chapter needs to appear in subsequent position, i.e., after at least one other item by the same author.

MWE:

pandoc -C -t plain <<EOT

Blah [@doe1; @doe2].

# References {-}

---
references:
- id: doe1
  type: book
  author:
  - family: Doe
    given: John
  issued:
    - year: 2020
  title: An anthology
- type: chapter
  id: doe2
  author:
    - family: Doe
      given: John
  container-title: An anthology
  editor:
    - family: Doe
      given: John
  issued:
    - year: 2020
  page: 63–79
  publisher: A publisher
  publisher-place: A place
  title: Some chapter  
...
EOT

Expected = output of pandoc’s current main branch:

Blah (Doe 2020a, 2020b).

References

Doe, John. 2020a. An Anthology.

———. 2020b. “Some Chapter.” In An Anthology, edited by John Doe, 63–79.
A place: A publisher.

Actual citeproc branch output:

Blah (Doe 2020a; 2020b).

References

Doe, John. 2020a. An Anthology.

———. 2020b. “Some Chapter.” In An Anthology, ———edited by, 63–79. A
place: A publisher.

Disambiguation error

I'm referring to this discussion on the mailing list. After checking it looks like citeproc's interpretation of disambiguate="true" is not correct.

This here should show the issue:

--- 
title: Test
csl: chicago-note-bibliography.csl
references:
- type: book
  id: doe1
  author:
  - family: Doe
    given: John
  issued:
    date-parts:
    - - 2020
      - 12
      - 1
  title: 'A first book by John Doe'
- type: book
  id: doe2
  author:
  - family: Doe
    given: John
  issued:
    date-parts:
    - - 2021
      - 1
      - 1
  title: 'A second book by John Doe'
- type: book
  id: smith
  author:
  - family: Smith
    given: Jane
  issued:
    date-parts:
    - - 2021
      - 1
      - 1
  title: 'A first book by Jane Smith'
---

Test [@doe1]
Test [@doe2]
Test [@smith]

I'm converting with: pandoc file.md -o file.html --citeproc

Result is:

Weird comma in front of the page using DIN1505

Hi there,

I am using the https://github.com/citation-style-language/styles/blob/master/din-1505-2.csl for my citations.
I always get a wierd comma in front of my year and I have no idea how to get rid of it

the citation in the markdownfile (in an footnote):
[^3f261b3e]: @Spitau2019

the bibtex entry

@Book{Spitau2019,
author    = {Spitau, Marcel},
title     = {Papier Im Griff - Vom Informationsbesitzer Zum Informationsbenutzer Mit Classei},
publisher = {Independently Published},
isbn      = {978-1-092-11316-8},
address   = {Printed in the United States of America},
year      = {2019},
}

And this is how it looks like...
https://i.imgur.com/RYI68U0.png

I do not understand WHY there is the comma and I want to get rid of it.

Any ideas?

pandoc 2.11.3
Compiled with pandoc-types 1.22, texmath 0.12.1, skylighting 0.10.2,
citeproc 0.3.0.3, ipynb 0.1.0.1

This is my commandline to get an PDF:

pandoc md-Mein-Buch.md -s --bibliography /home/marcel/Dropbox/1_arbeit/zettelkasten/literatur.bib --pdf-engine=xelatex --citeproc --csl /home/marcel/.pandoc/din-1505-2.csl -o pdf-Mein-Buch.pdf

Bold formatting seems to break punctuation moving and merging

I have the following example:

--- 
title: Test
csl: chicago-note-bibliography-always-short-with-label.csl
references:
- type: article-journal
  id: doe1
  author:
  - family: Doe
    given: John
  issued:
    date-parts:
    - - 2020
      - 12
      - 1
  title: 'A first article by John Doe'
- type: article-journal
  id: doe2
  author:
  - family: Doe
    given: John
  issued:
    date-parts:
    - - 2020
      - 12
      - 1
  title: 'A second article by John Doe?'
---

Test [@doe1]
Test [@doe2]

Style: chicago-note-bibliography-always-short-with-label.zip

The resulting bibliography looks like this:

As you can see the punctuation stays outside the quotes, and they don't get merged. If I remove the font-weight="bold" on line 1387 everything behaves as expected. I don't know if that is on purpose, but bold formatting seems to prevent punctuation moving and merging.

Punctuation moving mechanism

I've brought up this issue a while ago on the pandoc-citeproc issue tracker: While the punctuation moving mechanism is quite nice, it's also not sufficient in some situations. To deal with this I've built a lua filter that essentially works in the opposite direction. Instead of moving punctuation inside the quotes from after the citation, it extracts the punctuation from the quotation and moves it after the citation.

Would it be possible to include such a feature here, or is that out of scope?

comma instead of semicolon with collapsing and locator on second citation

The Chicago Manua of Style requires that multiple sources are separated by semicolons, rather than commas, if pages are given for at least one of them.

The way to achieve this behaviour used to be to prefix the semicolon that separates multiple sources in the Markdown source code by \;. Pandoc then omitted the comma. It no longer does.

pandoc -C -t plain <<EOF
---
references:
    - type: book
      id: one
      author: Jane Doe
      issued:
          date-parts:
              - - 2010
      title: The joy of citations
    - type: book
      id: two
      author: Jane Doe
      issued:
          date-parts:
              - - 2020
      title: Citations in times of COVID-19
---

P [@one\;; @two]!
EOF

This outputs:

P (Jane Doe 2010;, 2020)!
[...]

The output I expected is:

P (Jane Doe 2010; 2020)!
[...]

I am using pandoc v2.11.2 on macOS 11.1.

If somebody else stumbles upon this thread, this Lua filter replaces ";," by ";" in citations:

function Cite (cite)
    for i = 1, #cite.c do
        local els = cite.c[i]
        for j = 1, #els do
            local el = els[j]
            if el.t == 'Str' then el.c = el.c:gsub(';,', ';') end
        end
    end
    return cite
end

Thanks a for looking into this.

Multiple bibliographies

Now, with the new citeproc library being more tightly integrated into pandoc proper, would that open ways to new features like multiple bibliographies? I'm thinking of having one bibliography with primary sources, another with secondary sources; or, per-section bibliographies, etc.

Field value ignored with using "raw" date type with CSL-YAML

When using CSL-YAML the following record with date field works as expected:

---
references:
- id: agenda
  title: "Meeting agenda"
  author:
  - literal: "Institution XYZ"
  issued: [[2020,2,11]]
...

However, when trying to use a "raw" date format, the value of "issued" segment is ignored:

---
references:
- id: agenda
  title: "Meeting agenda"
  author:
  - literal: "Institution XYZ"
  issued:
  - raw: "2020-2-11"
...

If my understanding is correct the case above should work in the same way as it does in CSL-JSON, e.g.:

"issued": {
     "raw": "2020-2-11"
}

Regression: DOIs with "https://doi.org/" prefix

Many style guides, including the Chicago Manual, require DOIs in bibliographies to be shown with a https://doi.org/ prefix. Unlike the "old" pandoc, the "new" pandoc from the "master" branch, source downloaded and self-compiled earlier today, outputs this prefix, but does not include it in the actual link. The DOI part itself is still clickable, but the appearance is weird, in particular if the DOI part but not the prefix is coloured and/or underlined.

MWE, including both a regular DOI and a shortDOI:

pandoc -C -t markdown-citations <<EOT

Blah [@doe1; @doe2].

# References {-}

---
references:
- id: doe1
  type: article
  author:
  - family: Doe
    given: John
  DOI: 10/abcde
  issued:
    - year: 2020
  title: An article
- id: doe2
  type: article
  author:
  - family: Doe
    given: John
  DOI: 10.1109/5.771073
  issued:
    - year: 2020
  title: Another article
...
EOT

Actual output:

Blah (Doe 2020a, 2020b).

References {#references .unnumbered}
==========

::: {#refs .references .csl-bib-body .hanging-indent}
::: {#ref-doe1 .csl-entry}
Doe, John. 2020a. "An Article."
https://doi.org/[10/abcde](https://doi.org/10/abcde).
:::

::: {#ref-doe2 .csl-entry}
---------. 2020b. "Another Article."
https://doi.org/[10.1109/5.771073](https://doi.org/10.1109/5.771073).
:::
:::

Expected output = output from the "old" pandoc:

Blah (Doe 2020a, 2020b).

References {#references .unnumbered}
==========

::: {#refs .references .hanging-indent}
::: {#ref-doe1}
Doe, John. 2020a. "An Article." <https://doi.org/10/abcde>.
:::

::: {#ref-doe2}
---------. 2020b. "Another Article." <https://doi.org/10.1109/5.771073>.
:::
:::

Same effect with latex and html output formats.

While we’re at it, something else that might be improved is one detail concerning the URL form of shortDOIs:

https://www.doi.org/resources.html states: ”The shortDOI Service is a public service for creating shortcuts to DOI names, which can be very long strings. The service creates short handles of the form 10/abcde and enables short HTTP URIs of the form http://doi.org/abcde that are ideal for use in email, blogs, mobile messaging and more.”

So the shortest URL form of a shortDOI is https://doi.org/abcde rather than https://doi.org/10/abcde. Would it be possible for pandoc to remove, only for shortDOIs, and only if the CSL style used requests the use of the URL form, the 10/ part?

(As to distinguishing regular DOIs and shortDOIs: shortDOIs invariably begin with 10/ whereas regular DOIs start with 10., see here.)

Can support for bibtex's "and others" be improved?

BibTeX allows to use the special string and others in author names, which is rendered as "et al.". Pandoc's bibtex parser already treats this specially

toName _ [Str "others"] =
  return emptyName{ nameLiteral = Just "others" }

while citeproc seems to treat that name just as any other. Is it possible (and sensible) to mirror bibtex's behavior here? If so, where would be a good place to put the code?

For reference: this issue is motivated by openjournals/joss-reviews#3000.

UTF-8 BOM encoded CSL files

Not sure if this is expected behaviour, I only saw it as one of my CSL files was failing while another was fine and I found this. If a CSL file is saved as UTF-8 + BOM, it works fine when used via pandoc-citeproc:

➜ pandoc --bibliography testbib.json --csl cell.csl test.md
<h1 id="introduction">Introduction</h1>
<p>Here is <em>some</em> text <span class="citation" data-cites="drew2019">(Drew, 2019)</span> to <strong>test</strong> Lua debugging…</p>
<div id="refs" class="references" role="doc-bibliography">
<div id="ref-drew2019">
<p>Drew, L. (2019). The mouse in the video game. Nature <em>567</em>, 158–160.</p>
</div>
</div>

But when the same CSL file is used by citeproc there is an error:

➜ citeproc -s cell.csl -r testbib.json testcite.json
CiteprocXMLError: 1:1 (0)-1:2 (1): Found content after root element: EventContent (ContentText "\65279")

Resaving the CSL without the BOM and then citeproc works fine:

➜ citeproc -s cell.csl -r testbib.json testcite.json
{
  "citations": [
    "(Drew, 2019)",
    "(Bertalmío <i>et al.</i>, 2020)"
  ],
  "bibliography": [
    [
      "bertalmio2020",
      "Bertalmío, M., Gomez-Villa, A., Martín, A., Vazquez-Corral, J., Kane, D., and Malo, J. (2020). Evidence for the intrinsically nonlinear nature of receptive fields in vision.Scientific Reports<i>10</i>, 16277."
    ],
    [
      "cavanagh2020",
      "Cavanagh, S.E., Lam, N.H., Murray, J.D., Hunt, L.T., and Kennerley, S.W. (2020). A circuit mechanism for decision-making biases and nmda receptor hypofunction.eLife<i>9</i>."
    ],
    [
      "drew2019",
      "Drew, L. (2019). The mouse in the video game.Nature<i>567</i>, 158–160."
    ]
  ],
  "warnings": []
}

Resaving at UTF8 is an easy fix, but just thought I'd mention it here.

Tested with this style: https://www.zotero.org/styles/cell

And this json for citeproc command:

{
	"citations": [
		{
			"citationItems": [
				{
					"id": "drew2019"
				}
			]
		},
		{
			"citationItems": [
				{
					"id": "bertalmio2020"
				}
			]
		}
	]
}

JSON bibliography:

[
  {
    "DOI": "10.1038/d41586-019-00791-w",
    "author": [
      {
        "family": "Drew",
        "given": "Liam"
      }
    ],
    "container-title": "Nature",
    "id": "drew2019",
    "issue": "7747",
    "issued": {
      "date-parts": [
        [
          2019
        ]
      ]
    },
    "keyword": "VR; virtual reality; mice",
    "page": "158-160",
    "title": "The mouse in the video game",
    "type": "article-journal",
    "volume": "567"
  },
  {
    "DOI": "10.1038/s41598-020-73113-0",
    "author": [
      {
        "family": "Bertalmío",
        "given": "M."
      },
      {
        "family": "Gomez-Villa",
        "given": "A."
      },
      {
        "family": "Martín",
        "given": "A."
      },
      {
        "family": "Vazquez-Corral",
        "given": "J."
      },
      {
        "family": "Kane",
        "given": "D."
      },
      {
        "family": "Malo",
        "given": "J."
      }
    ],
    "container-title": "Scientific reports",
    "id": "bertalmio2020",
    "issue": "1",
    "issued": {
      "date-parts": [
        [
          2020
        ]
      ]
    },
    "page": "16277",
    "title": "Evidence for the intrinsically nonlinear nature of receptive fields in vision",
    "type": "article-journal",
    "volume": "10"
  },
  {
    "DOI": "10.7554/elife.53664",
    "author": [
      {
        "family": "Cavanagh",
        "given": "Sean Edward"
      },
      {
        "family": "Lam",
        "given": "Norman H."
      },
      {
        "family": "Murray",
        "given": "John D."
      },
      {
        "family": "Hunt",
        "given": "Laurence Tudor"
      },
      {
        "family": "Kennerley",
        "given": "Steven Wayne"
      }
    ],
    "container-title": "eLife",
    "id": "cavanagh2020",
    "issued": {
      "date-parts": [
        [
          2020
        ]
      ]
    },
    "title": "A circuit mechanism for decision-making biases and nmda receptor hypofunction",
    "type": "article-journal",
    "volume": "9"
  }
]

Treatment of unknown variables / non-strict mode

Currently, this library ignores unknown variables. Citeproc-js just treats them as string variables. While that probably should not be the default, what about a non-strict mode where unknown variables are just treated as string variables?

Citation sorting for numeric style

When using a numeric style like nature.csl for citations, the references are not cited by order of citation. Instead they are cited by the order in the .bib file that I use. That seems broken and not respecting the CSL. See the screenshot of the HTML output that I get.

The files that I used to test this are below and the nature.csl file is the latest. The compilation command was:

pandoc --standalone --citeproc -o test.html test.md

The output of pandoc --version is,

pandoc 2.11
Compiled with pandoc-types 1.22, texmath 0.12.0.3, skylighting 0.10.0.2

`test.html`

`test.md`

---
title: Citation sorting test
author:
    - Some Author
    - Some Coauthor
bibliography: [./refs.bib]
csl: [./nature.csl]
link-citations: true
---

# Main text

This is some text [@Mandelbrot:1983kx;@Virot2020].
There are many citations in this sample text [@Biewener1989aa], but none of them are related to the text [@Mcmahon1973].

There is some more text [@Coleman1998] that refers to more citations [@Biewener1989aa].
The problem [@Niklas:1994eu] is that the citations are not sorted by their order of appearance [@Alexander1979aa].

`refs.bib`

@article{Mcmahon1973,
	author = {McMahon, Thomas},
	date-added = {2021-02-08 09:18:47 -0500},
	date-modified = {2021-02-08 09:18:47 -0500},
	journal = {Science},
	number = {4079},
	pages = {1201--1204},
	publisher = {American Association for the Advancement of Science},
	title = {Size and shape in biology: elastic criteria impose limits on biological proportions, and consequently on metabolic rates},
	volume = {179},
	year = {1973}}

@article{Alexander1979aa,
	author = {Alexander, R and Jayes, AS and Maloiy, GMO and Wathuta, EM},
	date-added = {2021-02-08 09:18:47 -0500},
	date-modified = {2021-02-08 09:18:47 -0500},
	journal = {Journal of Zoology},
	number = {3},
	pages = {305--314},
	publisher = {Wiley Online Library},
	title = {Allometry of the limb bones of mammals from shrews (Sorex) to elephant (Loxodonta)},
	volume = {189},
	year = {1979}}

@book{Mandelbrot:1983kx,
	author = {Mandelbrot, Benoit B},
	date-added = {2021-02-08 09:18:47 -0500},
	date-modified = {2021-02-08 09:18:47 -0500},
	publisher = {Macmillan},
	title = {The fractal geometry of nature},
	year = {1983}}

@article{Biewener1989aa,
	author = {Biewener, Andrew A},
	date-added = {2021-02-08 09:18:47 -0500},
	date-modified = {2021-02-08 09:18:47 -0500},
	journal = {Bioscience},
	number = {11},
	pages = {776--783},
	publisher = {JSTOR},
	title = {Mammalian terrestrial locomotion and size},
	volume = {39},
	year = {1989}}

@article{Niklas:1994eu,
	author = {Niklas, KJ},
	date-added = {2021-02-08 09:18:47 -0500},
	date-modified = {2021-02-08 09:18:47 -0500},
	journal = {Evolution},
	pages = {44--54},
	publisher = {JSTOR},
	title = {The scaling of plant and animal body mass, length, and diameter},
	year = {1994}}

@article{Coleman1998,
	author = {Coleman, Michael J and Ruina, Andy},
	date-added = {2021-02-08 09:18:47 -0500},
	date-modified = {2021-02-08 09:18:47 -0500},
	journal = {Physical Review Letters},
	number = {16},
	pages = {3658},
	publisher = {APS},
	title = {An uncontrolled walking toy that cannot stand still},
	volume = {80},
	year = {1998}}

@article{Virot2020,
	author = {Virot, E and Spandan, V and Niu, L and van Rees, WM and Mahadevan, L},
	date-added = {2021-02-08 09:18:47 -0500},
	date-modified = {2021-02-08 09:18:47 -0500},
	journal = {Physical Review Letters},
	number = {5},
	pages = {058102},
	publisher = {APS},
	title = {Elastohydrodynamic Scaling Law for Heart Rates},
	volume = {125},
	year = {2020}}

How to add terms and types? (for 1.0.2)

Great to see that this is already so advanced.

I've just wanted to add the new variables, terms, and types we've added in 1.0.2. (1.0.2 has been released just today.)

Obviously, I haven't had the time to extensively study the code, but: Am I right to assume that, unlike with variables, there's no hardcoded list of terms and types?

jgm / citeproc Goto Github PK

citeproc's Introduction

citeproc

How to use it

The citeproc executable

Known bugs and limitations

citeproc's People

Contributors

Stargazers

Watchers

Forkers

citeproc's Issues

Background / non-working workaround

output when using depecrated pandoc-citeproc filter

Reference list

Earlier

Recently

The Problem

Questions

test.html

test.md

refs.bib

Recommend Projects

Recommend Topics

Recommend Org

output when using depecrated `pandoc-citeproc` filter

`test.html`

`test.md`

`refs.bib`