2247 Commits
Author SHA1 Message Date
John MacFarlane 5cff32c575 OPML writer: respect --wrap options & --columns in contents of notes.
Closes #9297.
2024-01-04 16:45:02 -07:00
John MacFarlane 87533e2d04 Add command test for #9293. 2023-12-30 22:09:13 -07:00
John MacFarlane b1a1f04168 ODT/opendocument writers: properly handle highlighting styles.
These styles were going into an office:styles element in content.xml,
but this is invalid.  Instead they must go in styles.xml. See #9287.

The variable `highlighting-styles` no longer has any effect on
the default opendocument template, and highlighting styles are
not included in opendocument output.
2023-12-26 22:12:44 -08:00
John MacFarlane 81521cee0d Markdown writer: add table identifier at end of caption if present.
Suggested at #9279.
2023-12-22 11:14:09 -08:00
John MacFarlane 36d1ec3e45 Org reader: parse caption and label for grid tables.
Closes #9279.
2023-12-20 16:55:10 -08:00
John MacFarlane 907b1192d1 Remove html5shiv from default HTML5 template.
I don't think we need to support IE < 9 any more!
2023-12-20 10:10:21 -08:00
Edwin Török 712d746320 fix(docx): sort inline elements in schema order
Fixes #9273

```
[
    {
        "Description": "The element has unexpected child element 'http://schemas.openxmlformats.org/wordprocessingml/2006/main:b'.",
        "Path": {
            "NamespacesDefinitions": [
                "xmlns:w=\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\""
            ],
            "Namespaces": {

            },
            "XPath": "/w:document[1]/w:body[1]/w:p[1]/w:r[7]/w:rPr[1]",
            "PartUri": "/word/document.xml"
        },
        "Id": "Sch_UnexpectedElementContentExpectingComplex",
        "ErrorType": "Schema"
    }
]
```

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-19 15:00:23 -08:00
John MacFarlane 0e86a84e33 LaTeX writer: omit superfluous page locator label...
when used with `--natbib` or `--biblatex`. These will treat a
bare number as a page locator, and they will be able to localize
it.

We borrow the code for stripping the locator label from the suffix
from Citeproc code. Note that the recognition of the locator label
is locale-sensitive; if `lang` is `de`, then `S. 33` is a page
reference, and `p. 33` is not!

Closes #9275.
2023-12-18 23:27:40 -08:00
Edwin Török dbd1019a97 fix(docx): fix validation error on endnotePr
Copying `endnotePr` causes validation errors, because it is now
referencing something that doesn't exist in the document:
```
  {
        "FilePath": "test/docx/golden/custom_style_reference.docx",
        "ValidationErrors": "[{\"Description\":\"Element 'w:endnote' referenced by 'endnote@http://schemas.openxmlformats.org/wordprocessingml/2006/main:id' does not exist in part '/MainDocumentPart/EndnotesPart'. The reference value is '0'.\",\"Path\":{\"NamespacesDefinitions\":[\"xmlns:w=\\\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\\\"\"],\"Namespaces\":{},\"XPath\":\"/w:settings[1]/w:endnotePr[1]/w:endnote[2]\",\"PartUri\":\"/word/settings.xml\"},\"Id\":\"Sem_MissingReferenceElement\",\"ErrorType\":\"Semantic\"},{\"Description\":\"Element 'w:endnote' referenced by 'endnote@http://schemas.openxmlformats.org/wordprocessingml/2006/main:id' does not exist in part '/MainDocumentPart/EndnotesPart'. The reference value is '-1'.\",\"Path\":{\"NamespacesDefinitions\":[\"xmlns:w=\\\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\\\"\"],\"Namespaces\":{},\"XPath\":\"/w:settings[1]/w:endnotePr[1]/w:endnote[1]\",\"PartUri\":\"/word/settings.xml\"},\"Id\":\"Sem_MissingReferenceElement\",\"ErrorType\":\"Semantic\"}]"
  }
```

For now don't copy this element, it wasn't copied before, and it doesn't
seem necessary to fix the ordering problems we had with settings.

Fixes: c9bf4da74 ("Docx writer: ensure that elements in settings are ordered correctly.")

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-18 17:15:03 -08:00
Edwin Török 1e4775f069 fix(docx): fix validation error on w:tblHeader
```
    {
        "FilePath": "test/docx/golden/tables.docx",
        "ValidationErrors": "[{\"Description\":\"The attribute 'http://schemas.openxmlformats.org/wordprocessingml/2006/main:val' has invalid value 'true'. The Enumeration constraint failed.\",\"Path\":{\"NamespacesDefinitions\":[\"xmlns:w=\\\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\\\"\"],\"Namespaces\":{},\"XPath\":\"/w:document[1]/w:body[1]/w:tbl[1]/w:tr[1]/w:trPr[1]/w:tblHeader[1]\",\"PartUri\":\"/word/document.xml\"},\"Id\":\"Sch_AttributeValueDataTypeDetailed\",\"ErrorType\":\"Schema\"}]"
    }
```

Although this one might actually be a bug in Open-XML-SDK similar to
this, or a subtle difference between standard versions:
https://github.com/dotnet/Open-XML-SDK/issues/780

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-18 17:15:03 -08:00
Edwin Török fe8847bdaa fix(docx): use left vs start consistently
They are equivalent, but OOXML-Validator complains:
```
  {
        "FilePath": "test/docx/golden/tables_separated_with_rawblock.docx",
        "ValidationErrors": "[{\"Description\":\"The attribute 'http://schemas.openxmlformats.org/wordprocessingml/2006/main:val' has invalid value 'start'. The Enumeration constraint failed.\",\"Path\":{\"NamespacesDefinitions\":[\"xmlns:w=\\\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\\\"\"],\"Namespaces\":{},\"XPath\":\"/w:document[1]/w:body[1]/w:tbl[2]/w:tblPr[1]/w:jc[1]\",\"PartUri\":\"/word/document.xml\"},\"Id\":\"Sch_AttributeValueDataTypeDetailed\",\"ErrorType\":\"Schema\"},{\"Description\":\"The attribute 'http://schemas.openxmlformats.org/wordprocessingml/2006/main:val' has invalid value 'start'. The Enumeration constraint failed.\",\"Path\":{\"NamespacesDefinitions\":[\"xmlns:w=\\\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\\\"\"],\"Namespaces\":{},\"XPath\":\"/w:document[1]/w:body[1]/w:tbl[1]/w:tblPr[1]/w:jc[1]\",\"PartUri\":\"/word/document.xml\"},\"Id\":\"Sch_AttributeValueDataTypeDetailed\",\"ErrorType\":\"Schema\"}]"
    }
```

pandoc already uses 'left' elsewhere, so be consistent, we still produce
the transitional schema, not the strict one which would have the 'start'
attribute.

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-18 17:15:03 -08:00
Edwin Török 95dde29c5e fix(docx): fix validation error on inline w:i/w:iCs order
From `make validate-docx-golden-tests2`:
```
  {
        "FilePath": "test/docx/golden/definition_list.docx",
        "ValidationErrors": "[{\"Description\":\"The element has unexpected child element 'http://schemas.openxmlformats.org/wordprocessingml/2006/main:i'.\",\"Path\":{\"NamespacesDefinitions\":[\"xmlns:w=\\\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\\\"\"],\"Namespaces\":{},\"XPath\":\"/w:document[1]/w:body[1]/w:p[3]/w:r[3]/w:rPr[1]\",\"PartUri\":\"/word/document.xml\"},\"Id\":\"Sch_UnexpectedElementContentExpectingComplex\",\"ErrorType\":\"Schema\"}]"
    },
```

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-18 17:15:03 -08:00
Edwin Török c5780857e5 fix(docx): fix OOXMLValidator error on KeywordTok output
xmllint doesn't warn about this (maybe because the tag is empty?), but
the order doesn't match wml.xsd:
```
    <w:rPr>
      <w:color w:val="007020"/>
      <w:b/>
    </w:rPr>
```

And OOXMLValidatorCLI does warn about it:
```
{
        "Description": "The element has unexpected child element 'http://schemas.openxmlformats.org/wordprocessingml/2006/main:b'.",
        "Path": {
            "NamespacesDefinitions": [
                "xmlns:w=\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\""
            ],
            "Namespaces": {

            },
            "XPath": "/w:styles[1]/w:style[40]/w:rPr[1]",
            "PartUri": "/word/styles.xml"
        },
        "Id": "Sch_UnexpectedElementContentExpectingComplex",
        "ErrorType": "Schema"
    }
```

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-18 17:15:03 -08:00
Edwin Török 333b0dd2ff fix(docx): fix validation error on w:annotationRef
annotationRef is not valid for `w:rPr`, only for `w:r` according to
wml.xsd.

See https://github.com/jgm/pandoc/issues/9269

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-18 14:08:53 -08:00
Edwin Török 2477fb50ff test(docx): also compare number of children
This detects if a tag gets added/removed, or moves within the hierarchy.

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-18 14:08:53 -08:00
Edwin Török ded8132d30 fix(docx): fix validation error in w:nsid
The length here seems to refer to length in bytes (so twice as long in
hex):
```
./tmp/numbering-pretty.xml:4: element nsid: Schemas validity error : Element '{http://schemas.openxmlformats.org/wordprocessingml/2006/main}nsid', attribute '{http://schemas.openxmlformats.org/wordprocessingml/2006/main}val': [facet 'length'] The value 'A990' has a length of '2'; this differs from the allowed length of '4'.
```

[This](https://learn.microsoft.com/en-us/dotnet/api/documentformat.openxml.wordprocessing.nsid?view=openxml-2.8.1) also documents the longer values.

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-18 12:30:20 -08:00
John MacFarlane f7e31adc97 Docx writer: fixed validation errors in tables.
Closes #9266.
2023-12-18 10:18:57 -08:00
John MacFarlane 9c5e6aeedc Docx writer: fix validation error.
The elements in pPr in lists were not properly ordered.
This doesn't seem to cause problems for Word, but it makes
validation fail and may pose problems for other consumers of
docx.  Closes #9265.
2023-12-18 09:54:14 -08:00
John MacFarlane c9bf4da74f Docx writer: ensure that elements in settings are ordered correctly.
The elements must occur in a specific order. This was being
messed up when integrating a custom reference.docx. Closes #9264.
2023-12-17 17:03:14 -08:00
Edwin Török 5875de3f86 test/docx/golden: regenerate
Using `make test TESTARGS=--accept`

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-17 10:07:10 -08:00
John MacFarlane fd73880a87 Typst writer: emit ; after typst code...
unless followed by space.  Otherwise there's the potential
that the typst code will swallow up a following character.
Closes #9252.
2023-12-14 12:28:04 -08:00
John MacFarlane b46d00343a Depend on texmath 0.12.8.6.
This gives us less verbose typst math output.
2023-12-12 19:08:30 -08:00
John MacFarlane ea9317c73d Typst template fixes. 2023-12-11 10:18:37 -08:00
Hikaru Ibayashi 3be253fb90 LaTeX writer: fix bug with big footnotes inside emphasis (#9168)
Closes #8982.
2023-12-10 09:29:45 -08:00
John MacFarlane aa95770743 Typst writer: use quote for block quotes.
Remove custom definitiion of blockquote in default template.
2023-12-09 23:23:15 -08:00
John MacFarlane fb4cea44b9 Docx writer: Use different style for block quotes in notes.
Using "Footnote Block Text" for the style name, so it can be
given a different font size if footnotes are.

Closes #9243.
2023-12-08 13:34:33 -08:00
John MacFarlane 901359bf46 Docx writer: allow embedded fonts to be used in reference.docx.
Closes #6728.
2023-12-08 13:16:59 -08:00
John MacFarlane e46c3e68d2 Update typst-hs commit and typst writer test. 2023-12-08 13:10:33 -08:00
John MacFarlane dbbd70d4f8 Add tests for alerts gfm <-> docbook. 2023-12-05 16:52:49 -08:00
John MacFarlane fe351f1fd5 Fine-tuning on alerts.
Added a test to show that we can convert smoothly between
gfm, rst, and asciidoc alerts.
2023-12-05 09:38:47 -08:00
John MacFarlane 41da8ad9e0 HTML5 writer footnote changes (aria-role, element type).
* To conform to validator's expectations, `doc-footnote` role is used
  with `aside` and `doc-endnotes` with `section`.
* `aside` is used only for notes at ends of sections or blocks;
  if all the notes come at the end of the document, `section` is
  used so we can have the `doc-endnotes` role.
2023-12-04 09:22:58 -08:00
John MacFarlane 80ea048bb2 LaTeX reader: handle otherlanguage environment...
...and language-name environments like
`\begin{french}...\end{french}`.
Closes #9202.
2023-11-29 21:22:03 -08:00
Stephan Meijer 2e8ecb3fba Docx reader: unwrap content of shaped textboxes...
* #9214 text in shape format test document

* #9214 support Text in Shape Format

* #9214 remove irrelevant code
2023-11-29 18:59:09 -08:00
John MacFarlane 6149cec27a Update command test for SVG embedding changes. 2023-11-25 11:39:22 -05:00
John MacFarlane a2a0f73eb8 LaTeX writer: handle identifiers inside heading contents.
`\phantomsection` can't be used in this case, so we need `\hypertarget`.
Closes #9209.
2023-11-22 23:24:55 -05:00
John MacFarlane 49e64c1451 Man reader: properly handle .sp macro inside lists and block quotes.
Closes #9201.
2023-11-19 10:45:05 -08:00
Hikaru Ibayashi a3fba6d83e LaTeX reader: fix theorem label parsing (#9198)
Closes #8872.

In the current implementation , theorem labels in `theoremEnvironment` are
determined by the `LastLabel` in the current state.
This approach works well when the `\label{label_name}` is placed at the end
of the theorem body. However, when a label is placed at the beginning of the
theorem (another common practice) and additional labels follow in the theorem body,
`theoremEnvironment` incorrectly picks the last label (e.g., `\label{item2}` in #8872).

This patch addresses the issue by extracting the label extraction independently of the `LastLabel` state.
2023-11-17 18:49:03 -08:00
John MacFarlane 3591733e19 Typst writer: better handling of tables with captions.
We now put these in a figure with a caption argument.

Closes #9194.
2023-11-15 21:17:06 -08:00
John MacFarlane 13e1b49224 HTML reader: Fix handling of invalidly nested sublists.
This revises the fix to #8150 (and the test case) and closes #9187.

HTML in the (invalid) form:

    <ul>
      <li>L1</li>
      <ul>
        <li>L1.1</li>
      </ul>
    </ul>

is treated by browsers like

    <ul>
      <li>L1
        <ul>
          <li>L1.1</li>
        </ul>
      </li>
    </ul>

not

    <ul>
      <li>L1
      <li><ul>
          <li>L1.1</li>
        </ul>
      </li>
    </ul>

as pandoc previously assumed.

This change will give a similar treatment to

    <ul>
      <li>L1</li>
      <p>foobar</p>
    </ul>

which also seems to match browser behavior.
2023-11-15 14:40:00 -08:00
John MacFarlane cd48bf4059 Markdown reader: don't change newlines to spaces in math.
Preserve them: otherwise we can get unwanted results if there's
a `%` comment. Closes #9193.
2023-11-15 10:06:22 -08:00
Stephan Meijer 56f7f11473 ODT reader: Support attr text:continue-numbering (#8998)
Closes #8979.
2023-11-06 19:15:37 -08:00
John MacFarlane 76cd115d3b Markdown reader: make attributes work with reference links.
Closes #9171.
2023-11-05 21:27:03 -08:00
John MacFarlane bb36f127e2 LaTeX template: include bookmark package unconditionally.
This package produces better PDF bookmarks than hyperref
and does it on the first pass.

As a consequence, we now default to running LaTeX only
once in producing a PDF (instead of twice). If a table
of contents is present, we still have to run three times
to get the page numbers.
2023-11-03 10:40:38 -07:00
John MacFarlane 5132f1ef33 T.P.RoffChar: escape - as \-.
The groff_man (7)` man page indicates that `-` characters will be
treated as typographic hyphens and are not appropriate for cases
where the output should be copy-pasteable as an ASCII
hyphen-minus character.  (E.g. in command line options.)

However, until a recent update groff man did not actually do this;
it treated `-` and `\-` the same.  With the new update (1.23.0)
the two are distinguished (see https://lwn.net/Articles/947941/
for background), so now it is important that pandoc escape `-`.

This reverts ee60ba5252.
That change was motivated by a problem with backslash-escaping
`-` in a filename for .PSPIC. That's simply a separate issue;
we shouldn't do the normal escapes in such a context. It has
been addressed in the previous commit.
2023-11-02 19:36:25 -07:00
John MacFarlane 0f3211c339 Shared: improve isTightList...
so that it recognizes an item containing only a list which is
itself tight as potentially an item in a tight list.

Closes #9161.
2023-10-28 10:07:13 -07:00
John MacFarlane e2081ef9df Update typst-reader test for new typst-hs. 2023-10-27 10:34:28 -07:00
John MacFarlane 5ed86e69c4 Make reference-section-title work with jats+element_citations.
Closes #9021.
2023-10-26 17:04:55 -07:00
Julia Diaz f2a22e71df Modify JATS reader to handle BITS too (#9138)
Add provision for title-group, book, book-part-wrapper, book-meta,
book-part-meta, book-title, book-title-group, index, toc, legend,
title, collection-meta
2023-10-26 15:18:33 -07:00
John MacFarlane b671e0dfe5 Org writer: escape literal *, |, # at beginning of line with ZWS.
Closes #9159.
2023-10-26 09:18:56 -07:00
John MacFarlane fbf9194e34 DokuWiki reader: parse <code> and <file> as block-level code.
Previously we treated them as inline code in some contexts,
but that is not how DokuWiki works.

Closes #9154.
2023-10-22 10:59:07 -07:00