Commit Graph
2264 Commits
Author SHA1 Message Date
John MacFarlane 03bb426a4a Shared: makeSections behavior changes.
+ When the optional base level parameter is provided, we no
  longer ensure that the sequence of heading levels is gapless
  [behavior change]. Instead, we set the lowest heading level to
  the specified base level, and adjust the others accordingly. If
  an author wants to skip a level, e.g. from level 1 to level 3,
  they can do that. In general, the heading levels specified
  in the source document are preserved; `makeSections` only
  puts them into a hierarchical structure. Closes #9398.

+ Section numbers are now assigned so that the top level
  gets `1`, no matter what heading level is used. So, even
  if the top heading level is 2, numbers will be `1`, `2`, etc.
  rather than `0.1`, `0.2`, as in the past. Closes #5071.

+ We revert to the old behavior when the `--number-offset` option
  is used. So, for example, if a document begins with a level-3
  heading, and `--number-offset=1,2` is used, the top-level section
  numbers will be `1.2.1`, `1.2.2`, etc. This is mainly for
  backwards-compatibility.
2024-02-03 22:19:29 -08:00
Tomas Dahlqvist 163be37dd6 Using internal column widths in pptx writer tables (#9392)
The table writer used to only divide all available width evenly
for all columns. In this update the code uses the incoming widths
if they are available. If they are not set the earlier even
distribution is used. Some of the golden templates are adjusted
slightly because of different rounding when using the new
calculation model.

Closes #5706
2024-02-01 10:37:12 -08:00
John MacFarlane c1f87fdb37 Typst writer: handle labels and citaiton ids with spaces...
and other special characters. In these cases, we produce an
explicit `label()` rather than using `<>` or `@`.

Cloess #9387.
2024-01-29 12:25:09 -08:00
John MacFarlane 97eb927339 Typst writer: avoid illegal labels.
Closes #9387.
2024-01-29 11:45:15 -08:00
John MacFarlane 75d013f73e LaTeX reader: generate relative widths for \linewidth, \textheight.
Closes #9388.
2024-01-29 11:11:45 -08:00
John MacFarlane 8b539dbf1d Makefile: change default rule to build THEN test.
That way we can play with the executable while the tests complete.
2024-01-29 11:11:39 -08:00
John MacFarlane b6ac32b116 Typst writer escaping improvements.
We no longer escape `(`. The reason we did this before (#9137)
has been addressed in another way (#9252).

We only escape `=`, `+`, `-` at the beginning of a line.

We now also escape `/` at the beginning of a line.

This should reduce unnecessary escapes.

Closes #9386.
2024-01-29 10:53:51 -08:00
John MacFarlane caa73134bc Regenerate epub tests with latest changes to EPUB reader. 2024-01-27 10:34:00 -08:00
John MacFarlane 8d09c4aff4 EPUB reader: don't put # characters in identifiers. 2024-01-26 23:58:42 -08:00
John MacFarlane d9a70d2eaf Add some kerns where needed between quotes in LaTeX output.
Closes #9371.
2024-01-25 09:41:47 -08:00
Gokul Rajiv 254ebeb68b add testcase 2024-01-23 14:13:51 -08:00
Albert Krewinkel 57774d79e8 Org writer: escape special lines in code blocks
Fixes: #9218
2024-01-22 18:13:34 +01:00
John MacFarlane f43ffabd3c LaTeX writer: create valid table even when table is empty.
Closes #9350.
2024-01-20 10:19:32 -08:00
John MacFarlane df1944f774 Markdown writer: fix output for pipe tables...
...with a huge number of columns. Previously we got invalid pipe
tables when the number of table columns exceeded the setting of
`--columns`.

Closes #9346.
2024-01-18 08:35:21 -08:00
John MacFarlane 18f0b70d40 DocBook reader: better handling of <procedure> and <substeps>.
`<procedure>` now gets parsed as an ordered list, and
`<substeps>` as a sublist. Closes #9341.
2024-01-16 11:58:42 -08:00
George Stagg 032f7c832f Add readMan to Text.Pandoc.Readers exports 2024-01-16 11:24:13 -08:00
John MacFarlane 9f58a55280 Typst reader: fix handling of \overline.
Due to a typo, it was being incorrectly rendered as an `\underset`.
Closes #9294.
2024-01-05 22:09:36 -08:00
John MacFarlane 5cff32c575 OPML writer: respect --wrap options & --columns in contents of notes.
Closes #9297.
2024-01-04 16:45:02 -07:00
John MacFarlane 87533e2d04 Add command test for #9293. 2023-12-30 22:09:13 -07:00
John MacFarlane b1a1f04168 ODT/opendocument writers: properly handle highlighting styles.
These styles were going into an office:styles element in content.xml,
but this is invalid.  Instead they must go in styles.xml. See #9287.

The variable `highlighting-styles` no longer has any effect on
the default opendocument template, and highlighting styles are
not included in opendocument output.
2023-12-26 22:12:44 -08:00
John MacFarlane 81521cee0d Markdown writer: add table identifier at end of caption if present.
Suggested at #9279.
2023-12-22 11:14:09 -08:00
John MacFarlane 36d1ec3e45 Org reader: parse caption and label for grid tables.
Closes #9279.
2023-12-20 16:55:10 -08:00
John MacFarlane 907b1192d1 Remove html5shiv from default HTML5 template.
I don't think we need to support IE < 9 any more!
2023-12-20 10:10:21 -08:00
Edwin Török 712d746320 fix(docx): sort inline elements in schema order
Fixes #9273

```
[
    {
        "Description": "The element has unexpected child element 'http://schemas.openxmlformats.org/wordprocessingml/2006/main:b'.",
        "Path": {
            "NamespacesDefinitions": [
                "xmlns:w=\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\""
            ],
            "Namespaces": {

            },
            "XPath": "/w:document[1]/w:body[1]/w:p[1]/w:r[7]/w:rPr[1]",
            "PartUri": "/word/document.xml"
        },
        "Id": "Sch_UnexpectedElementContentExpectingComplex",
        "ErrorType": "Schema"
    }
]
```

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-19 15:00:23 -08:00
John MacFarlane 0e86a84e33 LaTeX writer: omit superfluous page locator label...
when used with `--natbib` or `--biblatex`. These will treat a
bare number as a page locator, and they will be able to localize
it.

We borrow the code for stripping the locator label from the suffix
from Citeproc code. Note that the recognition of the locator label
is locale-sensitive; if `lang` is `de`, then `S. 33` is a page
reference, and `p. 33` is not!

Closes #9275.
2023-12-18 23:27:40 -08:00
Edwin Török dbd1019a97 fix(docx): fix validation error on endnotePr
Copying `endnotePr` causes validation errors, because it is now
referencing something that doesn't exist in the document:
```
  {
        "FilePath": "test/docx/golden/custom_style_reference.docx",
        "ValidationErrors": "[{\"Description\":\"Element 'w:endnote' referenced by 'endnote@http://schemas.openxmlformats.org/wordprocessingml/2006/main:id' does not exist in part '/MainDocumentPart/EndnotesPart'. The reference value is '0'.\",\"Path\":{\"NamespacesDefinitions\":[\"xmlns:w=\\\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\\\"\"],\"Namespaces\":{},\"XPath\":\"/w:settings[1]/w:endnotePr[1]/w:endnote[2]\",\"PartUri\":\"/word/settings.xml\"},\"Id\":\"Sem_MissingReferenceElement\",\"ErrorType\":\"Semantic\"},{\"Description\":\"Element 'w:endnote' referenced by 'endnote@http://schemas.openxmlformats.org/wordprocessingml/2006/main:id' does not exist in part '/MainDocumentPart/EndnotesPart'. The reference value is '-1'.\",\"Path\":{\"NamespacesDefinitions\":[\"xmlns:w=\\\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\\\"\"],\"Namespaces\":{},\"XPath\":\"/w:settings[1]/w:endnotePr[1]/w:endnote[1]\",\"PartUri\":\"/word/settings.xml\"},\"Id\":\"Sem_MissingReferenceElement\",\"ErrorType\":\"Semantic\"}]"
  }
```

For now don't copy this element, it wasn't copied before, and it doesn't
seem necessary to fix the ordering problems we had with settings.

Fixes: c9bf4da74 ("Docx writer: ensure that elements in settings are ordered correctly.")

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-18 17:15:03 -08:00
Edwin Török 1e4775f069 fix(docx): fix validation error on w:tblHeader
```
    {
        "FilePath": "test/docx/golden/tables.docx",
        "ValidationErrors": "[{\"Description\":\"The attribute 'http://schemas.openxmlformats.org/wordprocessingml/2006/main:val' has invalid value 'true'. The Enumeration constraint failed.\",\"Path\":{\"NamespacesDefinitions\":[\"xmlns:w=\\\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\\\"\"],\"Namespaces\":{},\"XPath\":\"/w:document[1]/w:body[1]/w:tbl[1]/w:tr[1]/w:trPr[1]/w:tblHeader[1]\",\"PartUri\":\"/word/document.xml\"},\"Id\":\"Sch_AttributeValueDataTypeDetailed\",\"ErrorType\":\"Schema\"}]"
    }
```

Although this one might actually be a bug in Open-XML-SDK similar to
this, or a subtle difference between standard versions:
https://github.com/dotnet/Open-XML-SDK/issues/780

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-18 17:15:03 -08:00
Edwin Török fe8847bdaa fix(docx): use left vs start consistently
They are equivalent, but OOXML-Validator complains:
```
  {
        "FilePath": "test/docx/golden/tables_separated_with_rawblock.docx",
        "ValidationErrors": "[{\"Description\":\"The attribute 'http://schemas.openxmlformats.org/wordprocessingml/2006/main:val' has invalid value 'start'. The Enumeration constraint failed.\",\"Path\":{\"NamespacesDefinitions\":[\"xmlns:w=\\\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\\\"\"],\"Namespaces\":{},\"XPath\":\"/w:document[1]/w:body[1]/w:tbl[2]/w:tblPr[1]/w:jc[1]\",\"PartUri\":\"/word/document.xml\"},\"Id\":\"Sch_AttributeValueDataTypeDetailed\",\"ErrorType\":\"Schema\"},{\"Description\":\"The attribute 'http://schemas.openxmlformats.org/wordprocessingml/2006/main:val' has invalid value 'start'. The Enumeration constraint failed.\",\"Path\":{\"NamespacesDefinitions\":[\"xmlns:w=\\\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\\\"\"],\"Namespaces\":{},\"XPath\":\"/w:document[1]/w:body[1]/w:tbl[1]/w:tblPr[1]/w:jc[1]\",\"PartUri\":\"/word/document.xml\"},\"Id\":\"Sch_AttributeValueDataTypeDetailed\",\"ErrorType\":\"Schema\"}]"
    }
```

pandoc already uses 'left' elsewhere, so be consistent, we still produce
the transitional schema, not the strict one which would have the 'start'
attribute.

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-18 17:15:03 -08:00
Edwin Török 95dde29c5e fix(docx): fix validation error on inline w:i/w:iCs order
From `make validate-docx-golden-tests2`:
```
  {
        "FilePath": "test/docx/golden/definition_list.docx",
        "ValidationErrors": "[{\"Description\":\"The element has unexpected child element 'http://schemas.openxmlformats.org/wordprocessingml/2006/main:i'.\",\"Path\":{\"NamespacesDefinitions\":[\"xmlns:w=\\\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\\\"\"],\"Namespaces\":{},\"XPath\":\"/w:document[1]/w:body[1]/w:p[3]/w:r[3]/w:rPr[1]\",\"PartUri\":\"/word/document.xml\"},\"Id\":\"Sch_UnexpectedElementContentExpectingComplex\",\"ErrorType\":\"Schema\"}]"
    },
```

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-18 17:15:03 -08:00
Edwin Török c5780857e5 fix(docx): fix OOXMLValidator error on KeywordTok output
xmllint doesn't warn about this (maybe because the tag is empty?), but
the order doesn't match wml.xsd:
```
    <w:rPr>
      <w:color w:val="007020"/>
      <w:b/>
    </w:rPr>
```

And OOXMLValidatorCLI does warn about it:
```
{
        "Description": "The element has unexpected child element 'http://schemas.openxmlformats.org/wordprocessingml/2006/main:b'.",
        "Path": {
            "NamespacesDefinitions": [
                "xmlns:w=\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\""
            ],
            "Namespaces": {

            },
            "XPath": "/w:styles[1]/w:style[40]/w:rPr[1]",
            "PartUri": "/word/styles.xml"
        },
        "Id": "Sch_UnexpectedElementContentExpectingComplex",
        "ErrorType": "Schema"
    }
```

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-18 17:15:03 -08:00
Edwin Török 333b0dd2ff fix(docx): fix validation error on w:annotationRef
annotationRef is not valid for `w:rPr`, only for `w:r` according to
wml.xsd.

See https://github.com/jgm/pandoc/issues/9269

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-18 14:08:53 -08:00
Edwin Török 2477fb50ff test(docx): also compare number of children
This detects if a tag gets added/removed, or moves within the hierarchy.

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-18 14:08:53 -08:00
Edwin Török ded8132d30 fix(docx): fix validation error in w:nsid
The length here seems to refer to length in bytes (so twice as long in
hex):
```
./tmp/numbering-pretty.xml:4: element nsid: Schemas validity error : Element '{http://schemas.openxmlformats.org/wordprocessingml/2006/main}nsid', attribute '{http://schemas.openxmlformats.org/wordprocessingml/2006/main}val': [facet 'length'] The value 'A990' has a length of '2'; this differs from the allowed length of '4'.
```

[This](https://learn.microsoft.com/en-us/dotnet/api/documentformat.openxml.wordprocessing.nsid?view=openxml-2.8.1) also documents the longer values.

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-18 12:30:20 -08:00
John MacFarlane f7e31adc97 Docx writer: fixed validation errors in tables.
Closes #9266.
2023-12-18 10:18:57 -08:00
John MacFarlane 9c5e6aeedc Docx writer: fix validation error.
The elements in pPr in lists were not properly ordered.
This doesn't seem to cause problems for Word, but it makes
validation fail and may pose problems for other consumers of
docx.  Closes #9265.
2023-12-18 09:54:14 -08:00
John MacFarlane c9bf4da74f Docx writer: ensure that elements in settings are ordered correctly.
The elements must occur in a specific order. This was being
messed up when integrating a custom reference.docx. Closes #9264.
2023-12-17 17:03:14 -08:00
Edwin Török 5875de3f86 test/docx/golden: regenerate
Using `make test TESTARGS=--accept`

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-17 10:07:10 -08:00
John MacFarlane fd73880a87 Typst writer: emit ; after typst code...
unless followed by space.  Otherwise there's the potential
that the typst code will swallow up a following character.
Closes #9252.
2023-12-14 12:28:04 -08:00
John MacFarlane b46d00343a Depend on texmath 0.12.8.6.
This gives us less verbose typst math output.
2023-12-12 19:08:30 -08:00
John MacFarlane ea9317c73d Typst template fixes. 2023-12-11 10:18:37 -08:00
Hikaru Ibayashi 3be253fb90 LaTeX writer: fix bug with big footnotes inside emphasis (#9168)
Closes #8982.
2023-12-10 09:29:45 -08:00
John MacFarlane aa95770743 Typst writer: use quote for block quotes.
Remove custom definitiion of blockquote in default template.
2023-12-09 23:23:15 -08:00
John MacFarlane fb4cea44b9 Docx writer: Use different style for block quotes in notes.
Using "Footnote Block Text" for the style name, so it can be
given a different font size if footnotes are.

Closes #9243.
2023-12-08 13:34:33 -08:00
John MacFarlane 901359bf46 Docx writer: allow embedded fonts to be used in reference.docx.
Closes #6728.
2023-12-08 13:16:59 -08:00
John MacFarlane e46c3e68d2 Update typst-hs commit and typst writer test. 2023-12-08 13:10:33 -08:00
John MacFarlane dbbd70d4f8 Add tests for alerts gfm <-> docbook. 2023-12-05 16:52:49 -08:00
John MacFarlane fe351f1fd5 Fine-tuning on alerts.
Added a test to show that we can convert smoothly between
gfm, rst, and asciidoc alerts.
2023-12-05 09:38:47 -08:00
John MacFarlane 41da8ad9e0 HTML5 writer footnote changes (aria-role, element type).
* To conform to validator's expectations, `doc-footnote` role is used
  with `aside` and `doc-endnotes` with `section`.
* `aside` is used only for notes at ends of sections or blocks;
  if all the notes come at the end of the document, `section` is
  used so we can have the `doc-endnotes` role.
2023-12-04 09:22:58 -08:00
John MacFarlane 80ea048bb2 LaTeX reader: handle otherlanguage environment...
...and language-name environments like
`\begin{french}...\end{french}`.
Closes #9202.
2023-11-29 21:22:03 -08:00
Stephan Meijer 2e8ecb3fba Docx reader: unwrap content of shaped textboxes...
* #9214 text in shape format test document

* #9214 support Text in Shape Format

* #9214 remove irrelevant code
2023-11-29 18:59:09 -08:00