Commit Graph

14045 Commits

Author SHA1 Message Date
John MacFarlane de3b22c356 EPUB writer: use SOURCE_DATE_EPOCH for modification date.
This enables reproducible builds.
2021-02-10 23:13:33 -08:00
John MacFarlane 359881d779 T.P.Class: Add getSourceDateEpoch [API change].
This attempts to read the SOURCE_DATE_EPOCH environment variable
and parse a UTC time from it (treating it as a unix date stamp,
see https://reproducible-builds.org/specs/source-date-epoch/).

If the variable is not set or can't be parsed as a unix date
stamp, then the function returns the current date.
2021-02-10 23:04:49 -08:00
John MacFarlane 8ca191604d Add new unexported module T.P.XMLParser.
This exports functions that uses xml-conduit's parser to
produce an xml-light Element or [Content].  This allows
existing pandoc code to use a better parser without
much modification.

The new parser is used in all places where xml-light's
parser was previously used.  Benchmarks show a significant
performance improvement in parsing XML-based formats
(especially ODT and FB2).

Note that the xml-light types use String, so the
conversion from xml-conduit types involves a lot
of extra allocation.  It would be desirable to
avoid that in the future by gradually switching
to using xml-conduit directly. This can be done
module by module.

The new parser also reports errors, which we report
when possible.

A new constructor PandocXMLError has been added to
PandocError in T.P.Error [API change].

Closes #7091, which was the main stimulus.

These changes revealed the need for some changes
in the tests.  The docbook-reader.docbook test
lacked definitions for the entities it used; these
have been added. And the docx golden tests have been
updated, because the new parser does not preserve
the order of attributes.

Add entity defs to docbook-reader.docbook.

Update golden tests for docx.
2021-02-10 22:04:11 -08:00
John MacFarlane 9994ad977d Use lts-17.2 resolver (with ghc 8.10.3). 2021-02-08 10:11:06 -08:00
John MacFarlane f70795dc5e ODT reader: finer-grained errors on parse failure.
See #7091.
2021-02-08 09:39:59 -08:00
John MacFarlane 5cd1c1001f ODT reader: give more information if zip can't be unpacked. 2021-02-08 09:39:59 -08:00
Nils Carlson 69b7401e31 DocBook reader: Support informalfigure (#7079)
Add support for informalfigure.
2021-02-08 09:36:58 -08:00
Albert Krewinkel d202f7eb77 Avoid unnecessary use of NoImplicitPrelude pragma (#7089) 2021-02-07 10:02:35 -08:00
Albert Krewinkel f7be8d0964 pandoc.cabal: use common stanza to reduce duplication (#7086) 2021-02-07 08:33:43 -08:00
Nixon Enraght-Moony bab5d10ea7 Document no template fallback for absolute path (#7088)
See jgm/pandoc#7077
2021-02-07 08:30:34 -08:00
John MacFarlane 8e9131db4e Markdown reader: improved handling of mmd link attributes in references.
Previously they only worked for links that had titles.  Closes #7080.
2021-02-06 21:52:12 -08:00
John MacFarlane 0ab3e4048c stack.yaml - use commonmark-0.1.1.4 for GHC 9 2021-02-06 19:00:22 -08:00
John MacFarlane b63d579ba2 CI: use haskell/actions/setup.
actions/haskell-setup is no longer maintained.
2021-02-06 19:00:00 -08:00
Albert Krewinkel 51c3c93f0f CI: use cabal 2.2 when building with GHC 8.0.2. (#7085) 2021-02-06 18:09:05 -08:00
Albert Krewinkel a5169f68b2 Lua filters: use same function names in Haskell and Lua 2021-02-04 19:07:59 +01:00
Albert Krewinkel 57e56ed55c doc/lua-filters.md: improve docs for pandoc.mediabag.insert 2021-02-04 19:07:59 +01:00
Albert Krewinkel 364fe4a03b doc/lua-filters.md: fix, improve docs for pandoc.mediabag.fetch 2021-02-04 15:32:35 +01:00
Nick Berendsen b79aba6ea1 ePub writer: belongs-to-collection metadata (#7063) 2021-02-03 09:00:18 -08:00
Andrew Dunning 4de9edb8e8 LaTeX template: Update to iftex package (#7073)
Load the iftex package directly rather than via the ifxetex and ifluatex compatibility
wrappers, which have been merged into a single package that is part of the LaTeX core.
The capitalization of the commands has been changed for compatibility with older
versions of TeX Live that have the version of iftex by the Persian TeX Group. This had
been removed in
<https://github.com/jgm/pandoc/commit/2845794c0c31b2ef1f3e6a73bb5b109da4c74f37>
for compatibility with BasicTeX, but that is no longer an issue.
2021-02-03 08:54:11 -08:00
John MacFarlane e6c7fcc598 Fixed some compiler warnings in tests. 2021-02-02 21:09:10 -08:00
Albert Krewinkel 6f79042502 Add tests for search_path_separator 2021-02-02 21:04:30 -08:00
Albert Krewinkel e0bf4bfe82 Check that all documented functions are present.
Rely on tests in the module package to check the correctness of each
function.
2021-02-02 21:04:30 -08:00
Albert Krewinkel 2ace15964b Fix stack config 2021-02-02 21:04:30 -08:00
Albert Krewinkel ebb8f23b66 Use hslua-module-path 0.1.0 2021-02-02 21:04:30 -08:00
Albert Krewinkel f84512228e Improve docs for directory, normalize 2021-02-02 21:04:30 -08:00
Albert Krewinkel 61b108d527 Lua: add module "pandoc.path"
The module allows to work with file paths in a convenient and
platform-independent manner.

Closes: #6001
Closes: #6565
2021-02-02 21:04:30 -08:00
John MacFarlane 2415b2680a Test suite: a more robust way of testing the executable.
Mmny of our tests require running the pandoc
executable. This is problematic for a few different reasons.
First, cabal-install will sometimes run the test suite
after building the library but before building the executable,
which means the executable isn't in place for the tests.
One can work around that by first building, then building
and running the tests, but that's fragile.  Second,
we have to find the executable. So far, we've done that
using a function findPandoc that attempts to locate it
relative to the test executable (which can be located
using findExecutablePath).  But the logic here is delicate
and work with every combination of options.

To solve both problems, we add an `--emulate` option to
the `test-pandoc` executable.  When `--emulate` occurs
as the first argument passed to `test-pandoc`, the
program simply emulates the regular pandoc executable,
using the rest of the arguments (after `--emulate`).
Thus,

    test-pandoc --emulate -f markdown -t latex

is just like

    pandoc -f markdown -t latex

Since all the work is done by library functions,
implementing this emulation just takes a couple lines
of code and should be entirely reliable.

With this change, we can test the pandoc executable
by running the test program itself (locatable using
findExecutablePath) with the `--emulate` option.
This removes the need for the fragile `findPandoc`
step, and it means we can run our integration tests
even when we're just building the library, not the
executable.

Part of this change involved simplifying some complex
handling to set environment variables for dynamic
library paths.  I have tested a build with
`--enable-dynamic-executable`, and it works, but
further testing may be needed.
2021-02-02 20:36:51 -08:00
John MacFarlane ec8509295a Add parseOptionsFromArgs [API change, addition].
Exported by Text.Pandoc.App.
2021-02-02 17:00:03 -08:00
John MacFarlane af77241b17 cabal.project - more heap space 2021-02-02 14:47:27 -08:00
John MacFarlane d8e4bbca4c shell.nix - install zlib 2021-02-02 14:47:14 -08:00
John MacFarlane defad03ce1 Remove superfluous import 2021-02-02 14:46:59 -08:00
John MacFarlane 84d46e3523 Add default.bibtex to repository. 2021-02-01 18:53:59 -08:00
John MacFarlane 02d3c71e72 BibTeX writer: use doclayout and doctemplate.
This change allows bibtex/biblatex output to wrap as other
formats do, depending on the settings of `--wrap` and `--columns`.

It also introduces default templates for bibtex and biblatex,
which allow for using the variables `header-include`, `include-before`
or `include-after` (or alternatively the command line options
`--include-in-header`, `--include-before-body`, `--include-after-body`)
to insert content into the generated bibtex/biblatex.

This change requires a change in the return type of the unexported
`T.P.Citeproc.writeBibTeXString` from `Text` to `Doc Text`.

Closes #7068.
2021-02-01 18:05:20 -08:00
John MacFarlane b239c89a82 BibTeX writer fixes. Closes #7067.
+ Require citeproc 0.3.0.7, which correctly titlecases when titles
  contain non-ASCII characters.
+ Correctly handle 'pages' (= 'page' in CSL).
+ Correctly handle BibLaTeX 'langid' (= 'language' in CSL).
+ In BibTeX output, protect foreign titles since there's no language
  field.
2021-02-01 11:23:07 -08:00
John MacFarlane 22faea15c2 Makefile: give allocation data in benchmarks. 2021-01-31 18:18:53 -08:00
John MacFarlane aa60d9b230 Add shell.nix. 2021-01-31 12:06:38 -08:00
John MacFarlane d1875b69ec RST reader: fix handling of header in CSV tables.
The interpretation of this line is not affected
by the delim option. Closes #7064.
2021-01-31 12:05:46 -08:00
Albert Krewinkel 9c8ff53b54 CslJson writer: fix compiler warning 2021-01-31 14:37:47 +01:00
John MacFarlane a9adb29648 Require citeproc 0.3.0.6. 2021-01-30 19:09:08 -08:00
John MacFarlane 6695917258 CslJson writer: output [] if no references in input,
instead of raising a PandocAppError as before.
2021-01-30 18:10:22 -08:00
John MacFarlane fe06437ba4 Use tasty-bench instead of criterion for benchmarks.
It is much lighter-weight.
2021-01-30 18:01:14 -08:00
John MacFarlane a6c7469c85 Use latest citeproc.
The fix to jgm/citeproc#49 also closes #7060.
2021-01-30 13:56:43 -08:00
John MacFarlane 9223788a05 Markdown writer: handle math right before digit.
We insert an HTML comment to avoid a `$` right before
a digit, which pandoc will not recognize as a math delimiter.
2021-01-29 18:29:17 -08:00
Albert Krewinkel d82fe52a02 JATS templates: tag author.name as string-name
The partitioning the components of a name into surname, given names,
etc. is not always possible or not available. Using `author.name` allows
to give the full name as a fallback to be used when `author.surname` is
not available.
2021-01-29 09:51:20 +01:00
Albert Krewinkel 300b9b0ea3 JATS writer: escape special chars in reference elements.
Prevents the generation of invalid markup if a citation element contains
an ampersand or another character with a special meaning in XML.
2021-01-29 09:51:20 +01:00
John MacFarlane 2ca61a5350 Improve docs for cite-method. 2021-01-27 09:22:50 -08:00
John MacFarlane 98c2a52b4e Clean up BibTeX parsing.
Previously there was a messy code path that gave strange
results in some cases, not passing through raw tex but
trying to extract a string content.  This was an artefact
of trying to handle some special bibtex-specific commands
in the BibTeX reader. Now we just handle these in the
LaTeX reader and simplify parsing in the BibTeX reader.
This does mean that more raw tex will be passed through
(and currently this is not sensitive to the `raw_tex`
extension; this should be fixed).

Closes #7049.
2021-01-26 22:45:57 -08:00
Mauro Bieg 12bc662535 LaTeX writer: change BCP47 lang tag from jp to ja
fixes #7047
2021-01-26 15:29:33 -08:00
Albert Krewinkel 490065f3ed Lua: always load built-in Lua scripts from default data-dir
The Lua modules `pandoc` and `pandoc.List` are now always loaded from the
system's default data directory. Loading from a different directory by
overriding the default path, e.g. via `--data-dir`, is no longer supported to
avoid unexpected behavior and to address security concerns.
2021-01-26 09:43:56 -08:00
John MacFarlane 3f9eee473d Update AUTHORS.md. 2021-01-23 15:54:38 -08:00