mirror of
https://github.com/Tencent/WeKnora.git
synced 2026-11-03 22:43:26 +08:00
Ochan Kwonandwizardchen
c9a342d965
Remote http(s) images written as <img src="..."> reached no resolver.
Markdown images went through ResolveRemoteImages, HTML data-URIs and HTML
relative paths each had their own pass, but nothing claimed the remaining
combination, so those images were dropped without a log line and the
multimodal stage was skipped for documents that rely on HTML for layout.
ResolveRemoteImages now scans HTML img tags with a quoted src as a second
pass and rewrites only the src attribute value, leaving the tag intact
because the surrounding markup is why these documents use HTML at all.
The per-image work (whitelist, SSRF, download, icon filter, upload) moves
into one helper so the two syntaxes cannot drift apart on policy. Each
pass carries its own image budget; sharing one would let HTML images eat
into the number of Markdown images a document already gets resolved.
An attribute value is trimmed, HTML-unescaped and its scheme lowercased
before use, because HTML permits padding, spells a query separator &
and tolerates any scheme case, while every fetcher downstream compares the
scheme byte-for-byte. Markdown targets get none of that normalization, so
no Markdown reference resolves differently than it does today. A
whitelisted host keeps serving the image from where it already lives, and
its span is rewritten with the normalized URL rather than the raw attribute
text: later stages both locate the image by searching the document for its
serving URL and fetch that same string, so the two have to agree.
The src attribute must now be preceded by whitespace, so a hyphenated
attribute name is no longer mistaken for it. Among HTML tags this cuts two
ways: one carrying both a hyphenated *-src and a real src now resolves the
real src instead of the placeholder, while one carrying only a hyphenated
*-src is no longer resolved at all. Recovering the latter needs an
attribute parser rather than a regex, and resolving a lazy-loading
placeholder while the real image goes unvisited is the worse of the two
failures. Both halves are pinned by tests.
Image-derived text also has to survive into the answer, which is why this
touches internal/searchutil. The chat enricher and the image_info filters
matched Markdown links only, so an HTML-only passage looked image-free:
every entry was dropped and the model received bare markup with the
caption and OCR text stripped, even though the analysis had run. Both now
recognise HTML img tags through one shared regex, so a tag the parser
stores can always be matched back. Both syntaxes are located against the
original passage and spliced in a single pass; scanning one regex over the
other's output let a metadata block that itself quotes an image reference
be rescanned and duplicated. The chat variant still enriches only images
present in the passage, so orphan entries stay out.
Known limits:
- An unquoted src and a srcset-only tag are not matched, nor is a tag
whose attribute values hold a bare > or a quoted src= of their own.
- imgHTMLDataURI still matches src without a boundary, so a data-src
holding a base64 placeholder is consumed as if it were the src and the
whole tag replaced. Worth the same fix, but it changes what an existing
lazy-loading document resolves, so it belongs on its own.
- A reference shown as a documentation example is rewritten like any
other. Recognising code spans needs a Markdown parser; a hand-rolled
scanner risks the far worse failure of treating a document's real
screenshots as examples.
- Rewriting a whitelisted src writes the unescaped URL back into an HTML
attribute, so a query parameter whose name spells an entity renders
differently than the author wrote it. Keeping the encoded form would
mean unescaping on every matcher instead.
- Image text reaches the model only when the whole reference falls inside
the retrieved window; an img tag is several times longer than a
Markdown link and straddles a chunk boundary more often.
- A reference repeated in one document is fetched once per occurrence,
which is what gives each chunk its own analysed copy.
- Nothing bounds how long one document's fetches may take in total.
- Parent-context pruning removes only Markdown images.
Co-authored-by: wizardchen <wizardchen@tencent.com>