* Revert "fix(route/twitter): send SearchTimeline over POST (#23359)"
This reverts commit 980db92c22.
* fix: include SearchTimeline for getClientTransactionId
* fix(route:twitter): Add X-Client-Transaction-Id header for url UserTweetsAndReplies
* fix(route:twitter): Add cache for clientTransaction auth key
* fix(route:twitter): cache transaction keys and bump x-client-transaction-id 0.3.2
Use RSSHub cache.tryGet for key/animationKey instead of an in-process ClientTransaction instance, and bump x-client-transaction-id to 0.3.2 for the updated X homepage URL.
Cloudflare Workers KV rejects put() calls whose expirationTtl is below
60 seconds. Some callers use shorter TTLs that work on the memory and
Redis backends, e.g. the 1-second lock release in the Twitter web API
utils and the data258 route, so on Workers those calls throw
"Invalid expiration_ttl of 1".
Clamp the TTL passed to KV in the kv cache module and in the worker
globalCache.set path. The requested TTL is still stored in the cacheTtl
sidecar key.
Based on #23400 and #23403 by @EngDawood.
Co-authored-by: Dawood Saleh <52585641+EngDawood@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat: add full-text DR.dk news RSS routes
* fix(route/dr): wrap pubDate with parseDate
* fix(route/dr): remove test file flagged as orphan
* fix(route/dr): consolidate full-text feeds into /dr/:category?
- merge seven near-identical routes into one parameterized route
defaulting to senestenyt, with feed slugs as parameter options and
one radar rule per section
- use ofetch instead of got in utils
- throw inside cache.tryGet and fall back outside so a temporary
failure never caches the RSS summary
- render FactBoxComponent and ImageCollectionComponent
- only parse startDate as pubDate and use the article pubDate
* feat(route/czechstepbystep): add kratke-ceske-zpravy
Adds a new RSS route for Krátké české zprávy (short Czech news) from CzechStepByStep.
Each feed item includes:
- Embedded YouTube video rendered with RSSHub's official YouTube renderer
- Full transcript under "Text zprávy"
- Wordwall online exercise link
- Downloadable worksheet (.docx / .pdf) as link and enclosure
- Publication date parsed from the article page
* refactor(route/czechstepbystep): simplify article parsing
Extract per-section helpers (video, transcript, exercise, worksheet) to reduce detail-parsing complexity, replace brute-force selector fallbacks with single stable anchors, and inline the YouTube iframe in the template to drop the cross-namespace youtube utils dependency.
Co-authored-by: deepseek-v4-pro <noreply@deepseek.com>
* refactor(route/czechstepbystep): drop iframe referrerpolicy
Remove the referrerpolicy attribute from the embedded YouTube iframe per the no-referrer-policy route rule; RSSHub middleware injects the policy automatically.
Co-authored-by: deepseek-v4-pro <noreply@deepseek.com>
* refactor(route/czechstepbystep): render video via youtube utils
Replace the hand-written YouTube nocookie iframe in the description template
with the exported renderYoutube helper from lib/routes/youtube/utils.tsx, as
requested in review.
As well as removing the duplicated markup, the shared renderer honours
config.youtube.videoEmbedUrl, so self-hosted embed proxies keep working.
Co-authored-by: deepseek-v4-flash-vision-exp <noreply@deepseek.com>
* refactor(route/czechstepbystep): harden article parsing
Three robustness fixes so a site change degrades the feed instead of
breaking it:
- extractYouTubeId returns undefined when no embed is found, rather than
throwing, so an article without a video still renders its transcript,
exercise and worksheet. The description template already treats videoId
as optional.
- Per-item article fetches are awaited inside a try/catch, so one
unreachable article no longer rejects the whole Promise.all and leaves
the feed empty.
- The "Text zprávy:" transcript marker is matched after trimming, so
incidental leading or trailing whitespace no longer silently drops the
transcript section.
Co-authored-by: deepseek-v4-flash-vision-exp <noreply@deepseek.com>
---------
Co-authored-by: deepseek-v4-pro <noreply@deepseek.com>
About 70% of posts open with the featured image inside the body as
img.wp-image-<featured_media>, so prepending it showed the same picture
twice. Only prepend it when the body doesn't contain it.
Claude-Session: https://claude.ai/code/session_017gmEyEJ8ufxFDtbwMCXNiL
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
`@/utils/got` has been an ofetch-based imitation of got since e5a9fd46e.
Its responses only carry `data` and `body`, so reading `response.headers`
throws `TypeError: Cannot read properties of undefined`.
Switch the call sites that read response headers to `ofetch.raw()` and
read them through the Fetch `Headers` API (`getSetCookie()` / `get()`),
using `_data` for the body.
- Pass `redirect: 'manual'` where the code relied on got's
`followRedirect: false` (trow, informs, nhentai login), which the fake
got ignored
- Force `responseType: 'json'` for the 163 playlist API, which responds
with `text/plain`
- github/contributors: pagination through the `link` header works again
instead of silently returning only the first page
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
sortRoutes returned 0 whenever two paths had the same segment kinds up to
the shorter length, e.g. `/articles` vs `/radios/:category?`, or the empty
path `''` vs any path. That tie relation is not transitive, so toSorted
could register a literal route after a param route that matches it,
depending on the module import order:
- /sehuatang/user/:uid was served by /sehuatang/:subforumid?/:type?
- /gcores/radios/preview was served by /gcores/radios/:category?
Compare paths lexicographically by segment rank (literal < regex param <
plain param). When one path runs out of segments, it goes first unless it
ends with a regex param, which may span several segments, so
/discuz/:link{.+} still comes after the longer discuz paths.
Close#18335
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The site publishes an Atom feed at /feed/all/, but every entry is truncated
to a 128 character summary and none of them carry category tags. This route
consumes that feed and enriches it:
- Full text: the body is fetched from `article.article > div.body` on each
detail page, which is 3x to 659x the length of the published summary.
- Categories: tags are extracted from `#tags a` on the detail page.
Measured against the 20 entries currently in the feed: all 20 have a unique
link, a pubDate and an author; 17 have at least one tag.
* fix(utils): parse abbreviated relative date units
YouTube serves a compact metadata variant to some sessions, where a video
published a year ago reads `1y ago` instead of `1 year ago`. Neither `y`,
`w` nor `mo` was recognised, so those strings fell through to the plain
Day.js parse and came back as `Invalid Date`.
`mo` needs a lookahead on the minutes pattern too: the bare `m` matched
the `mo` of a month, so `11 months ago` resolved to eleven months *and*
eleven minutes ago, and `3mo ago` to three minutes ago.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(route/youtube): add live streams
`/youtube/live` searches for ongoing broadcasts through the Data API, so
there is no way to follow a channel's schedule or its past streams. Read
the channel's Live tab instead, which lists all three, and let the route
params pick which of them end up in the feed. No API key needed.
A stream is classified by its thumbnail badge: an ongoing one carries a
`LIVE` badge and a scheduled one an `Upcoming` badge. The absence of a
duration badge is deliberately not used as the signal, because YouTube
groups durations past a thousand hours, e.g. `20,772:51:34` — the same
reason the duration attachment now strips separators before summing.
Scheduled streams have no publish date, so they take the time they are
scheduled for. YouTube renders it in the time zone youtubei.js reports,
which is the local one, hence the plain local parse.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(utils): hoist parse-date test helper to module scope
`p` captured nothing from the `describe` scope, which tripped
`unicorn/consistent-function-scoping`. Moving it out also lets its
parameters be derived from `parseRelativeDate` instead of `any[]`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(route/youtube): add stream descriptions
The lockups on the Live tab carry only a title, so every item in
`/youtube/streams` had an empty body. Add `includeDescription`, which
fills it from the player response. That is one request per stream, hence
a default of false and only the streams that survive the state filter
being fetched.
A stream can be unplayable, e.g. a members-only one, so a failed lookup
falls back to an empty description instead of taking the whole feed
down. The cached value is wrapped in an object because an empty string
comes back out of `cache.tryGet` as a literal `""`, and its expiration
is deliberately not renewed on a hit, so an edited description still
reaches a feed that is polled steadily.
`isIncluded` becomes `isEnabled` to give the new parameter a default of
false while the `include*` ones keep parsing exactly as before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(route/youtube): categorize streams instead of filtering them
The `includeLive` / `includeUpcoming` / `includeCompleted` route params
reimplemented, per route, what the common `filter_category` params
already do. Drop them and expose the state that drove them as the item
category, so `/youtube/streams/@GawrGura?filterout_category=completed`
replaces `/youtube/streams/@GawrGura/includeCompleted=false`.
The category lives in `getStreamsByChannelId` rather than in the shared
`lockupViewToItem`, which also serves the Videos tab and playlists where
an uploaded video is not a `completed` stream.
With the filtering gone, `allowEmpty` goes too. It only ever covered a
feed emptied by those params: a channel that has never streamed has no
Live tab at all, so `getLiveStreams()` throws rather than returning
nothing, and the empty check runs before the `filter_category` pass. An
empty list now means the lockup parsing broke, which should surface.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
tunnel-agent (used by request / request-promise, e.g. instagram-private-api with IG_PROXY) sends a CONNECT request with legacy options whose path is the tunnel target authority (host:port), not a URL path. Rebuilding a URL from those options produced http://127.0.0.1i.instagram.com:443/ and node then resolved that bogus hostname (ENOTFOUND). Hand CONNECT requests to node as-is: they cannot be turned into a URL, and UA/referer/proxy-agent injection does not apply to a proxy handshake.
Fixes#16970
X serves the SearchTimeline GraphQL operation over POST only and answers GET
with a bare 404 and an empty body, which makes /twitter/keyword fail even
though the resolved query ID is correct.
Verified against the X API with a valid auth_token cookie: a GET request using
the query ID and the 40 featureSwitches read from the current web bundle still
returns 404 on x.com, api.x.com, twitter.com and api.twitter.com, while the
same request sent as POST returns 200.
twitterGot now accepts an optional method/queryId, moving the parameters into a
JSON body when POST is requested. paginationTweets uses it for the operations
in POST_ENDPOINTS, currently only SearchTimeline; every other operation keeps
using GET since they still work that way.
Co-authored-by: Geunho Bang <gh.bang@vinucommerce.com>
* feat(route): add PetaPixel
The official feed only carries excerpts. Reads the WordPress REST API and
returns full posts with all images (largest srcset candidate), author and
categories, optionally filtered by topic.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(route/petapixel): use toSorted when picking the largest srcset candidate
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(route/petapixel): drop the radar target so the optional category is honoured
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(route/petapixel): strip lazy-loading attributes from embedded iframes too
Readers that never trigger lazy loading show an empty box where the embed
should be, the same way they do for images.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(route): add PetaPixel — address review
Drops the Array.isArray guards on the top-level responses and the explicit
accept header; this host serves JSON with browser-like headers too. Image
handling keeps picking the largest srcset candidate, and only the attributes
describing the variant it replaced are removed.
Adds a guard the route did need: when an _embed sub-request fails WordPress
inlines an error object in place of the term list, which this site now does
for tags, and the category mapping threw on it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017gmEyEJ8ufxFDtbwMCXNiL
* feat(route): add PetaPixel — drop decodeHTML
The middleware already runs decodeXML on the feed and item titles, and
WordPress only emits numeric entities there. PetaPixel posts only carry
categories, none of which contains an entity.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017gmEyEJ8ufxFDtbwMCXNiL
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
