opensource/RSSHub
3
0
Fork 0
mirror of https://github.com/DIYgod/RSSHub.git synced 2026-09-24 23:12:34 +08:00
Code Issues Packages Projects Releases Wiki Activity
9,070 Commits 13 Branches 0 Tags
5b6d7cb5242e03de416b06410598baafbef7b79c
Commit Graph
137 Commits
Author SHA1 Message Date
dependabot[bot] 3ff763ee8d chore(deps): bump puppeteer from 19.8.3 to 19.11.0 (#12385) 
* chore(deps): bump puppeteer from 19.8.3 to 19.11.0

Bumps [puppeteer](https://github.com/puppeteer/puppeteer) from 19.8.3 to 19.11.0.
- [Release notes](https://github.com/puppeteer/puppeteer/releases)
- [Changelog](https://github.com/puppeteer/puppeteer/blob/main/release-please-config.json)
- [Commits](https://github.com/puppeteer/puppeteer/compare/puppeteer-v19.8.3...puppeteer-v19.11.0)

---
updated-dependencies:
- dependency-name: puppeteer
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* test: fix puppeteer exitCode

* feat: use new headless chrome

https://developer.chrome.com/articles/new-headless/

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-04-26 03:14:45 +08:00
DIYgod f60744241c feat: remove torrent search function 2023-03-18 21:50:26 +00:00
Tony a9d6957e60 feat(route): wxkol (#12010) 
* feat(route): wxkol

* fix: use async pool
 2023-03-03 01:59:45 +08:00
Tony e348914bf5 refactor: move json feed rendering to separated module (#11938) 2023-02-22 20:47:04 +09:00
Tony 589ddb3a43 refactor(eslint): add back no-return-await (#11877) 
* chore: add back no-return-await

* chore: fix action warning

* chore: fix all no-return-await
 2023-02-16 04:30:05 +08:00
Tony a66cbcf6ee feat(config)!: unsafe domain toggle (#11588) 2023-01-10 11:45:05 +00:00
DIYgod ab2f61824d feat: output quote data for twitter and jike 2023-01-09 16:25:26 +00:00
Tony d2ab8ab3c0 feat(core): json feed (#11494) 
* feat(core): json feed

* docs: json feed description

* fix: remove extra comma if no elements exist after author

* test: json feed test

* feat: generate json feed without art-template

* test: fix coverage

* style: 4 spaces json

* docs: fix parameter
 2023-01-02 21:35:36 +08:00
Rongrong d60d89ea87 feat(core): unify proxy; optimize request-wrapper (#11472) 
* feat(core): unify proxy; optimize request-wrapper

Signed-off-by: Rongrong <i@rong.moe>

* Fix DeepScan issue

Signed-off-by: Rongrong <i@rong.moe>

* Simplify protocol existence judgment

Signed-off-by: Rongrong <i@rong.moe>

* chore: regenerate yarn.lock

Signed-off-by: Rongrong <i@rong.moe>
 2022-12-23 01:13:01 +08:00
Rongrong 9264d7afbe feat(core/utils/puppeteer-utils): Cookie utils (#11471) 
Signed-off-by: Rongrong <i@rong.moe>

Signed-off-by: Rongrong <i@rong.moe>
 2022-12-20 19:51:33 +08:00
Rongrong 1381377e91 fix(core/utils/puppeteer): proxy-chain error (#11470) 
HTTPS/SOCKS proxy is not supported by proxy-chain and will cause an
error unconditionally. If an HTTPS/SOCKS proxy does not need
authentication, explicitly bypassing proxy-chain is a must. If it does
need, making a warning and continuing without proxy seems to be the best
choice.

Signed-off-by: Rongrong <i@rong.moe>

Signed-off-by: Rongrong <i@rong.moe>
 2022-12-19 23:39:11 +08:00
Tony 6316ff1a4f feat(route): freewechat (#11419) 2022-12-09 05:25:44 +08:00
Derek c07c960957 fix: restore /hrbeu/ugs/news.js (#11224) 
* fix: restore /heu/ugs/news.js

* Update lib/v2/heu/ugs/news.js

* Update lib/v2/heu/ugs/news.js

* Update lib/v2/heu/ugs/news.js

* feat(require): update require

* refactor: mise fixes

* rename namespace to hrbeu
- hrbeu.edu.cn is the primary domain name as it's where the contact us mail hosted on

* increase code readability
- use template literals over .concat()
- use .includes() to check the existence of sth

* remove fake publish time

* remove duplicated title in description

* fix: fetch article from wechat mp

* fix: codeql
 2022-11-09 22:31:19 +08:00
evnydd0sf fdc59d2d65 feat: 增加设计学院研创、后浪模块；支持获取微信文章内容；修复重复条目bug (#10994) 
* 南京艺术学院研究生院

* Update lib/v2/nua/gra.js

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/v2/nua/gra.js

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/v2/nua/gra.js

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/v2/nua/radar.js

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/v2/nua/radar.js

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/v2/nua/gra.js

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/v2/nua/gra.js

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update docs/university.md

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update docs/university.md

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* 优化代码

* 优化

* update

* 补充南京艺术学院

* fix

* fix

* fix

* 更新南京艺术学院数据源。

* 更新南京艺术学院数据源。

* fix

* fix

* update

* fix doc

* fix

* Update lib/v2/nua/radar.js

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* fix: yarn.lock permission

* style: auto format

* fix double fetch same page

* Update dc.js

* Revert "Merge branch 'master' of https://github.com/evnydd0sf/RSS-Hub"

This reverts commit dd56f9469b, reversing
changes made to d900050eb8.

* fix

* Revert "fix"

This reverts commit 820ff88120.

* Revert "Revert "Merge branch 'master' of https://github.com/evnydd0sf/RSS-Hub""

This reverts commit ce67e0b53c.

* 增加微信公众号解析引导。

* 修正url

* 适配微信公众号内容。

* 适配站内文章、微信公号号文章和未知文章引导。

* Update wechat-mp.js

* fix fetch wechat bug

* 增加设计学院研创、后浪模块

* bugfix

* Revert "Merge branch 'DIYgod:master' into master"

This reverts commit 102cb48db6, reversing
changes made to ccac4be518.

* Update university.md

* Revert "Revert "Merge branch 'DIYgod:master' into master""

This reverts commit c8841910b8.

* fix fetch listname

* fix: switch default case
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2022-10-05 21:32:49 +08:00
Tony 6251a2dc02 fix(core): parse date utils (#10789) 
* fix(core): parse date utils

* fix: regex
 2022-09-14 21:59:32 +08:00
DIYgod 9329a44c80 feat(core): support proxy config for pupperteer (#10714) 
* feat: support proxy config for pupperteer

* test: add puppeteer proxy detection

* fix: package.json

* test: fix regex
 2022-09-07 00:07:04 +08:00
Tony 9fa248c324 fix(route): nmpa feed items on 3rd party site (#10146) 
* fix(route): nmpa feed items on 3rd party site

* fix: regex unescaped .
 2022-07-06 00:46:05 +08:00
Atlas Quan bd80574f76 feat(route): add slowmist news (#10044) 
* feat(route): add slowmist news

* feat(utils): remove useless ul for code section

* docs: add slowmist news route docs

* docs(quick-start): new router under /lib/v2

* test(utils): add test fixArticleContent for new wechat-mp code

* feat(router): update radar source of slowmist
 2022-07-04 23:44:58 +08:00
Tony 2f26479fc9 feat(route): sspai series update (#9901) 2022-06-05 23:59:10 +08:00
github-actions[bot] 5bc8be9ac5 style: auto format 2022-06-05 14:42:33 +00:00
Tony 450e522167 feat(utils): decode cf protected email string (#9900) 
* feat(utils): decode cf protected email string

* Update cf-email.js
 2022-06-05 22:40:57 +08:00
Tony a5fd8de30e fix(docker): puppeteer stealth not working in docker (#9896) 2022-06-05 17:09:34 +08:00
MisLink df6f4caf8b feat(route): add 北京大学国家发展研究院 - 观点 (#9804) 
* feat(route): add 北京大学国家发展研究院 - 观点

Signed-off-by: MisLink <gjq.uoiai@outlook.com>

* Fix https issue

Signed-off-by: MisLink <gjq.uoiai@outlook.com>

* fetch full text from wechat-mp and pku news

Signed-off-by: MisLink <gjq.uoiai@outlook.com>

* Fix ci

Signed-off-by: MisLink <gjq.uoiai@outlook.com>

* refactor: sort new route
 2022-05-27 19:12:04 +08:00
Tony 0f31bfa8b9 fix(utils): parse relative date with meridiem (#9775) 
* fix(utils): parse relative date with meridiem

* fix: use regex
 2022-05-17 20:21:47 +08:00
Rongrong 23fcb6bc5a feat(core/utils/request-wrapper): request logging (#9691) 
Signed-off-by: Rongrong <i@rong.moe>
 2022-05-04 12:36:48 +10:00
Tony 0b544e1395 feat(utils): puppeteer-extra-plugin-stealth (#9676) 2022-05-03 01:17:17 +08:00
Rongrong 7a6be9a229 feat(core): customizable Chromium executable path (#9670) 
* feat(core): customizable Chromium executable path

also build Chromium-bundled Docker image for arm/arm64

Signed-off-by: Rongrong <i@rong.moe>

* chore: fix typo

Signed-off-by: Rongrong <i@rong.moe>

* chore(CI/test): using build matrix

Signed-off-by: Rongrong <i@rong.moe>

* docs(install): fix punctuation

Signed-off-by: Rongrong <i@rong.moe>
 2022-05-01 21:00:29 +08:00
Tony 34b58ebc64 fix(utils): request without hostname (#9649) 2022-04-28 19:35:31 +08:00
任平生andblankyu 9d9926d0bf fix(utils): 修复抓取微信已删除文章时遇到的报错 (#9589) 
* fix(utils): 支持微信公众号单图片文章抓取

* fix(utils): 支持输出微信公众号转载文章阅读原文链接

* fix(utils): 修复抓取微信已删除文章时遇到的报错

* refactor: migrate to v2

Co-authored-by: blankyu(于海洋) <blankyu@tencent.com>
 2022-04-21 21:40:16 +08:00
dependabot[bot]andTonyRL dd4a216648 chore(deps): bump socks-proxy-agent from 6.1.1 to 6.2.0 (#9572) 
* chore(deps): bump socks-proxy-agent from 6.1.1 to 6.2.0

Bumps [socks-proxy-agent](https://github.com/TooTallNate/node-socks-proxy-agent) from 6.1.1 to 6.2.0.
- [Release notes](https://github.com/TooTallNate/node-socks-proxy-agent/releases)
- [Changelog](https://github.com/TooTallNate/node-socks-proxy-agent/blob/master/CHANGELOG.md)
- [Commits](https://github.com/TooTallNate/node-socks-proxy-agent/compare/v6.1.1...v6.2.0)

---
updated-dependencies:
- dependency-name: socks-proxy-agent
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix: use dot notation

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: TonyRL <TonyRL@users.noreply.github.com>
 2022-04-21 03:54:53 +08:00
Rongrong 0522c63d5f fix(core/utils): invalid request header Server (#9582) 
Signed-off-by: Rongrong <15956627+Rongronggg9@users.noreply.github.com>
 2022-04-20 22:32:46 +08:00
任平生 f3e069d399 fix(utils): 支持微信公众号单图片文章抓取; 增加输出阅读原文链接 (#9557) 
* fix(utils): 支持微信公众号单图片文章抓取

* fix(utils): 支持输出微信公众号转载文章阅读原文链接
 2022-04-19 01:29:45 +08:00
Levi ZimandTony 958be6266e feat(route): 山东大学（威海）新闻网 (#9537) 
* feat(sduwh): add extractors.

* feat(route): add route for 山东大学（威海）新闻网

* docs: for route sduwh/news

* docs: for route sduwh/news

(cherry picked from commit 831830167a)

* feat(radar): for route 山东大学（威海）新闻网

* refactor: change `got.get` to `got`.

* refactor: prefer `parseDate()` to `new Date()`

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* fix: incomplete URL substring sanitization.

Make CodeQL happy.

* fix(radar): fix target field.

* fix: change route /sduwh to /sdu/wh

* fix: remove superfluous slash character in url.

* feat: look for exact date first.

* feat: extract exact date from news extractor.

* feat: extract exact date from view extractor.

* feat: extractor for www.sdrj.sdu.edu.cn

* refactor: semantic separation of sduwh with sdu

* feat(radar): more accurate name

* docs: update documentation

* refactor: migrate to v2

* refactor: fix deprecated url.resolve

* fix: update docs url

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* fix: sdu not working routes

* fix: accurate `ctx.state.data.url`

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* fix: better error handling for extractors.

* fix: timezone

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* fix: better error handling.

Co-authored-by: Tony <TonyRL@users.noreply.github.com>
 2022-04-17 00:01:39 +08:00
任平生 eb467afae1 fix(utils): 支持将微信公众号转载文章的正文抓取回来 (#9534) 
* feat: 修正日期时间匹配规则、移除一些不必要评论元素

* fix(route)(fortunechina): 修正财富中文网1、双语文章中文内容重复问题；2、移除 kol 大头像

* fix(route)(wechat): 支持将微信公众号转载文章的正文抓取回来
 2022-04-15 19:23:09 +08:00
Tony bafb3534e1 feat(utils): random user agent (#9449) 
* feat(utils): random ua

* chore: bump rand-user-agent to 1.0.58(no more deps)
 2022-04-12 17:51:07 +08:00
Ethan Shen a25fd4b67f fix(utils): wrong date with same weekday in parse-date (#9506) 2022-04-10 23:40:35 +08:00
Chenxing LuoandTony ef94bcde8e feat(route): add two blogs: Stratechery & Miris Whispers (#9496) 
* Add two blogs

* fix(utils): add parseDate timezone common-config

* Update lib/v2/stratechery/index.js

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update docs/blog.md

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update docs/en/blog.md

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/v2/miris/blog.js
 2022-04-08 23:48:28 +08:00
Rongrong 74e1f88a32 feat(core)(utils/wechat-mp): normalize URL (#9497) 
Signed-off-by: Rongrong <15956627+Rongronggg9@users.noreply.github.com>
 2022-04-08 18:42:52 +08:00
Rongrong a79cc20ec1 feat(utils): add utils for WeChat MP (#9487) 
Motivation:
There are multiple routes that need to fetch articles from WeChat MP.
However, letting them fetch articles by themselves could potentially
lead to cache key collisions. Even if cache key collisions do not occur,
un-normalized URL could potentially lead to duplicated requests.
What's more, articles from WeChat MP have weird formats and need to be
fixed. Creating a universal function to do this work can create some
ease for new route contributors.

Note:
In order to make this PR atomic as much as possible, I did not touch
those broken routes. Once this PR is merged, I will try to fix them.

Signed-off-by: Rongrong <15956627+Rongronggg9@users.noreply.github.com>
 2022-04-07 21:46:15 +08:00
Rongrong 5b35471e39 fix(core): offending RFC4287 (#9441) 
* fix(core): offending RFC4287

should not leave `<updated>` blank when `<published>` is not blank
these two fields MUST conform to the "date-time" production in RFC3339

Signed-off-by: Rongrong <15956627+Rongronggg9@users.noreply.github.com>

* test(common-utils): complete tests

Signed-off-by: Rongrong <15956627+Rongronggg9@users.noreply.github.com>

* test(template): restrict expected value of `pubDate`

Signed-off-by: Rongrong <15956627+Rongronggg9@users.noreply.github.com>
 2022-04-02 17:44:45 +08:00
Rongrong 95db3b4e99 fix(core): torrent searching error (#9407) 
Signed-off-by: Rongrong <15956627+Rongronggg9@users.noreply.github.com>
 2022-03-29 21:12:16 +08:00
Tony cd151f45ab fix(utils): parseDate custom format not working 2022-03-28 22:25:40 +08:00
Ethan Shen a3396029ec fix(utils): wrong last weekday for relative date (#9397) 2022-03-27 23:39:44 +08:00
Ethan Shen e42bdf3d97 fix(utils): typo in parse-date (#9391) 
* fix(utils): typo in parse-date

* fix: add `后日`
 2022-03-26 19:44:27 +08:00
Ethan Shen d5648b37ee fix(utils): parse relative dates with multiple time units (#9365) 
* fix(utils): parse relative dates with multiple time units

* docs: remove warning

* fix: add more characters to match

* fix: rename to parse-date
 2022-03-26 16:45:42 +08:00
Rongrong c48ca6bd5b fix(core): invalid feed fields (#9286) 
Signed-off-by: Rongrong <15956627+Rongronggg9@users.noreply.github.com>
 2022-03-22 02:13:15 +08:00
Tony 2c2bf4ed09 fix: typo in puppeteer options 2022-03-06 23:42:32 +08:00
DIYgod 35c9834049 fix: puppeteer userDataDir 2021-11-28 01:21:42 +00:00
NeverBehave 0792f7ba25 feat(core): first attempt to init script standard (#8224) 
- lazy load
- rate limit per path
- init .debug.json support
- docs
- maintainer
- radar
 2021-09-22 05:41:00 -07:00
Daniel Li (李丹阳)andSukka d77a039f05 style(eslint): add no-implicit-coercion rule (#8175) 
* refactor: add no-implicit-coercion rule for ESLint

* fix: errors from deepscan

* fix: errors from deepscan

* fix: errors from deepscan

* fix: errors from deepscan

* fix: errors from deepscan

* Update docs/en/joinus/quick-start.md

Co-authored-by: Sukka <isukkaw@gmail.com>

* Update docs/joinus/quick-start.md

Co-authored-by: Sukka <isukkaw@gmail.com>

* Update lib/routes/av01/tag.js

Co-authored-by: Sukka <isukkaw@gmail.com>

* Update lib/routes/gov/taiwan/mnd.js

Co-authored-by: Sukka <isukkaw@gmail.com>

* Update lib/routes/ps/product.js

Co-authored-by: Sukka <isukkaw@gmail.com>

* refactor: minify html string

Co-authored-by: Sukka <isukkaw@gmail.com>
 2021-09-15 21:22:11 +08:00