kashiro

kashiro выполнил(а) push в master в jbell730/anya

1f61810979 Externalize and expand headline aliases Move the hard-coded ALIASES dict into resources/aliases.toml (loaded with stdlib tomllib), expand it with agencies, international bodies, and country names, and fix two correctness bugs in the naive replace(): - Whole-word matching so 'united states' no longer collapses inside 'united statesman', and 'inflation rate' no longer mangles 'inflation rates'. - Longest-phrase-first application so 'president of the united states' resolves to 'potus' before 'united states' fires. - Correct a stopword collision: 'united nations' -> 'un' was silently dropped because 'un' is a stopword; it now maps to 'unitednations'. Aliases are threaded through normalize_headline -> is_headline -> _build_headlines -> prepare_headlines and loaded in main.py. Includes unit tests and README coverage.

1 день назад

kashiro выполнил(а) push в master в jbell730/anya

10492326c7 Merge remote changes (thresholds, extra feeds) with SSR extractor
ad03cd96ce Add SSR/embedded-JSON extraction and restore Forbes services/ssr.py walks JSON-LD and Next.js __NEXT_DATA__ for article-like objects (headline, or title+url, or name with article signals), recovering JS-rendered sites without a browser. prepare_headlines favors embedded JSON over regex link scraping. Forbes is re-added as a live proof.
Сравнить 2 коммитов »

2 дней назад

kashiro выполнил(а) push в master в jbell730/anya

951ee4ebd8 Add international and regional RSS sources Add English-language international feeds: DW, Le Monde (English ed.), France24, Euronews, Der Spiegel, El Pais (EN), The Japan Times, SCMP, ABC News Australia, and CBC. Nikkei offers no clean public RSS, so Japan Times + SCMP cover Japan/Asia instead.

3 дней назад

kashiro выполнил(а) push в master в jbell730/anya

c761127503 Swap HTML sources for RSS feeds; drop outlets without RSS Replace JS-heavy/paywalled HTML sources with verified RSS feeds (CNN, Fox, NBC, NPR, CBS, NYT, Time, Axios). Drop Reuters, AP, Forbes, MSN and US News, which offer no public/working RSS and are not reliably scrapable.

3 дней назад

kashiro выполнил(а) push в master в jbell730/anya

b15bc58452 Add RSS/Atom support, expand sources and excluded phrases - services/feeds.py parses RSS 2.0/1.0 and Atom into (title, date) entries - prepare_headlines auto-detects feeds by content and parses them directly - strip feeds./rss./moxie. subdomains so feeds collapse to the outlet domain - add 7 verified RSS feeds (BBC, Guardian, Al Jazeera, WaPo, The Hill, Vox, CNBC) - get_sources skips blank lines and # comments - broaden excluded-phrase list (skip links, share/follow, newsletter, utility)

3 дней назад

kashiro выполнил(а) push в master в jbell730/anya

9920a96748 Filter boilerplate link text via configurable excluded phrases Add resources/excluded_phrases.txt (skip links, privacy/consent, legal boilerplate) and match it case- and punctuation-insensitively as a contiguous run of words. Phrases live in a data file so they can be extended without touching code.

3 дней назад

kashiro выполнил(а) push в master в jbell730/anya

1b616210a6 Add browser-like request headers for resilience News sites often reject minimal-bot requests; send a full browser header set (User-Agent, Accept, Sec-Fetch-*, client hints). Accept-Encoding is left to requests/urllib3 to avoid undecodable brotli responses.

3 дней назад

kashiro выполнил(а) push в master в jbell730/anya

3 дней назад

kashiro выполнил(а) push в master в jbell730/anya

8f2d8430d6 Group headlines into stories and filter by source count and date - Track source domain and publication time on each headline - Cluster near-duplicate headlines into stories (union-find) - Output stories covered by >=N distinct sources (--min-sources) - Add --since/--until date window - Ignore __pycache__ build artifacts

3 дней назад

a9f81efe28 Fix APP__ env prefix so .env database URL is honored. Figment was prefixed with APP_, so APP__DATABASE__URL never mapped to database.url and the server kept logging in as the default app role. Also walk up to the workspace .env and log a redacted DSN.

1 месяц назад