soj2<dZddlZdedefdZdedefdZded ed edefd Zded efd Zded ededefdZ dedefdZ dedefdZ dZ dS)aK MarkdownV2 (Telegram Bot API flavor) -> HTML converter. The original codebase was written against python-telegram-bot and formats all of its text using Bot API's MarkdownV2 dialect (see ``constants.TG_DEFAULT_PARSE_MODE`` and ``src/utils/string_utils.py``'s ``escape_valid_markdown_chars`` / ``escape_invalid_markdown_chars`` helpers). Earlier versions of this compat layer parsed MarkdownV2 directly into raw ``telethon.tl.types.MessageEntity*`` objects and passed them via ``formatting_entities=``, bypassing Telethon's own parser entirely. That approach turned out to be too fragile in practice (specific formatting - plain hyperlinks in particular - silently failed to render, without a reproducible root cause found through static analysis alone). This module instead converts Bot-API-flavored MarkdownV2 into the HTML dialect Telethon's own, battle-tested ``parse_mode='html'`` understands (``telethon/extensions/html.py``), and lets Telethon do the actual parsing, entity construction, and (for mentions) access-hash resolution - the same code path used by every other Telethon-based bot, rather than a hand-rolled one specific to this project. Supported syntax (https://core.telegram.org/bots/api#markdownv2-style): *bold* _italic_ __underline__ ~strikethrough~ ||spoiler|| `inline code` ```lang pre block ``` [text](http://example.com) -> text link [text](tg://user?id=12345) -> inline mention of a user by id >quoted line -> blockquote (consecutive '>' lines) >quoted line ending in || -> expandable/collapsible blockquote (this codebase's own convention, see resources/phrases_en.py's surround_with_expandable_quote) Any character preceded by ``\`` is treated as a literal character. Nesting is supported by recursively converting the inner text of each construct - since the output is HTML, nesting falls out naturally from properly nested tags rather than needing manual offset/length bookkeeping. Ntextreturnc.tj|dS)zHTML-escape literal text so it can't be misread as markup by Telethon's HTML parser. Does not escape quotes, since this text never ends up inside an HTML attribute.Fquote)_htmlescape)rs /app/src/tg_compat/_markdown.py_escaper 1s <E * * **urlcBd|vs|dr|Sd|zS)a A link whose url has no recognized scheme (e.g. a misconfigured env var storing "t.me/xxx" instead of "https://t.me/xxx") risks being silently rejected by Telegram's servers - the rest of the message still sends, but that one link renders as plain text with no visible error anywhere. tg://... links (used for user mentions, already a recognized scheme) are left untouched; anything else with no "scheme://" prefix gets "https://" prepended, matching how a browser address bar treats a bare domain. z://ztg:zhttps://) startswith)r s r _normalize_urlr8s- ||s~~e,,|  r sstarttokenc|}t|}t|}||kr0||dkr|dz }||||z|kr|S|dz }||k0dS)z@Find the first unescaped occurrence of `token` at/after `start`.\)len)rrrintok_lens r _find_unescapedrGsw A AA%jjG a%% Q44<< FA  QW_  & &H Q a%% 2r rc|||dzdkr&|d|dz}|dkr|dzndS||dkr&|d|dz}|dkr|dzndS||dkrOt||dzd}|dkr3||dz|d zd krt||d zd }|dkr|dzSdSdS) a If a self-contained, opaque construct starts at s[i] - a ```pre``` block, a `code` span, or a [text](url) link - return the index right after it ends. Otherwise return None, meaning s[i] is just an ordinary character. Mirrors the matching branches of _to_html() closely enough to identify the same spans, but only measures their extent rather than converting them - conversion still happens in the normal course of _to_html() once whatever *outer* delimiter search called this has been resolved. ```rN`r[]r())findr)rrend close_bracket close_parens r _atomic_span_endr*Vs QU|uffUAE""))sQww-ts{{ffS!a%  ))sQww-ts{{'1q5#66 B  1]Q%69J%J#Ks#R#R)!]Q->DDKb  "Q&t 4r delimc|}t|}t|}||krE||dkr|dz }||||z|kr|St||}||}@|dz }||kEdS)a Like _find_unescaped, but treats [text](url), `code`, and ```pre``` spans as opaque - a markdown-special character living inside one of those (almost always inside a URL) is never a candidate closing delimiter for whatever *outer* span is being closed here, matching how a real MarkdownV2 parser treats them. Without this, a raw, unescaped underscore inside a URL nested in (for example) an italic span reads, to a plain left-to-right scan, exactly like that span's own closing "_" - so this codebase's "_Posted by [u/{}]({author_url}) on [r/{}]({sub_url})_" pattern (reddit_service.py) broke on any Reddit username containing an underscore: the italic span closed early, mid-way through the username's URL, which truncated the link before its own closing ")" - so the link-parsing branch in _to_html() never found one and the whole thing fell back to literal, broken-looking [brackets](and parens) in the sent message, with the swallowed "_" visibly missing from the leaked URL text (Telegram then auto-linkified that leaked plain-text URL on its own, which is what made it look like *some* kind of working link despite being neither the intended link nor the intended text). rrNrr)rr*)rrr+rrdlenskip_tos r _find_closing_delimr/qs, A AA u::D a%% Q44<< FA  QT\?e # #H"1a((  A  Q a%% 2r c(d|vr|Sg}d}t|}||kr_||dkr-|dz|kr$|||dz|dz }n ||||dz }||k_d|S)a Undo MarkdownV2 backslash-escaping ("\X" -> "X"), for the few substrings that are taken as a raw slice instead of being run back through _to_html() - a link's URL, and the contents of `code`/```pre``` blocks. Real Bot API MarkdownV2 (what PTB sends to Telegram's own server-side parser) unescapes "\X" to "X" uniformly, wherever it appears in a message - including inside a link's URL and inside code/pre text. But this codebase's escape_invalid_markdown_chars/escape_valid_markdown_chars helpers (src/service/message_service.py) run over whole, already-built messages - deeplinks and all - without excluding text that happens to sit inside a [text](url), `code`, or ```pre``` construct. So a URL like "https://t.me/x" commonly arrives here as "https://t\.me/x" (the "." got escaped along with the rest of the message), and a deeplink's base64 payload commonly picks up an escaped "\=" from its padding. Without this, those literal backslashes were passed straight through: into the href attribute (producing a URL Telegram silently refuses to treat as clickable - the message still sends, the link just isn't a link) for links, and visibly in the rendered text for code/pre. Bold, italic, blockquotes etc. were never affected, since those recurse through _to_html() (which already unescapes via the `ch == "\"` branch below) - only these raw-slice cases skipped that step. rrrr)rappendjoin)routrrs r _unescaper5s2 4 C A D A a%% 7d??q1uqyy JJtAE{ # # # FAA JJtAw    FA a%% 773<<r sourcecN dt}gd.fd }|krl}|dkr:dz|kr1tdzdz O|dkr`dksdz d krJg}}||kr|dkr|dz }d |}|d kr ||d|}nP||||dz|kr|dzdkr|dz}n|}n||kr |dkd }|r0|d d r|d dd |d <d}d |}|rdnd} d| dt |d|dzdkr`ddz} | d kr)t|dz dz| } d} d | vr8| d d\} }| rtd| Ds| } |} | rkdt| dtt| dnYdtt| d| dz(|dkrddz} | d kr)t|dz vdz| }dtt|d| dz|dkrtdzd}|d kr͉|dz|dzdkrt|dzd }|d krdz|}tt|dz|}d!tj |d"dt |d#|dzщt|dz dzd kr9|d d$d %rt|dz Ddzd&kr9|d&d'd&%ret|dz |d(kr7|d(d)rt|dz |d*kr7|d*d+r߉t|dz |d,kr7|d,d-rt|dz Et|dz |kldS)/z...`. Returns True if a full match was consumed (i pointer already advanced), False if there was no valid closing delimiter (caller should emit the opening character literally and advance by one). rF<>z.consume_wrappeds"U!&!c%jj.'BB "995q3u::~+, $s$E$$$%%% 8E??### ;;;; #g,, tr rrrr: rFz||Tz expandablez rr c3>K|]}|VdS)N)isspace).0cs r z_to_html..s*)J)J!!))++)J)J)J)J)J)Jr z
z
z
z
r!zzr"r#r$r%z rdsJ++Z+#+#++++       s 3 s s    6$3$s$3$3$$$$N%C%C%%%%PvSvSvvvvr     r