Support http/https proxy for imap #9

Closed
opened 2026-04-27 22:10:56 +07:00 by thuanle · 3 comments
Owner
No description provided.
codex was assigned by thuanle 2026-04-27 22:11:15 +07:00
Collaborator

Đề xuất hoàn thiện requirement cho Support http/https proxy for imap:

Scope

  • Mở rộng IMAP_PROXY_URL từ SOCKS5-only thành hỗ trợ 3 scheme:
    • socks5://
    • http://
    • https://
  • Đây chỉ áp dụng cho IMAP ingress.
  • OpenClaw HTTP client vẫn đi direct như requirement hiện tại, không kế thừa proxy hệ thống và không dùng IMAP_PROXY_URL.

Contract mong muốn

  • Nếu IMAP_PROXY_URL không set: giữ nguyên direct dial hiện tại.
  • Nếu set socks5://: đi qua SOCKS5 proxy.
  • Nếu set http:// hoặc https://: đi qua HTTP CONNECT tunnel tới IMAP_HOST:IMAP_PORT.
  • Nếu proxy URL có user:pass@, bridge phải dùng credential đó để authenticate với proxy.
  • Với https://, chặng từ bridge tới proxy phải có TLS trước khi tạo CONNECT tunnel.
  • Nếu URL sai format, scheme không hỗ trợ, auth lỗi, CONNECT lỗi, hoặc proxy/tunnel fail:
    • current connect attempt phải fail rõ ràng,
    • log phải có error đủ để debug,
    • không được silently fallback sang direct dial,
    • reconnect loop của watcher vẫn tiếp tục như behavior hiện tại.

Config examples

  • IMAP_PROXY_URL="socks5://user:pass@proxy-host:port"
  • IMAP_PROXY_URL="http://user:pass@proxy-host:port"
  • IMAP_PROXY_URL="https://user:pass@proxy-host:port"

Acceptance criteria

  1. IMAP_PROXY_URL=socks5://... -> IMAP connection đi qua SOCKS5 path.
  2. IMAP_PROXY_URL=http://... -> IMAP connection đi qua HTTP CONNECT tunnel.
  3. IMAP_PROXY_URL=https://... -> IMAP connection đi qua HTTPS proxy + CONNECT tunnel.
  4. Proxy auth qua user:pass@ hoạt động cho các scheme hỗ trợ auth.
  5. IMAP_PROXY_URL sai format hoặc scheme không hỗ trợ -> fail rõ ràng, không fallback sang direct.
  6. Proxy auth / CONNECT / tunnel failure -> fail rõ ràng, reconnect loop vẫn tiếp tục.
  7. Khi không set IMAP_PROXY_URL, behavior direct hiện tại không đổi.

Test matrix tối thiểu

  1. socks5://... -> dùng SOCKS5 path.
  2. http://... -> dùng HTTP CONNECT path.
  3. https://... -> dùng HTTPS CONNECT path.
  4. Proxy URL có auth -> request dùng đúng credential.
  5. Scheme không hỗ trợ -> error rõ ràng.
  6. Proxy URL malformed -> error rõ ràng.
  7. Proxy tunnel/auth fail -> không direct fallback.
  8. Proxy unset -> direct path vẫn pass như cũ.

Nếu muốn giữ scope nhỏ hơn cho v1.1 thì có thể tách thành 2 phase:

  • phase 1: http:// support
  • phase 2: https:// support

Nhưng nếu để title là Support http/https proxy for imap thì req nên chốt cả 2 scheme ngay từ đầu để tránh mismatch giữa issue title và behavior thực tế.

Đề xuất hoàn thiện requirement cho `Support http/https proxy for imap`: ## Scope - Mở rộng `IMAP_PROXY_URL` từ SOCKS5-only thành hỗ trợ 3 scheme: - `socks5://` - `http://` - `https://` - Đây chỉ áp dụng cho IMAP ingress. - OpenClaw HTTP client vẫn đi direct như requirement hiện tại, không kế thừa proxy hệ thống và không dùng `IMAP_PROXY_URL`. ## Contract mong muốn - Nếu `IMAP_PROXY_URL` không set: giữ nguyên direct dial hiện tại. - Nếu set `socks5://`: đi qua SOCKS5 proxy. - Nếu set `http://` hoặc `https://`: đi qua HTTP CONNECT tunnel tới `IMAP_HOST:IMAP_PORT`. - Nếu proxy URL có `user:pass@`, bridge phải dùng credential đó để authenticate với proxy. - Với `https://`, chặng từ bridge tới proxy phải có TLS trước khi tạo CONNECT tunnel. - Nếu URL sai format, scheme không hỗ trợ, auth lỗi, CONNECT lỗi, hoặc proxy/tunnel fail: - current connect attempt phải fail rõ ràng, - log phải có error đủ để debug, - không được silently fallback sang direct dial, - reconnect loop của watcher vẫn tiếp tục như behavior hiện tại. ## Config examples - `IMAP_PROXY_URL="socks5://user:pass@proxy-host:port"` - `IMAP_PROXY_URL="http://user:pass@proxy-host:port"` - `IMAP_PROXY_URL="https://user:pass@proxy-host:port"` ## Acceptance criteria 1. `IMAP_PROXY_URL=socks5://...` -> IMAP connection đi qua SOCKS5 path. 2. `IMAP_PROXY_URL=http://...` -> IMAP connection đi qua HTTP CONNECT tunnel. 3. `IMAP_PROXY_URL=https://...` -> IMAP connection đi qua HTTPS proxy + CONNECT tunnel. 4. Proxy auth qua `user:pass@` hoạt động cho các scheme hỗ trợ auth. 5. `IMAP_PROXY_URL` sai format hoặc scheme không hỗ trợ -> fail rõ ràng, không fallback sang direct. 6. Proxy auth / CONNECT / tunnel failure -> fail rõ ràng, reconnect loop vẫn tiếp tục. 7. Khi không set `IMAP_PROXY_URL`, behavior direct hiện tại không đổi. ## Test matrix tối thiểu 1. `socks5://...` -> dùng SOCKS5 path. 2. `http://...` -> dùng HTTP CONNECT path. 3. `https://...` -> dùng HTTPS CONNECT path. 4. Proxy URL có auth -> request dùng đúng credential. 5. Scheme không hỗ trợ -> error rõ ràng. 6. Proxy URL malformed -> error rõ ràng. 7. Proxy tunnel/auth fail -> không direct fallback. 8. Proxy unset -> direct path vẫn pass như cũ. Nếu muốn giữ scope nhỏ hơn cho v1.1 thì có thể tách thành 2 phase: - phase 1: `http://` support - phase 2: `https://` support Nhưng nếu để title là `Support http/https proxy for imap` thì req nên chốt cả 2 scheme ngay từ đầu để tránh mismatch giữa issue title và behavior thực tế.
Collaborator

Final Requirements

Summary: Extend IMAP_PROXY_URL to support http:// and https:// proxy schemes (via HTTP CONNECT tunnel) in addition to the existing socks5:// support.

Scope:

  • Refactor dialTLSViaSOCKS5 into a generic proxy dialer that handles all three schemes.
  • http:// — plain TCP to proxy, then HTTP CONNECT tunnel to IMAP_HOST:IMAP_PORT.
  • https:// — TLS to proxy first, then HTTP CONNECT tunnel to IMAP_HOST:IMAP_PORT.
  • socks5:// — keep existing behavior unchanged.
  • Proxy auth via user:pass@ in the URL for all schemes that support it.
  • On any proxy/tunnel failure (bad URL, unsupported scheme, auth error, CONNECT failure): fail the connection attempt explicitly with a descriptive error log — never silently fall back to direct dial.
  • Reconnect loop (Run) continues retrying on failure as it does today.
  • OpenClaw HTTP client remains direct — it does not use IMAP_PROXY_URL.

Out of scope:

  • System-level proxy env vars (HTTP_PROXY, HTTPS_PROXY) — not inherited.
  • SOCKS4 or any scheme other than socks5, http, https.
  • Changes to the OpenClaw outbound HTTP path.

Acceptance criteria:

  1. IMAP_PROXY_URL=socks5://... → IMAP connection uses existing SOCKS5 path (no regression).
  2. IMAP_PROXY_URL=http://... → IMAP connection goes through HTTP CONNECT tunnel.
  3. IMAP_PROXY_URL=https://... → IMAP connection goes through TLS-to-proxy then CONNECT tunnel.
  4. user:pass@ in proxy URL → credentials are used to authenticate with the proxy.
  5. Unsupported scheme or malformed URL → clear error, no direct-dial fallback.
  6. Proxy auth / CONNECT / tunnel failure → clear error log, reconnect loop continues.
  7. IMAP_PROXY_URL unset → existing direct-dial behavior unchanged.

Test matrix (minimum):

# Scenario Expected
1 socks5://... SOCKS5 path
2 http://... HTTP CONNECT path
3 https://... HTTPS CONNECT path
4 URL with user:pass@ Auth credentials used
5 Unsupported scheme (e.g. ftp://) Clear error, no fallback
6 Malformed URL Clear error, no fallback
7 Proxy/tunnel failure Error logged, no direct fallback
8 IMAP_PROXY_URL empty/unset Direct dial, no regression

Note: Both http:// and https:// are included in this issue. The implementation difference is minimal (TLS before CONNECT for https), and the issue title explicitly covers both.

## Final Requirements **Summary:** Extend `IMAP_PROXY_URL` to support `http://` and `https://` proxy schemes (via HTTP CONNECT tunnel) in addition to the existing `socks5://` support. **Scope:** - Refactor `dialTLSViaSOCKS5` into a generic proxy dialer that handles all three schemes. - `http://` — plain TCP to proxy, then HTTP CONNECT tunnel to `IMAP_HOST:IMAP_PORT`. - `https://` — TLS to proxy first, then HTTP CONNECT tunnel to `IMAP_HOST:IMAP_PORT`. - `socks5://` — keep existing behavior unchanged. - Proxy auth via `user:pass@` in the URL for all schemes that support it. - On any proxy/tunnel failure (bad URL, unsupported scheme, auth error, CONNECT failure): fail the connection attempt explicitly with a descriptive error log — never silently fall back to direct dial. - Reconnect loop (`Run`) continues retrying on failure as it does today. - OpenClaw HTTP client remains direct — it does **not** use `IMAP_PROXY_URL`. **Out of scope:** - System-level proxy env vars (`HTTP_PROXY`, `HTTPS_PROXY`) — not inherited. - SOCKS4 or any scheme other than `socks5`, `http`, `https`. - Changes to the OpenClaw outbound HTTP path. **Acceptance criteria:** 1. `IMAP_PROXY_URL=socks5://...` → IMAP connection uses existing SOCKS5 path (no regression). 2. `IMAP_PROXY_URL=http://...` → IMAP connection goes through HTTP CONNECT tunnel. 3. `IMAP_PROXY_URL=https://...` → IMAP connection goes through TLS-to-proxy then CONNECT tunnel. 4. `user:pass@` in proxy URL → credentials are used to authenticate with the proxy. 5. Unsupported scheme or malformed URL → clear error, no direct-dial fallback. 6. Proxy auth / CONNECT / tunnel failure → clear error log, reconnect loop continues. 7. `IMAP_PROXY_URL` unset → existing direct-dial behavior unchanged. **Test matrix (minimum):** | # | Scenario | Expected | |---|----------|----------| | 1 | `socks5://...` | SOCKS5 path | | 2 | `http://...` | HTTP CONNECT path | | 3 | `https://...` | HTTPS CONNECT path | | 4 | URL with `user:pass@` | Auth credentials used | | 5 | Unsupported scheme (e.g. `ftp://`) | Clear error, no fallback | | 6 | Malformed URL | Clear error, no fallback | | 7 | Proxy/tunnel failure | Error logged, no direct fallback | | 8 | `IMAP_PROXY_URL` empty/unset | Direct dial, no regression | > **Note:** Both `http://` and `https://` are included in this issue. The implementation difference is minimal (TLS before CONNECT for `https`), and the issue title explicitly covers both.
Collaborator

Implementation Plan

Approach: Refactor dialTLSViaSOCKS5 into a generic dialTLSViaProxy that dispatches by URL scheme.

Changes in internal/mail/imap.go:

  1. Rename + refactor: dialTLSViaSOCKS5dialTLSViaProxy

    • Parse URL once, switch on scheme
    • socks5 → existing SOCKS5 logic (extracted to helper)
    • http / https → new dialViaCONNECT
    • Unknown scheme → clear error
    • After getting raw conn, common TLS handshake to IMAP server
  2. New dialViaCONNECT(u, targetAddr):

    • Dial proxy (plain TCP for http, TLS for https)
    • Send CONNECT target:port HTTP/1.1 with optional Basic auth
    • Read response, verify 200 status
    • Return tunnel connection (wrapped with proxyConn if bufio buffered data)
  3. New proxyConn type:

    • Wraps net.Conn + bufio.Reader to drain buffered data from CONNECT response before TLS handshake
  4. Update NewIMAPWatcher: dialIMAPViaProxy: dialTLSViaProxy

Changes in internal/mail/imap_test.go:

  • Update existing SOCKS5 test references
  • Add fake TCP proxy listener tests for HTTP CONNECT
  • Add TLS fake proxy tests for HTTPS CONNECT
  • Add proxy auth test
  • Add unsupported scheme / malformed URL tests

No new dependencies — uses only net/http, crypto/tls, bufio, encoding/base64 from stdlib.

## Implementation Plan **Approach:** Refactor `dialTLSViaSOCKS5` into a generic `dialTLSViaProxy` that dispatches by URL scheme. **Changes in `internal/mail/imap.go`:** 1. **Rename + refactor:** `dialTLSViaSOCKS5` → `dialTLSViaProxy` - Parse URL once, switch on scheme - `socks5` → existing SOCKS5 logic (extracted to helper) - `http` / `https` → new `dialViaCONNECT` - Unknown scheme → clear error - After getting raw conn, common TLS handshake to IMAP server 2. **New `dialViaCONNECT(u, targetAddr)`:** - Dial proxy (plain TCP for `http`, TLS for `https`) - Send `CONNECT target:port HTTP/1.1` with optional Basic auth - Read response, verify 200 status - Return tunnel connection (wrapped with `proxyConn` if bufio buffered data) 3. **New `proxyConn` type:** - Wraps `net.Conn` + `bufio.Reader` to drain buffered data from CONNECT response before TLS handshake 4. **Update `NewIMAPWatcher`:** `dialIMAPViaProxy: dialTLSViaProxy` **Changes in `internal/mail/imap_test.go`:** - Update existing SOCKS5 test references - Add fake TCP proxy listener tests for HTTP CONNECT - Add TLS fake proxy tests for HTTPS CONNECT - Add proxy auth test - Add unsupported scheme / malformed URL tests **No new dependencies** — uses only `net/http`, `crypto/tls`, `bufio`, `encoding/base64` from stdlib.
Sign in to join this conversation.
No labels
3 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: thuanle/claw-email#9