Merge pull request 'docs: reorganize README and technical overview' (#23 ) from docs/readme-docs-reorg into main

Reviewed-on: #23
docs: reorganize readme and architecture docs
2026-04-29 00:52:41 +07:00 · 2026-04-29 00:50:08 +07:00 · 2026-04-29 00:38:11 +07:00 · 2026-04-29 00:27:30 +07:00 · 2026-04-28 23:09:01 +07:00 · 2026-04-28 11:12:29 +07:00
11 changed files with 635 additions and 111 deletions
@@ -0,0 +1,89 @@
+---
+name: create-adr
+description: Use when a change introduces or updates a durable cross-cutting technical decision that future work in this repository must preserve, such as dependency, storage, integration, workflow, or quality-gate policy changes
+---
+
+# Create ADR
+
+## Overview
+
+Create a new ADR in `docs/adr/**` when the current task locks in a durable technical decision.
+Do not use this for bug fixes, styling changes, routine refactors, or ordinary test additions.
+
+## Use This Skill For
+
+- introducing or removing a major dependency
+- choosing or changing a storage strategy
+- defining release, branch, or workflow policy
+- changing how agents, jobs, or external tools are integrated
+- adopting or tightening repository quality gates
+- changing a behavior-defining contract future work must preserve
+
+## Do Not Use This Skill For
+
+- bug fixes
+- styling-only changes
+- routine refactors
+- ordinary test additions
+- temporary plans or execution notes
+
+## Workflow
+
+1. Check whether an active ADR already covers the same decision.
+   - Read `docs/adr/README.md` and scan existing files under `docs/adr/`.
+   - If the current change is only implementing an already-recorded decision, do not create a new ADR.
+2. If the new decision changes or replaces an older ADR, create a new ADR that supersedes it.
+   - Do not rewrite the meaning of an active ADR in place.
+3. Find the next numeric ID under `docs/adr/`.
+4. Create `docs/adr/NNNN-short-kebab-title.md`.
+5. Use this template:
+
+```md
+---
+type: ADR
+id: "NNNN"
+title: "Short decision title"
+status: active
+date: YYYY-MM-DD
+---
+
+## Context
+
+What changed? Why does this decision need to exist now?
+
+## Decision
+
+**State the decision clearly in one or two sentences.**
+
+## Options considered
+
+- **Option A** (chosen): description with pros and cons
+- **Option B**: description with pros and cons
+- **Option C**: description with pros and cons
+
+## Consequences
+
+What becomes easier?
+What becomes harder?
+What risks were accepted?
+What would trigger re-evaluation?
+
+## Advice
+
+Optional. Capture important external guidance or review input.
+```
+
+6. Add the new ADR to the index in `docs/adr/README.md`.
+7. Commit the ADR in the same change as the implementation it documents.
+8. Mention the ADR in the issue comment and PR body under `Docs/ADR`.
+
+## Rule Of Thumb
+
+If the decision changes how future work should be done across the repository, it probably deserves an ADR.
+
+## Repository Rules
+
+- Keep one decision per file.
+- Name files `NNNN-short-kebab-title.md`.
+- Keep the `docs/adr/README.md` index aligned with the files on disk.
+- Prefer concise, durable wording over implementation detail.
@@ -0,0 +1,3 @@
+interface:
+  display_name: "Create ADR"
+  short_description: "Record a new architecture decision for durable repo changes"
@@ -33,6 +33,9 @@ description: Phase 2 skill. Use when issue scope is locked by Product Owner (Pha
 3. **Write plan and post on issue**
   - List implementation steps with verification criteria.
   - List files to change.
+   - State whether `README.md` or `.env.example` must change.
+   - State whether an ADR is required or not required.
+   - If ADR is required or recommended, invoke `create-adr` before finalizing the implementation summary and PR body.
   - Flag risks or tradeoffs.
   - Post plan as comment: `mcp__gitea__.issue_write` (`method: "add_comment"`).
   - **Wait for "Approved" before coding.** Stop here.
@@ -48,18 +51,32 @@ description: Phase 2 skill. Use when issue scope is locked by Product Owner (Pha
     git worktree add .worktrees/<branch-name> main
     ```
   - Implement following the plan (surgical changes only).
+   - Update `README.md` and `.env.example` when config, endpoint, or runtime behavior changes.
+   - If the change introduces a durable behavior, contract, architecture, or workflow decision, use `create-adr` and create/update `docs/adr/**`.
   - Run validation:
     ```
     gofmt -l .
-     go vet ./...
     go test ./...
+     go vet ./...
+     staticcheck ./...
     ```
+   - Post an implementation summary comment on the issue with sections:
+     - `Summary`
+     - `Changes`
+     - `Test Plan`
+     - `Refactor`
+     - `Docs/ADR`
+     - `Risks/Follow-ups`

 5. **Create PR**
   - Use `mcp__gitea__.pull_request_write` (`method: "create"`).
   - PR body must include:
-     - Summary of changes
-     - Test evidence (exact commands and output)
+     - `Summary`
+     - `Changes`
+     - `Test Plan` with exact commands and output
+     - `Refactor`
+     - `Docs/ADR`
+     - `Risks/Follow-ups`
     - `Fixes #<issue-id>`
   - Add reviewer.

@@ -76,9 +93,11 @@ description: Phase 2 skill. Use when issue scope is locked by Product Owner (Pha
 - Use Gitea MCP tools for all issue/PR operations.
 - Preserve existing assignees when adding yourself unless the user explicitly asks to replace them.
 - Never edit remote files directly — always work on local branch.
+- Satisfy the repo Definition of Done before claiming completion.
+- Missing required tests, docs, `.env.example` updates, or ADRs is a quality failure unless explicitly justified in both the issue comment and PR body.
 - Never claim completion without fresh verification output.
 - Keep changes strictly scoped to the approved plan.
- If the change needs durable documentation, create/update `docs/adr/**`. Do not commit `docs/superpowers/**` planning artifacts.
+- If the change needs durable documentation, use `create-adr` and create/update `docs/adr/**`. Do not commit `docs/superpowers/**` planning artifacts.

 ## Next skill
 After PR is created, the user invokes `/review-pr <pr-number>` on the Reviewer agent.
@@ -47,8 +47,9 @@ description: Phase 2 skill. Use when a PR is created or updated. Review PR again
   - Run checks:
     ```
     gofmt -l .
-     go vet ./...
     go test ./...
+     go vet ./...
+     staticcheck ./...
     ```
   - Remove worktree when done:
     ```
@@ -57,6 +58,11 @@ description: Phase 2 skill. Use when a PR is created or updated. Review PR again

 5. **Evaluate**
   - Does PR address **all** Final Requirements from the issue?
+   - Does PR satisfy the repo Definition of Done?
+     - relevant tests for behavior/config changes
+     - required `README.md` / `.env.example` updates
+     - ADR present when the change locks in a durable decision
+     - required implementation evidence exists on the issue and in the PR body
   - Does PR contain unrelated changes or refactoring? → FAIL immediately.
   - Code correctness, style, test coverage.

@@ -85,6 +91,7 @@ Validation:
 - gofmt -l .: OK
 - go vet ./...: OK
 - go test ./...: OK
+- staticcheck ./...: OK

 Non-blocking:
 - <optional note>
@@ -106,6 +113,7 @@ Validation:
 - Do not add yourself as reviewer twice; check requested reviewers and existing reviews first.
 - Do NOT edit code on the PR.
 - FAIL if PR contains unrelated changes or scope creep.
+- FAIL if the PR misses required DoD evidence, docs, or ADR work for the accepted change.
 - Post review proactively if verdict is clear.
 - Do not post a verdict until PR discussion comments have been checked.
 - Mention validation commands you actually ran.
@@ -27,6 +27,8 @@ description: Phase 1 skill. Use when a Gitea issue is created or updated and nee
   - Is the problem statement clear?
   - Is the scope well-defined or too broad?
   - Is it technically feasible?
+   - What acceptance criteria will prove the change is done?
+   - Will the change require docs updates or a durable ADR?
   - If needed, read relevant code: `mcp__gitea__.get_file_contents`.

 3. **Decide**
@@ -34,8 +36,10 @@ description: Phase 1 skill. Use when a Gitea issue is created or updated and nee
     - Post clarifying questions: `mcp__gitea__.issue_write` (`method: "add_comment"`).
     - Stop. Wait for user to respond, then re-run this skill.
   - **Clear and feasible:**
-     - Post Final Requirements (see format below).
-     - If the change likely needs a durable ADR, add a note: "ADR recommended: this change locks in a behavior/contract that future work must preserve."
+     - Post Final Requirements (see format below) with specific acceptance criteria.
+     - State whether docs updates are expected.
+     - State whether an ADR is required, recommended, or not needed.
+     - If ADR is `required` or `recommended`, tell the Developer to use `create-adr` during implementation.
     - This locks the scope and signals Phase 2 can begin.

 ## Final Requirements format
@@ -55,11 +59,18 @@ description: Phase 1 skill. Use when a Gitea issue is created or updated and nee
 **Acceptance criteria:**
 - <verifiable check 1>
 - <verifiable check 2>
+
+**Documentation impact:**
+- <README.md / .env.example updates required or not required>
+
+**ADR:**
+- <required / recommended / not needed, with one-line reason>
 ```

 ## Rules
 - Use Gitea MCP tools for all operations.
 - Do NOT write code, create branches, or create PRs.
+- Make acceptance criteria specific enough that Dev and Reviewer can verify Definition of Done without guessing.
 - Do NOT proceed to Phase 2 — that is a separate skill.

 ## Next skill
@@ -6,11 +6,10 @@ on:
 jobs:
  fmt:
    runs-on: linux
+    container:
+      image: git.thuanle.me/public/node-go
    steps:
      - uses: actions/checkout@v4
-      - uses: actions/setup-go@v5
-        with:
-          go-version-file: go.mod
      - run: |
          out=$(gofmt -l .)
          if [ -n "$out" ]; then
@@ -21,28 +20,25 @@ jobs:

  test:
    runs-on: linux
+    container:
+      image: git.thuanle.me/public/node-go
    steps:
      - uses: actions/checkout@v4
-      - uses: actions/setup-go@v5
-        with:
-          go-version-file: go.mod
      - run: go test ./...

  vet:
    runs-on: linux
+    container:
+      image: git.thuanle.me/public/node-go
    steps:
      - uses: actions/checkout@v4
-      - uses: actions/setup-go@v5
-        with:
-          go-version-file: go.mod
      - run: go vet ./...

  staticcheck:
    runs-on: linux
+    container:
+      image: git.thuanle.me/public/node-go
    steps:
      - uses: actions/checkout@v4
-      - uses: actions/setup-go@v5
-        with:
-          go-version-file: go.mod
      - run: go install honnef.co/go/tools/cmd/staticcheck@2025.1.1
      - run: $(go env GOPATH)/bin/staticcheck ./...
@@ -3,16 +3,19 @@
 ## 1. Coding Principles

 ### 1.1 Simplicity First
+
 - No out-of-scope features.
 - No abstractions for single-use code.
 - If 50 lines work instead of 200, write 50.

 ### 1.2 Surgical Changes
+
 - Only change lines directly related to the request.
 - Never refactor adjacent code unless asked.
 - Remove orphaned imports/variables/functions your changes created.

 ### 1.3 Goal-Driven Execution
+
 - Define success criteria before coding.
 - Verify each step before moving to the next.

@@ -25,6 +28,24 @@ All work starts from a Gitea issue. Two phases, each with a dedicated role:
 1. **Phase 1 — Requirement Refinement** (Product Owner): Clarify and lock scope.
 2. **Phase 2 — Requirement Implementation** (Developer & Reviewer): Plan, code, review, merge.

+### 2.1 Definition of Done
+
+A change is done only when all of the following are true:
+
+- The issue has locked **Final Requirements** with verifiable acceptance criteria.
+- The implementation stays within approved scope; unrelated refactors are not allowed.
+- Behavior, contract, or config changes have relevant tests added or updated, or the change record explains why tests were not added.
+- Validation passes with fresh output: `gofmt -l .`, `go test ./...`, `go vet ./...`, `staticcheck ./...`.
+- `README.md` and `.env.example` and related documents are updated when config, endpoint, or runtime behavior changes.
+- A new ADR is added in `docs/adr/**` when the change introduces a durable architecture, contract, behavior, or workflow decision.
+- The implementation evidence is posted on the issue and reflected in the PR body:
+  - what changed
+  - what tests were run
+  - whether any refactor was included
+  - whether docs or ADRs were added or not needed
+  - risks or follow-ups
+- After merge, the linked issue is verified closed.
+
 ---

 ## 3. Phase 1: Requirement Refinement
@@ -52,13 +73,16 @@ All work starts from a Gitea issue. Two phases, each with a dedicated role:

 1. Read the locked issue and any clarification comments.
 2. Read relevant code to understand current state.
-3. Write an implementation plan — list steps, files to change, risks.
+3. Write an implementation plan — list steps, files to change, risks, verification criteria, and any docs/ADR impact.
 4. **Comment the plan on the issue. Wait for "Approved" before coding.**
 5. Once approved:
   - Create a feature branch from `main`.
   - Implement following the plan (surgical changes only).
-   - Run validation: `go test ./...`, `go vet ./...`, `gofmt -l .`.
-   - Create PR with `Fixes #<issue>` in the body so Gitea auto-closes the issue on merge.
+   - Update `README.md` and `.env.example` if config, endpoint, or runtime behavior changes.
+   - If the change locks in a durable decision, write/update an ADR in `docs/adr/**`.
+   - Run validation: `gofmt -l .`, `go test ./...`, `go vet ./...`, `staticcheck ./...`.
+   - Post an implementation summary comment on the issue with sections: `Summary`, `Changes`, `Test Plan`, `Refactor`, `Docs/ADR`, `Risks/Follow-ups`.
+   - Create PR with the same evidence in the body plus `Fixes #<issue>` so Gitea auto-closes the issue on merge.
   - After merge, verify the issue is closed. If not (e.g. missing keyword), close it manually with a summary comment.
 6. **Own the PR until merged:**
   - Monitor for review feedback.
@@ -71,10 +95,15 @@ All work starts from a Gitea issue. Two phases, each with a dedicated role:

 1. Re-read the original issue — verify PR addresses the locked requirement.
 2. Read PR diff, review history, and PR discussion comments.
-3. Validate in worktree if needed (`.worktrees/pr-<number>`, run `go test`, `go vet`, `gofmt`).
+3. Validate in worktree if needed (`.worktrees/pr-<number>`, run `gofmt -l .`, `go test ./...`, `go vet ./...`, `staticcheck ./...`).
 4. Check for scope creep — FAIL if PR contains unrelated changes or refactoring.
-5. Write verdict — start with **PASS** or **FAIL**.
-6. Post review proactively if verdict is clear:
+5. Verify the repo Definition of Done:
+   - relevant tests exist for behavior/config changes
+   - docs and `.env.example` were updated when required
+   - ADR exists when a durable decision was introduced
+   - the issue comment and PR body contain the required implementation evidence
+6. Write verdict — start with **PASS** or **FAIL**.
+7. Post review proactively if verdict is clear:
   - `PASS` → `APPROVED`
   - `FAIL` → `REQUEST_CHANGES`
   - Non-blocking note → `COMMENT`
@@ -83,10 +112,9 @@ All work starts from a Gitea issue. Two phases, each with a dedicated role:

 ## 5. Gitea Tools

- If git remote contains `git.thuanle.me`, ALWAYS use Gitea MCP tools.
+- Always use Gitea MCP tools in this project.
 - Scope: read PR/issue, list comments, post replies, create/edit PRs, reviews.
 - Use `tea` CLI only as fallback when MCP is unavailable.
- Do not use `gh` for Gitea repositories.
 - Local CLI tools (`git`, `go`, `rg`) are fine for local validation.

 ---
@@ -99,8 +127,10 @@ PASS
 No blocking issues found.

 Validation:
+- gofmt -l .: OK
 - go test ./...: OK
 - go vet ./...: OK
+- staticcheck ./...: OK

 Non-blocking:
 - <optional note>
@@ -113,7 +143,7 @@ FAIL
 2. [Medium] <issue with path:line and impact>

 Validation:
- go test ./...: OK
+- <list only the commands you actually ran and their result>
 ```

 ---
@@ -130,6 +160,7 @@ Validation:
 ## 8. Comment Style

 - Use readable multiline markdown.
- Use short sections (Summary, Changes, Test Plan).
+- Use short sections (`Summary`, `Changes`, `Test Plan`).
+- Implementation completion comments should also include `Refactor`, `Docs/ADR`, and `Risks/Follow-ups`.
 - Concise bullet points and explicit line breaks.
 - No bracketed author/tool tags.
@@ -1,38 +1,26 @@
 # Email-OpenClaw Bridge

-Tự động xử lý email bằng OpenClaw AI theo luồng:
+Bridge tự động xử lý email bằng OpenClaw AI theo luồng:

-```
-Ingress (IMAP) → Safety Checks → External Rules → Apply Context → Dispatch (OpenClaw) → Callback → Egress (SMTP reply)
+```text
+Ingress (IMAP) → Safety Checks → External Rules → Dispatch (OpenClaw) → Callback → Egress (SMTP reply)
 ```

-## Tính năng
+README này tập trung vào onboarding và vận hành cơ bản. Chi tiết kỹ thuật hiện hành được tách sang `docs/architecture/overview.md`.

- **IMAP IDLE** — theo dõi mailbox realtime, nhận email mới tức thì
- **Threading** — giữ nguyên email thread qua `In-Reply-To` / `References`
- **Idempotency** — chống duplicate theo `message_id`
- **Anti-loop & Whitelist** — bỏ qua email từ chính hệ thống, chỉ xử lý sender được phép
- **External Rules** — pipeline configurable: whitelist, blocklist, và rule metadata cho dispatch
- **Retry** — tối đa 3 lần với backoff 1s → 5s → 15s cho cả OpenClaw và SMTP
- **Stateful** — SQLite lưu trạng thái pipeline, không mất dữ liệu khi restart
+## Tính năng chính

-## Kiến trúc
+- Theo dõi mailbox realtime bằng IMAP IDLE
+- Giữ thread email qua `In-Reply-To` và `References`
+- Chống duplicate theo `message_id`
+- Chặn anti-loop và hỗ trợ whitelist/blocklist
+- Gửi callback vào bridge rồi phản hồi lại qua SMTP
+- Lưu trạng thái pipeline bằng SQLite để không mất tiến trình khi restart

-```
-cmd/bridge/main.go           Entrypoint
-internal/
-├── config/                   Đọc cấu hình từ .env
-├── database/                 SQLite + GORM, auto-migrate
-├── logging/                  Structured JSON logging (slog)
-├── api/                      HTTP server (Gin): health checks, webhook callback
-├── mail/                     IMAP ingress + SMTP egress
-└── ai_client/                OpenClaw HTTP dispatch
-```
-
-## Yêu cầu
+## Yêu cầu tối thiểu

 - Go 1.23+
- Docker & Docker Compose (cho deploy)
+- Docker và Docker Compose nếu chạy bằng container

 ## Quickstart

@@ -43,7 +31,7 @@ cp .env.example .env
 # Sửa .env với thông tin IMAP/SMTP/OpenClaw thực tế
 ```

-### 2. Chạy local (development)
+### 2. Chạy local

 ```bash
 go run ./cmd/bridge
@@ -56,51 +44,39 @@ docker compose up -d
 docker logs -f claw-email-bridge
 ```

-## Cấu hình (.env)
+## Cấu hình `.env`

 | Biến | Bắt buộc | Mô tả |
 |---|:---:|---|
 | `IMAP_HOST` | ✅ | IMAP server hostname |
-| `IMAP_PORT` | ✅ | IMAP port (thường `993`) |
+| `IMAP_PORT` | ✅ | IMAP port, thường là `993` |
 | `IMAP_USER` | ✅ | IMAP username |
 | `IMAP_PASS` | ✅ | IMAP password |
 | `SMTP_HOST` | ✅ | SMTP server hostname |
-| `SMTP_PORT` | ✅ | SMTP port (thường `587`) |
+| `SMTP_PORT` | ✅ | SMTP port, thường là `587` |
 | `SMTP_USER` | ✅ | SMTP username |
 | `SMTP_PASS` | ✅ | SMTP password |
 | `OPENCLAW_URL` | ✅ | OpenClaw API endpoint |
-| `OPENCLAW_API_KEY` | | OpenClaw API key (nếu cần) |
+| `OPENCLAW_API_KEY` | | OpenClaw API key nếu cần |
 | `BRIDGE_CALLBACK_TOKEN` | ✅ | Token xác thực webhook callback |
-| `SYSTEM_EMAIL` | ✅ | Email của hệ thống (dùng cho anti-loop) |
+| `SYSTEM_EMAIL` | ✅ | Email của bridge, dùng cho anti-loop |
 | `WHITELIST_EMAILS` | | Danh sách email được phép, phân cách bằng dấu phẩy |
-| `IMAP_PROXY_URL` | | SOCKS5 proxy cho IMAP (vd: `socks5://user:pass@host:port`) |
-| `LISTEN_ADDR` | | Địa chỉ bind HTTP server cục bộ (mặc định `:8080`) |
-| `CALLBACK_BASE_URL` | | Base URL public để tạo `callback_url` gửi OpenClaw (mặc định theo `LISTEN_ADDR`, ví dụ `http://localhost:8080`) |
+| `BLOCKLIST_EMAILS` | | Danh sách email bị chặn, phân cách bằng dấu phẩy |
+| `IMAP_PROXY_URL` | | Proxy cho IMAP, hỗ trợ `socks5://`, `http://`, `https://` |
+| `LISTEN_ADDR` | | Địa chỉ bind HTTP server, mặc định `:8080` |
+| `CALLBACK_BASE_URL` | | Base URL public để tạo `callback_url`, mặc định suy ra từ `LISTEN_ADDR` |

-## Health Checks
+## Health checks

 | Endpoint | Mục đích |
 |---|---|
-| `GET /healthz` | Liveness — process còn sống |
-| `GET /readyz` | Readiness — DB writable + config hợp lệ |
+| `GET /healthz` | Liveness check, xác nhận process còn sống |
+| `GET /readyz` | Readiness check, xác nhận DB writable và config bắt buộc hợp lệ |

-## Pipeline trạng thái
+## Đọc thêm

-```
-RECEIVED → AI_PROCESSING → CALLBACK_DONE → COMPLETED
-    ↓           ↓                ↓
- IGNORED      FAILED        SMTP_RETRYING → FAILED
-```
-
-## Công nghệ
-
-| Thành phần | Thư viện |
-|---|---|
-| HTTP Server | [gin-gonic/gin](https://github.com/gin-gonic/gin) |
-| Database | [GORM](https://gorm.io) + [modernc.org/sqlite](https://pkg.go.dev/modernc.org/sqlite) (pure Go, no CGO) |
-| IMAP | [emersion/go-imap](https://github.com/emersion/go-imap) |
-| Logging | [log/slog](https://pkg.go.dev/log/slog) (stdlib) |
-| Config | [joho/godotenv](https://github.com/joho/godotenv) |
+- [Technical overview](docs/architecture/overview.md)
+- [Architecture Decision Records](docs/adr/README.md)

 ## License

@@ -0,0 +1,101 @@
+# Technical Overview
+
+## Mục đích
+
+Tài liệu này là technical overview hiện hành của `Email-OpenClaw Bridge`. `README.md` chỉ giữ phần onboarding và vận hành cơ bản; phần này giữ các chi tiết kỹ thuật bền vững hơn về flow, package layout, state machine và công nghệ chính.
+
+## Luồng xử lý
+
+Bridge xử lý email theo pipeline sau:
+
+```text
+Ingress (IMAP) → Safety Checks → External Rules → Dispatch (OpenClaw) → Callback → Egress (SMTP reply)
+```
+
+Các giai đoạn runtime chính:
+
+1. IMAP watcher theo dõi `INBOX` bằng IDLE và fetch email chưa đọc.
+2. Safety checks loại bỏ mail từ chính hệ thống và chặn duplicate theo `message_id`.
+3. External rules pipeline áp whitelist/blocklist và sinh `DispatchContext`.
+4. Thread resolution gắn `thread_id` dựa trên quan hệ email đã biết.
+5. Task được lưu vào SQLite với trạng thái pipeline hiện tại.
+6. OpenClaw nhận request xử lý và callback về bridge.
+7. Callback cập nhật `AIResponse`, sau đó bridge gửi phản hồi lại qua SMTP.
+
+## Package layout
+
+- `cmd/bridge`
+  - Entrypoint của process, wiring config, DB, IMAP watcher, HTTP router và SMTP egress.
+- `internal/config`
+  - Đọc `.env`, chuẩn hóa default như `LISTEN_ADDR`, `CALLBACK_BASE_URL`, và validate biến bắt buộc.
+- `internal/database`
+  - Khởi tạo SQLite qua GORM, định nghĩa `Task` model, repository helpers và health check DB.
+- `internal/mail`
+  - Chứa IMAP ingress, thread resolution, proxy support cho IMAP, retry logic và SMTP egress.
+- `internal/rules`
+  - External rules pipeline, whitelist/blocklist rules và rule metadata dùng cho dispatch.
+- `internal/ai_client`
+  - HTTP client gọi OpenClaw và gửi metadata/callback information ra ngoài.
+- `internal/api`
+  - HTTP router cho `healthz`, `readyz` và `POST /callback`.
+- `internal/logging`
+  - Helper tạo structured logger theo `task_uuid`, `thread_id`, `message_id`.
+
+## Task state machine
+
+Task được lưu trong SQLite và đi qua các trạng thái sau:
+
+```text
+RECEIVED → AI_PROCESSING → CALLBACK_DONE → COMPLETED
+    ↓           ↓                ↓
+ IGNORED      FAILED        SMTP_RETRYING → FAILED
+```
+
+- `IGNORED`
+  - Mail bị anti-loop, duplicate, hoặc bị external rules reject.
+- `RECEIVED`
+  - Task đã được lưu và sẵn sàng dispatch.
+- `AI_PROCESSING`
+  - Request đã được gửi sang OpenClaw.
+- `CALLBACK_DONE`
+  - Callback hợp lệ đã cập nhật `AIResponse`.
+- `SMTP_RETRYING`
+  - Đang retry gửi email phản hồi.
+- `COMPLETED`
+  - Toàn bộ pipeline kết thúc thành công.
+- `FAILED`
+  - Hết retry hoặc gặp lỗi không recoverable.
+
+## HTTP và health behavior
+
+- `GET /healthz`
+  - Trả `200 {"status":"ok"}` nếu process đang sống.
+- `GET /readyz`
+  - Trả `200` khi DB writable và `BRIDGE_CALLBACK_TOKEN` có mặt.
+  - Trả `503` nếu DB không writable hoặc config bắt buộc cho callback chưa sẵn sàng.
+- `POST /callback`
+  - Yêu cầu header `X-Bridge-Token`.
+  - Nhận payload callback từ OpenClaw, cập nhật task và kích hoạt SMTP egress bất đồng bộ.
+
+## Công nghệ chính
+
+- Go
+  - Runtime chính, phù hợp với service nhỏ gọn và dễ deploy.
+- SQLite + GORM
+  - Giữ state cục bộ đơn giản, không cần service DB riêng cho v1.
+- `github.com/emersion/go-imap/v2`
+  - Hỗ trợ IMAP IDLE và xử lý mailbox theo flow hiện tại.
+- `gin-gonic/gin`
+  - Cung cấp HTTP router tối giản cho health endpoints và callback.
+- `log/slog`
+  - Structured logging chuẩn thư viện chuẩn của Go.
+
+## ADR liên quan
+
+- [ADR 0001: IMAP thread resolution prefers In-Reply-To and falls back to References](../adr/0001-imap-thread-resolution.md)
+- [ADR 0002: Ingress uses a configurable external rules pipeline before dispatch](../adr/0002-external-rules-pipeline.md)
+- [ADR index](../adr/README.md)
+
+## Tài liệu lịch sử
+
+- Historical design input: [docs/specs/2026-04-24-email-openclaw-bridge-design.md](../specs/2026-04-24-email-openclaw-bridge-design.md)
@@ -1,13 +1,16 @@
 package mail

 import (
+	"bufio"
 	"context"
 	"crypto/tls"
+	"encoding/base64"
 	"encoding/json"
 	"fmt"
 	"io"
 	"log/slog"
 	"net"
+	"net/http"
 	"net/url"
 	"strings"
 	"time"
@@ -59,6 +62,10 @@ func (d timeoutContextDialer) DialContext(ctx context.Context, network, address

 var socks5DialerFactory = proxy.SOCKS5

+var proxyTLSConfigForTest = func(host string) *tls.Config {
+	return &tls.Config{ServerName: host}
+}
+
 // NewIMAPWatcher creates a new IMAP watcher.
 func NewIMAPWatcher(cfg *config.Config, db *gorm.DB) *IMAPWatcher {
 	var externalRules []rules.Rule
@@ -73,7 +80,7 @@ func NewIMAPWatcher(cfg *config.Config, db *gorm.DB) *IMAPWatcher {
 		cfg:              cfg,
 		db:               db,
 		dialIMAP:         imapclient.DialTLS,
-		dialIMAPViaProxy: dialTLSViaSOCKS5,
+		dialIMAPViaProxy: dialTLSViaProxy,
 		rules:            rules.NewPipeline(externalRules),
 	}
 }
@@ -193,34 +200,20 @@ func (w *IMAPWatcher) connectAndWatch(ctx context.Context) error {
 	}
 }

-func dialTLSViaSOCKS5(addr, proxyURL string, options *imapclient.Options) (*imapclient.Client, error) {
+func dialTLSViaProxy(addr, proxyURL string, options *imapclient.Options) (*imapclient.Client, error) {
 	u, err := url.Parse(proxyURL)
 	if err != nil {
 		return nil, fmt.Errorf("parse proxy url: %w", err)
 	}
-	if u.Scheme != "socks5" {
-		return nil, fmt.Errorf("unsupported proxy scheme: %s", u.Scheme)
-	}
-
-	var auth *proxy.Auth
-	if u.User != nil {
-		pw, _ := u.User.Password()
-		auth = &proxy.Auth{User: u.User.Username(), Password: pw}
-	}
-
-	dialer, err := socks5DialerFactory("tcp", u.Host, auth, timeoutContextDialer{timeout: imapDialTimeout})
-	if err != nil {
-		return nil, fmt.Errorf("create socks5 dialer: %w", err)
-	}
-
-	dialCtx, dialCancel := context.WithTimeout(context.Background(), imapDialTimeout)
-	defer dialCancel()

 	var conn net.Conn
-	if cd, ok := dialer.(proxy.ContextDialer); ok {
-		conn, err = cd.DialContext(dialCtx, "tcp", addr)
-	} else {
-		conn, err = dialer.Dial("tcp", addr)
+	switch u.Scheme {
+	case "socks5":
+		conn, err = dialViaSOCKS5(u, addr)
+	case "http", "https":
+		conn, err = dialViaCONNECT(u, addr)
+	default:
+		return nil, fmt.Errorf("unsupported proxy scheme: %s", u.Scheme)
 	}
 	if err != nil {
 		return nil, fmt.Errorf("proxy dial: %w", err)
@@ -243,6 +236,96 @@ func dialTLSViaSOCKS5(addr, proxyURL string, options *imapclient.Options) (*imap
 	return imapclient.New(tlsConn, options), nil
 }

+func dialViaSOCKS5(u *url.URL, targetAddr string) (net.Conn, error) {
+	var auth *proxy.Auth
+	if u.User != nil {
+		pw, _ := u.User.Password()
+		auth = &proxy.Auth{User: u.User.Username(), Password: pw}
+	}
+
+	dialer, err := socks5DialerFactory("tcp", u.Host, auth, timeoutContextDialer{timeout: imapDialTimeout})
+	if err != nil {
+		return nil, fmt.Errorf("create socks5 dialer: %w", err)
+	}
+
+	dialCtx, dialCancel := context.WithTimeout(context.Background(), imapDialTimeout)
+	defer dialCancel()
+
+	if cd, ok := dialer.(proxy.ContextDialer); ok {
+		return cd.DialContext(dialCtx, "tcp", targetAddr)
+	}
+	return dialer.Dial("tcp", targetAddr)
+}
+
+// proxyConn wraps a net.Conn, draining buffered data from a bufio.Reader
+// before delegating reads to the underlying connection.
+type proxyConn struct {
+	net.Conn
+	reader io.Reader
+}
+
+func (c *proxyConn) Read(b []byte) (int, error) {
+	return c.reader.Read(b)
+}
+
+// dialToProxy is the function used to establish a TCP connection to the proxy.
+var dialToProxy = func(ctx context.Context, network, addr string) (net.Conn, error) {
+	return (&net.Dialer{Timeout: imapDialTimeout}).DialContext(ctx, network, addr)
+}
+
+func dialViaCONNECT(u *url.URL, targetAddr string) (net.Conn, error) {
+	ctx, cancel := context.WithTimeout(context.Background(), imapDialTimeout)
+	defer cancel()
+
+	conn, err := dialToProxy(ctx, "tcp", u.Host)
+	if err != nil {
+		return nil, fmt.Errorf("dial proxy %s: %w", u.Host, err)
+	}
+
+	if u.Scheme == "https" {
+		tlsConn := tls.Client(conn, proxyTLSConfigForTest(u.Hostname()))
+		if err := tlsConn.HandshakeContext(ctx); err != nil {
+			_ = tlsConn.Close()
+			return nil, fmt.Errorf("tls handshake to proxy: %w", err)
+		}
+		conn = tlsConn
+	}
+
+	_ = conn.SetDeadline(time.Now().Add(imapDialTimeout))
+
+	connectReq := fmt.Sprintf("CONNECT %s HTTP/1.1\r\nHost: %s\r\n", targetAddr, targetAddr)
+	if u.User != nil {
+		username := u.User.Username()
+		password, _ := u.User.Password()
+		creds := base64.StdEncoding.EncodeToString([]byte(username + ":" + password))
+		connectReq += fmt.Sprintf("Proxy-Authorization: Basic %s\r\n", creds)
+	}
+	connectReq += "\r\n"
+
+	if _, err := fmt.Fprint(conn, connectReq); err != nil {
+		_ = conn.Close()
+		return nil, fmt.Errorf("send connect: %w", err)
+	}
+
+	br := bufio.NewReader(conn)
+	resp, err := http.ReadResponse(br, nil)
+	if err != nil {
+		_ = conn.Close()
+		return nil, fmt.Errorf("read connect response: %w", err)
+	}
+	if resp.StatusCode != http.StatusOK {
+		_ = conn.Close()
+		return nil, fmt.Errorf("proxy connect failed: %s", resp.Status)
+	}
+
+	_ = conn.SetDeadline(time.Time{})
+
+	if br.Buffered() > 0 {
+		return &proxyConn{Conn: conn, reader: io.MultiReader(br, conn)}, nil
+	}
+	return conn, nil
+}
+
 // fetchUnseen searches for UNSEEN messages and processes each one.
 func (w *IMAPWatcher) fetchUnseen(c *imapclient.Client) error {
 	criteria := &imap.SearchCriteria{
@@ -1,10 +1,20 @@
 package mail

 import (
+	"bufio"
 	"context"
+	"crypto/ecdsa"
+	"crypto/elliptic"
+	"crypto/rand"
+	"crypto/tls"
+	"crypto/x509"
+	"encoding/base64"
 	"errors"
+	"fmt"
 	"io"
+	"math/big"
 	"net"
+	"net/http"
 	"strings"
 	"testing"
 	"time"
@@ -73,13 +83,22 @@ func TestConnectAndWatch_UsesProxyDial_WhenProxySet(t *testing.T) {
 	}
 }

-func TestDialTLSViaSOCKS5_RejectsNonSocks5Scheme(t *testing.T) {
-	_, err := dialTLSViaSOCKS5("imap.example.com:993", "http://proxy:8080", nil)
+func TestDialTLSViaProxy_RejectsUnsupportedScheme(t *testing.T) {
+	_, err := dialTLSViaProxy("imap.example.com:993", "ftp://proxy:21", nil)
 	if err == nil || !strings.Contains(err.Error(), "unsupported proxy scheme") {
 		t.Fatalf("expected unsupported scheme error, got %v", err)
 	}
 }

+func TestDialTLSViaProxy_RejectsMalformedURL(t *testing.T) {
+	_, err := dialTLSViaProxy("imap.example.com:993", "://bad", nil)
+	if err == nil {
+		t.Fatal("expected error for malformed URL")
+	}
+}
+
+// --- SOCKS5 path ---
+
 type fakeContextDialer struct{}

 func (fakeContextDialer) Dial(network, address string) (net.Conn, error) {
@@ -93,7 +112,7 @@ func (fakeContextDialer) DialContext(ctx context.Context, network, address strin
 	return nil, errors.New("used dialcontext")
 }

-func TestDialTLSViaSOCKS5_UsesDialContextWithTimeout(t *testing.T) {
+func TestDialTLSViaProxy_SOCKS5_UsesDialContextWithTimeout(t *testing.T) {
 	orig := socks5DialerFactory
 	t.Cleanup(func() { socks5DialerFactory = orig })

@@ -104,7 +123,7 @@ func TestDialTLSViaSOCKS5_UsesDialContextWithTimeout(t *testing.T) {
 		return fakeContextDialer{}, nil
 	}

-	_, err := dialTLSViaSOCKS5("imap.example.com:993", "socks5://127.0.0.1:1080", nil)
+	_, err := dialTLSViaProxy("imap.example.com:993", "socks5://127.0.0.1:1080", nil)
 	if err == nil || !strings.Contains(err.Error(), "proxy dial: used dialcontext") {
 		t.Fatalf("expected DialContext path, got %v", err)
 	}
@@ -143,7 +162,7 @@ func (d contextConnDialer) DialContext(ctx context.Context, network, address str
 	return d.conn, nil
 }

-func TestDialTLSViaSOCKS5_SetsDeadlineForTLSHandshake(t *testing.T) {
+func TestDialTLSViaProxy_SOCKS5_SetsDeadlineForTLSHandshake(t *testing.T) {
 	orig := socks5DialerFactory
 	t.Cleanup(func() { socks5DialerFactory = orig })

@@ -152,7 +171,7 @@ func TestDialTLSViaSOCKS5_SetsDeadlineForTLSHandshake(t *testing.T) {
 		return contextConnDialer{conn: conn}, nil
 	}

-	_, err := dialTLSViaSOCKS5("imap.example.com:993", "socks5://127.0.0.1:1080", nil)
+	_, err := dialTLSViaProxy("imap.example.com:993", "socks5://127.0.0.1:1080", nil)
 	if err == nil || !strings.Contains(err.Error(), "tls handshake") {
 		t.Fatalf("expected tls handshake error, got %v", err)
 	}
@@ -160,3 +179,191 @@ func TestDialTLSViaSOCKS5_SetsDeadlineForTLSHandshake(t *testing.T) {
 		t.Fatal("expected TLS handshake deadline to be set")
 	}
 }
+
+// --- HTTP CONNECT path ---
+
+func startFakeProxy(t *testing.T, handler func(host string) bool) (addr string, cleanup func()) {
+	t.Helper()
+	ln, err := net.Listen("tcp", "127.0.0.1:0")
+	if err != nil {
+		t.Fatalf("listen: %v", err)
+	}
+	done := make(chan struct{})
+	go func() {
+		defer close(done)
+		for {
+			conn, err := ln.Accept()
+			if err != nil {
+				return
+			}
+			go func(c net.Conn) {
+				defer c.Close()
+				br := bufio.NewReader(c)
+				req, err := http.ReadRequest(br)
+				if err != nil {
+					return
+				}
+				if handler(req.URL.Host) {
+					fmt.Fprintf(c, "HTTP/1.1 200 OK\r\n\r\n")
+				} else {
+					fmt.Fprintf(c, "HTTP/1.1 403 Forbidden\r\n\r\n")
+				}
+			}(conn)
+		}
+	}()
+	return ln.Addr().String(), func() {
+		ln.Close()
+		<-done
+	}
+}
+
+func TestDialTLSViaProxy_HTTPConnect_ReachesTLSHandshake(t *testing.T) {
+	addr, cleanup := startFakeProxy(t, func(host string) bool { return true })
+	defer cleanup()
+
+	_, err := dialTLSViaProxy("imap.example.com:993", fmt.Sprintf("http://%s", addr), nil)
+	if err == nil || !strings.Contains(err.Error(), "tls handshake") {
+		t.Fatalf("expected tls handshake error (CONNECT succeeded), got %v", err)
+	}
+}
+
+func TestDialTLSViaProxy_HTTPConnect_RejectsOnProxyFailure(t *testing.T) {
+	addr, cleanup := startFakeProxy(t, func(host string) bool { return false })
+	defer cleanup()
+
+	_, err := dialTLSViaProxy("imap.example.com:993", fmt.Sprintf("http://%s", addr), nil)
+	if err == nil || !strings.Contains(err.Error(), "proxy connect failed") {
+		t.Fatalf("expected proxy connect failed error, got %v", err)
+	}
+}
+
+func TestDialTLSViaProxy_HTTPConnect_SendsProxyAuth(t *testing.T) {
+	var gotAuth string
+	ln, err := net.Listen("tcp", "127.0.0.1:0")
+	if err != nil {
+		t.Fatalf("listen: %v", err)
+	}
+	go func() {
+		defer ln.Close()
+		conn, err := ln.Accept()
+		if err != nil {
+			return
+		}
+		defer conn.Close()
+		br := bufio.NewReader(conn)
+		req, err := http.ReadRequest(br)
+		if err != nil {
+			return
+		}
+		gotAuth = req.Header.Get("Proxy-Authorization")
+		fmt.Fprintf(conn, "HTTP/1.1 200 OK\r\n\r\n")
+	}()
+
+	_, _ = dialTLSViaProxy("imap.example.com:993",
+		fmt.Sprintf("http://testuser:testpass@%s", ln.Addr().String()), nil)
+
+	expected := "Basic " + base64.StdEncoding.EncodeToString([]byte("testuser:testpass"))
+	if gotAuth != expected {
+		t.Fatalf("expected auth %q, got %q", expected, gotAuth)
+	}
+}
+
+func TestDialTLSViaProxy_HTTPConnect_DialError(t *testing.T) {
+	origDial := dialToProxy
+	t.Cleanup(func() { dialToProxy = origDial })
+
+	dialToProxy = func(ctx context.Context, network, addr string) (net.Conn, error) {
+		return nil, errors.New("dial refused")
+	}
+
+	_, err := dialTLSViaProxy("imap.example.com:993", "http://127.0.0.1:1", nil)
+	if err == nil || !strings.Contains(err.Error(), "dial proxy") {
+		t.Fatalf("expected dial proxy error, got %v", err)
+	}
+}
+
+// --- HTTPS CONNECT path ---
+
+func generateTestCert(t *testing.T) tls.Certificate {
+	t.Helper()
+	key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
+	if err != nil {
+		t.Fatalf("generate key: %v", err)
+	}
+	tmpl := &x509.Certificate{
+		SerialNumber: big.NewInt(1),
+		NotBefore:    time.Now(),
+		NotAfter:     time.Now().Add(time.Hour),
+		KeyUsage:     x509.KeyUsageDigitalSignature,
+		DNSNames:     []string{"127.0.0.1"},
+		IPAddresses:  []net.IP{net.ParseIP("127.0.0.1")},
+	}
+	certDER, err := x509.CreateCertificate(rand.Reader, tmpl, tmpl, &key.PublicKey, key)
+	if err != nil {
+		t.Fatalf("create cert: %v", err)
+	}
+	cert, err := x509.ParseCertificate(certDER)
+	if err != nil {
+		t.Fatalf("parse cert: %v", err)
+	}
+	return tls.Certificate{
+		Certificate: [][]byte{certDER},
+		PrivateKey:  key,
+		Leaf:        cert,
+	}
+}
+
+func TestDialTLSViaProxy_HTTPSConnect_ReachesTLSHandshake(t *testing.T) {
+	cert := generateTestCert(t)
+
+	ln, err := net.Listen("tcp", "127.0.0.1:0")
+	if err != nil {
+		t.Fatalf("listen: %v", err)
+	}
+	tlsLn := tls.NewListener(ln, &tls.Config{
+		Certificates: []tls.Certificate{cert},
+	})
+	done := make(chan struct{})
+	go func() {
+		defer close(done)
+		for {
+			conn, err := tlsLn.Accept()
+			if err != nil {
+				return
+			}
+			go func(c net.Conn) {
+				defer c.Close()
+				br := bufio.NewReader(c)
+				req, err := http.ReadRequest(br)
+				if err != nil {
+					return
+				}
+				if req.URL.Host != "" {
+					fmt.Fprintf(c, "HTTP/1.1 200 OK\r\n\r\n")
+				}
+			}(conn)
+		}
+	}()
+	defer func() {
+		tlsLn.Close()
+		<-done
+	}()
+
+	// Override dialToProxy to return raw TCP, but inject InsecureSkipVerify
+	// so the TLS handshake to proxy succeeds with the test cert.
+	origDial := dialToProxy
+	t.Cleanup(func() { dialToProxy = origDial })
+
+	origProxyTLS := proxyTLSConfigForTest
+	t.Cleanup(func() { proxyTLSConfigForTest = origProxyTLS })
+
+	proxyTLSConfigForTest = func(host string) *tls.Config {
+		return &tls.Config{ServerName: host, InsecureSkipVerify: true}
+	}
+
+	_, err = dialTLSViaProxy("imap.example.com:993",
+		fmt.Sprintf("https://%s", ln.Addr().String()), nil)
+	if err == nil || !strings.Contains(err.Error(), "tls handshake") {
+		t.Fatalf("expected tls handshake error (HTTPS CONNECT succeeded), got %v", err)
+	}
+}
Author	SHA1	Message	Date
thuanle	376057361f	Merge pull request 'docs: reorganize README and technical overview' (#23 ) from docs/readme-docs-reorg into main Reviewed-on: #23	2026-04-29 00:52:41 +07:00
thuanle	a499841b1e	docs: reorganize readme and architecture docs CI / staticcheck (pull_request) Successful in 1m39s Details CI / fmt (pull_request) Successful in 5s Details CI / test (pull_request) Successful in 1m27s Details CI / vet (pull_request) Successful in 57s Details	2026-04-29 00:50:08 +07:00
thuanle	fac874e747	skill create-adr	2026-04-29 00:38:11 +07:00
thuanle	c616fcf037	update DoD	2026-04-29 00:27:30 +07:00
thuanle	0d9230bd5f	Merge pull request 'ci: use node-go image instead of setup-go to reduce CI time' (#22 ) from feat/issue-11-ci-node-go-image into main Reviewed-on: #22	2026-04-28 23:09:01 +07:00
thuanleandClaude Opus 4.7	c08ae01d0a	ci: use node-go image instead of setup-go to reduce CI time CI / fmt (pull_request) Successful in 4s Details CI / test (pull_request) Successful in 1m24s Details CI / staticcheck (pull_request) Successful in 1m18s Details CI / vet (pull_request) Successful in 3m47s Details Switch all CI jobs (fmt, test, vet, staticcheck) to use the pre-built git.thuanle.me/public/node-go container image, removing the dynamic Go setup step from each job. Closes #11 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>	2026-04-28 11:12:29 +07:00
thuanle	193868ca23	Merge pull request 'feat: support http/https proxy for IMAP' (#21 ) from feat/issue-9-http-proxy into main Reviewed-on: #21 Reviewed-by: codex <39+codex@noreply.localhost>	2026-04-28 11:09:02 +07:00
thuanle	200bba7549	Merge branch 'main' into feat/issue-9-http-proxy CI / fmt (pull_request) Successful in 4m42s Details CI / vet (pull_request) Successful in 10m35s Details CI / test (pull_request) Successful in 12m55s Details CI / staticcheck (pull_request) Successful in 11m2s Details	2026-04-28 11:08:53 +07:00
thuanleandClaude Opus 4.7	6af6da1d99	fix: typo Alway -> Always in AGENT.md CI / fmt (pull_request) Successful in 4m58s Details CI / test (pull_request) Successful in 11m0s Details Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>	2026-04-28 11:07:31 +07:00
thuanleandClaude Opus 4.7	f978ed44b8	feat: support http/https proxy for IMAP via HTTP CONNECT tunnel CI / fmt (pull_request) Successful in 4m45s Details CI / test (pull_request) Successful in 10m47s Details Extend IMAP_PROXY_URL to accept http:// and https:// schemes in addition to the existing socks5:// support. HTTP CONNECT tunneling is used for both new schemes, with TLS to the proxy for https://. Proxy auth via user:pass@ in the URL is supported for all schemes. Refactors dialTLSViaSOCKS5 into a generic dialTLSViaProxy dispatcher. Fixes #9 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>	2026-04-28 10:56:57 +07:00
thuanle	b5941e5aa1	update agent instruction	2026-04-28 09:58:47 +07:00