29 Commits
Author SHA1 Message Date
thuanle 376057361f Merge pull request 'docs: reorganize README and technical overview' (#23) from docs/readme-docs-reorg into main
Reviewed-on: #23
2026-04-29 00:52:41 +07:00
thuanle a499841b1e docs: reorganize readme and architecture docs
CI / staticcheck (pull_request) Successful in 1m39s
CI / fmt (pull_request) Successful in 5s
CI / test (pull_request) Successful in 1m27s
CI / vet (pull_request) Successful in 57s
2026-04-29 00:50:08 +07:00
thuanle fac874e747 skill create-adr 2026-04-29 00:38:11 +07:00
thuanle c616fcf037 update DoD 2026-04-29 00:27:30 +07:00
thuanle 0d9230bd5f Merge pull request 'ci: use node-go image instead of setup-go to reduce CI time' (#22) from feat/issue-11-ci-node-go-image into main
Reviewed-on: #22
2026-04-28 23:09:01 +07:00
thuanleandClaude Opus 4.7 c08ae01d0a ci: use node-go image instead of setup-go to reduce CI time
CI / fmt (pull_request) Successful in 4s
CI / test (pull_request) Successful in 1m24s
CI / staticcheck (pull_request) Successful in 1m18s
CI / vet (pull_request) Successful in 3m47s
Switch all CI jobs (fmt, test, vet, staticcheck) to use the
pre-built git.thuanle.me/public/node-go container image, removing
the dynamic Go setup step from each job.

Closes #11

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-28 11:12:29 +07:00
thuanle 193868ca23 Merge pull request 'feat: support http/https proxy for IMAP' (#21) from feat/issue-9-http-proxy into main
Reviewed-on: #21
Reviewed-by: codex <39+codex@noreply.localhost>
2026-04-28 11:09:02 +07:00
thuanle 200bba7549 Merge branch 'main' into feat/issue-9-http-proxy
CI / fmt (pull_request) Successful in 4m42s
CI / vet (pull_request) Successful in 10m35s
CI / test (pull_request) Successful in 12m55s
CI / staticcheck (pull_request) Successful in 11m2s
2026-04-28 11:08:53 +07:00
thuanleandClaude Opus 4.7 6af6da1d99 fix: typo Alway -> Always in AGENT.md
CI / fmt (pull_request) Successful in 4m58s
CI / test (pull_request) Successful in 11m0s
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-28 11:07:31 +07:00
thuanleandClaude Opus 4.7 f978ed44b8 feat: support http/https proxy for IMAP via HTTP CONNECT tunnel
CI / fmt (pull_request) Successful in 4m45s
CI / test (pull_request) Successful in 10m47s
Extend IMAP_PROXY_URL to accept http:// and https:// schemes in addition
to the existing socks5:// support. HTTP CONNECT tunneling is used for
both new schemes, with TLS to the proxy for https://. Proxy auth via
user:pass@ in the URL is supported for all schemes.

Refactors dialTLSViaSOCKS5 into a generic dialTLSViaProxy dispatcher.

Fixes #9

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-28 10:56:57 +07:00
thuanle b5941e5aa1 update agent instruction 2026-04-28 09:58:47 +07:00
thuanle 6bbc107050 Merge pull request 'ci: add vet and pinned staticcheck checks' (#20) from feat/issue-18-ci-vet-staticcheck into main
Reviewed-on: #20
Reviewed-by: codex <39+codex@noreply.localhost>
2026-04-28 09:55:00 +07:00
thuanle b73b399c55 chore: remove superpowers spec artifact from PR scope
CI / fmt (pull_request) Successful in 4m42s
CI / vet (pull_request) Successful in 10m42s
CI / test (pull_request) Successful in 12m54s
CI / staticcheck (pull_request) Successful in 10m56s
2026-04-28 08:14:05 +07:00
thuanleandClaude Opus 4.7 6c6a6280b6 ci: add pinned staticcheck job to PR workflow
CI / fmt (pull_request) Successful in 4m45s
CI / vet (pull_request) Successful in 10m46s
CI / test (pull_request) Successful in 12m54s
CI / staticcheck (pull_request) Successful in 10m55s
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-28 07:53:23 +07:00
thuanleandClaude Opus 4.7 3eb64338c9 ci: add go vet job to PR workflow
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-28 07:51:04 +07:00
thuanle c112ac37c9 docs: add design spec for issue 18 CI checks
Record the approved design to add go vet and pinned staticcheck jobs to PR CI with minimal workflow changes.
2026-04-28 07:25:52 +07:00
thuanle 61280ba9a6 Merge pull request 'Add configurable external rules pipeline for ingress' (#19) from feat/external-rules-pipeline into main
Reviewed-on: #19
Reviewed-by: codex <39+codex@noreply.localhost>
2026-04-28 07:14:45 +07:00
thuanleandClaude Opus 4.7 094bd35de4 style: gofmt client_test.go
CI / fmt (pull_request) Successful in 4m46s
CI / test (pull_request) Successful in 12m58s
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-28 07:12:49 +07:00
thuanleandClaude Opus 4.7 c127018bc9 fix: add dispatch_context to schema docs and test buildDispatchMetadata
CI / fmt (pull_request) Failing after 4m45s
CI / test (pull_request) Successful in 10m47s
- Add dispatch_context column to requirements.md task schema section
- Export buildDispatchMetadata for test coverage
- Add tests: context forwarding, task_uuid overwrite protection, empty context

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-28 07:08:19 +07:00
thuanleandClaude Opus 4.7 5e7a479991 fix: protect task_uuid from DispatchContext overwrite and update acceptance criteria
CI / fmt (pull_request) Successful in 4m44s
CI / test (pull_request) Successful in 12m48s
- Skip task_uuid key when merging DispatchContext into dispatch metadata
- Update requirements.md acceptance criteria and test matrix for external rules

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-28 07:01:20 +07:00
thuanleandClaude Opus 4.7 562d3b01c3 fix: reorder ingress checks and update requirements.md
CI / fmt (pull_request) Successful in 4m43s
CI / test (pull_request) Failing after 21m37s
- Move idempotency before external rules per ADR-0002
- Update requirements.md: new pipeline flow, BLOCKLIST_EMAILS,
  dispatch_context metadata

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-28 06:53:05 +07:00
thuanleandClaude Opus 4.7 92121278bb docs: add ADR-0002, update flow and add BLOCKLIST_EMAILS
CI / fmt (pull_request) Successful in 4m47s
CI / test (pull_request) Successful in 12m54s
- ADR-0002: external rules pipeline architecture
- README: updated flow to show safety checks → rules → dispatch
- .env.example: added BLOCKLIST_EMAILS

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-28 06:39:50 +07:00
thuanleandClaude Opus 4.7 aa05e3f7c0 feat: wire external rules pipeline into ingress and dispatch
- Add DispatchContext field to Task model for rule metadata storage
- Add BlocklistEmails config (BLOCKLIST_EMAILS env var)
- Wire rules.Pipeline into IMAPWatcher, replacing inline whitelist check
- Pass rule metadata through DispatchContext to OpenClaw dispatch
- Remove isWhitelisted method (now handled by WhitelistRule)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-28 06:38:37 +07:00
thuanleandClaude Opus 4.7 ab992abc26 feat(rules): add BlocklistRule
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-28 06:26:56 +07:00
thuanleandClaude Opus 4.7 8006e6d64c feat(rules): add Rule interface, Pipeline, and WhitelistRule
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-28 06:26:01 +07:00
thuanle 2233fd232a update workflow - auto assign yourself as assignee/reviewer 2026-04-28 06:23:28 +07:00
thuanle 3985b9dae3 Merge pull request 'Require PR discussion comments in reviewer workflow' (#17) from fix/reviewer-pr-discussion-comments into main
Reviewed-on: #17
2026-04-28 06:13:27 +07:00
thuanle 28224ccb9f docs: require PR discussion comments in reviews
CI / fmt (pull_request) Successful in 4m46s
CI / test (pull_request) Successful in 10m45s
2026-04-28 06:10:55 +07:00
thuanle 668b64d594 Merge pull request 'Add ADR workflow, update skills, rename do-task to implement-task' (#15) from feat/adr-workflow into main
Reviewed-on: #15
2026-04-28 05:56:21 +07:00
28 changed files with 1134 additions and 327 deletions
+89
View File
@@ -0,0 +1,89 @@
---
name: create-adr
description: Use when a change introduces or updates a durable cross-cutting technical decision that future work in this repository must preserve, such as dependency, storage, integration, workflow, or quality-gate policy changes
---
# Create ADR
## Overview
Create a new ADR in `docs/adr/**` when the current task locks in a durable technical decision.
Do not use this for bug fixes, styling changes, routine refactors, or ordinary test additions.
## Use This Skill For
- introducing or removing a major dependency
- choosing or changing a storage strategy
- defining release, branch, or workflow policy
- changing how agents, jobs, or external tools are integrated
- adopting or tightening repository quality gates
- changing a behavior-defining contract future work must preserve
## Do Not Use This Skill For
- bug fixes
- styling-only changes
- routine refactors
- ordinary test additions
- temporary plans or execution notes
## Workflow
1. Check whether an active ADR already covers the same decision.
- Read `docs/adr/README.md` and scan existing files under `docs/adr/`.
- If the current change is only implementing an already-recorded decision, do not create a new ADR.
2. If the new decision changes or replaces an older ADR, create a new ADR that supersedes it.
- Do not rewrite the meaning of an active ADR in place.
3. Find the next numeric ID under `docs/adr/`.
4. Create `docs/adr/NNNN-short-kebab-title.md`.
5. Use this template:
```md
---
type: ADR
id: "NNNN"
title: "Short decision title"
status: active
date: YYYY-MM-DD
---
## Context
What changed? Why does this decision need to exist now?
## Decision
**State the decision clearly in one or two sentences.**
## Options considered
- **Option A** (chosen): description with pros and cons
- **Option B**: description with pros and cons
- **Option C**: description with pros and cons
## Consequences
What becomes easier?
What becomes harder?
What risks were accepted?
What would trigger re-evaluation?
## Advice
Optional. Capture important external guidance or review input.
```
6. Add the new ADR to the index in `docs/adr/README.md`.
7. Commit the ADR in the same change as the implementation it documents.
8. Mention the ADR in the issue comment and PR body under `Docs/ADR`.
## Rule Of Thumb
If the decision changes how future work should be done across the repository, it probably deserves an ADR.
## Repository Rules
- Keep one decision per file.
- Name files `NNNN-short-kebab-title.md`.
- Keep the `docs/adr/README.md` index aligned with the files on disk.
- Prefer concise, durable wording over implementation detail.
@@ -0,0 +1,3 @@
interface:
display_name: "Create ADR"
short_description: "Record a new architecture decision for durable repo changes"
@@ -1,53 +0,0 @@
---
name: do-task
description: Use when handling a Gitea issue/task end-to-end, including validation, planning, implementation, PR creation, and review-feedback iteration.
---
# Do Task
## Overview
Process a single Gitea issue from analysis to merge-ready PR with explicit decision gates. Prefer Gitea MCP tools for all issue/PR actions.
## Workflow
1. **Load issue**
- Read issue by ID from Gitea.
- Extract: problem, expected behavior, impact, acceptance checks.
2. **Validate technical correctness**
- Verify against current code/docs/tests.
- Decide:
- **Valid issue** -> continue to planning.
- **Not valid / out of scope** -> comment rationale on issue and stop.
3. **If valid: plan and comment**
- Create implementation plan using Superpowers planning flow.
- Post summary plan to the issue before coding.
4. **Implement**
- Use Superpowers execution flow (TDD + verification before completion).
- Keep changes scoped strictly to issue requirements.
5. **Create PR**
- Open PR from feature branch.
- PR description must include:
- Summary of changes
- Test evidence (exact commands)
- `Fixes #<issue-id>` (or `Closes #<issue-id>`)
- Add reviewer: `codex`.
6. **Review feedback loop**
- Read all feedback.
- For each item:
- If technically valid -> implement + re-test + reply.
- If not valid -> reply with concise technical reasoning.
- Do not blindly accept external feedback without verification.
## Required Rules
- Use Gitea MCP tools first for issue/PR/review operations.
- Do not use `tea` or other CLI Gitea clients unless MCP is unavailable.
- Never claim completion without fresh verification output.
## Quick Command Pattern
- Invoke as: `/do-task <issue-id>`
- Example: `/do-task 12`
+26 -4
View File
@@ -23,6 +23,8 @@ description: Phase 2 skill. Use when issue scope is locked by Product Owner (Pha
- Read issue: `mcp__gitea__.issue_read` (`method: "get"`). - Read issue: `mcp__gitea__.issue_read` (`method: "get"`).
- Read comments, find the **Final Requirements** comment: `mcp__gitea__.issue_read` (`method: "get_comments"`). - Read comments, find the **Final Requirements** comment: `mcp__gitea__.issue_read` (`method: "get_comments"`).
- If Final Requirements not found → stop, tell user to validate issue first. - If Final Requirements not found → stop, tell user to validate issue first.
- Read current Gitea user: `mcp__gitea__.get_me`.
- If current user is not already in the issue assignee list, update the issue with `mcp__gitea__.issue_write` (`method: "update"`) and set `assignees` to the existing assignees plus the current username.
2. **Read codebase** 2. **Read codebase**
- Use `mcp__gitea__.get_file_contents`, `mcp__gitea__.get_repository_tree` to understand current state. - Use `mcp__gitea__.get_file_contents`, `mcp__gitea__.get_repository_tree` to understand current state.
@@ -31,6 +33,9 @@ description: Phase 2 skill. Use when issue scope is locked by Product Owner (Pha
3. **Write plan and post on issue** 3. **Write plan and post on issue**
- List implementation steps with verification criteria. - List implementation steps with verification criteria.
- List files to change. - List files to change.
- State whether `README.md` or `.env.example` must change.
- State whether an ADR is required or not required.
- If ADR is required or recommended, invoke `create-adr` before finalizing the implementation summary and PR body.
- Flag risks or tradeoffs. - Flag risks or tradeoffs.
- Post plan as comment: `mcp__gitea__.issue_write` (`method: "add_comment"`). - Post plan as comment: `mcp__gitea__.issue_write` (`method: "add_comment"`).
- **Wait for "Approved" before coding.** Stop here. - **Wait for "Approved" before coding.** Stop here.
@@ -46,18 +51,32 @@ description: Phase 2 skill. Use when issue scope is locked by Product Owner (Pha
git worktree add .worktrees/<branch-name> main git worktree add .worktrees/<branch-name> main
``` ```
- Implement following the plan (surgical changes only). - Implement following the plan (surgical changes only).
- Update `README.md` and `.env.example` when config, endpoint, or runtime behavior changes.
- If the change introduces a durable behavior, contract, architecture, or workflow decision, use `create-adr` and create/update `docs/adr/**`.
- Run validation: - Run validation:
``` ```
gofmt -l . gofmt -l .
go vet ./...
go test ./... go test ./...
go vet ./...
staticcheck ./...
``` ```
- Post an implementation summary comment on the issue with sections:
- `Summary`
- `Changes`
- `Test Plan`
- `Refactor`
- `Docs/ADR`
- `Risks/Follow-ups`
5. **Create PR** 5. **Create PR**
- Use `mcp__gitea__.pull_request_write` (`method: "create"`). - Use `mcp__gitea__.pull_request_write` (`method: "create"`).
- PR body must include: - PR body must include:
- Summary of changes - `Summary`
- Test evidence (exact commands and output) - `Changes`
- `Test Plan` with exact commands and output
- `Refactor`
- `Docs/ADR`
- `Risks/Follow-ups`
- `Fixes #<issue-id>` - `Fixes #<issue-id>`
- Add reviewer. - Add reviewer.
@@ -72,10 +91,13 @@ description: Phase 2 skill. Use when issue scope is locked by Product Owner (Pha
## Rules ## Rules
- Use Gitea MCP tools for all issue/PR operations. - Use Gitea MCP tools for all issue/PR operations.
- Preserve existing assignees when adding yourself unless the user explicitly asks to replace them.
- Never edit remote files directly — always work on local branch. - Never edit remote files directly — always work on local branch.
- Satisfy the repo Definition of Done before claiming completion.
- Missing required tests, docs, `.env.example` updates, or ADRs is a quality failure unless explicitly justified in both the issue comment and PR body.
- Never claim completion without fresh verification output. - Never claim completion without fresh verification output.
- Keep changes strictly scoped to the approved plan. - Keep changes strictly scoped to the approved plan.
- If the change needs durable documentation, create/update `docs/adr/**`. Do not commit `docs/superpowers/**` planning artifacts. - If the change needs durable documentation, use `create-adr` and create/update `docs/adr/**`. Do not commit `docs/superpowers/**` planning artifacts.
## Next skill ## Next skill
After PR is created, the user invokes `/review-pr <pr-number>` on the Reviewer agent. After PR is created, the user invokes `/review-pr <pr-number>` on the Reviewer agent.
-133
View File
@@ -1,133 +0,0 @@
---
name: review-pr
description: Review pull requests in Gitea repositories using Gitea MCP for server-side PR metadata, comments, reviews, diffs, workflow state, and PR comments. Use local CLI tools such as git and test runners only for local repository inspection and validation. Use when the user asks to "Review PR", "review pull request", "check PR", "recheck PR after updates", or "post review comment". Inspect PR metadata and discussion, compare the PR head against its base branch, produce a clear PASS/FAIL verdict, and optionally post the result back to the PR.
---
# Review PR
## Local Preference
For repositories on `git.thuanle.me`, prefer the `mcp__gitea__` tools for Gitea-hosted pull request metadata, comments, reviews, server-side diffs, repository contents, branches, commits, workflow runs, and PR comments. Do not use `tea` or other command-line Gitea API clients unless the user explicitly asks for them or the MCP tools are unavailable for the required Gitea operation. Other local CLI tools such as `git`, `go`, `rg`, test runners, and formatters are OK for local repository inspection and validation.
## Workflow
Follow this sequence:
1. Identify the PR and current repo context.
- Use the current repository when the user is already inside it.
- If the user gives only a PR number and the repo context is ambiguous, ask a short clarifying question.
- Do not ask follow-up questions when the repository, PR, and requested reviewer action are already clear.
- If the user has already given a standing instruction earlier in the same thread such as "if you're confident, post it", carry that instruction forward for later re-reviews until the user changes it.
- Read the PR first with `mcp__gitea__.pull_request_read` using `method: "get"`, and read issue comments with `mcp__gitea__.issue_read` using `method: "get_comments"`.
- Read reviews with `mcp__gitea__.pull_request_read` using `method: "get_reviews"` when review state matters.
- Note the PR status exactly. If it is already merged or closed, say so explicitly before continuing.
2. Read the review history before judging the latest update.
- Find the latest blocking review comments first.
- If the author says they addressed feedback in a follow-up commit, focus on the new commits after that discussion.
- Still sanity-check the current full PR diff before concluding `PASS`.
3. Compare against the real base branch.
- Use PR metadata from MCP as the source of truth for base/head refs and SHAs.
- Prefer the PR base branch from MCP metadata; use `main` only when the base branch is not obvious from PR metadata.
4. Inspect the code changes.
- Use `mcp__gitea__.pull_request_read` with `method: "get_diff"` to read the PR diff.
- Use `mcp__gitea__.get_file_contents`, `mcp__gitea__.get_repository_tree`, and commit/branch MCP reads for surrounding context.
- If the user asks for a re-review after updates, optionally diff the latest fix commit range as a helper view, but do not skip the full PR sanity check.
5. Validate behavior where it matters.
- Prefer an isolated worktree when validation should run on the PR snapshot instead of the currently checked out branch.
- Use a repository-local worktree under `.worktrees/` by default, for example `git worktree add .worktrees/pr-<pr> origin/<head>`. Before creating it, run `git worktree list` and confirm the repo-local `.worktrees/` path.
- For `/Users/tm/working/thuanle/crypto/crypto-price-bot`, PR validation worktrees must be under `.worktrees/pr-<pr>` from the repository root. Do not use `/tmp` for this repo; if `.worktrees/` is unavailable, stop and ask the user before using any fallback path.
- Run validation commands inside that worktree.
- Remove it afterward with `git worktree remove .worktrees/pr-<pr>` when it is no longer needed.
- For other repositories, if `git worktree add` fails because the environment blocks writes to `.git/worktrees`, fall back to exporting a temporary snapshot for validation and say that you used the fallback.
- Run repo-appropriate checks when feasible.
- Prefer targeted checks first, then broader validation such as `go test ./...` and `go vet ./...` in Go repositories.
- Say explicitly when validation could not be run.
6. Write the verdict for the user.
- Start with `PASS` or `FAIL`.
- For `FAIL`, list blocking findings first, ordered by severity.
- Include file and line references whenever possible.
- Explain the concrete behavior impact, not style preferences.
- For `PASS`, say that no blocking issues were found and then add any non-blocking notes.
7. Post a review result proactively by default.
- When acting as a reviewer, prefer an official PR review over a plain issue comment if the result should affect PR state.
- Default stance: if the verdict is clear and you are confident in it, post the review without waiting for another prompt from the user.
- If the user has already asked you to post the result, or has given a standing instruction such as "if you're confident, post it", treat that as continuing authority for later re-reviews in the same thread unless the user revokes or narrows it.
- Ask the user only when there is a real ambiguity you cannot safely resolve, such as unclear repo/PR context, unclear whether they explicitly want a local-only verdict instead of a posted review, or a materially uncertain finding.
- If the user explicitly asks for a local verdict only, do not post.
- Use `mcp__gitea__.pull_request_review_write` for review posting.
- For normal review results, prefer a single `create` call with the final `state` and `body`. This is the default path for `PASS`, `FAIL`, and neutral review comments.
- Use `submit` only when you intentionally need a multi-step pending review flow, such as staging a pending review first and finalizing it later.
- Never send the same review narrative through both `create` and `submit`; that is a process bug and can surface as duplicate review content in the UI.
- Map the verdict to review state:
- `PASS` -> `APPROVED`
- `FAIL` -> `REQUEST_CHANGES`
- neutral/non-blocking note only -> `COMMENT`
- Use `mcp__gitea__.issue_write` with `method: "add_comment"` only for plain discussion comments that should not change PR review status.
- Do not add bracketed author/tool tags such as `[codex]`.
- Start directly with the review result or author response, depending on your role in the thread.
- Keep the review/comment concise and direct.
- Mention the validation commands you actually ran when posting review results.
## Command Patterns
Use these MCP operations directly:
- `mcp__gitea__.pull_request_read` with `method: "get"`
- `mcp__gitea__.pull_request_read` with `method: "get_diff"`
- `mcp__gitea__.pull_request_read` with `method: "get_reviews"`
- `mcp__gitea__.pull_request_read` with `method: "get_review_comments"` when a specific review thread is needed
- `mcp__gitea__.issue_read` with `method: "get_comments"`
- `mcp__gitea__.get_file_contents`
- `mcp__gitea__.get_repository_tree`
- `mcp__gitea__.get_commit`
- `mcp__gitea__.actions_run_read` for workflow status and logs
- `mcp__gitea__.pull_request_review_write` with `method: "create"` for the normal one-shot final review path
- Set the final `state` on `create`
- Put the final review text on `create`
- `mcp__gitea__.pull_request_review_write` with `method: "submit"` only when finishing an intentionally pending review
- `mcp__gitea__.issue_write` with `method: "add_comment"` when posting a plain PR comment
Do not use `tea` for Gitea PR review unless the user explicitly requests it or MCP cannot provide the needed data. Local CLI usage such as `git diff`, `git worktree`, and test commands remains acceptable when it materially improves local validation.
## Comment Template
Use this structure when posting a review result:
```text
PASS
No blocking issues found in the latest update.
Validation:
- <command 1>
- <command 2>
Non-blocking note:
- <optional note>
```
```text
FAIL
1. [High] <blocking issue with path:line and impact>
2. [Medium] <blocking issue with path:line and impact>
Validation:
- <command 1>
- <command 2>
```
When acting as a reviewer and the user asks to approve/reject or otherwise post the result, also submit the matching official review state:
- `PASS` -> `APPROVED`
- `FAIL` -> `REQUEST_CHANGES`
- neutral note only -> `COMMENT`
When responding as the PR author rather than reviewer, do not use `PASS`/`FAIL`; write a concise author response that explains what feedback was addressed and which validation ran.
Do not add filler. Keep the review specific, technical, and actionable.
+17 -3
View File
@@ -22,13 +22,17 @@ description: Phase 2 skill. Use when a PR is created or updated. Review PR again
1. **Load PR and issue** 1. **Load PR and issue**
- Read PR: `mcp__gitea__.pull_request_read` (`method: "get"`). - Read PR: `mcp__gitea__.pull_request_read` (`method: "get"`).
- If PR is already merged or closed, say so and stop.
- Read current Gitea user: `mcp__gitea__.get_me`.
- Read existing reviews: `mcp__gitea__.pull_request_read` (`method: "get_reviews"`).
- If the current user is not already requested as a reviewer and has not already reviewed this PR, add them with `mcp__gitea__.pull_request_write` (`method: "add_reviewers"`).
- Identify the linked issue from PR body (`Fixes #<id>` or `Closes #<id>`). - Identify the linked issue from PR body (`Fixes #<id>` or `Closes #<id>`).
- Read issue comments, find the **Final Requirements** comment: `mcp__gitea__.issue_read` (`method: "get_comments"`). - Read issue comments, find the **Final Requirements** comment: `mcp__gitea__.issue_read` (`method: "get_comments"`).
- Read existing reviews: `mcp__gitea__.pull_request_read` (`method: "get_reviews"`). - Read PR discussion comments: `mcp__gitea__.issue_read` (`method: "get_comments"`), using the PR number as the index.
- If PR is already merged or closed, say so and stop.
2. **Read review history** (for re-reviews) 2. **Read review history** (for re-reviews)
- Get review comments: `mcp__gitea__.pull_request_read` (`method: "get_review_comments"`). - Get review comments: `mcp__gitea__.pull_request_read` (`method: "get_review_comments"`).
- Treat reviews, inline review comments, and PR discussion comments as separate context sources.
- If author addressed feedback in a follow-up commit, focus on new commits but still sanity-check the full diff. - If author addressed feedback in a follow-up commit, focus on new commits but still sanity-check the full diff.
3. **Get the diff** 3. **Get the diff**
@@ -43,8 +47,9 @@ description: Phase 2 skill. Use when a PR is created or updated. Review PR again
- Run checks: - Run checks:
``` ```
gofmt -l . gofmt -l .
go vet ./...
go test ./... go test ./...
go vet ./...
staticcheck ./...
``` ```
- Remove worktree when done: - Remove worktree when done:
``` ```
@@ -53,6 +58,11 @@ description: Phase 2 skill. Use when a PR is created or updated. Review PR again
5. **Evaluate** 5. **Evaluate**
- Does PR address **all** Final Requirements from the issue? - Does PR address **all** Final Requirements from the issue?
- Does PR satisfy the repo Definition of Done?
- relevant tests for behavior/config changes
- required `README.md` / `.env.example` updates
- ADR present when the change locks in a durable decision
- required implementation evidence exists on the issue and in the PR body
- Does PR contain unrelated changes or refactoring? → FAIL immediately. - Does PR contain unrelated changes or refactoring? → FAIL immediately.
- Code correctness, style, test coverage. - Code correctness, style, test coverage.
@@ -81,6 +91,7 @@ Validation:
- gofmt -l .: OK - gofmt -l .: OK
- go vet ./...: OK - go vet ./...: OK
- go test ./...: OK - go test ./...: OK
- staticcheck ./...: OK
Non-blocking: Non-blocking:
- <optional note> - <optional note>
@@ -99,9 +110,12 @@ Validation:
## Rules ## Rules
- Use Gitea MCP tools for all PR/review operations. - Use Gitea MCP tools for all PR/review operations.
- Do not add yourself as reviewer twice; check requested reviewers and existing reviews first.
- Do NOT edit code on the PR. - Do NOT edit code on the PR.
- FAIL if PR contains unrelated changes or scope creep. - FAIL if PR contains unrelated changes or scope creep.
- FAIL if the PR misses required DoD evidence, docs, or ADR work for the accepted change.
- Post review proactively if verdict is clear. - Post review proactively if verdict is clear.
- Do not post a verdict until PR discussion comments have been checked.
- Mention validation commands you actually ran. - Mention validation commands you actually ran.
- No bracketed author/tool tags. - No bracketed author/tool tags.
- Accept `docs/adr/**` when it records a direct, durable decision tied to the issue scope. - Accept `docs/adr/**` when it records a direct, durable decision tied to the issue scope.
+13 -2
View File
@@ -27,6 +27,8 @@ description: Phase 1 skill. Use when a Gitea issue is created or updated and nee
- Is the problem statement clear? - Is the problem statement clear?
- Is the scope well-defined or too broad? - Is the scope well-defined or too broad?
- Is it technically feasible? - Is it technically feasible?
- What acceptance criteria will prove the change is done?
- Will the change require docs updates or a durable ADR?
- If needed, read relevant code: `mcp__gitea__.get_file_contents`. - If needed, read relevant code: `mcp__gitea__.get_file_contents`.
3. **Decide** 3. **Decide**
@@ -34,8 +36,10 @@ description: Phase 1 skill. Use when a Gitea issue is created or updated and nee
- Post clarifying questions: `mcp__gitea__.issue_write` (`method: "add_comment"`). - Post clarifying questions: `mcp__gitea__.issue_write` (`method: "add_comment"`).
- Stop. Wait for user to respond, then re-run this skill. - Stop. Wait for user to respond, then re-run this skill.
- **Clear and feasible:** - **Clear and feasible:**
- Post Final Requirements (see format below). - Post Final Requirements (see format below) with specific acceptance criteria.
- If the change likely needs a durable ADR, add a note: "ADR recommended: this change locks in a behavior/contract that future work must preserve." - State whether docs updates are expected.
- State whether an ADR is required, recommended, or not needed.
- If ADR is `required` or `recommended`, tell the Developer to use `create-adr` during implementation.
- This locks the scope and signals Phase 2 can begin. - This locks the scope and signals Phase 2 can begin.
## Final Requirements format ## Final Requirements format
@@ -55,11 +59,18 @@ description: Phase 1 skill. Use when a Gitea issue is created or updated and nee
**Acceptance criteria:** **Acceptance criteria:**
- <verifiable check 1> - <verifiable check 1>
- <verifiable check 2> - <verifiable check 2>
**Documentation impact:**
- <README.md / .env.example updates required or not required>
**ADR:**
- <required / recommended / not needed, with one-line reason>
``` ```
## Rules ## Rules
- Use Gitea MCP tools for all operations. - Use Gitea MCP tools for all operations.
- Do NOT write code, create branches, or create PRs. - Do NOT write code, create branches, or create PRs.
- Make acceptance criteria specific enough that Dev and Reviewer can verify Definition of Done without guessing.
- Do NOT proceed to Phase 2 — that is a separate skill. - Do NOT proceed to Phase 2 — that is a separate skill.
## Next skill ## Next skill
+1
View File
@@ -18,6 +18,7 @@ OPENCLAW_API_KEY=
BRIDGE_CALLBACK_TOKEN=a-strong-random-token BRIDGE_CALLBACK_TOKEN=a-strong-random-token
SYSTEM_EMAIL=bridge@example.com SYSTEM_EMAIL=bridge@example.com
WHITELIST_EMAILS=user1@example.com,user2@example.com WHITELIST_EMAILS=user1@example.com,user2@example.com
BLOCKLIST_EMAILS=
# === Optional === # === Optional ===
IMAP_PROXY_URL= IMAP_PROXY_URL=
+21 -6
View File
@@ -6,11 +6,10 @@ on:
jobs: jobs:
fmt: fmt:
runs-on: linux runs-on: linux
container:
image: git.thuanle.me/public/node-go
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version-file: go.mod
- run: | - run: |
out=$(gofmt -l .) out=$(gofmt -l .)
if [ -n "$out" ]; then if [ -n "$out" ]; then
@@ -21,9 +20,25 @@ jobs:
test: test:
runs-on: linux runs-on: linux
container:
image: git.thuanle.me/public/node-go
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version-file: go.mod
- run: go test ./... - run: go test ./...
vet:
runs-on: linux
container:
image: git.thuanle.me/public/node-go
steps:
- uses: actions/checkout@v4
- run: go vet ./...
staticcheck:
runs-on: linux
container:
image: git.thuanle.me/public/node-go
steps:
- uses: actions/checkout@v4
- run: go install honnef.co/go/tools/cmd/staticcheck@2025.1.1
- run: $(go env GOPATH)/bin/staticcheck ./...
+42 -11
View File
@@ -3,16 +3,19 @@
## 1. Coding Principles ## 1. Coding Principles
### 1.1 Simplicity First ### 1.1 Simplicity First
- No out-of-scope features. - No out-of-scope features.
- No abstractions for single-use code. - No abstractions for single-use code.
- If 50 lines work instead of 200, write 50. - If 50 lines work instead of 200, write 50.
### 1.2 Surgical Changes ### 1.2 Surgical Changes
- Only change lines directly related to the request. - Only change lines directly related to the request.
- Never refactor adjacent code unless asked. - Never refactor adjacent code unless asked.
- Remove orphaned imports/variables/functions your changes created. - Remove orphaned imports/variables/functions your changes created.
### 1.3 Goal-Driven Execution ### 1.3 Goal-Driven Execution
- Define success criteria before coding. - Define success criteria before coding.
- Verify each step before moving to the next. - Verify each step before moving to the next.
@@ -25,6 +28,24 @@ All work starts from a Gitea issue. Two phases, each with a dedicated role:
1. **Phase 1 — Requirement Refinement** (Product Owner): Clarify and lock scope. 1. **Phase 1 — Requirement Refinement** (Product Owner): Clarify and lock scope.
2. **Phase 2 — Requirement Implementation** (Developer & Reviewer): Plan, code, review, merge. 2. **Phase 2 — Requirement Implementation** (Developer & Reviewer): Plan, code, review, merge.
### 2.1 Definition of Done
A change is done only when all of the following are true:
- The issue has locked **Final Requirements** with verifiable acceptance criteria.
- The implementation stays within approved scope; unrelated refactors are not allowed.
- Behavior, contract, or config changes have relevant tests added or updated, or the change record explains why tests were not added.
- Validation passes with fresh output: `gofmt -l .`, `go test ./...`, `go vet ./...`, `staticcheck ./...`.
- `README.md` and `.env.example` and related documents are updated when config, endpoint, or runtime behavior changes.
- A new ADR is added in `docs/adr/**` when the change introduces a durable architecture, contract, behavior, or workflow decision.
- The implementation evidence is posted on the issue and reflected in the PR body:
- what changed
- what tests were run
- whether any refactor was included
- whether docs or ADRs were added or not needed
- risks or follow-ups
- After merge, the linked issue is verified closed.
--- ---
## 3. Phase 1: Requirement Refinement ## 3. Phase 1: Requirement Refinement
@@ -52,13 +73,16 @@ All work starts from a Gitea issue. Two phases, each with a dedicated role:
1. Read the locked issue and any clarification comments. 1. Read the locked issue and any clarification comments.
2. Read relevant code to understand current state. 2. Read relevant code to understand current state.
3. Write an implementation plan — list steps, files to change, risks. 3. Write an implementation plan — list steps, files to change, risks, verification criteria, and any docs/ADR impact.
4. **Comment the plan on the issue. Wait for "Approved" before coding.** 4. **Comment the plan on the issue. Wait for "Approved" before coding.**
5. Once approved: 5. Once approved:
- Create a feature branch from `main`. - Create a feature branch from `main`.
- Implement following the plan (surgical changes only). - Implement following the plan (surgical changes only).
- Run validation: `go test ./...`, `go vet ./...`, `gofmt -l .`. - Update `README.md` and `.env.example` if config, endpoint, or runtime behavior changes.
- Create PR with `Fixes #<issue>` in the body so Gitea auto-closes the issue on merge. - If the change locks in a durable decision, write/update an ADR in `docs/adr/**`.
- Run validation: `gofmt -l .`, `go test ./...`, `go vet ./...`, `staticcheck ./...`.
- Post an implementation summary comment on the issue with sections: `Summary`, `Changes`, `Test Plan`, `Refactor`, `Docs/ADR`, `Risks/Follow-ups`.
- Create PR with the same evidence in the body plus `Fixes #<issue>` so Gitea auto-closes the issue on merge.
- After merge, verify the issue is closed. If not (e.g. missing keyword), close it manually with a summary comment. - After merge, verify the issue is closed. If not (e.g. missing keyword), close it manually with a summary comment.
6. **Own the PR until merged:** 6. **Own the PR until merged:**
- Monitor for review feedback. - Monitor for review feedback.
@@ -70,11 +94,16 @@ All work starts from a Gitea issue. Two phases, each with a dedicated role:
**Constraint:** Reviewer must NOT edit code on the PR. **Constraint:** Reviewer must NOT edit code on the PR.
1. Re-read the original issue — verify PR addresses the locked requirement. 1. Re-read the original issue — verify PR addresses the locked requirement.
2. Read PR diff and discussion history. 2. Read PR diff, review history, and PR discussion comments.
3. Validate in worktree if needed (`.worktrees/pr-<number>`, run `go test`, `go vet`, `gofmt`). 3. Validate in worktree if needed (`.worktrees/pr-<number>`, run `gofmt -l .`, `go test ./...`, `go vet ./...`, `staticcheck ./...`).
4. Check for scope creep — FAIL if PR contains unrelated changes or refactoring. 4. Check for scope creep — FAIL if PR contains unrelated changes or refactoring.
5. Write verdict — start with **PASS** or **FAIL**. 5. Verify the repo Definition of Done:
6. Post review proactively if verdict is clear: - relevant tests exist for behavior/config changes
- docs and `.env.example` were updated when required
- ADR exists when a durable decision was introduced
- the issue comment and PR body contain the required implementation evidence
6. Write verdict — start with **PASS** or **FAIL**.
7. Post review proactively if verdict is clear:
- `PASS``APPROVED` - `PASS``APPROVED`
- `FAIL``REQUEST_CHANGES` - `FAIL``REQUEST_CHANGES`
- Non-blocking note → `COMMENT` - Non-blocking note → `COMMENT`
@@ -83,10 +112,9 @@ All work starts from a Gitea issue. Two phases, each with a dedicated role:
## 5. Gitea Tools ## 5. Gitea Tools
- If git remote contains `git.thuanle.me`, ALWAYS use Gitea MCP tools. - Always use Gitea MCP tools in this project.
- Scope: read PR/issue, list comments, post replies, create/edit PRs, reviews. - Scope: read PR/issue, list comments, post replies, create/edit PRs, reviews.
- Use `tea` CLI only as fallback when MCP is unavailable. - Use `tea` CLI only as fallback when MCP is unavailable.
- Do not use `gh` for Gitea repositories.
- Local CLI tools (`git`, `go`, `rg`) are fine for local validation. - Local CLI tools (`git`, `go`, `rg`) are fine for local validation.
--- ---
@@ -99,8 +127,10 @@ PASS
No blocking issues found. No blocking issues found.
Validation: Validation:
- gofmt -l .: OK
- go test ./...: OK - go test ./...: OK
- go vet ./...: OK - go vet ./...: OK
- staticcheck ./...: OK
Non-blocking: Non-blocking:
- <optional note> - <optional note>
@@ -113,7 +143,7 @@ FAIL
2. [Medium] <issue with path:line and impact> 2. [Medium] <issue with path:line and impact>
Validation: Validation:
- go test ./...: OK - <list only the commands you actually ran and their result>
``` ```
--- ---
@@ -130,6 +160,7 @@ Validation:
## 8. Comment Style ## 8. Comment Style
- Use readable multiline markdown. - Use readable multiline markdown.
- Use short sections (Summary, Changes, Test Plan). - Use short sections (`Summary`, `Changes`, `Test Plan`).
- Implementation completion comments should also include `Refactor`, `Docs/ADR`, and `Risks/Follow-ups`.
- Concise bullet points and explicit line breaks. - Concise bullet points and explicit line breaks.
- No bracketed author/tool tags. - No bracketed author/tool tags.
+29 -52
View File
@@ -1,37 +1,26 @@
# Email-OpenClaw Bridge # Email-OpenClaw Bridge
Tự động xử lý email bằng OpenClaw AI theo luồng: Bridge tự động xử lý email bằng OpenClaw AI theo luồng:
``` ```text
Ingress (IMAP) → Dispatch (OpenClaw) → Callback → Egress (SMTP reply) Ingress (IMAP) → Safety Checks → External Rules → Dispatch (OpenClaw) → Callback → Egress (SMTP reply)
``` ```
## Tính năng README này tập trung vào onboarding và vận hành cơ bản. Chi tiết kỹ thuật hiện hành được tách sang `docs/architecture/overview.md`.
- **IMAP IDLE** — theo dõi mailbox realtime, nhận email mới tức thì ## Tính năng chính
- **Threading** — giữ nguyên email thread qua `In-Reply-To` / `References`
- **Idempotency** — chống duplicate theo `message_id`
- **Anti-loop & Whitelist** — bỏ qua email từ chính hệ thống, chỉ xử lý sender được phép
- **Retry** — tối đa 3 lần với backoff 1s → 5s → 15s cho cả OpenClaw và SMTP
- **Stateful** — SQLite lưu trạng thái pipeline, không mất dữ liệu khi restart
## Kiến trúc - Theo dõi mailbox realtime bằng IMAP IDLE
- Giữ thread email qua `In-Reply-To``References`
- Chống duplicate theo `message_id`
- Chặn anti-loop và hỗ trợ whitelist/blocklist
- Gửi callback vào bridge rồi phản hồi lại qua SMTP
- Lưu trạng thái pipeline bằng SQLite để không mất tiến trình khi restart
``` ## Yêu cầu tối thiểu
cmd/bridge/main.go Entrypoint
internal/
├── config/ Đọc cấu hình từ .env
├── database/ SQLite + GORM, auto-migrate
├── logging/ Structured JSON logging (slog)
├── api/ HTTP server (Gin): health checks, webhook callback
├── mail/ IMAP ingress + SMTP egress
└── ai_client/ OpenClaw HTTP dispatch
```
## Yêu cầu
- Go 1.23+ - Go 1.23+
- Docker & Docker Compose (cho deploy) - Docker Docker Compose nếu chạy bằng container
## Quickstart ## Quickstart
@@ -42,7 +31,7 @@ cp .env.example .env
# Sửa .env với thông tin IMAP/SMTP/OpenClaw thực tế # Sửa .env với thông tin IMAP/SMTP/OpenClaw thực tế
``` ```
### 2. Chạy local (development) ### 2. Chạy local
```bash ```bash
go run ./cmd/bridge go run ./cmd/bridge
@@ -55,51 +44,39 @@ docker compose up -d
docker logs -f claw-email-bridge docker logs -f claw-email-bridge
``` ```
## Cấu hình (.env) ## Cấu hình `.env`
| Biến | Bắt buộc | Mô tả | | Biến | Bắt buộc | Mô tả |
|---|:---:|---| |---|:---:|---|
| `IMAP_HOST` | ✅ | IMAP server hostname | | `IMAP_HOST` | ✅ | IMAP server hostname |
| `IMAP_PORT` | ✅ | IMAP port (thường `993`) | | `IMAP_PORT` | ✅ | IMAP port, thường `993` |
| `IMAP_USER` | ✅ | IMAP username | | `IMAP_USER` | ✅ | IMAP username |
| `IMAP_PASS` | ✅ | IMAP password | | `IMAP_PASS` | ✅ | IMAP password |
| `SMTP_HOST` | ✅ | SMTP server hostname | | `SMTP_HOST` | ✅ | SMTP server hostname |
| `SMTP_PORT` | ✅ | SMTP port (thường `587`) | | `SMTP_PORT` | ✅ | SMTP port, thường `587` |
| `SMTP_USER` | ✅ | SMTP username | | `SMTP_USER` | ✅ | SMTP username |
| `SMTP_PASS` | ✅ | SMTP password | | `SMTP_PASS` | ✅ | SMTP password |
| `OPENCLAW_URL` | ✅ | OpenClaw API endpoint | | `OPENCLAW_URL` | ✅ | OpenClaw API endpoint |
| `OPENCLAW_API_KEY` | | OpenClaw API key (nếu cần) | | `OPENCLAW_API_KEY` | | OpenClaw API key nếu cần |
| `BRIDGE_CALLBACK_TOKEN` | ✅ | Token xác thực webhook callback | | `BRIDGE_CALLBACK_TOKEN` | ✅ | Token xác thực webhook callback |
| `SYSTEM_EMAIL` | ✅ | Email của hệ thống (dùng cho anti-loop) | | `SYSTEM_EMAIL` | ✅ | Email của bridge, dùng cho anti-loop |
| `WHITELIST_EMAILS` | | Danh sách email được phép, phân cách bằng dấu phẩy | | `WHITELIST_EMAILS` | | Danh sách email được phép, phân cách bằng dấu phẩy |
| `IMAP_PROXY_URL` | | SOCKS5 proxy cho IMAP (vd: `socks5://user:pass@host:port`) | | `BLOCKLIST_EMAILS` | | Danh sách email bị chặn, phân cách bằng dấu phẩy |
| `LISTEN_ADDR` | | Địa chỉ bind HTTP server cục bộ (mặc định `:8080`) | | `IMAP_PROXY_URL` | | Proxy cho IMAP, hỗ trợ `socks5://`, `http://`, `https://` |
| `CALLBACK_BASE_URL` | | Base URL public để tạo `callback_url` gửi OpenClaw (mặc định theo `LISTEN_ADDR`, ví dụ `http://localhost:8080`) | | `LISTEN_ADDR` | | Địa chỉ bind HTTP server, mặc định `:8080` |
| `CALLBACK_BASE_URL` | | Base URL public để tạo `callback_url`, mặc định suy ra từ `LISTEN_ADDR` |
## Health Checks ## Health checks
| Endpoint | Mục đích | | Endpoint | Mục đích |
|---|---| |---|---|
| `GET /healthz` | Liveness process còn sống | | `GET /healthz` | Liveness check, xác nhận process còn sống |
| `GET /readyz` | Readiness DB writable + config hợp lệ | | `GET /readyz` | Readiness check, xác nhận DB writable config bắt buộc hợp lệ |
## Pipeline trạng thái ## Đọc thêm
``` - [Technical overview](docs/architecture/overview.md)
RECEIVED → AI_PROCESSING → CALLBACK_DONE → COMPLETED - [Architecture Decision Records](docs/adr/README.md)
↓ ↓ ↓
IGNORED FAILED SMTP_RETRYING → FAILED
```
## Công nghệ
| Thành phần | Thư viện |
|---|---|
| HTTP Server | [gin-gonic/gin](https://github.com/gin-gonic/gin) |
| Database | [GORM](https://gorm.io) + [modernc.org/sqlite](https://pkg.go.dev/modernc.org/sqlite) (pure Go, no CGO) |
| IMAP | [emersion/go-imap](https://github.com/emersion/go-imap) |
| Logging | [log/slog](https://pkg.go.dev/log/slog) (stdlib) |
| Config | [joho/godotenv](https://github.com/joho/godotenv) |
## License ## License
+30
View File
@@ -0,0 +1,30 @@
---
type: ADR
id: "0002"
title: "Ingress uses a configurable external rules pipeline before dispatch"
status: active
date: 2026-04-28
---
## Context
The IMAP ingress flow previously hardcoded whitelist and idempotency checks directly in processMessage. There was no extension point for operator-defined policy such as sender blocking or context-based routing. The flow mixed internal safety rules (anti-loop) with configurable policy (whitelist).
## Decision
Separate internal safety checks (anti-loop, idempotency) from external configurable rules. External rules run in a pipeline after safety checks. Each rule implements a `Rule` interface that evaluates `EmailContext` and returns `RuleResult` (accept/reject + metadata). Rule metadata is persisted as `DispatchContext` on the Task and passed to OpenClaw dispatch.
Pipeline order: anti-loop (internal) → idempotency (internal) → external rules → threading → save → dispatch.
## Options considered
- **Pipeline with Rule interface** (chosen): extensible, testable, each rule is isolated.
- **Keep everything in processMessage**: simpler but not configurable or testable.
- **Plugin-based rules via config files**: more flexible but over-engineered for current needs.
## Consequences
New rules can be added by implementing the Rule interface and registering in config.
Whitelist and blocklist are now env-configurable and independently testable.
Future rules (domain routing, context selection) fit the same pipeline.
Task model has a new DispatchContext column (auto-migrated).
+1
View File
@@ -73,3 +73,4 @@ Optional. Capture important external input or review notes.
| ID | Title | Status | | ID | Title | Status |
|----|-------|--------| |----|-------|--------|
| 0001 | IMAP thread resolution prefers In-Reply-To and falls back to References | active | | 0001 | IMAP thread resolution prefers In-Reply-To and falls back to References | active |
| 0002 | Ingress uses a configurable external rules pipeline before dispatch | active |
+101
View File
@@ -0,0 +1,101 @@
# Technical Overview
## Mục đích
Tài liệu này là technical overview hiện hành của `Email-OpenClaw Bridge`. `README.md` chỉ giữ phần onboarding và vận hành cơ bản; phần này giữ các chi tiết kỹ thuật bền vững hơn về flow, package layout, state machine và công nghệ chính.
## Luồng xử lý
Bridge xử lý email theo pipeline sau:
```text
Ingress (IMAP) → Safety Checks → External Rules → Dispatch (OpenClaw) → Callback → Egress (SMTP reply)
```
Các giai đoạn runtime chính:
1. IMAP watcher theo dõi `INBOX` bằng IDLE và fetch email chưa đọc.
2. Safety checks loại bỏ mail từ chính hệ thống và chặn duplicate theo `message_id`.
3. External rules pipeline áp whitelist/blocklist và sinh `DispatchContext`.
4. Thread resolution gắn `thread_id` dựa trên quan hệ email đã biết.
5. Task được lưu vào SQLite với trạng thái pipeline hiện tại.
6. OpenClaw nhận request xử lý và callback về bridge.
7. Callback cập nhật `AIResponse`, sau đó bridge gửi phản hồi lại qua SMTP.
## Package layout
- `cmd/bridge`
- Entrypoint của process, wiring config, DB, IMAP watcher, HTTP router và SMTP egress.
- `internal/config`
- Đọc `.env`, chuẩn hóa default như `LISTEN_ADDR`, `CALLBACK_BASE_URL`, và validate biến bắt buộc.
- `internal/database`
- Khởi tạo SQLite qua GORM, định nghĩa `Task` model, repository helpers và health check DB.
- `internal/mail`
- Chứa IMAP ingress, thread resolution, proxy support cho IMAP, retry logic và SMTP egress.
- `internal/rules`
- External rules pipeline, whitelist/blocklist rules và rule metadata dùng cho dispatch.
- `internal/ai_client`
- HTTP client gọi OpenClaw và gửi metadata/callback information ra ngoài.
- `internal/api`
- HTTP router cho `healthz`, `readyz``POST /callback`.
- `internal/logging`
- Helper tạo structured logger theo `task_uuid`, `thread_id`, `message_id`.
## Task state machine
Task được lưu trong SQLite và đi qua các trạng thái sau:
```text
RECEIVED → AI_PROCESSING → CALLBACK_DONE → COMPLETED
↓ ↓ ↓
IGNORED FAILED SMTP_RETRYING → FAILED
```
- `IGNORED`
- Mail bị anti-loop, duplicate, hoặc bị external rules reject.
- `RECEIVED`
- Task đã được lưu và sẵn sàng dispatch.
- `AI_PROCESSING`
- Request đã được gửi sang OpenClaw.
- `CALLBACK_DONE`
- Callback hợp lệ đã cập nhật `AIResponse`.
- `SMTP_RETRYING`
- Đang retry gửi email phản hồi.
- `COMPLETED`
- Toàn bộ pipeline kết thúc thành công.
- `FAILED`
- Hết retry hoặc gặp lỗi không recoverable.
## HTTP và health behavior
- `GET /healthz`
- Trả `200 {"status":"ok"}` nếu process đang sống.
- `GET /readyz`
- Trả `200` khi DB writable và `BRIDGE_CALLBACK_TOKEN` có mặt.
- Trả `503` nếu DB không writable hoặc config bắt buộc cho callback chưa sẵn sàng.
- `POST /callback`
- Yêu cầu header `X-Bridge-Token`.
- Nhận payload callback từ OpenClaw, cập nhật task và kích hoạt SMTP egress bất đồng bộ.
## Công nghệ chính
- Go
- Runtime chính, phù hợp với service nhỏ gọn và dễ deploy.
- SQLite + GORM
- Giữ state cục bộ đơn giản, không cần service DB riêng cho v1.
- `github.com/emersion/go-imap/v2`
- Hỗ trợ IMAP IDLE và xử lý mailbox theo flow hiện tại.
- `gin-gonic/gin`
- Cung cấp HTTP router tối giản cho health endpoints và callback.
- `log/slog`
- Structured logging chuẩn thư viện chuẩn của Go.
## ADR liên quan
- [ADR 0001: IMAP thread resolution prefers In-Reply-To and falls back to References](../adr/0001-imap-thread-resolution.md)
- [ADR 0002: Ingress uses a configurable external rules pipeline before dispatch](../adr/0002-external-rules-pipeline.md)
- [ADR index](../adr/README.md)
## Tài liệu lịch sử
- Historical design input: [docs/specs/2026-04-24-email-openclaw-bridge-design.md](../specs/2026-04-24-email-openclaw-bridge-design.md)
+23 -1
View File
@@ -73,7 +73,7 @@ func (d *Dispatcher) Dispatch(task *database.Task) {
SessionID: task.ThreadID, SessionID: task.ThreadID,
History: history, History: history,
CallbackURL: callbackURL, CallbackURL: callbackURL,
Metadata: map[string]string{"task_uuid": task.TaskUUID}, Metadata: buildDispatchMetadata(task),
} }
// Update status to AI_PROCESSING before first attempt. // Update status to AI_PROCESSING before first attempt.
@@ -195,3 +195,25 @@ func (d *Dispatcher) buildHistory(threadID string) ([]historyEntry, error) {
return history, nil return history, nil
} }
// ExportBuildDispatchMetadata exports buildDispatchMetadata for testing.
func ExportBuildDispatchMetadata(task *database.Task) map[string]string {
return buildDispatchMetadata(task)
}
// buildDispatchMetadata constructs the metadata map for OpenClaw dispatch,
// merging the task UUID with any DispatchContext from the rules pipeline.
func buildDispatchMetadata(task *database.Task) map[string]string {
metadata := map[string]string{"task_uuid": task.TaskUUID}
if task.DispatchContext != "" {
var ctx map[string]string
if err := json.Unmarshal([]byte(task.DispatchContext), &ctx); err == nil {
for k, v := range ctx {
if k != "task_uuid" {
metadata[k] = v
}
}
}
}
return metadata
}
+47
View File
@@ -12,6 +12,53 @@ import (
"thuanle.me/claw-email-bridge/internal/database" "thuanle.me/claw-email-bridge/internal/database"
) )
func TestBuildDispatchMetadata_ForwardsContext(t *testing.T) {
task := &database.Task{
TaskUUID: "uuid-ctx-test",
DispatchContext: `{"rule":"whitelist","source":"env"}`,
}
metadata := ai_client.ExportBuildDispatchMetadata(task)
if metadata["task_uuid"] != "uuid-ctx-test" {
t.Errorf("expected task_uuid=uuid-ctx-test, got %q", metadata["task_uuid"])
}
if metadata["rule"] != "whitelist" {
t.Errorf("expected rule=whitelist, got %q", metadata["rule"])
}
if metadata["source"] != "env" {
t.Errorf("expected source=env, got %q", metadata["source"])
}
}
func TestBuildDispatchMetadata_ProtectsTaskUUID(t *testing.T) {
task := &database.Task{
TaskUUID: "original-uuid",
DispatchContext: `{"task_uuid":"malicious-override","extra":"data"}`,
}
metadata := ai_client.ExportBuildDispatchMetadata(task)
if metadata["task_uuid"] != "original-uuid" {
t.Errorf("task_uuid should not be overwritten, got %q", metadata["task_uuid"])
}
if metadata["extra"] != "data" {
t.Errorf("expected extra=data, got %q", metadata["extra"])
}
}
func TestBuildDispatchMetadata_NoContext(t *testing.T) {
task := &database.Task{
TaskUUID: "uuid-no-ctx",
}
metadata := ai_client.ExportBuildDispatchMetadata(task)
if len(metadata) != 1 {
t.Errorf("expected 1 key, got %d", len(metadata))
}
if metadata["task_uuid"] != "uuid-no-ctx" {
t.Errorf("expected task_uuid=uuid-no-ctx, got %q", metadata["task_uuid"])
}
}
// Test matrix #2: OpenClaw fail 2 lần, lần 3 thành công → COMPLETED, attempt_openclaw = 3. // Test matrix #2: OpenClaw fail 2 lần, lần 3 thành công → COMPLETED, attempt_openclaw = 3.
func TestDispatch_RetryThenSuccess(t *testing.T) { func TestDispatch_RetryThenSuccess(t *testing.T) {
tdb := database.NewTestDB(t) tdb := database.NewTestDB(t)
+9
View File
@@ -31,6 +31,7 @@ type Config struct {
BridgeCallbackToken string BridgeCallbackToken string
SystemEmail string SystemEmail string
WhitelistEmails []string WhitelistEmails []string
BlocklistEmails []string
// Optional // Optional
IMAPProxyURL string IMAPProxyURL string
@@ -74,6 +75,14 @@ func Load() (*Config, error) {
} }
} }
if raw := os.Getenv("BLOCKLIST_EMAILS"); raw != "" {
for _, email := range strings.Split(raw, ",") {
if trimmed := strings.TrimSpace(email); trimmed != "" {
cfg.BlocklistEmails = append(cfg.BlocklistEmails, trimmed)
}
}
}
if cfg.ListenAddr == "" { if cfg.ListenAddr == "" {
cfg.ListenAddr = ":8080" cfg.ListenAddr = ":8080"
} }
+1
View File
@@ -24,6 +24,7 @@ type Task struct {
Sender string Sender string
Subject string Subject string
BodyPlain string BodyPlain string
DispatchContext string // JSON metadata from external rules pipeline.
AIResponse string AIResponse string
Status string `gorm:"index;not null"` Status string `gorm:"index;not null"`
AttemptOpenClaw int `gorm:"default:0"` AttemptOpenClaw int `gorm:"default:0"`
+147 -51
View File
@@ -1,12 +1,16 @@
package mail package mail
import ( import (
"bufio"
"context" "context"
"crypto/tls" "crypto/tls"
"encoding/base64"
"encoding/json"
"fmt" "fmt"
"io" "io"
"log/slog" "log/slog"
"net" "net"
"net/http"
"net/url" "net/url"
"strings" "strings"
"time" "time"
@@ -21,6 +25,7 @@ import (
"thuanle.me/claw-email-bridge/internal/config" "thuanle.me/claw-email-bridge/internal/config"
"thuanle.me/claw-email-bridge/internal/database" "thuanle.me/claw-email-bridge/internal/database"
"thuanle.me/claw-email-bridge/internal/logging" "thuanle.me/claw-email-bridge/internal/logging"
"thuanle.me/claw-email-bridge/internal/rules"
) )
// IMAPWatcher monitors an IMAP mailbox via IDLE and processes new emails. // IMAPWatcher monitors an IMAP mailbox via IDLE and processes new emails.
@@ -32,6 +37,8 @@ type IMAPWatcher struct {
dialIMAP func(addr string, options *imapclient.Options) (*imapclient.Client, error) dialIMAP func(addr string, options *imapclient.Options) (*imapclient.Client, error)
dialIMAPViaProxy func(addr, proxyURL string, options *imapclient.Options) (*imapclient.Client, error) dialIMAPViaProxy func(addr, proxyURL string, options *imapclient.Options) (*imapclient.Client, error)
rules *rules.Pipeline
// onReceived is called after a task is saved as RECEIVED. // onReceived is called after a task is saved as RECEIVED.
// This will be wired to the OpenClaw dispatch in a future step. // This will be wired to the OpenClaw dispatch in a future step.
OnReceived func(task *database.Task) OnReceived func(task *database.Task)
@@ -55,13 +62,26 @@ func (d timeoutContextDialer) DialContext(ctx context.Context, network, address
var socks5DialerFactory = proxy.SOCKS5 var socks5DialerFactory = proxy.SOCKS5
var proxyTLSConfigForTest = func(host string) *tls.Config {
return &tls.Config{ServerName: host}
}
// NewIMAPWatcher creates a new IMAP watcher. // NewIMAPWatcher creates a new IMAP watcher.
func NewIMAPWatcher(cfg *config.Config, db *gorm.DB) *IMAPWatcher { func NewIMAPWatcher(cfg *config.Config, db *gorm.DB) *IMAPWatcher {
var externalRules []rules.Rule
if len(cfg.WhitelistEmails) > 0 {
externalRules = append(externalRules, rules.NewWhitelistRule(cfg.WhitelistEmails))
}
if len(cfg.BlocklistEmails) > 0 {
externalRules = append(externalRules, rules.NewBlocklistRule(cfg.BlocklistEmails))
}
return &IMAPWatcher{ return &IMAPWatcher{
cfg: cfg, cfg: cfg,
db: db, db: db,
dialIMAP: imapclient.DialTLS, dialIMAP: imapclient.DialTLS,
dialIMAPViaProxy: dialTLSViaSOCKS5, dialIMAPViaProxy: dialTLSViaProxy,
rules: rules.NewPipeline(externalRules),
} }
} }
@@ -180,34 +200,20 @@ func (w *IMAPWatcher) connectAndWatch(ctx context.Context) error {
} }
} }
func dialTLSViaSOCKS5(addr, proxyURL string, options *imapclient.Options) (*imapclient.Client, error) { func dialTLSViaProxy(addr, proxyURL string, options *imapclient.Options) (*imapclient.Client, error) {
u, err := url.Parse(proxyURL) u, err := url.Parse(proxyURL)
if err != nil { if err != nil {
return nil, fmt.Errorf("parse proxy url: %w", err) return nil, fmt.Errorf("parse proxy url: %w", err)
} }
if u.Scheme != "socks5" {
return nil, fmt.Errorf("unsupported proxy scheme: %s", u.Scheme)
}
var auth *proxy.Auth
if u.User != nil {
pw, _ := u.User.Password()
auth = &proxy.Auth{User: u.User.Username(), Password: pw}
}
dialer, err := socks5DialerFactory("tcp", u.Host, auth, timeoutContextDialer{timeout: imapDialTimeout})
if err != nil {
return nil, fmt.Errorf("create socks5 dialer: %w", err)
}
dialCtx, dialCancel := context.WithTimeout(context.Background(), imapDialTimeout)
defer dialCancel()
var conn net.Conn var conn net.Conn
if cd, ok := dialer.(proxy.ContextDialer); ok { switch u.Scheme {
conn, err = cd.DialContext(dialCtx, "tcp", addr) case "socks5":
} else { conn, err = dialViaSOCKS5(u, addr)
conn, err = dialer.Dial("tcp", addr) case "http", "https":
conn, err = dialViaCONNECT(u, addr)
default:
return nil, fmt.Errorf("unsupported proxy scheme: %s", u.Scheme)
} }
if err != nil { if err != nil {
return nil, fmt.Errorf("proxy dial: %w", err) return nil, fmt.Errorf("proxy dial: %w", err)
@@ -230,6 +236,96 @@ func dialTLSViaSOCKS5(addr, proxyURL string, options *imapclient.Options) (*imap
return imapclient.New(tlsConn, options), nil return imapclient.New(tlsConn, options), nil
} }
func dialViaSOCKS5(u *url.URL, targetAddr string) (net.Conn, error) {
var auth *proxy.Auth
if u.User != nil {
pw, _ := u.User.Password()
auth = &proxy.Auth{User: u.User.Username(), Password: pw}
}
dialer, err := socks5DialerFactory("tcp", u.Host, auth, timeoutContextDialer{timeout: imapDialTimeout})
if err != nil {
return nil, fmt.Errorf("create socks5 dialer: %w", err)
}
dialCtx, dialCancel := context.WithTimeout(context.Background(), imapDialTimeout)
defer dialCancel()
if cd, ok := dialer.(proxy.ContextDialer); ok {
return cd.DialContext(dialCtx, "tcp", targetAddr)
}
return dialer.Dial("tcp", targetAddr)
}
// proxyConn wraps a net.Conn, draining buffered data from a bufio.Reader
// before delegating reads to the underlying connection.
type proxyConn struct {
net.Conn
reader io.Reader
}
func (c *proxyConn) Read(b []byte) (int, error) {
return c.reader.Read(b)
}
// dialToProxy is the function used to establish a TCP connection to the proxy.
var dialToProxy = func(ctx context.Context, network, addr string) (net.Conn, error) {
return (&net.Dialer{Timeout: imapDialTimeout}).DialContext(ctx, network, addr)
}
func dialViaCONNECT(u *url.URL, targetAddr string) (net.Conn, error) {
ctx, cancel := context.WithTimeout(context.Background(), imapDialTimeout)
defer cancel()
conn, err := dialToProxy(ctx, "tcp", u.Host)
if err != nil {
return nil, fmt.Errorf("dial proxy %s: %w", u.Host, err)
}
if u.Scheme == "https" {
tlsConn := tls.Client(conn, proxyTLSConfigForTest(u.Hostname()))
if err := tlsConn.HandshakeContext(ctx); err != nil {
_ = tlsConn.Close()
return nil, fmt.Errorf("tls handshake to proxy: %w", err)
}
conn = tlsConn
}
_ = conn.SetDeadline(time.Now().Add(imapDialTimeout))
connectReq := fmt.Sprintf("CONNECT %s HTTP/1.1\r\nHost: %s\r\n", targetAddr, targetAddr)
if u.User != nil {
username := u.User.Username()
password, _ := u.User.Password()
creds := base64.StdEncoding.EncodeToString([]byte(username + ":" + password))
connectReq += fmt.Sprintf("Proxy-Authorization: Basic %s\r\n", creds)
}
connectReq += "\r\n"
if _, err := fmt.Fprint(conn, connectReq); err != nil {
_ = conn.Close()
return nil, fmt.Errorf("send connect: %w", err)
}
br := bufio.NewReader(conn)
resp, err := http.ReadResponse(br, nil)
if err != nil {
_ = conn.Close()
return nil, fmt.Errorf("read connect response: %w", err)
}
if resp.StatusCode != http.StatusOK {
_ = conn.Close()
return nil, fmt.Errorf("proxy connect failed: %s", resp.Status)
}
_ = conn.SetDeadline(time.Time{})
if br.Buffered() > 0 {
return &proxyConn{Conn: conn, reader: io.MultiReader(br, conn)}, nil
}
return conn, nil
}
// fetchUnseen searches for UNSEEN messages and processes each one. // fetchUnseen searches for UNSEEN messages and processes each one.
func (w *IMAPWatcher) fetchUnseen(c *imapclient.Client) error { func (w *IMAPWatcher) fetchUnseen(c *imapclient.Client) error {
criteria := &imap.SearchCriteria{ criteria := &imap.SearchCriteria{
@@ -343,15 +439,7 @@ func (w *IMAPWatcher) processMessage(c *imapclient.Client, buf *imapclient.Fetch
return return
} }
// 2. Whitelist: skip if sender not in allowed list. // 2. Idempotency: skip if message_id already exists.
if !w.isWhitelisted(sender) {
log.Info("imap: sender not whitelisted, ignoring", "sender", sender)
w.saveIgnored(messageID, sender, subject, "not whitelisted")
w.markSeen(c, buf.UID)
return
}
// 3. Idempotency: skip if message_id already exists.
existing, err := database.FindByMessageID(w.db, messageID) existing, err := database.FindByMessageID(w.db, messageID)
if err != nil { if err != nil {
log.Error("imap: db lookup failed", "error", err) log.Error("imap: db lookup failed", "error", err)
@@ -363,6 +451,19 @@ func (w *IMAPWatcher) processMessage(c *imapclient.Client, buf *imapclient.Fetch
return return
} }
// 3. External rules: evaluate operator-configurable policy.
ruleResult := w.rules.Evaluate(rules.EmailContext{
Sender: sender,
Subject: subject,
MessageID: messageID,
})
if !ruleResult.Accepted {
log.Info("imap: rejected by external rules", "sender", sender, "reason", ruleResult.Reason)
w.saveIgnored(messageID, sender, subject, ruleResult.Reason)
w.markSeen(c, buf.UID)
return
}
// 4. Threading: determine thread_id. // 4. Threading: determine thread_id.
threadID := messageID threadID := messageID
inReplyTo := "" inReplyTo := ""
@@ -404,13 +505,14 @@ func (w *IMAPWatcher) processMessage(c *imapclient.Client, buf *imapclient.Fetch
// 5. Save task with status RECEIVED. // 5. Save task with status RECEIVED.
taskUUID := uuid.New().String() taskUUID := uuid.New().String()
task := &database.Task{ task := &database.Task{
TaskUUID: taskUUID, TaskUUID: taskUUID,
ThreadID: threadID, ThreadID: threadID,
MessageID: messageID, MessageID: messageID,
Sender: sender, Sender: sender,
Subject: subject, Subject: subject,
BodyPlain: bodyPlain, BodyPlain: bodyPlain,
Status: database.StatusReceived, DispatchContext: marshalDispatchContext(ruleResult.Metadata),
Status: database.StatusReceived,
} }
taskLog := logging.TaskLogger(taskUUID, threadID, messageID) taskLog := logging.TaskLogger(taskUUID, threadID, messageID)
@@ -453,19 +555,13 @@ func (w *IMAPWatcher) saveIgnored(messageID, sender, subject, reason string) {
} }
} }
// isWhitelisted checks if the sender is in the allowed list. // marshalDispatchContext serializes rule metadata to JSON.
// Empty whitelist means all senders are allowed. func marshalDispatchContext(metadata map[string]string) string {
func (w *IMAPWatcher) isWhitelisted(sender string) bool { if len(metadata) == 0 {
if len(w.cfg.WhitelistEmails) == 0 { return ""
return true
} }
lower := strings.ToLower(sender) data, _ := json.Marshal(metadata)
for _, email := range w.cfg.WhitelistEmails { return string(data)
if strings.ToLower(email) == lower {
return true
}
}
return false
} }
// markSeen flags a message as \Seen in IMAP. // markSeen flags a message as \Seen in IMAP.
+213 -6
View File
@@ -1,10 +1,20 @@
package mail package mail
import ( import (
"bufio"
"context" "context"
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"crypto/tls"
"crypto/x509"
"encoding/base64"
"errors" "errors"
"fmt"
"io" "io"
"math/big"
"net" "net"
"net/http"
"strings" "strings"
"testing" "testing"
"time" "time"
@@ -73,13 +83,22 @@ func TestConnectAndWatch_UsesProxyDial_WhenProxySet(t *testing.T) {
} }
} }
func TestDialTLSViaSOCKS5_RejectsNonSocks5Scheme(t *testing.T) { func TestDialTLSViaProxy_RejectsUnsupportedScheme(t *testing.T) {
_, err := dialTLSViaSOCKS5("imap.example.com:993", "http://proxy:8080", nil) _, err := dialTLSViaProxy("imap.example.com:993", "ftp://proxy:21", nil)
if err == nil || !strings.Contains(err.Error(), "unsupported proxy scheme") { if err == nil || !strings.Contains(err.Error(), "unsupported proxy scheme") {
t.Fatalf("expected unsupported scheme error, got %v", err) t.Fatalf("expected unsupported scheme error, got %v", err)
} }
} }
func TestDialTLSViaProxy_RejectsMalformedURL(t *testing.T) {
_, err := dialTLSViaProxy("imap.example.com:993", "://bad", nil)
if err == nil {
t.Fatal("expected error for malformed URL")
}
}
// --- SOCKS5 path ---
type fakeContextDialer struct{} type fakeContextDialer struct{}
func (fakeContextDialer) Dial(network, address string) (net.Conn, error) { func (fakeContextDialer) Dial(network, address string) (net.Conn, error) {
@@ -93,7 +112,7 @@ func (fakeContextDialer) DialContext(ctx context.Context, network, address strin
return nil, errors.New("used dialcontext") return nil, errors.New("used dialcontext")
} }
func TestDialTLSViaSOCKS5_UsesDialContextWithTimeout(t *testing.T) { func TestDialTLSViaProxy_SOCKS5_UsesDialContextWithTimeout(t *testing.T) {
orig := socks5DialerFactory orig := socks5DialerFactory
t.Cleanup(func() { socks5DialerFactory = orig }) t.Cleanup(func() { socks5DialerFactory = orig })
@@ -104,7 +123,7 @@ func TestDialTLSViaSOCKS5_UsesDialContextWithTimeout(t *testing.T) {
return fakeContextDialer{}, nil return fakeContextDialer{}, nil
} }
_, err := dialTLSViaSOCKS5("imap.example.com:993", "socks5://127.0.0.1:1080", nil) _, err := dialTLSViaProxy("imap.example.com:993", "socks5://127.0.0.1:1080", nil)
if err == nil || !strings.Contains(err.Error(), "proxy dial: used dialcontext") { if err == nil || !strings.Contains(err.Error(), "proxy dial: used dialcontext") {
t.Fatalf("expected DialContext path, got %v", err) t.Fatalf("expected DialContext path, got %v", err)
} }
@@ -143,7 +162,7 @@ func (d contextConnDialer) DialContext(ctx context.Context, network, address str
return d.conn, nil return d.conn, nil
} }
func TestDialTLSViaSOCKS5_SetsDeadlineForTLSHandshake(t *testing.T) { func TestDialTLSViaProxy_SOCKS5_SetsDeadlineForTLSHandshake(t *testing.T) {
orig := socks5DialerFactory orig := socks5DialerFactory
t.Cleanup(func() { socks5DialerFactory = orig }) t.Cleanup(func() { socks5DialerFactory = orig })
@@ -152,7 +171,7 @@ func TestDialTLSViaSOCKS5_SetsDeadlineForTLSHandshake(t *testing.T) {
return contextConnDialer{conn: conn}, nil return contextConnDialer{conn: conn}, nil
} }
_, err := dialTLSViaSOCKS5("imap.example.com:993", "socks5://127.0.0.1:1080", nil) _, err := dialTLSViaProxy("imap.example.com:993", "socks5://127.0.0.1:1080", nil)
if err == nil || !strings.Contains(err.Error(), "tls handshake") { if err == nil || !strings.Contains(err.Error(), "tls handshake") {
t.Fatalf("expected tls handshake error, got %v", err) t.Fatalf("expected tls handshake error, got %v", err)
} }
@@ -160,3 +179,191 @@ func TestDialTLSViaSOCKS5_SetsDeadlineForTLSHandshake(t *testing.T) {
t.Fatal("expected TLS handshake deadline to be set") t.Fatal("expected TLS handshake deadline to be set")
} }
} }
// --- HTTP CONNECT path ---
func startFakeProxy(t *testing.T, handler func(host string) bool) (addr string, cleanup func()) {
t.Helper()
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatalf("listen: %v", err)
}
done := make(chan struct{})
go func() {
defer close(done)
for {
conn, err := ln.Accept()
if err != nil {
return
}
go func(c net.Conn) {
defer c.Close()
br := bufio.NewReader(c)
req, err := http.ReadRequest(br)
if err != nil {
return
}
if handler(req.URL.Host) {
fmt.Fprintf(c, "HTTP/1.1 200 OK\r\n\r\n")
} else {
fmt.Fprintf(c, "HTTP/1.1 403 Forbidden\r\n\r\n")
}
}(conn)
}
}()
return ln.Addr().String(), func() {
ln.Close()
<-done
}
}
func TestDialTLSViaProxy_HTTPConnect_ReachesTLSHandshake(t *testing.T) {
addr, cleanup := startFakeProxy(t, func(host string) bool { return true })
defer cleanup()
_, err := dialTLSViaProxy("imap.example.com:993", fmt.Sprintf("http://%s", addr), nil)
if err == nil || !strings.Contains(err.Error(), "tls handshake") {
t.Fatalf("expected tls handshake error (CONNECT succeeded), got %v", err)
}
}
func TestDialTLSViaProxy_HTTPConnect_RejectsOnProxyFailure(t *testing.T) {
addr, cleanup := startFakeProxy(t, func(host string) bool { return false })
defer cleanup()
_, err := dialTLSViaProxy("imap.example.com:993", fmt.Sprintf("http://%s", addr), nil)
if err == nil || !strings.Contains(err.Error(), "proxy connect failed") {
t.Fatalf("expected proxy connect failed error, got %v", err)
}
}
func TestDialTLSViaProxy_HTTPConnect_SendsProxyAuth(t *testing.T) {
var gotAuth string
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatalf("listen: %v", err)
}
go func() {
defer ln.Close()
conn, err := ln.Accept()
if err != nil {
return
}
defer conn.Close()
br := bufio.NewReader(conn)
req, err := http.ReadRequest(br)
if err != nil {
return
}
gotAuth = req.Header.Get("Proxy-Authorization")
fmt.Fprintf(conn, "HTTP/1.1 200 OK\r\n\r\n")
}()
_, _ = dialTLSViaProxy("imap.example.com:993",
fmt.Sprintf("http://testuser:testpass@%s", ln.Addr().String()), nil)
expected := "Basic " + base64.StdEncoding.EncodeToString([]byte("testuser:testpass"))
if gotAuth != expected {
t.Fatalf("expected auth %q, got %q", expected, gotAuth)
}
}
func TestDialTLSViaProxy_HTTPConnect_DialError(t *testing.T) {
origDial := dialToProxy
t.Cleanup(func() { dialToProxy = origDial })
dialToProxy = func(ctx context.Context, network, addr string) (net.Conn, error) {
return nil, errors.New("dial refused")
}
_, err := dialTLSViaProxy("imap.example.com:993", "http://127.0.0.1:1", nil)
if err == nil || !strings.Contains(err.Error(), "dial proxy") {
t.Fatalf("expected dial proxy error, got %v", err)
}
}
// --- HTTPS CONNECT path ---
func generateTestCert(t *testing.T) tls.Certificate {
t.Helper()
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
t.Fatalf("generate key: %v", err)
}
tmpl := &x509.Certificate{
SerialNumber: big.NewInt(1),
NotBefore: time.Now(),
NotAfter: time.Now().Add(time.Hour),
KeyUsage: x509.KeyUsageDigitalSignature,
DNSNames: []string{"127.0.0.1"},
IPAddresses: []net.IP{net.ParseIP("127.0.0.1")},
}
certDER, err := x509.CreateCertificate(rand.Reader, tmpl, tmpl, &key.PublicKey, key)
if err != nil {
t.Fatalf("create cert: %v", err)
}
cert, err := x509.ParseCertificate(certDER)
if err != nil {
t.Fatalf("parse cert: %v", err)
}
return tls.Certificate{
Certificate: [][]byte{certDER},
PrivateKey: key,
Leaf: cert,
}
}
func TestDialTLSViaProxy_HTTPSConnect_ReachesTLSHandshake(t *testing.T) {
cert := generateTestCert(t)
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatalf("listen: %v", err)
}
tlsLn := tls.NewListener(ln, &tls.Config{
Certificates: []tls.Certificate{cert},
})
done := make(chan struct{})
go func() {
defer close(done)
for {
conn, err := tlsLn.Accept()
if err != nil {
return
}
go func(c net.Conn) {
defer c.Close()
br := bufio.NewReader(c)
req, err := http.ReadRequest(br)
if err != nil {
return
}
if req.URL.Host != "" {
fmt.Fprintf(c, "HTTP/1.1 200 OK\r\n\r\n")
}
}(conn)
}
}()
defer func() {
tlsLn.Close()
<-done
}()
// Override dialToProxy to return raw TCP, but inject InsecureSkipVerify
// so the TLS handshake to proxy succeeds with the test cert.
origDial := dialToProxy
t.Cleanup(func() { dialToProxy = origDial })
origProxyTLS := proxyTLSConfigForTest
t.Cleanup(func() { proxyTLSConfigForTest = origProxyTLS })
proxyTLSConfigForTest = func(host string) *tls.Config {
return &tls.Config{ServerName: host, InsecureSkipVerify: true}
}
_, err = dialTLSViaProxy("imap.example.com:993",
fmt.Sprintf("https://%s", ln.Addr().String()), nil)
if err == nil || !strings.Contains(err.Error(), "tls handshake") {
t.Fatalf("expected tls handshake error (HTTPS CONNECT succeeded), got %v", err)
}
}
+26
View File
@@ -0,0 +1,26 @@
package rules
import "strings"
// BlocklistRule rejects emails from configured senders.
type BlocklistRule struct {
emails map[string]bool
}
// NewBlocklistRule creates a blocklist rule from a list of email addresses.
func NewBlocklistRule(emails []string) *BlocklistRule {
m := make(map[string]bool, len(emails))
for _, e := range emails {
m[strings.ToLower(e)] = true
}
return &BlocklistRule{emails: m}
}
func (r *BlocklistRule) Name() string { return "blocklist" }
func (r *BlocklistRule) Evaluate(ctx EmailContext) RuleResult {
if r.emails[strings.ToLower(ctx.Sender)] {
return Reject("blocked sender")
}
return Accept()
}
+35
View File
@@ -0,0 +1,35 @@
package rules
import "testing"
func TestBlocklistRule_SenderBlocked_Rejects(t *testing.T) {
r := NewBlocklistRule([]string{"spam@test.com"})
result := r.Evaluate(EmailContext{Sender: "spam@test.com"})
if result.Accepted {
t.Fatal("expected reject for blocked sender")
}
}
func TestBlocklistRule_SenderNotBlocked_Accepts(t *testing.T) {
r := NewBlocklistRule([]string{"spam@test.com"})
result := r.Evaluate(EmailContext{Sender: "ok@test.com"})
if !result.Accepted {
t.Fatal("expected accept for non-blocked sender")
}
}
func TestBlocklistRule_EmptyList_Accepts(t *testing.T) {
r := NewBlocklistRule(nil)
result := r.Evaluate(EmailContext{Sender: "anyone@test.com"})
if !result.Accepted {
t.Fatal("empty blocklist should accept all")
}
}
func TestBlocklistRule_CaseInsensitive(t *testing.T) {
r := NewBlocklistRule([]string{"Spam@Test.com"})
result := r.Evaluate(EmailContext{Sender: "spam@test.com"})
if result.Accepted {
t.Fatal("blocklist should be case-insensitive")
}
}
+30
View File
@@ -0,0 +1,30 @@
package rules
// Pipeline evaluates a sequence of rules, stopping on first rejection.
type Pipeline struct {
rules []Rule
}
// NewPipeline creates a rules pipeline from the given rules.
func NewPipeline(rules []Rule) *Pipeline {
return &Pipeline{rules: rules}
}
// Evaluate runs all rules in order. Returns the first rejection or a merged accept.
func (p *Pipeline) Evaluate(ctx EmailContext) RuleResult {
merged := Accept()
for _, r := range p.rules {
result := r.Evaluate(ctx)
if !result.Accepted {
return result
}
for k, v := range result.Metadata {
if merged.Metadata == nil {
merged.Metadata = map[string]string{k: v}
} else {
merged.Metadata[k] = v
}
}
}
return merged
}
+106
View File
@@ -0,0 +1,106 @@
package rules
import (
"testing"
)
func TestPipeline_NoRules_Accepts(t *testing.T) {
p := NewPipeline(nil)
result := p.Evaluate(EmailContext{Sender: "anyone@test.com"})
if !result.Accepted {
t.Fatal("expected accept with no rules")
}
}
func TestPipeline_AllRulesAccept_Accepts(t *testing.T) {
allowAll := &stubRule{name: "allow-all", result: Accept()}
p := NewPipeline([]Rule{allowAll})
result := p.Evaluate(EmailContext{Sender: "a@b.com"})
if !result.Accepted {
t.Fatal("expected accept")
}
}
func TestPipeline_OneRuleRejects_Rejects(t *testing.T) {
rejector := &stubRule{name: "rejector", result: Reject("nope")}
p := NewPipeline([]Rule{rejector})
result := p.Evaluate(EmailContext{Sender: "a@b.com"})
if result.Accepted {
t.Fatal("expected reject")
}
if result.Reason != "nope" {
t.Fatalf("expected reason 'nope', got %s", result.Reason)
}
}
func TestPipeline_MergesMetadata(t *testing.T) {
r1 := &stubRule{name: "r1", result: Accept().WithMetadata("a", "1")}
r2 := &stubRule{name: "r2", result: Accept().WithMetadata("b", "2")}
p := NewPipeline([]Rule{r1, r2})
result := p.Evaluate(EmailContext{})
if result.Metadata["a"] != "1" {
t.Fatal("missing metadata a")
}
if result.Metadata["b"] != "2" {
t.Fatal("missing metadata b")
}
}
func TestPipeline_StopsOnFirstReject(t *testing.T) {
rejector := &stubRule{name: "rejector", result: Reject("stop")}
neverCalled := &trackingRule{}
p := NewPipeline([]Rule{rejector, neverCalled})
p.Evaluate(EmailContext{})
if neverCalled.called {
t.Fatal("expected second rule not to be called after rejection")
}
}
func TestWhitelistRule_EmptyList_Accepts(t *testing.T) {
r := NewWhitelistRule(nil)
result := r.Evaluate(EmailContext{Sender: "anyone@test.com"})
if !result.Accepted {
t.Fatal("empty whitelist should accept all")
}
}
func TestWhitelistRule_SenderInList_Accepts(t *testing.T) {
r := NewWhitelistRule([]string{"allowed@test.com"})
result := r.Evaluate(EmailContext{Sender: "allowed@test.com"})
if !result.Accepted {
t.Fatal("expected accept for whitelisted sender")
}
}
func TestWhitelistRule_SenderNotInList_Rejects(t *testing.T) {
r := NewWhitelistRule([]string{"allowed@test.com"})
result := r.Evaluate(EmailContext{Sender: "unknown@test.com"})
if result.Accepted {
t.Fatal("expected reject for non-whitelisted sender")
}
}
func TestWhitelistRule_CaseInsensitive(t *testing.T) {
r := NewWhitelistRule([]string{"Allowed@Test.com"})
result := r.Evaluate(EmailContext{Sender: "allowed@test.com"})
if !result.Accepted {
t.Fatal("whitelist should be case-insensitive")
}
}
// stubs
type stubRule struct {
name string
result RuleResult
}
func (s *stubRule) Name() string { return s.name }
func (s *stubRule) Evaluate(_ EmailContext) RuleResult { return s.result }
type trackingRule struct {
called bool
}
func (t *trackingRule) Name() string { return "tracker" }
func (t *trackingRule) Evaluate(_ EmailContext) RuleResult { t.called = true; return Accept() }
+41
View File
@@ -0,0 +1,41 @@
package rules
// EmailContext contains the data available to rules for evaluation.
type EmailContext struct {
Sender string
Subject string
MessageID string
}
// RuleResult is the output of evaluating a single rule.
type RuleResult struct {
Accepted bool
Reason string
Metadata map[string]string
}
// Accept returns a passing RuleResult.
func Accept() RuleResult {
return RuleResult{Accepted: true}
}
// Reject returns a failing RuleResult with a reason.
func Reject(reason string) RuleResult {
return RuleResult{Accepted: false, Reason: reason}
}
// WithMetadata attaches a key-value pair to the result.
func (r RuleResult) WithMetadata(key, value string) RuleResult {
if r.Metadata == nil {
r.Metadata = map[string]string{key: value}
} else {
r.Metadata[key] = value
}
return r
}
// Rule evaluates an email against operator-configurable policy.
type Rule interface {
Name() string
Evaluate(ctx EmailContext) RuleResult
}
+42
View File
@@ -0,0 +1,42 @@
package rules
import (
"testing"
)
func TestRuleResult_Accept(t *testing.T) {
r := Accept()
if !r.Accepted {
t.Fatal("expected Accepted=true")
}
if r.Reason != "" {
t.Fatal("expected no reason for accept")
}
}
func TestRuleResult_Reject(t *testing.T) {
r := Reject("blocked sender")
if r.Accepted {
t.Fatal("expected Accepted=false")
}
if r.Reason != "blocked sender" {
t.Fatalf("expected reason 'blocked sender', got %s", r.Reason)
}
}
func TestRuleResult_WithMetadata(t *testing.T) {
r := Accept().WithMetadata("routing", "high-priority")
if r.Metadata["routing"] != "high-priority" {
t.Fatalf("expected routing=high-priority, got %s", r.Metadata["routing"])
}
}
func TestEmailContext_Fields(t *testing.T) {
ctx := EmailContext{
Sender: "user@example.com",
Subject: "Test",
}
if ctx.Sender != "user@example.com" {
t.Fatal("sender mismatch")
}
}
+30
View File
@@ -0,0 +1,30 @@
package rules
import "strings"
// WhitelistRule accepts emails from configured senders.
// An empty list accepts all senders.
type WhitelistRule struct {
emails map[string]bool
}
// NewWhitelistRule creates a whitelist rule from a list of email addresses.
func NewWhitelistRule(emails []string) *WhitelistRule {
m := make(map[string]bool, len(emails))
for _, e := range emails {
m[strings.ToLower(e)] = true
}
return &WhitelistRule{emails: m}
}
func (r *WhitelistRule) Name() string { return "whitelist" }
func (r *WhitelistRule) Evaluate(ctx EmailContext) RuleResult {
if len(r.emails) == 0 {
return Accept()
}
if r.emails[strings.ToLower(ctx.Sender)] {
return Accept()
}
return Reject("not whitelisted")
}
+11 -5
View File
@@ -47,6 +47,7 @@ Bảng `tasks`:
- `status` TEXT - `status` TEXT
- `attempt_openclaw` INTEGER DEFAULT 0 - `attempt_openclaw` INTEGER DEFAULT 0
- `attempt_smtp` INTEGER DEFAULT 0 - `attempt_smtp` INTEGER DEFAULT 0
- `dispatch_context` TEXT (JSON metadata từ external rules pipeline, truyền qua OpenClaw dispatch)
- `last_error` TEXT NULL - `last_error` TEXT NULL
- `next_retry_at` DATETIME NULL (chỉ cần nếu retry xử lý theo scheduler/worker) - `next_retry_at` DATETIME NULL (chỉ cần nếu retry xử lý theo scheduler/worker)
- `created_at` DATETIME - `created_at` DATETIME
@@ -68,12 +69,12 @@ Danh sách trạng thái hợp lệ:
- Nếu có `IMAP_PROXY_URL` thì kết nối IMAP qua proxy; nếu không thì direct. - Nếu có `IMAP_PROXY_URL` thì kết nối IMAP qua proxy; nếu không thì direct.
- Khi có email mới: - Khi có email mới:
1. Anti-loop: nếu `sender` trùng email hệ thống -> `IGNORED`. 1. Anti-loop: nếu `sender` trùng email hệ thống -> `IGNORED`.
2. Whitelist: nếu không nằm trong danh sách cho phép -> `IGNORED`. 2. Idempotency: nếu `message_id` đã tồn tại -> bỏ qua.
3. Idempotency: nếu `message_id` đã tồn tại -> bỏ qua. 3. External rules: chạy pipeline các rule configurable (whitelist, blocklist, v.v.). Nếu reject -> `IGNORED`. Rule metadata được lưu vào `dispatch_context`.
4. Threading: 4. Threading:
- Nếu có `In-Reply-To`/`References`: tìm email cha trong DB, kế thừa `thread_id`. - Nếu có `In-Reply-To`/`References`: tìm email cha trong DB, kế thừa `thread_id`.
- Nếu không có: `thread_id = message_id`. - Nếu không có: `thread_id = message_id`.
5. Lưu DB với `status = RECEIVED`. 5. Lưu DB với `status = RECEIVED`, bao gồm `dispatch_context` từ rules.
4.2 Dispatch (OpenClaw) 4.2 Dispatch (OpenClaw)
- Lấy nội dung email hiện tại và history của cùng `thread_id` (chỉ các bản ghi `COMPLETED`). - Lấy nội dung email hiện tại và history của cùng `thread_id` (chỉ các bản ghi `COMPLETED`).
@@ -84,6 +85,7 @@ Danh sách trạng thái hợp lệ:
- `history` (optional) - `history` (optional)
- `callback_url` - `callback_url`
- `metadata.task_uuid` - `metadata.task_uuid`
- `metadata.*` từ `dispatch_context` (rule-derived context)
- Cập nhật `status = AI_PROCESSING`. - Cập nhật `status = AI_PROCESSING`.
Retry OpenClaw: Retry OpenClaw:
@@ -146,6 +148,7 @@ Required:
- `OPENCLAW_URL`, `OPENCLAW_API_KEY` (nếu dùng) - `OPENCLAW_URL`, `OPENCLAW_API_KEY` (nếu dùng)
- `BRIDGE_CALLBACK_TOKEN` - `BRIDGE_CALLBACK_TOKEN`
- `WHITELIST_EMAILS` (comma-separated) - `WHITELIST_EMAILS` (comma-separated)
- `BLOCKLIST_EMAILS` (comma-separated, optional)
- `SYSTEM_EMAIL` - `SYSTEM_EMAIL`
- `IMAP_PROXY_URL` (optional) - `IMAP_PROXY_URL` (optional)
@@ -166,7 +169,7 @@ Ví dụ:
3. OpenClaw/SMTP retry đúng thứ tự 1s -> 5s -> 15s. 3. OpenClaw/SMTP retry đúng thứ tự 1s -> 5s -> 15s.
4. Hết retry thì `FAILED` và có `last_error`. 4. Hết retry thì `FAILED` và có `last_error`.
5. Callback sai token trả 401 và không đổi state task. 5. Callback sai token trả 401 và không đổi state task.
6. Anti-loop + whitelist + idempotency hoạt động đúng. 6. Anti-loop + idempotency + external rules hoạt động đúng thứ tự.
10) Test matrix tối thiểu 10) Test matrix tối thiểu
@@ -175,4 +178,7 @@ Ví dụ:
3. SMTP fail hết retry -> `FAILED`, có `last_error`. 3. SMTP fail hết retry -> `FAILED`, có `last_error`.
4. Callback sai token -> 401, state không đổi. 4. Callback sai token -> 401, state không đổi.
5. Duplicate `message_id` -> không tạo task mới. 5. Duplicate `message_id` -> không tạo task mới.
6. Sender là chính hệ thống hoặc ngoài whitelist -> `IGNORED`. 6. Sender là chính hệ thống -> `IGNORED`.
7. Sender ngoài whitelist -> `IGNORED` bởi external rules.
8. Sender trong blocklist -> `IGNORED` bởi external rules.
9. Rule metadata được truyền qua dispatch context đến OpenClaw.