[FIX#508] parser rule.Error 의 Host 필드 누락 보정 — stale_relearn / host failure counter 정상화

[juhy0987]

연관 이슈

• Closes [FIX] parser rule.Error 의 Host 필드 누락 — stale_relearn / host failure counter 무력화 #508

구현 내용

라이브 회차 분석에서 v.daum.net 81건 + news.daum.net 34건 등 ~221건 page parse 실패가 누적됐음에도 stale_relearn 메커니즘이 단 한 번도 발화하지 않은 회귀 수정.

근본 원인

internal/processor/parser/rule/parser.go / discovery.go / extract.go 의 14개 &Error{} 발산 모두 URL: raw.URL 만 채우고 Host 필드 누락. → handleRuleError → recordStaleAndMaybeRelearn 의 host == "" 가드에 막혀 staleCounter.Record 가 호출되지 않음.

비교: resolver.go 의 ErrNoRule 두 지점만 정상 설정 (그래서 resolver 발산 ErrNoRule 흐름은 정상 동작).

변경 요약

1) 정공법 — parser.go / discovery.go / extract.go 의 모든 Error 발산에 Host 명시 설정 (commit f03a203)

• 신규 헬퍼 errorHost(rawURL string) string — resolver 의 extractHostPath 와 동일한 canonical 정규화 (u.Hostname() + strings.ToLower)
• 14개 발산 지점에 Host: errorHost(raw.URL) 추가:
  • parser.go 9개 (ParsePage 4개 + ParseLinks 5개)
  • discovery.go 4개 (PageLinkDiscovery)
  • extract.go 1개 (validateRaw)

2) 방어책 — worker.go fallback (commit 0d804b0)

• handleRuleError 진입 직후 rerr.Host == "" && raw != nil 이면 hostOf(raw.URL) 로 backfill
• 향후 새 Error 발산 지점에서 Host 를 까먹어도 stale_relearn / chromedp 자동 전환이 무력화되지 않도록 fail-safe
• worker 의 hostOf 도 strings.ToLower 추가 — resolver / parser.errorHost 와 canonical form 일치 (counter 키 일관성)

3) 단위 테스트 (commit 3407a34)

• ParsePage / ParseLinks 의 각 에러 경로 (ErrEmptySelector / ErrParseFailure / nil-rule ErrNoRule / validateRaw 빈 raw) 에서 rerr.Host 가 정상 설정되는지 검증
• 정규화 검증: 대문자 host → lowercase, port 포함 → port 제거
• 8개 신규 테스트 케이스 추가

라이브 검증 가능 지표

다음 라이브 회차에서 확인 (LLM quota 정상 상태 전제):

• "stale rule failure recorded" (DEBUG) 또는 "stale rule threshold reached — LLM relearn trigger eligible" (INFO) 로그 실제 발생
• v.daum.net 의 룰이 LLM relearn 으로 자동 회복

CI / 머지 게이트 점검

변경 영향 범위

• 영향 패키지/모듈:
  • internal/processor/parser/rule (parser.go / discovery.go / extract.go — Error 발산 14개)
  • internal/processor/parser/worker (handleRuleError fallback + hostOf 정규화)
  • 테스트 (test/internal/processor/parser/rule/parser_test.go — 8 신규 케이스)
• 위험도: Low — 기존 호출 시그니처 / 메시지 / 에러 코드 동일, Host 필드만 추가. 정규화 변경 (worker hostOf 의 lowercase) 은 resolver/parser.errorHost 와 일관성 통일이라 회귀 위험 낮음.

Required Status Checks

• 통과 확인 대상:
  • Commit Lint
  • PR Title Lint
  • Linked Issue Check
  • Format Check
  • Build
  • Test
  • Lint

롤백 계획

• 본 PR revert 만으로 원복 가능. 단, 원복 시 stale_relearn / host failure counter 가 다시 무력화 상태로 회귀 — 가급적 forward-fix 권장.

🤖 Generated with Claude Code

Summary by CodeRabbit

• Improvements

  • Error messages now include canonical host context information across parser operations.
  • Enhanced error handling to populate host context consistently, even when unavailable from primary sources.

• Tests

  • Added comprehensive test coverage for error host context scenarios, including uppercase normalization and port stripping.

[coderabbitai]

Warning

Rate limit exceeded

@juhy0987 has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 50 minutes and 11 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f57e0bfe-8288-4900-9ddf-58695db877b2

📥 Commits

Reviewing files that changed from the base of the PR and between 3407a34 and b1bfc74.

📒 Files selected for processing (5)

• internal/processor/parser/rule/discovery.go
• internal/processor/parser/rule/extract.go
• internal/processor/parser/rule/parser.go
• internal/processor/parser/worker/worker.go
• test/internal/processor/parser/rule/parser_test.go

📝 Walkthrough

Walkthrough

This PR fixes a critical bug where the Host field in parser rule.Error objects was not populated, rendering stale_relearn mechanisms and host failure counters non-functional. The fix introduces a canonical host extraction helper and ensures all error paths in discovery, extraction, and parsing layers include the Host field; adds worker-side fallback logic for resilience; and provides comprehensive error-path test coverage.

Changes

Error Host Field Propagation

Layer / File(s)	Summary
Host canonicalization helper and ParsePage/ParseLinks error construction internal/processor/parser/rule/parser.go	Introduce errorHost(rawURL string) to extract canonical hostnames (lowercased, port-stripped). Populate the Host field in all ParsePage and ParseLinks error returns: nil rule lookup, missing Title/MainContent selectors, HTML parse failures, empty content extraction, ItemContainer matching zero elements, and ItemContainer with no valid ItemLink found.
Host field additions in discovery and extract paths internal/processor/parser/rule/discovery.go, internal/processor/parser/rule/extract.go	Extend Host field population to LinkDiscovery.Discover() error paths (nil config, regex compile failure, extraction failure, empty results) and validateRaw() empty-content error via the same errorHost() canonicalization.
Worker error handling and host canonicalization fallback internal/processor/parser/worker/worker.go	Add strings import. In handleRuleError(), insert fallback logic to derive Host from raw.URL via hostOf() when rerr.Host is empty, ensuring downstream stale counter and LLM enqueue logic receive valid host context. Update hostOf() to return lowercased canonical hostname (port removed) for consistent counter keying.
Error-path test coverage for Host canonicalization test/internal/processor/parser/rule/parser_test.go	Add 8 new test functions (TestParser_ParsePage_ErrorHost_*, TestParser_ParseLinks_ErrorHost_*, TestParser_NilRuleSuccess_ErrorHost) covering empty selectors, parse failures, empty-raw cases, lowercasing, and port stripping; include documentation block linking to issue #508.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Possibly related PRs

• EinSofINTEREST/IssueTracker#464: Both PRs modify parser error construction in extract.go and ParsePage/ParseLinks error paths in parser.go, adding/propagating host-related error context.

Suggested labels

refactor

Poem

🐰 A host so small, was lost in the dark,
In errors that flew without any mark,
Now canonicalized, lowercased, and clean,
The counters awake—best fix that we've seen!
thump thump 🥕

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly references the specific issue being fixed (#508) and describes the main change: adding missing Host field to parser rule.Error and normalizing stale_relearn/host failure counter.
Linked Issues check	✅ Passed	The PR fully addresses all coding requirements from issue #508: adds Host field to all 14 Error generation points in parser.go/discovery.go/extract.go, implements fallback Host population in worker.go, and includes 8 test cases validating Host field population and canonical normalization.
Out of Scope Changes check	✅ Passed	All changes remain within scope of issue #508: Error Host field fixes, worker.hostOf normalization updates, and corresponding test coverage. No unrelated modifications detected.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/#508/parser-rule-error-host-field

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Copilot]

Pull request overview

이슈 #508 수정 PR로, rule.Error 구조체에 Host 필드를 빠뜨려 worker 의 staleCounter / failureCounter 가 빈 host 가드에 막혀 무력화되던 회귀를 바로잡습니다. parser/discovery/extract 의 14개 &Error{} 발산 지점 모두 Host 를 채우고, worker 측에는 fallback 보호 및 host 정규화(lowercase) 통일을 추가합니다.

Changes:

• parser.go/discovery.go/extract.go 의 모든 &Error{} 발산에 Host: errorHost(raw.URL) 명시 + errorHost 헬퍼 신설 (canonical: u.Hostname() + lowercase)
• worker.handleRuleError 에 rerr.Host == "" 시 hostOf(raw.URL) 로 backfill, hostOf 도 lowercase 통일
• parser 에러 경로별 rerr.Host 검증 단위 테스트 8건 추가

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
internal/processor/parser/rule/parser.go	errorHost 헬퍼 추가, ParsePage/ParseLinks 9개 Error 발산 지점에 Host 설정
internal/processor/parser/rule/discovery.go	PageLinkDiscovery 의 4개 Error 발산에 Host 설정
internal/processor/parser/rule/extract.go	validateRaw 의 Error 발산에 Host 설정
internal/processor/parser/worker/worker.go	handleRuleError 의 Host fallback 추가, hostOf 에 lowercase 정규화
test/internal/processor/parser/rule/parser_test.go	Host 필드 채움/정규화 검증 단위 테스트 8건 추가

[gemini-code-assist]

Code Review

This pull request implements canonical host extraction for rule.Error objects to ensure consistent error tracking across different components, addressing issue #508. The changes include adding an errorHost helper in the rule package, updating error returns to include the normalized host, and aligning the worker package's hostOf function with this canonical format. Comprehensive tests were added to verify host normalization, including lowercase conversion and port removal. Feedback recommends consolidating the duplicated host normalization logic into a single exported function to ensure a single source of truth and removing a redundant nil check in the errorHost function.

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/internal/processor/parser/rule/parser_test.go`:
- Around line 348-360: The test TestParser_ParsePage_ErrorHost_LowercasesHost
only asserts rerr.Host and can pass for other error codes; update it to assert
the specific error code (rule.ErrEmptySelector) before checking Host by adding
require.Equal(t, rule.ErrEmptySelector, rerr.Code) (or
require.ErrorIs/require.True with errors.Is(rerr, rule.ErrEmptySelector) if rerr
implements error wrapping) immediately after require.ErrorAs, then keep the Host
assertion; apply the same change to the sibling test covering lines 362-374 (the
other ParsePage error-host normalization test) so both pin the expected error
code.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b3b0f47e-06bd-40c2-8ef9-96e287054098

📥 Commits

Reviewing files that changed from the base of the PR and between 9e0fab5 and 3407a34.

📒 Files selected for processing (5)

• internal/processor/parser/rule/discovery.go
• internal/processor/parser/rule/extract.go
• internal/processor/parser/rule/parser.go
• internal/processor/parser/worker/worker.go
• test/internal/processor/parser/rule/parser_test.go