[REFAC#204] internal/parser → internal/processor/parser 이동 (메타 이슈 #195 완료)

[juhy0987]

연관 이슈

• Closes [REFAC] internal/parser → internal/processor/parser 이동 (이슈 #195 단계 4, 메타 이슈 close) #204
• Closes [REFAC] crawler → processor 정렬 — 파이프라인 단계별 디렉토리 재배치 (메타 이슈) #195

구현 내용

메타 이슈 #195 의 단계 4 (마지막) — internal/parser/ 를 internal/processor/parser/ 로 이동하여 정렬 원칙 완성. 본 PR 머지 시 모든 분리된 worker 프로세스 (fetcher / parser / validate) 가 internal/processor/<stage>/ 하위로 일관 정렬되어 메타 이슈 #195 도 함께 close.

코드 변경 (commit 1)

디렉토리 이동:

• internal/parser/ → internal/processor/parser/
  • parser.go (도메인 중립 인터페이스)
  • rule/{,llmgen,pathinfer,refiner} (DB-driven engine + LLM-driven 자동화)
  • worker/ (Claim Check ParserWorker + RawContentCleaner)

Import 갱신: 16개 .go 파일 (issuetracker/internal/parser → issuetracker/internal/processor/parser) sed 일괄 치환

Doc 참조 정리: internal/processor/fetcher/domain/general/types.go 와 internal/locks/processing_lock.go 의 path-mention comment 갱신

SQL comment: migrations/up/007_seed_parsing_rules.sql 갱신

테스트 mirror: test/internal/parser/ → test/internal/processor/parser/ (12개 test 파일)

문서 변경 (commit 2)

• docs/architecture/internal/parser/ → docs/architecture/internal/processor/parser/ (디렉토리 이동)
• 이동된 docs 의 sibling-link 깊이 +1 보정 (../locks/ → ../../locks/, 등)
• Inverse cross-link 갱신:
  • internal/processor/fetcher/{domain,core,README}.md 의 ../../parser/ → ../parser/ (sibling 관계로 단순화)
  • internal/storage/service.md / internal/locks/README.md 의 ../parser/ → ../processor/parser/
  • publisher.md / internal/README.md 의 (parser/README.md) → (processor/parser/README.md)
• docs/architecture/README.md 디렉토리 트리 — parser 를 processor 하위로 이동
• .claude/rules/01-architecture.md 디렉토리 트리 동일 정렬 + test mirror 갱신
• CLAUDE.md cmd/ 줄 — 실제 바이너리 목록 반영 ([REFAC] crawler 잔여 fetcher 영역 → processor/fetcher 이동 (이슈 #195 단계 3, 메타 이슈 close) #198 PR 에서 일부 누락된 부분 함께 commit)

변경 후 최종 구조

internal/
├── locks/                  ← 단계 무관 distributed lock
│   ├── ingestion_lock.go
│   └── processing_lock.go
├── processor/              ← 모든 파이프라인 stage 가 본 디렉토리 하위 (이슈 #195 정렬 완성)
│   ├── fetcher/            (이슈 #198)
│   │   ├── core/
│   │   ├── handler/
│   │   ├── implementation/{chromedp,goquery}
│   │   ├── domain/general/{fetcher,sources}
│   │   ├── rate_limiter/
│   │   └── worker/         (PoolManager + RetryScheduler)
│   ├── parser/             (이슈 #196 통합 + #204 이동)
│   │   ├── parser.go       (도메인 중립 인터페이스)
│   │   ├── rule/{,llmgen,pathinfer,refiner}
│   │   └── worker/         (ParserWorker + RawContentCleaner)
│   └── validate/           (기존)
├── publisher/              ← chained CrawlJob sync helper (worker 아님 — 정렬 제외)
├── scheduler/              ← seed CrawlJob producer (worker 아님 — 정렬 제외)
├── classifier/             ← 분류 클라이언트 (worker 아님)
└── storage/                ← repositories

internal/parser/ 가 더 이상 top-level 에 없음. 모든 분리 worker 프로세스 가 processor/<stage>/ 정렬 완료.

CI / 머지 게이트 점검

CI 운영 규약 및 Required Status Checks 단일 소스에 따라 작성합니다.

변경 영향 범위

• 영향 패키지/모듈: internal/parser/* (전체 제거), internal/processor/parser/* (전체 신규), cmd/issuetracker/main.go (import), internal/processor/fetcher/domain/general/types.go (path comment), internal/locks/processing_lock.go (path comment), 다수 docs
• 위험도(택1): Low-Medium
  • import 경로만 변경 — 동작 / 패키지명 / DB 스키마 / Kafka topic 변경 없음
  • 단계 3 ([REFAC] crawler 잔여 fetcher 영역 → processor/fetcher 이동 (이슈 #195 단계 3, 메타 이슈 close) #198) 와 동일 패턴 (path prefix 교체) — 검증된 작업 흐름
  • git mv 로 git history 보존
  • go build ./... && go test -race ./... 28 패키지 모두 통과 (failure 0)

Required Status Checks

• 통과 확인 대상 (PR Checks 탭에서 확인):
  • Commit Lint
  • PR Title Lint
  • Linked Issue Check
  • Format Check
  • Build
  • Test
  • Lint

롤백 계획

• git revert 2개 commit (코드 / 문서). DB / Redis / Kafka 영향 없음.
• 본 PR 머지 후 메타 이슈 [REFAC] crawler → processor 정렬 — 파이프라인 단계별 디렉토리 재배치 (메타 이슈) #195 도 함께 close — 후속 sub-issue 없음.

TODO

• (없음 — 본 PR 이 메타 이슈 [REFAC] crawler → processor 정렬 — 파이프라인 단계별 디렉토리 재배치 (메타 이슈) #195 의 마지막 단계)
• (선택) 추가 정리 (예: parser/rule/ flatten, domain/general/ rename) 는 별도 메타 이슈로 분리

논의 사항

• parser/rule/ 의 llmgen / pathinfer / refiner 가 함께 processor/parser/rule/ 하위로 이동 — LLM-driven 파싱 룰 자동화 인프라가 모두 단일 패키지 경계 안에 위치.
• internal/scheduler/ (seed producer) 와 internal/publisher/ (chained sync helper) 는 worker 가 아니므로 정렬 대상에서 제외 — 사용자 확인 후 명시 (이슈 본문의 "분리된 워커" 정의 = Kafka consumer + worker pool).

🤖 Generated with Claude Code

Summary by CodeRabbit

• Refactor

  • Reorganized internal module structure to improve code organization and maintainability.

• Documentation

  • Updated architecture documentation to reflect internal structural changes.

[coderabbitai]

Warning

Rate limit exceeded

@juhy0987 has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 29 minutes and 4 seconds before requesting another review.

To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 6abba2cb-7494-45d1-8ec4-f878ed766449

📥 Commits

Reviewing files that changed from the base of the PR and between 24e5a0b and 95abf68.

📒 Files selected for processing (4)

• docs/architecture/README.md
• docs/architecture/internal/processor/parser/README.md
• docs/architecture/internal/processor/parser/rule.md
• docs/architecture/pkg/queue.md

📝 Walkthrough

Walkthrough

The pull request relocates the parser implementation from internal/parser/ to internal/processor/parser/, consolidating all pipeline stage components under a unified internal/processor/ directory structure. Import paths are updated across source files, tests, and documentation to reference the new location. No functional logic, APIs, or data structures are modified.

Changes

Cohort / File(s)	Summary
Architecture Documentation .claude/rules/01-architecture.md, docs/architecture/README.md, docs/architecture/cmd/issuetracker.md, docs/architecture/internal/README.md, docs/architecture/internal/locks/README.md, docs/architecture/internal/processor/fetcher/*.md, docs/architecture/internal/processor/parser/*.md, docs/architecture/internal/publisher.md, docs/architecture/internal/storage/service.md, docs/architecture/pkg/*.md	Update cross-references and links from internal/parser/ to internal/processor/parser/ throughout architecture documentation; adjust directory tree listings and relative paths.
Go Source Files (Cmd & Core) cmd/issuetracker/main.go	Update parser package imports from issuetracker/internal/parser to issuetracker/internal/processor/parser in entry point wiring.
Go Source Files (Processor Parser) internal/processor/parser/rule/discovery.go, internal/processor/parser/rule/llmgen/generator.go, internal/processor/parser/rule/parser.go, internal/processor/parser/rule/refiner/llm_adapter.go, internal/processor/parser/rule/refiner/refiner.go, internal/processor/parser/worker/parser_worker.go	Update internal imports within parser module to reference new internal/processor/parser package hierarchy instead of internal/parser.
Go Source Files (Processor Fetcher) internal/processor/fetcher/domain/general/convert.go, internal/processor/fetcher/domain/general/types.go	Update parser package references in fetcher domain from issuetracker/internal/parser to issuetracker/internal/processor/parser.
Go Source Files (Core Utilities) internal/locks/processing_lock.go, CLAUDE.md	Update documentation and comments referencing parser worker location from internal/parser/worker to internal/processor/parser/worker; update cmd directory summary.
Test Files test/internal/processor/parser/rule/discovery_test.go, test/internal/processor/parser/rule/llmgen/generator_test.go, test/internal/processor/parser/rule/parser_test.go, test/internal/processor/parser/rule/pathinfer/*.md, test/internal/processor/parser/rule/refiner/refiner_test.go, test/internal/processor/parser/rule/resolver_test.go, test/internal/processor/parser/worker/cleanup_test.go	Update test imports from internal/parser/* to internal/processor/parser/* to match relocated implementation packages.
Database Migration migrations/up/007_seed_parsing_rules.sql	Update comment reference to parser entry point from internal/parser/rule.Parser to internal/processor/parser/rule.Parser.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~20 minutes

Possibly related issues

• [REFAC] crawler/parser → parser/rule 통합 (이슈 #195 단계 1) #196: Unified parser implementations (crawler/parser + internal/parser) into a single internal/parser module—directly related prior refactoring stage.

Possibly related PRs

• [REFAC#196] crawler/parser → parser/rule 디렉토리 통합 (메타 이슈 #195 단계 1) #201: Prior parser unification PR that merged internal/crawler/parser and internal/parser codebases—directly relevant to the same parser subsystem being relocated.
• [REFAC#197] crawler/worker 의 lock 인프라 → internal/locks 분리 (메타 이슈 #195 단계 2) #202: Extracted stage-agnostic lock infrastructure (IngestionLock, ProcessingLock) into internal/locks and updated parser worker dependencies—refactors the same parser worker wiring touched by this relocation.
• [REFAC#198] crawler/* → processor/fetcher/* 이동, 메타 이슈 #195 완료 #203: Stage 3 of the meta-refactor moving internal/crawler/{core,handler,implementation,domain,rate_limiter,worker} to internal/processor/fetcher/—parallel structural reorganization step.

Suggested labels

refactor

Poem

🐰 From parser's lonely perch, a hop and bound—
Now nested safe where processor stages sound,
Import paths shimmer, tests align with glee,
All pipeline children dwell in harmony. ✨

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and specifically describes the main change: moving internal/parser to internal/processor/parser as the final stage of refactoring issue #195.
Linked Issues check	✅ Passed	The PR fully satisfies both linked issues #204 and #195: all code (16 .go files with import updates, 12 test mirror paths) and documentation changes are properly implemented to move internal/parser to internal/processor/parser.
Out of Scope Changes check	✅ Passed	All changes are directly in scope: directory structure reorganization, import path updates, documentation/comment corrections, and test path adjustments—all aligned with the stated refactoring objectives with no extraneous modifications.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch refactor/#204/parser-to-processor

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Review rate limit: 0/1 reviews remaining, refill in 29 minutes and 4 seconds.}

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[gemini-code-assist]

Code Review

This pull request performs a major architectural reorganization by moving the internal/parser package to internal/processor/parser, ensuring all pipeline stages are grouped under the processor directory. The changes encompass widespread updates to import paths, documentation links, and test structures. Review feedback identifies several documentation inconsistencies where link labels were not updated to match the new paths and notes a misalignment in an ASCII architecture diagram caused by the longer directory names.

[coderabbitai]

Actionable comments posted: 4

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

docs/architecture/README.md (1)

3-5: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Fix minor wording inconsistency (“두 layer” → “두 레이어”).

Static analysis flags “두 layer 로” in this doc. For consistency with the rest of the Korean architecture writing, consider changing it to “두 레이어로”.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@docs/architecture/README.md` around lines 3 - 5, Replace the inconsistent
phrase "두 layer" with the Koreanized form "두 레이어" in the README content; search
for the exact token "두 layer" in the document and update it to "두 레이어" so the
wording matches the rest of the architecture docs and static analysis warnings
are resolved.

🧹 Nitpick comments (2)

docs/architecture/internal/locks/README.md (1)

56-62: 💤 Low value

Doc link granularity check for “parser/worker” row

The “호출 측” table updates to internal/processor/parser/worker, and the description matches ProcessingLock(StageParser). citeturn0relevant_code_snippet1turn0relevant_code_snippet2
One small consistency check: the row text includes /worker, but the href is ../processor/parser/README.md (not a worker-specific page). If a dedicated worker doc exists, consider linking directly to it to avoid confusion.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@docs/architecture/internal/locks/README.md` around lines 56 - 62, The parser
row currently labels the caller as "internal/processor/parser/worker" but links
to ../processor/parser/README.md; update the doc so the link and label are
consistent: either change the link to point to the worker-specific page (if one
exists, i.e., the worker doc for internal/processor/parser/worker) or change the
label to "internal/processor/parser" to match the README; ensure the description
still references ProcessingLock(StageParser) so readers can find the lock
behavior.

docs/architecture/internal/processor/fetcher/README.md (1)

53-56: ⚡ Quick win

Minor doc grammar/wording polish

Linking to internal/processor/parser/README.md for “Parser engine … 별도 패키지 …” is correct per the relocation, but the wording “Parser engine 은” looks like it may trigger the LanguageTool orthography hint. Consider adjusting phrasing to the project’s doc style (e.g., “Parser 엔진은” / “Parser 엔진은 …”) to reduce lint noise.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@docs/architecture/internal/processor/fetcher/README.md` around lines 53 - 56,
The phrase "Parser engine 은" in README.md should be changed to the project's
preferred wording to avoid LanguageTool orthography hints; update the sentence
that currently starts with "Parser engine 은 별도 패키지
(`internal/processor/parser/`) 로 분리됨" to use "Parser 엔진은" (or "Parser 엔진은 별도 패키지
(`internal/processor/parser/`)로 분리됨") and ensure spacing/punctuation follows
existing doc style (e.g., remove extra space before "로" if present) so the link
to ../parser/README.md remains intact.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@docs/architecture/internal/processor/parser/README.md`:
- Line 66: The relative link target in the README entry for
`internal/processor/fetcher/domain/general` is incorrect (it points to
`../processor/fetcher/domain.md`); update that link target to the correct
relative path (e.g., `../fetcher/domain.md`) so the link for the
`ConvertPageToContent` entry resolves to the actual `fetcher/domain.md`
document.
- Line 3: The README contains several intra-repo links using four-level relative
paths like ../../../../internal/... which incorrectly resolve; update every
occurrence in docs/architecture/internal/processor/parser/README.md (e.g., the
links shown around the current first instance and the other occurrences at lines
~20-22 and ~57-58) to use five-level upward segments ../../../../../internal/...
so they point to the repo-root internal/ paths consistently.

In `@docs/architecture/internal/processor/parser/rule.md`:
- Around line 5-6: Update broken relative code links in
docs/architecture/internal/processor/parser/rule.md by changing all occurrences
of "../../../../internal/..." to "../../../../../internal/..." so they correctly
point from this deeply nested docs path; specifically update links referencing
internal/processor/parser/parser.go, the internal/processor/parser/rule/
directory, and the Resolver.Invalidate anchor (and any other
"../../../../internal/..." occurrences on lines near 11-12, 40-44, 64-73) to use
the extra "../" segment; ensure every markdown link and reference in the file is
updated consistently.
- Around line 127-132: The two link targets in the “호출 측” bullets incorrectly
point to the same rule.md (self) — update the links so they point to the
intended worker docs/implementation for ParserWorker and issuetracker refiner;
specifically change the links for the bullets that mention
internal/processor/parser/worker.ParserWorker (the lines referencing
ParserWorker, ParsePage, ParseLinks, and ErrNoRule / llmGen.Enqueue) to the
correct worker README or specific worker doc (e.g., ../worker/README.md or the
worker's doc page) and change the cmd/issuetracker.buildRefiner bullet to the
correct issuetracker doc page so readers land on the actual implementation docs
rather than rule.md.

---

Outside diff comments:
In `@docs/architecture/README.md`:
- Around line 3-5: Replace the inconsistent phrase "두 layer" with the Koreanized
form "두 레이어" in the README content; search for the exact token "두 layer" in the
document and update it to "두 레이어" so the wording matches the rest of the
architecture docs and static analysis warnings are resolved.

---

Nitpick comments:
In `@docs/architecture/internal/locks/README.md`:
- Around line 56-62: The parser row currently labels the caller as
"internal/processor/parser/worker" but links to ../processor/parser/README.md;
update the doc so the link and label are consistent: either change the link to
point to the worker-specific page (if one exists, i.e., the worker doc for
internal/processor/parser/worker) or change the label to
"internal/processor/parser" to match the README; ensure the description still
references ProcessingLock(StageParser) so readers can find the lock behavior.

In `@docs/architecture/internal/processor/fetcher/README.md`:
- Around line 53-56: The phrase "Parser engine 은" in README.md should be changed
to the project's preferred wording to avoid LanguageTool orthography hints;
update the sentence that currently starts with "Parser engine 은 별도 패키지
(`internal/processor/parser/`) 로 분리됨" to use "Parser 엔진은" (or "Parser 엔진은 별도 패키지
(`internal/processor/parser/`)로 분리됨") and ensure spacing/punctuation follows
existing doc style (e.g., remove extra space before "로" if present) so the link
to ../parser/README.md remains intact.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 39be66e3-bcc6-4c32-8beb-34862cd8ff8e

📥 Commits

Reviewing files that changed from the base of the PR and between 66e9da5 and 24e5a0b.

📒 Files selected for processing (46)

• .claude/rules/01-architecture.md
• CLAUDE.md
• cmd/issuetracker/main.go
• docs/architecture/README.md
• docs/architecture/cmd/issuetracker.md
• docs/architecture/internal/README.md
• docs/architecture/internal/locks/README.md
• docs/architecture/internal/processor/fetcher/README.md
• docs/architecture/internal/processor/fetcher/core.md
• docs/architecture/internal/processor/fetcher/domain.md
• docs/architecture/internal/processor/parser/README.md
• docs/architecture/internal/processor/parser/rule.md
• docs/architecture/internal/publisher.md
• docs/architecture/internal/storage/service.md
• docs/architecture/pkg/config.md
• docs/architecture/pkg/links.md
• docs/architecture/pkg/llm.md
• docs/architecture/pkg/metrics.md
• docs/architecture/pkg/queue.md
• internal/locks/processing_lock.go
• internal/processor/fetcher/domain/general/convert.go
• internal/processor/fetcher/domain/general/types.go
• internal/processor/parser/parser.go
• internal/processor/parser/rule/discovery.go
• internal/processor/parser/rule/errors.go
• internal/processor/parser/rule/llmgen/dedup.go
• internal/processor/parser/rule/llmgen/generator.go
• internal/processor/parser/rule/llmgen/prompt.go
• internal/processor/parser/rule/parser.go
• internal/processor/parser/rule/pathinfer/llm.go
• internal/processor/parser/rule/pathinfer/pathinfer.go
• internal/processor/parser/rule/refiner/llm_adapter.go
• internal/processor/parser/rule/refiner/metrics.go
• internal/processor/parser/rule/refiner/refiner.go
• internal/processor/parser/rule/resolver.go
• internal/processor/parser/worker/cleanup.go
• internal/processor/parser/worker/parser_worker.go
• migrations/up/007_seed_parsing_rules.sql
• test/internal/processor/parser/rule/discovery_test.go
• test/internal/processor/parser/rule/llmgen/generator_test.go
• test/internal/processor/parser/rule/parser_test.go
• test/internal/processor/parser/rule/pathinfer/llm_test.go
• test/internal/processor/parser/rule/pathinfer/pathinfer_test.go
• test/internal/processor/parser/rule/refiner/refiner_test.go
• test/internal/processor/parser/rule/resolver_test.go
• test/internal/processor/parser/worker/cleanup_test.go

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.