[FEAT#61] neuron Phase 10 — channel-as-node graph hidden layer foundations (A-safe) - #62
Conversation
…ons (A-safe)
신규 모듈 src/graphlm/neuron/graph_channel.py:
- ChannelGraphLinear: paradigm 의 finest unit foundation
- weight (out, in) + adj (out, in) — per-edge learnable gate
- forward: y = (adj * weight) @ x
- adj_init="full" → standard Linear 와 forward 동치 (function preservation)
- adj_init="uniform_small" → uniform[0.05, 0.15] (Phase 2 sweet spot 패턴 channel-level)
- adj_init="zero"/"zeros" → ValueError (feedback_no_zero_init.md 규칙 적용)
- adj_sparsity / sparsify_adj / freeze_adjacency — Phase 11+ sparsification 학습 stub
신규 모듈 src/graphlm/neuron/graph_channel_demo.py:
- ChannelGraphMLPLM — 3 arch (plain / channel_full / channel_uniform_small) 통합
- train_channel_graph_mlp 학습 헬퍼
tests/neuron/test_graph_channel.py — 17 신규 테스트:
- shape, validation (positive int)
- adj_init values (full=1, uniform_small ∈ [0.05, 0.15])
- **0-init 거부 검증** — adj_init="zero" → ValueError with feedback_no_zero_init 메시지
- function preservation 수학적 검증 (adj=full + 같은 W → Linear atol=1e-5)
- gradient flow (weight + adj 양쪽)
- sparsify_adj / adj_sparsity 동작 + edge case
notebooks/02-function-level/09-phase10-channel-graph-foundations.ipynb:
- 3 × 2 sweep: arch ∈ {plain, channel_full, channel_uniform_small} × seed
- Phase 9 baseline (plain 2.1378, group_full 2.1391, group_identity 2.2797) 와 비교
- §6 학습된 adj 분포 (implicit pruning at edge-level 검증)
- §7 adj heatmap (256 × 256 channel × channel)
102/102 tests pass (기존 85 + 신규 17).
Phase 9 의 "0-init 금지" 규칙 첫 적용 — 새 학습 가능 자유도 (adj) 의
default 가 sweet spot 패턴 또는 function-preserving (1) 둘 중 하나.
|
Warning Review limit reached
More reviews will be available in 20 minutes and 3 seconds. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (3)
📒 Files selected for processing (6)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Code Review
This pull request introduces Phase 10 channel-as-node graph hidden layer foundations, adding the ChannelGraphLinear module, an MLP-LM demo, unit tests, and an experimental Jupyter notebook. A critical PyTorch runtime issue was identified in the sparsify_adj method, where performing an in-place masked_fill_ on a leaf parameter that requires gradients will raise an error; using copy_ is recommended to resolve this.
There was a problem hiding this comment.
Pull request overview
Phase 10 “channel-as-node” 그래프 히든 레이어 기초를 위해, nn.Linear의 각 weight entry를 edge로 재해석하고 per-edge 게이트(adj)를 학습하는 ChannelGraphLinear를 도입합니다. Phase 9에서 관측된 0-init vanishing 패턴을 재현하지 않도록 adj_init="zero"/"zeros"를 명시적으로 거부하는 설계가 포함됩니다.
Changes:
ChannelGraphLinear추가:effective_w = adj ⊙ weight기반 forward 및adj_sparsity/sparsify_adj/freeze_adjacency유틸 제공- 데모용 MLP-LM 학습 헬퍼(
ChannelGraphMLPLM,train_channel_graph_mlp) 추가 - 신규 pytest 17개로 init/검증/함수보존/gradient/sparsify/freeze 동작 검증 + Phase 10 노트북 추가
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/graphlm/neuron/graph_channel.py | ChannelGraphLinear 핵심 구현 및 adj 관련 유틸 추가 |
| src/graphlm/neuron/graph_channel_demo.py | 노트북에서 import해 실험 스윕을 수행하는 데모/학습 헬퍼 추가 |
| tests/neuron/test_graph_channel.py | ChannelGraphLinear 기능/안전성/회귀를 검증하는 테스트 17개 추가 |
| src/graphlm/neuron/init.py | ChannelGraphLinear 공개 API로 export |
| notebooks/02-function-level/09-phase10-channel-graph-foundations.ipynb | Phase 10 실험 스윕/시각화 노트북 추가 |
graph_channel.py: - import torch.nn.functional as F (Copilot #3301736141) — # noqa: N812 제거 + 다른 demo 와 import 스타일 통일 - 0-init 거부 에러 메시지에서 'feedback_no_zero_init.md' 제거 → repo-resolvable 'Phase 9 PR #60' 로 교체 (Copilot #3301736167) - module docstring 의 link 도 동일 정리 - sparsify_adj 에서 self.adj.data.masked_fill_ 명시 — leaf+requires_grad in-place 안전성 (gemini #3301728271) graph_group.py: - sparsify_adjacency 도 동일 .data 패턴으로 (consistency) notebooks/02-function-level/09-phase10-...ipynb: - 헤더의 broken link → Phase 9 PR #60 GitHub URL 로 (Copilot #3301736194) tests/neuron/test_graph_channel.py: - 메시지 변경에 따라 match='vanishing gradient' 로 갱신 102/102 tests pass.
- docs/figures/neuron/phase10/adj_distribution.png — 학습된 channel adj 분포 histogram - docs/figures/neuron/phase10/adj_heatmaps.png — channel × channel adj heatmap - docs/figures/neuron/phase10/loss_curves.png — 3 arch loss curve (mean ± σ) 새 규약: figure commit 은 해당 phase 의 feature branch 에서 진행 (feedback_image_commit_branch.md). Phase 6~9 의 main 직접 commit 패턴 polished.
- graph_channel.py 모듈 docstring 의 권장 옵션 list 갱신: 'uniform_small' → 'uniform_around_one' 권장으로 (Copilot #3302006547) uniform_small 은 anti-pattern 명시 - graph_channel.py ValueError 메시지에서 'uniform_small' 권장 → 'uniform_around_one' 으로 (gemini #3301992790) + Phase 10 PR #62 (magnitude rule) 참조 추가 - graph_channel_demo.py 모듈 docstring 의 '3 가지 architecture' → '4 가지' + uniform_around_one 설명 추가 (Copilot #3302006608) 104/104 tests pass.
연관 이슈
Closes #61
배경
Phase 9 (PR #60) 의 결정적 발견:
group_full≈ plain Linear (graph 구조 free)group_identity(block-diagonal sparse 시작) = +0.14 loss 열위 — 0-init vanishing 패턴 graph paradigm 에서도 재현0-init 금지규칙이를 바탕으로 사용자 vision (히든 레이어 = graph) 의 본질 axis = A (channel-as-node) 진입. 단순 0-init sparse 진입 = Phase 9 의 함정 재현 → A-safe (small random adj init).
구현 내용
1.
src/graphlm/neuron/graph_channel.py(신규)ChannelGraphLinear(in_features, out_features, adj_init)— paradigm 의 finest unit foundation:(out, in)+ adj(out, in)— per-edge learnable gatey = (adj * weight) @ x + biasadj_init="full"→ 모두 1 (standard Linear forward 동치, function preservation)adj_init="uniform_small"→ uniform[0.05, 0.15] (Phase 2 sweet spot 패턴 채널-edge 에 적용)adj_init="zero"/"zeros"→ ValueError with feedback_no_zero_init 메시지 (메모리 규칙 첫 적용)adj_sparsity,sparsify_adj,freeze_adjacency— Phase 11+ stub2.
src/graphlm/neuron/graph_channel_demo.py(신규)ChannelGraphMLPLM— 3 arch 비교용 MLP-LMtrain_channel_graph_mlp— sweep 학습 unit3.
tests/neuron/test_graph_channel.py— 17 신규 테스트feedback_no_zero_init포함 검증4.
notebooks/02-function-level/09-phase10-channel-graph-foundations.ipynb검증할 가설
CI / 머지 게이트 점검
make fmt통과make lint통과 (nbqa --fix 1건 자동 해소)make test통과 — 102 passed (기존 85 + 신규 17)변경 영향 범위 + 위험도
메모리 규칙 첫 적용
feedback_no_zero_init.md규칙이ChannelGraphLinear에 코드 레벨로 반영:adj_init="zero"옵션 자체가 ValueError + 메시지에 메모리 파일 명시Phase 11+ 계획