[FEAT#63] neuron Phase 11 — channel adj 의 scale-corrected init (uniform_around_one)

[juhy0987]

연관 이슈

Closes #63

배경 — Phase 10 의 결정적 발견 확장 검증

Phase 10 (PR #62) 결과:

• channel_full (adj=1) ≈ plain Linear + 약간 우위 (-0.0148)
• channel_uniform_small (adj ∈ [0.05, 0.15], Phase 2 sweet spot 패턴) = +0.18 열위
• 원인: effective_w = adj * W 에서 adj≈0.10 → magnitude 10% 수준 → forward 약화

→ 새 규칙: 0-init 금지 + 적정 magnitude 는 자유도의 의미적 위치에 의존

• residual gate (y = x + α·f(x)): ~0.10
• weight multiplier (y = (adj·W)x): ~1.0 ← 본 PR 의 직접 검증

본 phase = uniform[0.95, 1.05] 로 magnitude balance + 0-init 회피 + adj 학습 활성 동시 달성 가설 검증.

구현 내용

1. src/graphlm/neuron/graph_channel.py (변경)

• AdjInit literal 에 "uniform_around_one" 추가
• 동작: uniform[0.95, 1.05] — 1.0 근처 small noise
• 기존 "uniform_small" 은 anti-pattern (ablation 용) 으로 유지 + docstring 에 magnitude rule 위반 명시
• 모든 옵션의 trade-off docstring 갱신

2. src/graphlm/neuron/graph_channel_demo.py (변경)

• Arch literal + _make_linear 분기 추가

3. tests/neuron/test_graph_channel.py — 2 신규

• test_uniform_around_one_adj_init_range — [0.95, 1.05] + mean ≈ 1.0 검증
• test_uniform_around_one_function_preservation_approximate — effective_w/W magnitude ratio ≈ 1.0 (vs uniform_small ratio ~0.10)

4. notebooks/02-function-level/10-phase11-scale-corrected-init.ipynb

• 4 × 2 sweep: arch ∈ {plain, channel_full, channel_uniform_small, channel_uniform_around_one} × seed ∈ {42, 123}
• Phase 10 baseline (plain 2.1487, channel_full 2.1339, uniform_small 2.3268) 직접 비교
• 자동 verdict 코드 — around_one vs uniform_small (magnitude rule 입증), around_one vs channel_full (scale balance) 정량 판단
• §6 4 arch adj distribution 비교 (mean, std 표시)
• §7 4 arch loss curve mean ± σ

검증할 가설

1. uniform_around_one ≈ channel_full (scale balance + 0-init 회피 동시 작동)
2. uniform_around_one > uniform_small (magnitude rule 직접 입증)
3. 4 arch 의 명확한 ordering — 작은 task 에서도 magnitude 효과 가시?

CI / 머지 게이트 점검

• make fmt 통과
• make lint 통과 (nbqa --fix 1건)
• make test 통과 — 104 passed (기존 102 + 신규 2)

변경 영향 범위 + 위험도

• backbone 변경: adj_init 옵션 추가 + 기존 옵션 동작 무변경 → backward-compatible additive
• 신규 노트북 + 데모 분기 추가
• 위험도: 매우 낮음

메모리 규칙 update 사전 반영

feedback_no_zero_init.md 가 Phase 10 발견 (magnitude rule) 으로 업데이트됨 (본 PR 의 motivation).

• 새 자유도 도입 시 자동 검증: "이 자유도가 수식 어디에 있는가? residual gate / weight multiplier / scale"

Phase 12+ 계획 유지

• Phase 12: channel + group hybrid foundations
• Phase 13: Transformer 통합
• Phase 14: scale-up 정량 실험

Summary by CodeRabbit

• New Features

  • Added "uniform_around_one" initialization mode for channel graph linear layers, providing near-1.0 noise-based weight initialization.

• Documentation

  • Added Phase 11 experiment notebook comparing multiple neural network architectures on TinyShakespeare dataset with baseline comparisons.

• Tests

  • Added unit tests validating the new initialization mode produces expected adjacency distributions and preserves magnitude properties.

[coderabbitai]

Important

Review skipped

Review was skipped due to path filters

⛔ Files ignored due to path filters (2)

• docs/figures/neuron/phase11/adj_dist_compare.png is excluded by !**/*.png
• docs/figures/neuron/phase11/loss_curves.png is excluded by !**/*.png

CodeRabbit blocks several paths by default. You can override this behavior by explicitly including those paths in the path filters. For example, including **/dist/** will override the default block on the dist directory, by removing the pattern from both the lists.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b49f5d2f-4823-4b13-8232-b8b44de39641

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

📝 Walkthrough

Walkthrough

This PR implements the Phase 11 feature introducing a new uniform_around_one adjacency initialization mode for ChannelGraphLinear, alongside supporting type updates, demo integration, unit tests, and a comprehensive experiment notebook comparing four architectures on TinyShakespeare across two seeds.

Changes

uniform_around_one initialization and Phase 11 validation

Layer / File(s)	Summary
adj_init type contract and documentation src/graphlm/neuron/graph_channel.py	AdjInit Literal now includes "uniform_around_one", and class/parameter documentation updated to describe the new mode (uniform near 1.0 for scale balance), mark uniform_small as anti-pattern, and strengthen zero-init rejection rationale.
ChannelGraphLinear uniform_around_one initialization src/graphlm/neuron/graph_channel.py	Constructor initialization logic extended with conditional branch to generate adj values using uniform_(0.95, 1.05) for the new mode; error handling updated to reject both "zero" and "zeros".
Demo integration and unit tests src/graphlm/neuron/graph_channel_demo.py, tests/neuron/test_graph_channel.py	Demo module docstring and Arch type literal expanded to include "channel_uniform_around_one"; _make_linear factory extended with new conditional branch. Two unit tests verify adj init range with non-degenerate spread and magnitude-balance ratio around 1.0 versus much smaller ratio for uniform_small.
Phase 11 experiment notebook: sweep and analysis notebooks/02-function-level/10-phase11-scale-corrected-init.ipynb	Comprehensive 4-way × 2-seed training sweep comparing plain, channel_full, channel_uniform_small, and channel_uniform_around_one on TinyShakespeare. Prints per-run and aggregated losses with Phase 10 baseline comparisons and automatic verdict section. Plots adjacency fc1 distribution histograms for channel variants and smoothed mean ± std loss curves with Phase 10 baseline overlay. Concludes with Phase 12 decision framework.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related issues

• #63: The changes directly implement the Phase 11 feature specification: adding adj_init="uniform_around_one" initialization, verifying magnitude balance around 1.0, and running the four-architecture × two-seed experiment sweep to validate the scale-corrected initialization rule and confirm that uniform_small underperformance is driven by magnitude degradation.

Poem

🐰 A uniform dance near the one, so bright,
Adj values scattered [0.95, 1.05] tight!
No zeros allowed—gradients flow free,
Phase 11 sweeps show the symmetry,
Scale-balanced magic, hypothesis met!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly describes the primary change: introducing scale-corrected initialization (uniform_around_one) for channel adj in Phase 11. It is specific and reflects the main objective of the PR.
Linked Issues check	✅ Passed	The PR fully implements the requirements from issue #63: adds uniform_around_one initialization mode to ChannelGraphLinear, includes 2+ tests verifying range and function preservation, adds new architecture variant to the demo, and provides comprehensive 8-run experimental notebook with Phase 10 baseline comparisons and analysis.
Out of Scope Changes check	✅ Passed	All changes are directly aligned with issue #63 scope: graph_channel.py updates, graph_channel_demo.py variant addition, new unit tests, and Phase 11 experimental notebook. No unrelated modifications detected.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feature/#63/neuron-phase11-scale-corrected-init

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[gemini-code-assist]

Code Review

This pull request introduces a new scale-corrected initialization option, uniform_around_one, for ChannelGraphLinear across the codebase, including updates to the demo script, a new Phase 11 Jupyter notebook, and corresponding unit tests. It also documents uniform_small as an anti-pattern. The reviewer suggested updating the ValueError message for zero-initialization to recommend uniform_around_one instead of uniform_small to maintain consistency with these changes.

[Copilot]

Pull request overview

본 PR은 Phase 10에서 확인된 “weight multiplier 위치의 adj는 magnitude가 ~1.0이어야 한다”는 관찰을 검증하기 위해, ChannelGraphLinear의 adj_init에 scale-corrected 초기화(uniform_around_one, U[0.95, 1.05])를 추가합니다. 이를 통해 uniform_small(≈0.1 스케일)로 인해 effective_w = adj * W의 magnitude가 과도하게 줄어드는 문제를 완화/검증하는 흐름입니다.

Changes:

• adj_init="uniform_around_one" 옵션 추가 및 초기화 로직 구현 (U[0.95, 1.05])
• 데모 아키텍처 분기 확장 및 신규 테스트 2건 추가(범위/scale 보존 근사)
• Phase 11 실험 노트북 추가(4 arch × 2 seed sweep 및 자동 verdict/시각화)

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
src/graphlm/neuron/graph_channel.py	AdjInit에 uniform_around_one 추가 및 adj 초기화 분기 확장
src/graphlm/neuron/graph_channel_demo.py	데모용 Arch에 channel_uniform_around_one 추가 및 _make_linear 분기 확장
tests/neuron/test_graph_channel.py	uniform_around_one 범위/scale 근사 보존 검증 테스트 2건 추가
notebooks/02-function-level/10-phase11-scale-corrected-init.ipynb	Phase 11 sweep/비교/자동 판정 및 분포·loss curve 시각화 노트북 추가

Comments suppressed due to low confidence (1)

src/graphlm/neuron/graph_channel.py:103

• adj_init in {"zero","zeros"} 에 대한 ValueError 메시지가 현재 'full'/'uniform_small' 만 권장하고 있어, 새로 추가된 (그리고 Phase 11+ 권장으로 문서화된) 'uniform_around_one' 옵션을 사용하는 사용자가 혼란스러울 수 있습니다. 에러 메시지의 권장 옵션/설명을 'uniform_around_one' 까지 포함하도록 갱신하는 편이 일관됩니다.

                f"adj_init={adj_init!r} 는 금지됨 — 0-init 은 vanishing gradient 함정 "
                "(Phase 1 dead block / Phase 7 amplitude vanishing / Phase 9 block-diagonal "
                "에서 3차 재현). rationale: Phase 9 PR #60. "
                "'full' (function preserving) 또는 'uniform_small' (sweet spot 패턴) 사용 권장."
            )