fix: use .get() in get_leaf_nodes to avoid KeyError on leaf nodes by ChiragB254 · Pull Request #331 · VectifyAI/PageIndex

ChiragB254 · 2026-06-18T02:25:29Z

Summary

get_leaf_nodes() crashed with KeyError: 'nodes' when called on a tree built by the standard pipeline
list_to_tree() calls clean_node() which deletes the nodes key entirely from leaf nodes (rather than setting it to [])
Accessing structure['nodes'] directly on those nodes raises KeyError
Fix: replace structure['nodes'] with structure.get('nodes') — returns None (falsy) safely when the key is absent

Changes

pageindex/utils.py — one-line change in get_leaf_nodes:

# Beforeifnotstructure['nodes']: # KeyError on leaf nodes# Afterifnotstructure.get('nodes'): # safe: returns None if key absent

Why this is consistent

Every other nodes access in the codebase already uses .get():

format_structure → structure.get('nodes')
is_leaf_node → node.get('nodes')
print_tree → node.get('nodes')

Test

frompageindex.utilsimportget_leaf_nodes, list_to_treeflat= [
{'structure': '1', 'title': 'Chapter 1', 'start_index': 1, 'end_index': 5},
{'structure': '2', 'title': 'Chapter 2', 'start_index': 6, 'end_index': 10},
]
tree=list_to_tree(flat)
leaves=get_leaf_nodes(tree) # previously raised KeyError, now returns both nodesassertlen(leaves) ==2

Fixes#330

list_to_tree() deletes the 'nodes' key from leaf nodes entirely via clean_node(). Direct access via structure['nodes'] raises KeyError on these nodes. Using structure.get('nodes') returns None (falsy) safely, consistent with how 'nodes' is accessed elsewhere in the codebase. FixesVectifyAI#330

KylinMountain

Thanks @ChiragB254, nice catch — consistent with the rest of the codebase. Merging! 🙏

…ctifyAI#331) list_to_tree() deletes the 'nodes' key from leaf nodes entirely via clean_node(). Direct access via structure['nodes'] raises KeyError on these nodes. Using structure.get('nodes') returns None (falsy) safely, consistent with how 'nodes' is accessed elsewhere in the codebase. FixesVectifyAI#330

…on shims The new SDK copied the legacy indexing pipeline into pageindex/index/ instead of moving it, leaving two divergent copies of page_index.py / page_index_md.py / utils.py. They had already drifted (the legacy copy still compared IndexConfig booleans against 'yes' — a separate fix), and every pipeline change had to be applied twice. Make pageindex/index/ the single source of truth (same pattern as the LegacyCloudAPI shim for the 0.2.x cloud SDK): - pageindex/index/utils.py absorbs the 27 legacy-only helpers/classes (get_page_tokens, convert_page_to_int, ConfigLoader, PDF text helpers, ...) so it's the sole utils module. Reconciled the diverged funcs: kept the modern versions, backported the #331 get_leaf_nodes .get() fix, and restored remove_fields' max_len parameter (superset). - index/page_index*.py now import `from .utils import *`; index/legacy_utils.py (a re-export of the old top-level utils) deleted. - Top-level page_index.py / page_index_md.py / utils.py become thin re-export shims that emit PendingDeprecationWarning. The md_to_tree shim coerces legacy 'yes'/'no' string flags to bool (the canonical version is boolean-typed). - ConfigLoader no longer reads the deleted config.yaml; it builds defaults from IndexConfig (was an unconditional FileNotFoundError). - __init__.py and retrieve.py import from pageindex.index.* directly so `import pageindex` does not trip the shims. Adds tests/test_legacy_shims.py pinning the contract: clean top-level import doesn't warn, legacy submodule imports warn, symbols still resolve, shim and canonical share one implementation, the #331 fix and ConfigLoader-without-yaml both hold, and the md_to_tree coercion works. Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS

ChiragB254 · 2026-06-18T02:25:29Z

Summary

get_leaf_nodes() crashed with KeyError: 'nodes' when called on a tree built by the standard pipeline
list_to_tree() calls clean_node() which deletes the nodes key entirely from leaf nodes (rather than setting it to [])
Accessing structure['nodes'] directly on those nodes raises KeyError
Fix: replace structure['nodes'] with structure.get('nodes') — returns None (falsy) safely when the key is absent

Changes

pageindex/utils.py — one-line change in get_leaf_nodes:

# Beforeifnotstructure['nodes']: # KeyError on leaf nodes# Afterifnotstructure.get('nodes'): # safe: returns None if key absent

Why this is consistent

Every other nodes access in the codebase already uses .get():

format_structure → structure.get('nodes')
is_leaf_node → node.get('nodes')
print_tree → node.get('nodes')

Test

frompageindex.utilsimportget_leaf_nodes, list_to_treeflat= [
{'structure': '1', 'title': 'Chapter 1', 'start_index': 1, 'end_index': 5},
{'structure': '2', 'title': 'Chapter 2', 'start_index': 6, 'end_index': 10},
]
tree=list_to_tree(flat)
leaves=get_leaf_nodes(tree) # previously raised KeyError, now returns both nodesassertlen(leaves) ==2

Fixes#330

list_to_tree() deletes the 'nodes' key from leaf nodes entirely via clean_node(). Direct access via structure['nodes'] raises KeyError on these nodes. Using structure.get('nodes') returns None (falsy) safely, consistent with how 'nodes' is accessed elsewhere in the codebase. FixesVectifyAI#330

KylinMountain

Thanks @ChiragB254, nice catch — consistent with the rest of the codebase. Merging! 🙏

…ctifyAI#331) list_to_tree() deletes the 'nodes' key from leaf nodes entirely via clean_node(). Direct access via structure['nodes'] raises KeyError on these nodes. Using structure.get('nodes') returns None (falsy) safely, consistent with how 'nodes' is accessed elsewhere in the codebase. FixesVectifyAI#330

…on shims The new SDK copied the legacy indexing pipeline into pageindex/index/ instead of moving it, leaving two divergent copies of page_index.py / page_index_md.py / utils.py. They had already drifted (the legacy copy still compared IndexConfig booleans against 'yes' — a separate fix), and every pipeline change had to be applied twice. Make pageindex/index/ the single source of truth (same pattern as the LegacyCloudAPI shim for the 0.2.x cloud SDK): - pageindex/index/utils.py absorbs the 27 legacy-only helpers/classes (get_page_tokens, convert_page_to_int, ConfigLoader, PDF text helpers, ...) so it's the sole utils module. Reconciled the diverged funcs: kept the modern versions, backported the #331 get_leaf_nodes .get() fix, and restored remove_fields' max_len parameter (superset). - index/page_index*.py now import `from .utils import *`; index/legacy_utils.py (a re-export of the old top-level utils) deleted. - Top-level page_index.py / page_index_md.py / utils.py become thin re-export shims that emit PendingDeprecationWarning. The md_to_tree shim coerces legacy 'yes'/'no' string flags to bool (the canonical version is boolean-typed). - ConfigLoader no longer reads the deleted config.yaml; it builds defaults from IndexConfig (was an unconditional FileNotFoundError). - __init__.py and retrieve.py import from pageindex.index.* directly so `import pageindex` does not trip the shims. Adds tests/test_legacy_shims.py pinning the contract: clean top-level import doesn't warn, legacy submodule imports warn, symbols still resolve, shim and canonical share one implementation, the #331 fix and ConfigLoader-without-yaml both hold, and the md_to_tree coercion works. Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS

ChiragB254 · 2026-06-18T02:25:29Z

Summary

get_leaf_nodes() crashed with KeyError: 'nodes' when called on a tree built by the standard pipeline
list_to_tree() calls clean_node() which deletes the nodes key entirely from leaf nodes (rather than setting it to [])
Accessing structure['nodes'] directly on those nodes raises KeyError
Fix: replace structure['nodes'] with structure.get('nodes') — returns None (falsy) safely when the key is absent

Changes

pageindex/utils.py — one-line change in get_leaf_nodes:

# Beforeifnotstructure['nodes']: # KeyError on leaf nodes# Afterifnotstructure.get('nodes'): # safe: returns None if key absent

Why this is consistent

Every other nodes access in the codebase already uses .get():

format_structure → structure.get('nodes')
is_leaf_node → node.get('nodes')
print_tree → node.get('nodes')

Test

frompageindex.utilsimportget_leaf_nodes, list_to_treeflat= [
{'structure': '1', 'title': 'Chapter 1', 'start_index': 1, 'end_index': 5},
{'structure': '2', 'title': 'Chapter 2', 'start_index': 6, 'end_index': 10},
]
tree=list_to_tree(flat)
leaves=get_leaf_nodes(tree) # previously raised KeyError, now returns both nodesassertlen(leaves) ==2

Fixes#330

list_to_tree() deletes the 'nodes' key from leaf nodes entirely via clean_node(). Direct access via structure['nodes'] raises KeyError on these nodes. Using structure.get('nodes') returns None (falsy) safely, consistent with how 'nodes' is accessed elsewhere in the codebase. FixesVectifyAI#330

KylinMountain

Thanks @ChiragB254, nice catch — consistent with the rest of the codebase. Merging! 🙏

…ctifyAI#331) list_to_tree() deletes the 'nodes' key from leaf nodes entirely via clean_node(). Direct access via structure['nodes'] raises KeyError on these nodes. Using structure.get('nodes') returns None (falsy) safely, consistent with how 'nodes' is accessed elsewhere in the codebase. FixesVectifyAI#330

…on shims The new SDK copied the legacy indexing pipeline into pageindex/index/ instead of moving it, leaving two divergent copies of page_index.py / page_index_md.py / utils.py. They had already drifted (the legacy copy still compared IndexConfig booleans against 'yes' — a separate fix), and every pipeline change had to be applied twice. Make pageindex/index/ the single source of truth (same pattern as the LegacyCloudAPI shim for the 0.2.x cloud SDK): - pageindex/index/utils.py absorbs the 27 legacy-only helpers/classes (get_page_tokens, convert_page_to_int, ConfigLoader, PDF text helpers, ...) so it's the sole utils module. Reconciled the diverged funcs: kept the modern versions, backported the #331 get_leaf_nodes .get() fix, and restored remove_fields' max_len parameter (superset). - index/page_index*.py now import `from .utils import *`; index/legacy_utils.py (a re-export of the old top-level utils) deleted. - Top-level page_index.py / page_index_md.py / utils.py become thin re-export shims that emit PendingDeprecationWarning. The md_to_tree shim coerces legacy 'yes'/'no' string flags to bool (the canonical version is boolean-typed). - ConfigLoader no longer reads the deleted config.yaml; it builds defaults from IndexConfig (was an unconditional FileNotFoundError). - __init__.py and retrieve.py import from pageindex.index.* directly so `import pageindex` does not trip the shims. Adds tests/test_legacy_shims.py pinning the contract: clean top-level import doesn't warn, legacy submodule imports warn, symbols still resolve, shim and canonical share one implementation, the #331 fix and ConfigLoader-without-yaml both hold, and the md_to_tree coercion works. Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS

ChiragB254 · 2026-06-18T02:25:29Z

Summary

get_leaf_nodes() crashed with KeyError: 'nodes' when called on a tree built by the standard pipeline
list_to_tree() calls clean_node() which deletes the nodes key entirely from leaf nodes (rather than setting it to [])
Accessing structure['nodes'] directly on those nodes raises KeyError
Fix: replace structure['nodes'] with structure.get('nodes') — returns None (falsy) safely when the key is absent

Changes

pageindex/utils.py — one-line change in get_leaf_nodes:

# Beforeifnotstructure['nodes']: # KeyError on leaf nodes# Afterifnotstructure.get('nodes'): # safe: returns None if key absent

Why this is consistent

Every other nodes access in the codebase already uses .get():

format_structure → structure.get('nodes')
is_leaf_node → node.get('nodes')
print_tree → node.get('nodes')

Test

frompageindex.utilsimportget_leaf_nodes, list_to_treeflat= [
{'structure': '1', 'title': 'Chapter 1', 'start_index': 1, 'end_index': 5},
{'structure': '2', 'title': 'Chapter 2', 'start_index': 6, 'end_index': 10},
]
tree=list_to_tree(flat)
leaves=get_leaf_nodes(tree) # previously raised KeyError, now returns both nodesassertlen(leaves) ==2

Fixes#330

list_to_tree() deletes the 'nodes' key from leaf nodes entirely via clean_node(). Direct access via structure['nodes'] raises KeyError on these nodes. Using structure.get('nodes') returns None (falsy) safely, consistent with how 'nodes' is accessed elsewhere in the codebase. FixesVectifyAI#330

KylinMountain

Thanks @ChiragB254, nice catch — consistent with the rest of the codebase. Merging! 🙏

…ctifyAI#331) list_to_tree() deletes the 'nodes' key from leaf nodes entirely via clean_node(). Direct access via structure['nodes'] raises KeyError on these nodes. Using structure.get('nodes') returns None (falsy) safely, consistent with how 'nodes' is accessed elsewhere in the codebase. FixesVectifyAI#330

…on shims The new SDK copied the legacy indexing pipeline into pageindex/index/ instead of moving it, leaving two divergent copies of page_index.py / page_index_md.py / utils.py. They had already drifted (the legacy copy still compared IndexConfig booleans against 'yes' — a separate fix), and every pipeline change had to be applied twice. Make pageindex/index/ the single source of truth (same pattern as the LegacyCloudAPI shim for the 0.2.x cloud SDK): - pageindex/index/utils.py absorbs the 27 legacy-only helpers/classes (get_page_tokens, convert_page_to_int, ConfigLoader, PDF text helpers, ...) so it's the sole utils module. Reconciled the diverged funcs: kept the modern versions, backported the #331 get_leaf_nodes .get() fix, and restored remove_fields' max_len parameter (superset). - index/page_index*.py now import `from .utils import *`; index/legacy_utils.py (a re-export of the old top-level utils) deleted. - Top-level page_index.py / page_index_md.py / utils.py become thin re-export shims that emit PendingDeprecationWarning. The md_to_tree shim coerces legacy 'yes'/'no' string flags to bool (the canonical version is boolean-typed). - ConfigLoader no longer reads the deleted config.yaml; it builds defaults from IndexConfig (was an unconditional FileNotFoundError). - __init__.py and retrieve.py import from pageindex.index.* directly so `import pageindex` does not trip the shims. Adds tests/test_legacy_shims.py pinning the contract: clean top-level import doesn't warn, legacy submodule imports warn, symbols still resolve, shim and canonical share one implementation, the #331 fix and ConfigLoader-without-yaml both hold, and the md_to_tree coercion works. Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS

ChiragB254 · 2026-06-18T02:25:29Z

Summary

get_leaf_nodes() crashed with KeyError: 'nodes' when called on a tree built by the standard pipeline
list_to_tree() calls clean_node() which deletes the nodes key entirely from leaf nodes (rather than setting it to [])
Accessing structure['nodes'] directly on those nodes raises KeyError
Fix: replace structure['nodes'] with structure.get('nodes') — returns None (falsy) safely when the key is absent

Changes

pageindex/utils.py — one-line change in get_leaf_nodes:

# Beforeifnotstructure['nodes']: # KeyError on leaf nodes# Afterifnotstructure.get('nodes'): # safe: returns None if key absent

Why this is consistent

Every other nodes access in the codebase already uses .get():

format_structure → structure.get('nodes')
is_leaf_node → node.get('nodes')
print_tree → node.get('nodes')

Test

frompageindex.utilsimportget_leaf_nodes, list_to_treeflat= [
{'structure': '1', 'title': 'Chapter 1', 'start_index': 1, 'end_index': 5},
{'structure': '2', 'title': 'Chapter 2', 'start_index': 6, 'end_index': 10},
]
tree=list_to_tree(flat)
leaves=get_leaf_nodes(tree) # previously raised KeyError, now returns both nodesassertlen(leaves) ==2

Fixes#330

list_to_tree() deletes the 'nodes' key from leaf nodes entirely via clean_node(). Direct access via structure['nodes'] raises KeyError on these nodes. Using structure.get('nodes') returns None (falsy) safely, consistent with how 'nodes' is accessed elsewhere in the codebase. FixesVectifyAI#330

KylinMountain

Thanks @ChiragB254, nice catch — consistent with the rest of the codebase. Merging! 🙏

…ctifyAI#331) list_to_tree() deletes the 'nodes' key from leaf nodes entirely via clean_node(). Direct access via structure['nodes'] raises KeyError on these nodes. Using structure.get('nodes') returns None (falsy) safely, consistent with how 'nodes' is accessed elsewhere in the codebase. FixesVectifyAI#330

…on shims The new SDK copied the legacy indexing pipeline into pageindex/index/ instead of moving it, leaving two divergent copies of page_index.py / page_index_md.py / utils.py. They had already drifted (the legacy copy still compared IndexConfig booleans against 'yes' — a separate fix), and every pipeline change had to be applied twice. Make pageindex/index/ the single source of truth (same pattern as the LegacyCloudAPI shim for the 0.2.x cloud SDK): - pageindex/index/utils.py absorbs the 27 legacy-only helpers/classes (get_page_tokens, convert_page_to_int, ConfigLoader, PDF text helpers, ...) so it's the sole utils module. Reconciled the diverged funcs: kept the modern versions, backported the #331 get_leaf_nodes .get() fix, and restored remove_fields' max_len parameter (superset). - index/page_index*.py now import `from .utils import *`; index/legacy_utils.py (a re-export of the old top-level utils) deleted. - Top-level page_index.py / page_index_md.py / utils.py become thin re-export shims that emit PendingDeprecationWarning. The md_to_tree shim coerces legacy 'yes'/'no' string flags to bool (the canonical version is boolean-typed). - ConfigLoader no longer reads the deleted config.yaml; it builds defaults from IndexConfig (was an unconditional FileNotFoundError). - __init__.py and retrieve.py import from pageindex.index.* directly so `import pageindex` does not trip the shims. Adds tests/test_legacy_shims.py pinning the contract: clean top-level import doesn't warn, legacy submodule imports warn, symbols still resolve, shim and canonical share one implementation, the #331 fix and ConfigLoader-without-yaml both hold, and the md_to_tree coercion works. Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS

ChiragB254 · 2026-06-18T02:25:29Z

Summary

get_leaf_nodes() crashed with KeyError: 'nodes' when called on a tree built by the standard pipeline
list_to_tree() calls clean_node() which deletes the nodes key entirely from leaf nodes (rather than setting it to [])
Accessing structure['nodes'] directly on those nodes raises KeyError
Fix: replace structure['nodes'] with structure.get('nodes') — returns None (falsy) safely when the key is absent

Changes

pageindex/utils.py — one-line change in get_leaf_nodes:

# Beforeifnotstructure['nodes']: # KeyError on leaf nodes# Afterifnotstructure.get('nodes'): # safe: returns None if key absent

Why this is consistent

Every other nodes access in the codebase already uses .get():

format_structure → structure.get('nodes')
is_leaf_node → node.get('nodes')
print_tree → node.get('nodes')

Test

frompageindex.utilsimportget_leaf_nodes, list_to_treeflat= [
{'structure': '1', 'title': 'Chapter 1', 'start_index': 1, 'end_index': 5},
{'structure': '2', 'title': 'Chapter 2', 'start_index': 6, 'end_index': 10},
]
tree=list_to_tree(flat)
leaves=get_leaf_nodes(tree) # previously raised KeyError, now returns both nodesassertlen(leaves) ==2

Fixes#330

list_to_tree() deletes the 'nodes' key from leaf nodes entirely via clean_node(). Direct access via structure['nodes'] raises KeyError on these nodes. Using structure.get('nodes') returns None (falsy) safely, consistent with how 'nodes' is accessed elsewhere in the codebase. FixesVectifyAI#330

KylinMountain

Thanks @ChiragB254, nice catch — consistent with the rest of the codebase. Merging! 🙏

…ctifyAI#331) list_to_tree() deletes the 'nodes' key from leaf nodes entirely via clean_node(). Direct access via structure['nodes'] raises KeyError on these nodes. Using structure.get('nodes') returns None (falsy) safely, consistent with how 'nodes' is accessed elsewhere in the codebase. FixesVectifyAI#330

…on shims The new SDK copied the legacy indexing pipeline into pageindex/index/ instead of moving it, leaving two divergent copies of page_index.py / page_index_md.py / utils.py. They had already drifted (the legacy copy still compared IndexConfig booleans against 'yes' — a separate fix), and every pipeline change had to be applied twice. Make pageindex/index/ the single source of truth (same pattern as the LegacyCloudAPI shim for the 0.2.x cloud SDK): - pageindex/index/utils.py absorbs the 27 legacy-only helpers/classes (get_page_tokens, convert_page_to_int, ConfigLoader, PDF text helpers, ...) so it's the sole utils module. Reconciled the diverged funcs: kept the modern versions, backported the #331 get_leaf_nodes .get() fix, and restored remove_fields' max_len parameter (superset). - index/page_index*.py now import `from .utils import *`; index/legacy_utils.py (a re-export of the old top-level utils) deleted. - Top-level page_index.py / page_index_md.py / utils.py become thin re-export shims that emit PendingDeprecationWarning. The md_to_tree shim coerces legacy 'yes'/'no' string flags to bool (the canonical version is boolean-typed). - ConfigLoader no longer reads the deleted config.yaml; it builds defaults from IndexConfig (was an unconditional FileNotFoundError). - __init__.py and retrieve.py import from pageindex.index.* directly so `import pageindex` does not trip the shims. Adds tests/test_legacy_shims.py pinning the contract: clean top-level import doesn't warn, legacy submodule imports warn, symbols still resolve, shim and canonical share one implementation, the #331 fix and ConfigLoader-without-yaml both hold, and the md_to_tree coercion works. Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS

ChiragB254 · 2026-06-18T02:25:29Z

Summary

get_leaf_nodes() crashed with KeyError: 'nodes' when called on a tree built by the standard pipeline
list_to_tree() calls clean_node() which deletes the nodes key entirely from leaf nodes (rather than setting it to [])
Accessing structure['nodes'] directly on those nodes raises KeyError
Fix: replace structure['nodes'] with structure.get('nodes') — returns None (falsy) safely when the key is absent

Changes

pageindex/utils.py — one-line change in get_leaf_nodes:

# Beforeifnotstructure['nodes']: # KeyError on leaf nodes# Afterifnotstructure.get('nodes'): # safe: returns None if key absent

Why this is consistent

Every other nodes access in the codebase already uses .get():

format_structure → structure.get('nodes')
is_leaf_node → node.get('nodes')
print_tree → node.get('nodes')

Test

frompageindex.utilsimportget_leaf_nodes, list_to_treeflat= [
{'structure': '1', 'title': 'Chapter 1', 'start_index': 1, 'end_index': 5},
{'structure': '2', 'title': 'Chapter 2', 'start_index': 6, 'end_index': 10},
]
tree=list_to_tree(flat)
leaves=get_leaf_nodes(tree) # previously raised KeyError, now returns both nodesassertlen(leaves) ==2

Fixes#330

list_to_tree() deletes the 'nodes' key from leaf nodes entirely via clean_node(). Direct access via structure['nodes'] raises KeyError on these nodes. Using structure.get('nodes') returns None (falsy) safely, consistent with how 'nodes' is accessed elsewhere in the codebase. FixesVectifyAI#330

KylinMountain

Thanks @ChiragB254, nice catch — consistent with the rest of the codebase. Merging! 🙏

…ctifyAI#331) list_to_tree() deletes the 'nodes' key from leaf nodes entirely via clean_node(). Direct access via structure['nodes'] raises KeyError on these nodes. Using structure.get('nodes') returns None (falsy) safely, consistent with how 'nodes' is accessed elsewhere in the codebase. FixesVectifyAI#330

…on shims The new SDK copied the legacy indexing pipeline into pageindex/index/ instead of moving it, leaving two divergent copies of page_index.py / page_index_md.py / utils.py. They had already drifted (the legacy copy still compared IndexConfig booleans against 'yes' — a separate fix), and every pipeline change had to be applied twice. Make pageindex/index/ the single source of truth (same pattern as the LegacyCloudAPI shim for the 0.2.x cloud SDK): - pageindex/index/utils.py absorbs the 27 legacy-only helpers/classes (get_page_tokens, convert_page_to_int, ConfigLoader, PDF text helpers, ...) so it's the sole utils module. Reconciled the diverged funcs: kept the modern versions, backported the #331 get_leaf_nodes .get() fix, and restored remove_fields' max_len parameter (superset). - index/page_index*.py now import `from .utils import *`; index/legacy_utils.py (a re-export of the old top-level utils) deleted. - Top-level page_index.py / page_index_md.py / utils.py become thin re-export shims that emit PendingDeprecationWarning. The md_to_tree shim coerces legacy 'yes'/'no' string flags to bool (the canonical version is boolean-typed). - ConfigLoader no longer reads the deleted config.yaml; it builds defaults from IndexConfig (was an unconditional FileNotFoundError). - __init__.py and retrieve.py import from pageindex.index.* directly so `import pageindex` does not trip the shims. Adds tests/test_legacy_shims.py pinning the contract: clean top-level import doesn't warn, legacy submodule imports warn, symbols still resolve, shim and canonical share one implementation, the #331 fix and ConfigLoader-without-yaml both hold, and the md_to_tree coercion works. Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS

ChiragB254 · 2026-06-18T02:25:29Z

Summary

get_leaf_nodes() crashed with KeyError: 'nodes' when called on a tree built by the standard pipeline
list_to_tree() calls clean_node() which deletes the nodes key entirely from leaf nodes (rather than setting it to [])
Accessing structure['nodes'] directly on those nodes raises KeyError
Fix: replace structure['nodes'] with structure.get('nodes') — returns None (falsy) safely when the key is absent

Changes

pageindex/utils.py — one-line change in get_leaf_nodes:

# Beforeifnotstructure['nodes']: # KeyError on leaf nodes# Afterifnotstructure.get('nodes'): # safe: returns None if key absent

Why this is consistent

Every other nodes access in the codebase already uses .get():

format_structure → structure.get('nodes')
is_leaf_node → node.get('nodes')
print_tree → node.get('nodes')

Test

frompageindex.utilsimportget_leaf_nodes, list_to_treeflat= [
{'structure': '1', 'title': 'Chapter 1', 'start_index': 1, 'end_index': 5},
{'structure': '2', 'title': 'Chapter 2', 'start_index': 6, 'end_index': 10},
]
tree=list_to_tree(flat)
leaves=get_leaf_nodes(tree) # previously raised KeyError, now returns both nodesassertlen(leaves) ==2

Fixes#330

list_to_tree() deletes the 'nodes' key from leaf nodes entirely via clean_node(). Direct access via structure['nodes'] raises KeyError on these nodes. Using structure.get('nodes') returns None (falsy) safely, consistent with how 'nodes' is accessed elsewhere in the codebase. FixesVectifyAI#330

KylinMountain

Thanks @ChiragB254, nice catch — consistent with the rest of the codebase. Merging! 🙏

…ctifyAI#331) list_to_tree() deletes the 'nodes' key from leaf nodes entirely via clean_node(). Direct access via structure['nodes'] raises KeyError on these nodes. Using structure.get('nodes') returns None (falsy) safely, consistent with how 'nodes' is accessed elsewhere in the codebase. FixesVectifyAI#330

…on shims The new SDK copied the legacy indexing pipeline into pageindex/index/ instead of moving it, leaving two divergent copies of page_index.py / page_index_md.py / utils.py. They had already drifted (the legacy copy still compared IndexConfig booleans against 'yes' — a separate fix), and every pipeline change had to be applied twice. Make pageindex/index/ the single source of truth (same pattern as the LegacyCloudAPI shim for the 0.2.x cloud SDK): - pageindex/index/utils.py absorbs the 27 legacy-only helpers/classes (get_page_tokens, convert_page_to_int, ConfigLoader, PDF text helpers, ...) so it's the sole utils module. Reconciled the diverged funcs: kept the modern versions, backported the #331 get_leaf_nodes .get() fix, and restored remove_fields' max_len parameter (superset). - index/page_index*.py now import `from .utils import *`; index/legacy_utils.py (a re-export of the old top-level utils) deleted. - Top-level page_index.py / page_index_md.py / utils.py become thin re-export shims that emit PendingDeprecationWarning. The md_to_tree shim coerces legacy 'yes'/'no' string flags to bool (the canonical version is boolean-typed). - ConfigLoader no longer reads the deleted config.yaml; it builds defaults from IndexConfig (was an unconditional FileNotFoundError). - __init__.py and retrieve.py import from pageindex.index.* directly so `import pageindex` does not trip the shims. Adds tests/test_legacy_shims.py pinning the contract: clean top-level import doesn't warn, legacy submodule imports warn, symbols still resolve, shim and canonical share one implementation, the #331 fix and ConfigLoader-without-yaml both hold, and the md_to_tree coercion works. Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS

KylinMountain approved these changes Jul 3, 2026
View reviewed changes

KylinMountain merged commit cc7e43c into VectifyAI:devJul 3, 2026

Chirag254-isymply mentioned this pull request Jul 12, 2026
Bug: get_leaf_nodes raises KeyError on leaf nodes due to missing nodes key #330
Open