🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__ and upgrade minimum SQLAlchemy to >=1.4.36 - #443

Merged
tiangolo merged 5 commits into
fastapi:mainfrom
daniil-berg:construct-column-only-once
Oct 23, 2023

Conversation

@daniil-berg

@daniil-bergdaniil-berg commented Sep 9, 2022

Copy link
Copy Markdown
Contributor

Summary

There was a bug in the SQLModelMetaclass whereby the SQLAlchemy Column objects were constructed twice for each field defined on a table model: First in the meta class' __new__ method and then again in its __init__ method.

With these changes the get_column_from_field function is called only once for each field, namely in SQLModelMetaclass.__new__.

Example

Construct a table model with a foreign key explicitly constructed via the SQLAlchemy ForeignKey class:

fromsqlmodelimportField, SQLModel, create_enginefromsqlalchemy.sql.schemaimportForeignKeyclassUser(SQLModel, table=True):
id: int=Field(primary_key=True)
classPost(SQLModel, table=True):
id: int=Field(primary_key=True)
user_id: int=Field(
sa_column_args=(ForeignKey("user.id", ondelete="CASCADE"),)
)

Without the proposed changes, even just running these class definitions causes an error:

Traceback (most recent call last):
File "/home/daniil/coding/sqlmodel/experiments/__init__.py", line 12, in <module>
class Post(SQLModel, table=True):
File "/home/daniil/coding/sqlmodel/sqlmodel/main.py", line 329, in __init__
dict_used[field_name] = get_column_from_field(field_value)
File "/home/daniil/coding/sqlmodel/sqlmodel/main.py", line 457, in get_column_from_field
return Column(sa_type, *args, **kwargs) # type: ignore
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 1765, in __init__
self._init_items(*args)
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 144, in _init_items
spwd(self, **kw)
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 1047, in _set_parent_with_dispatch
self._set_parent(parent, **kw)
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 2559, in _set_parent
raise exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: This ForeignKey already has a parent !

This is fixed with this PR.

Add the following to the example:

sqlite_url="sqlite:///:memory:"engine=create_engine(sqlite_url, echo=True)
SQLModel.metadata.create_all(engine)

There is no more error and the SQL statements are as expected:

CREATETABLEuser (
id INTEGERNOT NULL, PRIMARY KEY (id)
)
CREATETABLEpost (
id INTEGERNOT NULL, user_id INTEGERNOT NULL, PRIMARY KEY (id), FOREIGN KEY(user_id) REFERENCES user (id) ON DELETE CASCADE
)

PS

The codecov bot is weird... I really don't see how coverage could possibly have decreased. The statement below is also inconsistent with the report, if you click on it. Oh well.

have `get_column_from_field` be called once for each field (in `SQLModelMetaclass.__new__`)
@codecov

codecovBot commented Sep 9, 2022

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (ea79c47) 98.49% compared to head (5ad7c6a) 97.76%.
Report is 66 commits behind head on main.

❗ Current head 5ad7c6a differs from pull request most recent head 50a3f76. Consider uploading reports for the commit 50a3f76 to get more accurate results

Additional details and impacted files
@@ Coverage Diff @@## main #443 +/- ##
==========================================
- Coverage 98.49% 97.76% -0.74% 
==========================================
Files 185 187 +2 Lines 5856 6266 +410 ==========================================
+ Hits 5768 6126 +358 - Misses 88 140 +52 
FilesCoverage Δ
sqlmodel/main.py84.83% <ø> (ø)

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@github-actions

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit 5ad7c6a at: https://631b234f16d4cf27863512be--sqlmodel.netlify.app

@github-actions

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit eee8e80 at: https://639cdfc1a7c2ed00568a9bc6--sqlmodel.netlify.app

@Vanderhoof

Vanderhoof commented Mar 3, 2023

Copy link
Copy Markdown

Hey is there a workaround for cascade FK definition now while this fix is being investigated? 😊

UPD, probably something like this:

fromsqlmodelimportField, SQLModel, create_enginefromsqlalchemyimportColumn, Integerfromsqlalchemy.sql.schemaimportForeignKeyclassUser(SQLModel, table=True):
id: int=Field(primary_key=True)
classPost(SQLModel, table=True):
id: int=Field(primary_key=True)
user_id: int=Field(
sa_column=Column("user_id", Integer, ForeignKey("user.id", ondelete="CASCADE")),)
)

@tiangolotiangolo added the bug Something isn't working label Oct 22, 2023
@tiangolotiangolo changed the title 🐛 Remove repeated column construction from SQLModelMetaclass.__init__🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__Oct 23, 2023
@github-actions

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit 50a3f76 at: https://7b43a033.sqlmodel.pages.dev

@tiangolo

Copy link
Copy Markdown
Member

Awesome, thanks a lot for the thorough explanation @daniil-berg! 🚀

I tweaked it and updated the internals a bit.

This will be available in the next version, 0.0.9. 🎉

@tiangolotiangolo changed the title 🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__ and upgrade minimum SQLAlchemy to >=1.4.36Oct 23, 2023
@tiangolo
tiangolo merged commit 9809b5b into fastapi:mainOct 23, 2023
shabani1 added a commit to lexy-ai/lexy that referenced this pull request Nov 3, 2023
# What
This updates SQLModel to version 0.0.11, which solves an
[issue](fastapi/sqlmodel#443) with foreign key
declaration, and another
[issue](fastapi/sqlmodel#58) with AsyncSession
type annotations.
# Why
Prior to this update, attempting to delete a document with an associated
index record would throw a foreign key violation error.
# Test plan
Add a document which results in an index record being created. Delete
the same document and verify that
- The document is deleted without error
- The index records associated with the document are also deleted
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugSomething isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@daniil-berg@Vanderhoof@tiangolo
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__ and upgrade minimum SQLAlchemy to >=1.4.36 - #443

Merged
tiangolo merged 5 commits into
fastapi:mainfrom
daniil-berg:construct-column-only-once
Oct 23, 2023

Conversation

@daniil-berg

@daniil-bergdaniil-berg commented Sep 9, 2022

Copy link
Copy Markdown
Contributor

Summary

There was a bug in the SQLModelMetaclass whereby the SQLAlchemy Column objects were constructed twice for each field defined on a table model: First in the meta class' __new__ method and then again in its __init__ method.

With these changes the get_column_from_field function is called only once for each field, namely in SQLModelMetaclass.__new__.

Example

Construct a table model with a foreign key explicitly constructed via the SQLAlchemy ForeignKey class:

fromsqlmodelimportField, SQLModel, create_enginefromsqlalchemy.sql.schemaimportForeignKeyclassUser(SQLModel, table=True):
id: int=Field(primary_key=True)
classPost(SQLModel, table=True):
id: int=Field(primary_key=True)
user_id: int=Field(
sa_column_args=(ForeignKey("user.id", ondelete="CASCADE"),)
)

Without the proposed changes, even just running these class definitions causes an error:

Traceback (most recent call last):
File "/home/daniil/coding/sqlmodel/experiments/__init__.py", line 12, in <module>
class Post(SQLModel, table=True):
File "/home/daniil/coding/sqlmodel/sqlmodel/main.py", line 329, in __init__
dict_used[field_name] = get_column_from_field(field_value)
File "/home/daniil/coding/sqlmodel/sqlmodel/main.py", line 457, in get_column_from_field
return Column(sa_type, *args, **kwargs) # type: ignore
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 1765, in __init__
self._init_items(*args)
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 144, in _init_items
spwd(self, **kw)
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 1047, in _set_parent_with_dispatch
self._set_parent(parent, **kw)
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 2559, in _set_parent
raise exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: This ForeignKey already has a parent !

This is fixed with this PR.

Add the following to the example:

sqlite_url="sqlite:///:memory:"engine=create_engine(sqlite_url, echo=True)
SQLModel.metadata.create_all(engine)

There is no more error and the SQL statements are as expected:

CREATETABLEuser (
id INTEGERNOT NULL, PRIMARY KEY (id)
)
CREATETABLEpost (
id INTEGERNOT NULL, user_id INTEGERNOT NULL, PRIMARY KEY (id), FOREIGN KEY(user_id) REFERENCES user (id) ON DELETE CASCADE
)

PS

The codecov bot is weird... I really don't see how coverage could possibly have decreased. The statement below is also inconsistent with the report, if you click on it. Oh well.

have `get_column_from_field` be called once for each field (in `SQLModelMetaclass.__new__`)
@codecov

codecovBot commented Sep 9, 2022

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (ea79c47) 98.49% compared to head (5ad7c6a) 97.76%.
Report is 66 commits behind head on main.

❗ Current head 5ad7c6a differs from pull request most recent head 50a3f76. Consider uploading reports for the commit 50a3f76 to get more accurate results

Additional details and impacted files
@@ Coverage Diff @@## main #443 +/- ##
==========================================
- Coverage 98.49% 97.76% -0.74% 
==========================================
Files 185 187 +2 Lines 5856 6266 +410 ==========================================
+ Hits 5768 6126 +358 - Misses 88 140 +52 
FilesCoverage Δ
sqlmodel/main.py84.83% <ø> (ø)

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@github-actions

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit 5ad7c6a at: https://631b234f16d4cf27863512be--sqlmodel.netlify.app

@github-actions

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit eee8e80 at: https://639cdfc1a7c2ed00568a9bc6--sqlmodel.netlify.app

@Vanderhoof

Vanderhoof commented Mar 3, 2023

Copy link
Copy Markdown

Hey is there a workaround for cascade FK definition now while this fix is being investigated? 😊

UPD, probably something like this:

fromsqlmodelimportField, SQLModel, create_enginefromsqlalchemyimportColumn, Integerfromsqlalchemy.sql.schemaimportForeignKeyclassUser(SQLModel, table=True):
id: int=Field(primary_key=True)
classPost(SQLModel, table=True):
id: int=Field(primary_key=True)
user_id: int=Field(
sa_column=Column("user_id", Integer, ForeignKey("user.id", ondelete="CASCADE")),)
)

@tiangolotiangolo added the bug Something isn't working label Oct 22, 2023
@tiangolotiangolo changed the title 🐛 Remove repeated column construction from SQLModelMetaclass.__init__🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__Oct 23, 2023
@github-actions

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit 50a3f76 at: https://7b43a033.sqlmodel.pages.dev

@tiangolo

Copy link
Copy Markdown
Member

Awesome, thanks a lot for the thorough explanation @daniil-berg! 🚀

I tweaked it and updated the internals a bit.

This will be available in the next version, 0.0.9. 🎉

@tiangolotiangolo changed the title 🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__ and upgrade minimum SQLAlchemy to >=1.4.36Oct 23, 2023
@tiangolo
tiangolo merged commit 9809b5b into fastapi:mainOct 23, 2023
shabani1 added a commit to lexy-ai/lexy that referenced this pull request Nov 3, 2023
# What
This updates SQLModel to version 0.0.11, which solves an
[issue](fastapi/sqlmodel#443) with foreign key
declaration, and another
[issue](fastapi/sqlmodel#58) with AsyncSession
type annotations.
# Why
Prior to this update, attempting to delete a document with an associated
index record would throw a foreign key violation error.
# Test plan
Add a document which results in an index record being created. Delete
the same document and verify that
- The document is deleted without error
- The index records associated with the document are also deleted
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugSomething isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@daniil-berg@Vanderhoof@tiangolo
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__ and upgrade minimum SQLAlchemy to >=1.4.36 - #443

Merged
tiangolo merged 5 commits into
fastapi:mainfrom
daniil-berg:construct-column-only-once
Oct 23, 2023

Conversation

@daniil-berg

@daniil-bergdaniil-berg commented Sep 9, 2022

Copy link
Copy Markdown
Contributor

Summary

There was a bug in the SQLModelMetaclass whereby the SQLAlchemy Column objects were constructed twice for each field defined on a table model: First in the meta class' __new__ method and then again in its __init__ method.

With these changes the get_column_from_field function is called only once for each field, namely in SQLModelMetaclass.__new__.

Example

Construct a table model with a foreign key explicitly constructed via the SQLAlchemy ForeignKey class:

fromsqlmodelimportField, SQLModel, create_enginefromsqlalchemy.sql.schemaimportForeignKeyclassUser(SQLModel, table=True):
id: int=Field(primary_key=True)
classPost(SQLModel, table=True):
id: int=Field(primary_key=True)
user_id: int=Field(
sa_column_args=(ForeignKey("user.id", ondelete="CASCADE"),)
)

Without the proposed changes, even just running these class definitions causes an error:

Traceback (most recent call last):
File "/home/daniil/coding/sqlmodel/experiments/__init__.py", line 12, in <module>
class Post(SQLModel, table=True):
File "/home/daniil/coding/sqlmodel/sqlmodel/main.py", line 329, in __init__
dict_used[field_name] = get_column_from_field(field_value)
File "/home/daniil/coding/sqlmodel/sqlmodel/main.py", line 457, in get_column_from_field
return Column(sa_type, *args, **kwargs) # type: ignore
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 1765, in __init__
self._init_items(*args)
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 144, in _init_items
spwd(self, **kw)
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 1047, in _set_parent_with_dispatch
self._set_parent(parent, **kw)
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 2559, in _set_parent
raise exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: This ForeignKey already has a parent !

This is fixed with this PR.

Add the following to the example:

sqlite_url="sqlite:///:memory:"engine=create_engine(sqlite_url, echo=True)
SQLModel.metadata.create_all(engine)

There is no more error and the SQL statements are as expected:

CREATETABLEuser (
id INTEGERNOT NULL, PRIMARY KEY (id)
)
CREATETABLEpost (
id INTEGERNOT NULL, user_id INTEGERNOT NULL, PRIMARY KEY (id), FOREIGN KEY(user_id) REFERENCES user (id) ON DELETE CASCADE
)

PS

The codecov bot is weird... I really don't see how coverage could possibly have decreased. The statement below is also inconsistent with the report, if you click on it. Oh well.

have `get_column_from_field` be called once for each field (in `SQLModelMetaclass.__new__`)
@codecov

codecovBot commented Sep 9, 2022

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (ea79c47) 98.49% compared to head (5ad7c6a) 97.76%.
Report is 66 commits behind head on main.

❗ Current head 5ad7c6a differs from pull request most recent head 50a3f76. Consider uploading reports for the commit 50a3f76 to get more accurate results

Additional details and impacted files
@@ Coverage Diff @@## main #443 +/- ##
==========================================
- Coverage 98.49% 97.76% -0.74% 
==========================================
Files 185 187 +2 Lines 5856 6266 +410 ==========================================
+ Hits 5768 6126 +358 - Misses 88 140 +52 
FilesCoverage Δ
sqlmodel/main.py84.83% <ø> (ø)

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@github-actions

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit 5ad7c6a at: https://631b234f16d4cf27863512be--sqlmodel.netlify.app

@github-actions

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit eee8e80 at: https://639cdfc1a7c2ed00568a9bc6--sqlmodel.netlify.app

@Vanderhoof

Vanderhoof commented Mar 3, 2023

Copy link
Copy Markdown

Hey is there a workaround for cascade FK definition now while this fix is being investigated? 😊

UPD, probably something like this:

fromsqlmodelimportField, SQLModel, create_enginefromsqlalchemyimportColumn, Integerfromsqlalchemy.sql.schemaimportForeignKeyclassUser(SQLModel, table=True):
id: int=Field(primary_key=True)
classPost(SQLModel, table=True):
id: int=Field(primary_key=True)
user_id: int=Field(
sa_column=Column("user_id", Integer, ForeignKey("user.id", ondelete="CASCADE")),)
)

@tiangolotiangolo added the bug Something isn't working label Oct 22, 2023
@tiangolotiangolo changed the title 🐛 Remove repeated column construction from SQLModelMetaclass.__init__🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__Oct 23, 2023
@github-actions

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit 50a3f76 at: https://7b43a033.sqlmodel.pages.dev

@tiangolo

Copy link
Copy Markdown
Member

Awesome, thanks a lot for the thorough explanation @daniil-berg! 🚀

I tweaked it and updated the internals a bit.

This will be available in the next version, 0.0.9. 🎉

@tiangolotiangolo changed the title 🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__ and upgrade minimum SQLAlchemy to >=1.4.36Oct 23, 2023
@tiangolo
tiangolo merged commit 9809b5b into fastapi:mainOct 23, 2023
shabani1 added a commit to lexy-ai/lexy that referenced this pull request Nov 3, 2023
# What
This updates SQLModel to version 0.0.11, which solves an
[issue](fastapi/sqlmodel#443) with foreign key
declaration, and another
[issue](fastapi/sqlmodel#58) with AsyncSession
type annotations.
# Why
Prior to this update, attempting to delete a document with an associated
index record would throw a foreign key violation error.
# Test plan
Add a document which results in an index record being created. Delete
the same document and verify that
- The document is deleted without error
- The index records associated with the document are also deleted
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugSomething isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@daniil-berg@Vanderhoof@tiangolo
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__ and upgrade minimum SQLAlchemy to >=1.4.36 - #443

Merged
tiangolo merged 5 commits into
fastapi:mainfrom
daniil-berg:construct-column-only-once
Oct 23, 2023

Conversation

@daniil-berg

@daniil-bergdaniil-berg commented Sep 9, 2022

Copy link
Copy Markdown
Contributor

Summary

There was a bug in the SQLModelMetaclass whereby the SQLAlchemy Column objects were constructed twice for each field defined on a table model: First in the meta class' __new__ method and then again in its __init__ method.

With these changes the get_column_from_field function is called only once for each field, namely in SQLModelMetaclass.__new__.

Example

Construct a table model with a foreign key explicitly constructed via the SQLAlchemy ForeignKey class:

fromsqlmodelimportField, SQLModel, create_enginefromsqlalchemy.sql.schemaimportForeignKeyclassUser(SQLModel, table=True):
id: int=Field(primary_key=True)
classPost(SQLModel, table=True):
id: int=Field(primary_key=True)
user_id: int=Field(
sa_column_args=(ForeignKey("user.id", ondelete="CASCADE"),)
)

Without the proposed changes, even just running these class definitions causes an error:

Traceback (most recent call last):
File "/home/daniil/coding/sqlmodel/experiments/__init__.py", line 12, in <module>
class Post(SQLModel, table=True):
File "/home/daniil/coding/sqlmodel/sqlmodel/main.py", line 329, in __init__
dict_used[field_name] = get_column_from_field(field_value)
File "/home/daniil/coding/sqlmodel/sqlmodel/main.py", line 457, in get_column_from_field
return Column(sa_type, *args, **kwargs) # type: ignore
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 1765, in __init__
self._init_items(*args)
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 144, in _init_items
spwd(self, **kw)
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 1047, in _set_parent_with_dispatch
self._set_parent(parent, **kw)
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 2559, in _set_parent
raise exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: This ForeignKey already has a parent !

This is fixed with this PR.

Add the following to the example:

sqlite_url="sqlite:///:memory:"engine=create_engine(sqlite_url, echo=True)
SQLModel.metadata.create_all(engine)

There is no more error and the SQL statements are as expected:

CREATETABLEuser (
id INTEGERNOT NULL, PRIMARY KEY (id)
)
CREATETABLEpost (
id INTEGERNOT NULL, user_id INTEGERNOT NULL, PRIMARY KEY (id), FOREIGN KEY(user_id) REFERENCES user (id) ON DELETE CASCADE
)

PS

The codecov bot is weird... I really don't see how coverage could possibly have decreased. The statement below is also inconsistent with the report, if you click on it. Oh well.

have `get_column_from_field` be called once for each field (in `SQLModelMetaclass.__new__`)
@codecov

codecovBot commented Sep 9, 2022

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (ea79c47) 98.49% compared to head (5ad7c6a) 97.76%.
Report is 66 commits behind head on main.

❗ Current head 5ad7c6a differs from pull request most recent head 50a3f76. Consider uploading reports for the commit 50a3f76 to get more accurate results

Additional details and impacted files
@@ Coverage Diff @@## main #443 +/- ##
==========================================
- Coverage 98.49% 97.76% -0.74% 
==========================================
Files 185 187 +2 Lines 5856 6266 +410 ==========================================
+ Hits 5768 6126 +358 - Misses 88 140 +52 
FilesCoverage Δ
sqlmodel/main.py84.83% <ø> (ø)

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@github-actions

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit 5ad7c6a at: https://631b234f16d4cf27863512be--sqlmodel.netlify.app

@github-actions

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit eee8e80 at: https://639cdfc1a7c2ed00568a9bc6--sqlmodel.netlify.app

@Vanderhoof

Vanderhoof commented Mar 3, 2023

Copy link
Copy Markdown

Hey is there a workaround for cascade FK definition now while this fix is being investigated? 😊

UPD, probably something like this:

fromsqlmodelimportField, SQLModel, create_enginefromsqlalchemyimportColumn, Integerfromsqlalchemy.sql.schemaimportForeignKeyclassUser(SQLModel, table=True):
id: int=Field(primary_key=True)
classPost(SQLModel, table=True):
id: int=Field(primary_key=True)
user_id: int=Field(
sa_column=Column("user_id", Integer, ForeignKey("user.id", ondelete="CASCADE")),)
)

@tiangolotiangolo added the bug Something isn't working label Oct 22, 2023
@tiangolotiangolo changed the title 🐛 Remove repeated column construction from SQLModelMetaclass.__init__🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__Oct 23, 2023
@github-actions

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit 50a3f76 at: https://7b43a033.sqlmodel.pages.dev

@tiangolo

Copy link
Copy Markdown
Member

Awesome, thanks a lot for the thorough explanation @daniil-berg! 🚀

I tweaked it and updated the internals a bit.

This will be available in the next version, 0.0.9. 🎉

@tiangolotiangolo changed the title 🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__ and upgrade minimum SQLAlchemy to >=1.4.36Oct 23, 2023
@tiangolo
tiangolo merged commit 9809b5b into fastapi:mainOct 23, 2023
shabani1 added a commit to lexy-ai/lexy that referenced this pull request Nov 3, 2023
# What
This updates SQLModel to version 0.0.11, which solves an
[issue](fastapi/sqlmodel#443) with foreign key
declaration, and another
[issue](fastapi/sqlmodel#58) with AsyncSession
type annotations.
# Why
Prior to this update, attempting to delete a document with an associated
index record would throw a foreign key violation error.
# Test plan
Add a document which results in an index record being created. Delete
the same document and verify that
- The document is deleted without error
- The index records associated with the document are also deleted
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugSomething isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@daniil-berg@Vanderhoof@tiangolo
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__ and upgrade minimum SQLAlchemy to >=1.4.36 - #443

Merged
tiangolo merged 5 commits into
fastapi:mainfrom
daniil-berg:construct-column-only-once
Oct 23, 2023

Conversation

@daniil-berg

@daniil-bergdaniil-berg commented Sep 9, 2022

Copy link
Copy Markdown
Contributor

Summary

There was a bug in the SQLModelMetaclass whereby the SQLAlchemy Column objects were constructed twice for each field defined on a table model: First in the meta class' __new__ method and then again in its __init__ method.

With these changes the get_column_from_field function is called only once for each field, namely in SQLModelMetaclass.__new__.

Example

Construct a table model with a foreign key explicitly constructed via the SQLAlchemy ForeignKey class:

fromsqlmodelimportField, SQLModel, create_enginefromsqlalchemy.sql.schemaimportForeignKeyclassUser(SQLModel, table=True):
id: int=Field(primary_key=True)
classPost(SQLModel, table=True):
id: int=Field(primary_key=True)
user_id: int=Field(
sa_column_args=(ForeignKey("user.id", ondelete="CASCADE"),)
)

Without the proposed changes, even just running these class definitions causes an error:

Traceback (most recent call last):
File "/home/daniil/coding/sqlmodel/experiments/__init__.py", line 12, in <module>
class Post(SQLModel, table=True):
File "/home/daniil/coding/sqlmodel/sqlmodel/main.py", line 329, in __init__
dict_used[field_name] = get_column_from_field(field_value)
File "/home/daniil/coding/sqlmodel/sqlmodel/main.py", line 457, in get_column_from_field
return Column(sa_type, *args, **kwargs) # type: ignore
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 1765, in __init__
self._init_items(*args)
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 144, in _init_items
spwd(self, **kw)
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 1047, in _set_parent_with_dispatch
self._set_parent(parent, **kw)
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 2559, in _set_parent
raise exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: This ForeignKey already has a parent !

This is fixed with this PR.

Add the following to the example:

sqlite_url="sqlite:///:memory:"engine=create_engine(sqlite_url, echo=True)
SQLModel.metadata.create_all(engine)

There is no more error and the SQL statements are as expected:

CREATETABLEuser (
id INTEGERNOT NULL, PRIMARY KEY (id)
)
CREATETABLEpost (
id INTEGERNOT NULL, user_id INTEGERNOT NULL, PRIMARY KEY (id), FOREIGN KEY(user_id) REFERENCES user (id) ON DELETE CASCADE
)

PS

The codecov bot is weird... I really don't see how coverage could possibly have decreased. The statement below is also inconsistent with the report, if you click on it. Oh well.

have `get_column_from_field` be called once for each field (in `SQLModelMetaclass.__new__`)
@codecov

codecovBot commented Sep 9, 2022

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (ea79c47) 98.49% compared to head (5ad7c6a) 97.76%.
Report is 66 commits behind head on main.

❗ Current head 5ad7c6a differs from pull request most recent head 50a3f76. Consider uploading reports for the commit 50a3f76 to get more accurate results

Additional details and impacted files
@@ Coverage Diff @@## main #443 +/- ##
==========================================
- Coverage 98.49% 97.76% -0.74% 
==========================================
Files 185 187 +2 Lines 5856 6266 +410 ==========================================
+ Hits 5768 6126 +358 - Misses 88 140 +52 
FilesCoverage Δ
sqlmodel/main.py84.83% <ø> (ø)

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@github-actions

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit 5ad7c6a at: https://631b234f16d4cf27863512be--sqlmodel.netlify.app

@github-actions

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit eee8e80 at: https://639cdfc1a7c2ed00568a9bc6--sqlmodel.netlify.app

@Vanderhoof

Vanderhoof commented Mar 3, 2023

Copy link
Copy Markdown

Hey is there a workaround for cascade FK definition now while this fix is being investigated? 😊

UPD, probably something like this:

fromsqlmodelimportField, SQLModel, create_enginefromsqlalchemyimportColumn, Integerfromsqlalchemy.sql.schemaimportForeignKeyclassUser(SQLModel, table=True):
id: int=Field(primary_key=True)
classPost(SQLModel, table=True):
id: int=Field(primary_key=True)
user_id: int=Field(
sa_column=Column("user_id", Integer, ForeignKey("user.id", ondelete="CASCADE")),)
)

@tiangolotiangolo added the bug Something isn't working label Oct 22, 2023
@tiangolotiangolo changed the title 🐛 Remove repeated column construction from SQLModelMetaclass.__init__🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__Oct 23, 2023
@github-actions

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit 50a3f76 at: https://7b43a033.sqlmodel.pages.dev

@tiangolo

Copy link
Copy Markdown
Member

Awesome, thanks a lot for the thorough explanation @daniil-berg! 🚀

I tweaked it and updated the internals a bit.

This will be available in the next version, 0.0.9. 🎉

@tiangolotiangolo changed the title 🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__ and upgrade minimum SQLAlchemy to >=1.4.36Oct 23, 2023
@tiangolo
tiangolo merged commit 9809b5b into fastapi:mainOct 23, 2023
shabani1 added a commit to lexy-ai/lexy that referenced this pull request Nov 3, 2023
# What
This updates SQLModel to version 0.0.11, which solves an
[issue](fastapi/sqlmodel#443) with foreign key
declaration, and another
[issue](fastapi/sqlmodel#58) with AsyncSession
type annotations.
# Why
Prior to this update, attempting to delete a document with an associated
index record would throw a foreign key violation error.
# Test plan
Add a document which results in an index record being created. Delete
the same document and verify that
- The document is deleted without error
- The index records associated with the document are also deleted
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugSomething isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@daniil-berg@Vanderhoof@tiangolo
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__ and upgrade minimum SQLAlchemy to >=1.4.36 - #443

Merged
tiangolo merged 5 commits into
fastapi:mainfrom
daniil-berg:construct-column-only-once
Oct 23, 2023

Conversation

@daniil-berg

@daniil-bergdaniil-berg commented Sep 9, 2022

Copy link
Copy Markdown
Contributor

Summary

There was a bug in the SQLModelMetaclass whereby the SQLAlchemy Column objects were constructed twice for each field defined on a table model: First in the meta class' __new__ method and then again in its __init__ method.

With these changes the get_column_from_field function is called only once for each field, namely in SQLModelMetaclass.__new__.

Example

Construct a table model with a foreign key explicitly constructed via the SQLAlchemy ForeignKey class:

fromsqlmodelimportField, SQLModel, create_enginefromsqlalchemy.sql.schemaimportForeignKeyclassUser(SQLModel, table=True):
id: int=Field(primary_key=True)
classPost(SQLModel, table=True):
id: int=Field(primary_key=True)
user_id: int=Field(
sa_column_args=(ForeignKey("user.id", ondelete="CASCADE"),)
)

Without the proposed changes, even just running these class definitions causes an error:

Traceback (most recent call last):
File "/home/daniil/coding/sqlmodel/experiments/__init__.py", line 12, in <module>
class Post(SQLModel, table=True):
File "/home/daniil/coding/sqlmodel/sqlmodel/main.py", line 329, in __init__
dict_used[field_name] = get_column_from_field(field_value)
File "/home/daniil/coding/sqlmodel/sqlmodel/main.py", line 457, in get_column_from_field
return Column(sa_type, *args, **kwargs) # type: ignore
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 1765, in __init__
self._init_items(*args)
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 144, in _init_items
spwd(self, **kw)
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 1047, in _set_parent_with_dispatch
self._set_parent(parent, **kw)
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 2559, in _set_parent
raise exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: This ForeignKey already has a parent !

This is fixed with this PR.

Add the following to the example:

sqlite_url="sqlite:///:memory:"engine=create_engine(sqlite_url, echo=True)
SQLModel.metadata.create_all(engine)

There is no more error and the SQL statements are as expected:

CREATETABLEuser (
id INTEGERNOT NULL, PRIMARY KEY (id)
)
CREATETABLEpost (
id INTEGERNOT NULL, user_id INTEGERNOT NULL, PRIMARY KEY (id), FOREIGN KEY(user_id) REFERENCES user (id) ON DELETE CASCADE
)

PS

The codecov bot is weird... I really don't see how coverage could possibly have decreased. The statement below is also inconsistent with the report, if you click on it. Oh well.

have `get_column_from_field` be called once for each field (in `SQLModelMetaclass.__new__`)
@codecov

codecovBot commented Sep 9, 2022

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (ea79c47) 98.49% compared to head (5ad7c6a) 97.76%.
Report is 66 commits behind head on main.

❗ Current head 5ad7c6a differs from pull request most recent head 50a3f76. Consider uploading reports for the commit 50a3f76 to get more accurate results

Additional details and impacted files
@@ Coverage Diff @@## main #443 +/- ##
==========================================
- Coverage 98.49% 97.76% -0.74% 
==========================================
Files 185 187 +2 Lines 5856 6266 +410 ==========================================
+ Hits 5768 6126 +358 - Misses 88 140 +52 
FilesCoverage Δ
sqlmodel/main.py84.83% <ø> (ø)

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@github-actions

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit 5ad7c6a at: https://631b234f16d4cf27863512be--sqlmodel.netlify.app

@github-actions

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit eee8e80 at: https://639cdfc1a7c2ed00568a9bc6--sqlmodel.netlify.app

@Vanderhoof

Vanderhoof commented Mar 3, 2023

Copy link
Copy Markdown

Hey is there a workaround for cascade FK definition now while this fix is being investigated? 😊

UPD, probably something like this:

fromsqlmodelimportField, SQLModel, create_enginefromsqlalchemyimportColumn, Integerfromsqlalchemy.sql.schemaimportForeignKeyclassUser(SQLModel, table=True):
id: int=Field(primary_key=True)
classPost(SQLModel, table=True):
id: int=Field(primary_key=True)
user_id: int=Field(
sa_column=Column("user_id", Integer, ForeignKey("user.id", ondelete="CASCADE")),)
)

@tiangolotiangolo added the bug Something isn't working label Oct 22, 2023
@tiangolotiangolo changed the title 🐛 Remove repeated column construction from SQLModelMetaclass.__init__🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__Oct 23, 2023
@github-actions

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit 50a3f76 at: https://7b43a033.sqlmodel.pages.dev

@tiangolo

Copy link
Copy Markdown
Member

Awesome, thanks a lot for the thorough explanation @daniil-berg! 🚀

I tweaked it and updated the internals a bit.

This will be available in the next version, 0.0.9. 🎉

@tiangolotiangolo changed the title 🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__ and upgrade minimum SQLAlchemy to >=1.4.36Oct 23, 2023
@tiangolo
tiangolo merged commit 9809b5b into fastapi:mainOct 23, 2023
shabani1 added a commit to lexy-ai/lexy that referenced this pull request Nov 3, 2023
# What
This updates SQLModel to version 0.0.11, which solves an
[issue](fastapi/sqlmodel#443) with foreign key
declaration, and another
[issue](fastapi/sqlmodel#58) with AsyncSession
type annotations.
# Why
Prior to this update, attempting to delete a document with an associated
index record would throw a foreign key violation error.
# Test plan
Add a document which results in an index record being created. Delete
the same document and verify that
- The document is deleted without error
- The index records associated with the document are also deleted
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugSomething isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@daniil-berg@Vanderhoof@tiangolo
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__ and upgrade minimum SQLAlchemy to >=1.4.36 - #443

Merged
tiangolo merged 5 commits into
fastapi:mainfrom
daniil-berg:construct-column-only-once
Oct 23, 2023

Conversation

@daniil-berg

@daniil-bergdaniil-berg commented Sep 9, 2022

Copy link
Copy Markdown
Contributor

Summary

There was a bug in the SQLModelMetaclass whereby the SQLAlchemy Column objects were constructed twice for each field defined on a table model: First in the meta class' __new__ method and then again in its __init__ method.

With these changes the get_column_from_field function is called only once for each field, namely in SQLModelMetaclass.__new__.

Example

Construct a table model with a foreign key explicitly constructed via the SQLAlchemy ForeignKey class:

fromsqlmodelimportField, SQLModel, create_enginefromsqlalchemy.sql.schemaimportForeignKeyclassUser(SQLModel, table=True):
id: int=Field(primary_key=True)
classPost(SQLModel, table=True):
id: int=Field(primary_key=True)
user_id: int=Field(
sa_column_args=(ForeignKey("user.id", ondelete="CASCADE"),)
)

Without the proposed changes, even just running these class definitions causes an error:

Traceback (most recent call last):
File "/home/daniil/coding/sqlmodel/experiments/__init__.py", line 12, in <module>
class Post(SQLModel, table=True):
File "/home/daniil/coding/sqlmodel/sqlmodel/main.py", line 329, in __init__
dict_used[field_name] = get_column_from_field(field_value)
File "/home/daniil/coding/sqlmodel/sqlmodel/main.py", line 457, in get_column_from_field
return Column(sa_type, *args, **kwargs) # type: ignore
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 1765, in __init__
self._init_items(*args)
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 144, in _init_items
spwd(self, **kw)
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 1047, in _set_parent_with_dispatch
self._set_parent(parent, **kw)
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 2559, in _set_parent
raise exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: This ForeignKey already has a parent !

This is fixed with this PR.

Add the following to the example:

sqlite_url="sqlite:///:memory:"engine=create_engine(sqlite_url, echo=True)
SQLModel.metadata.create_all(engine)

There is no more error and the SQL statements are as expected:

CREATETABLEuser (
id INTEGERNOT NULL, PRIMARY KEY (id)
)
CREATETABLEpost (
id INTEGERNOT NULL, user_id INTEGERNOT NULL, PRIMARY KEY (id), FOREIGN KEY(user_id) REFERENCES user (id) ON DELETE CASCADE
)

PS

The codecov bot is weird... I really don't see how coverage could possibly have decreased. The statement below is also inconsistent with the report, if you click on it. Oh well.

have `get_column_from_field` be called once for each field (in `SQLModelMetaclass.__new__`)
@codecov

codecovBot commented Sep 9, 2022

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (ea79c47) 98.49% compared to head (5ad7c6a) 97.76%.
Report is 66 commits behind head on main.

❗ Current head 5ad7c6a differs from pull request most recent head 50a3f76. Consider uploading reports for the commit 50a3f76 to get more accurate results

Additional details and impacted files
@@ Coverage Diff @@## main #443 +/- ##
==========================================
- Coverage 98.49% 97.76% -0.74% 
==========================================
Files 185 187 +2 Lines 5856 6266 +410 ==========================================
+ Hits 5768 6126 +358 - Misses 88 140 +52 
FilesCoverage Δ
sqlmodel/main.py84.83% <ø> (ø)

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@github-actions

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit 5ad7c6a at: https://631b234f16d4cf27863512be--sqlmodel.netlify.app

@github-actions

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit eee8e80 at: https://639cdfc1a7c2ed00568a9bc6--sqlmodel.netlify.app

@Vanderhoof

Vanderhoof commented Mar 3, 2023

Copy link
Copy Markdown

Hey is there a workaround for cascade FK definition now while this fix is being investigated? 😊

UPD, probably something like this:

fromsqlmodelimportField, SQLModel, create_enginefromsqlalchemyimportColumn, Integerfromsqlalchemy.sql.schemaimportForeignKeyclassUser(SQLModel, table=True):
id: int=Field(primary_key=True)
classPost(SQLModel, table=True):
id: int=Field(primary_key=True)
user_id: int=Field(
sa_column=Column("user_id", Integer, ForeignKey("user.id", ondelete="CASCADE")),)
)

@tiangolotiangolo added the bug Something isn't working label Oct 22, 2023
@tiangolotiangolo changed the title 🐛 Remove repeated column construction from SQLModelMetaclass.__init__🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__Oct 23, 2023
@github-actions

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit 50a3f76 at: https://7b43a033.sqlmodel.pages.dev

@tiangolo

Copy link
Copy Markdown
Member

Awesome, thanks a lot for the thorough explanation @daniil-berg! 🚀

I tweaked it and updated the internals a bit.

This will be available in the next version, 0.0.9. 🎉

@tiangolotiangolo changed the title 🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__ and upgrade minimum SQLAlchemy to >=1.4.36Oct 23, 2023
@tiangolo
tiangolo merged commit 9809b5b into fastapi:mainOct 23, 2023
shabani1 added a commit to lexy-ai/lexy that referenced this pull request Nov 3, 2023
# What
This updates SQLModel to version 0.0.11, which solves an
[issue](fastapi/sqlmodel#443) with foreign key
declaration, and another
[issue](fastapi/sqlmodel#58) with AsyncSession
type annotations.
# Why
Prior to this update, attempting to delete a document with an associated
index record would throw a foreign key violation error.
# Test plan
Add a document which results in an index record being created. Delete
the same document and verify that
- The document is deleted without error
- The index records associated with the document are also deleted
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugSomething isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@daniil-berg@Vanderhoof@tiangolo
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__ and upgrade minimum SQLAlchemy to >=1.4.36 - #443

Merged
tiangolo merged 5 commits into
fastapi:mainfrom
daniil-berg:construct-column-only-once
Oct 23, 2023

Conversation

@daniil-berg

@daniil-bergdaniil-berg commented Sep 9, 2022

Copy link
Copy Markdown
Contributor

Summary

There was a bug in the SQLModelMetaclass whereby the SQLAlchemy Column objects were constructed twice for each field defined on a table model: First in the meta class' __new__ method and then again in its __init__ method.

With these changes the get_column_from_field function is called only once for each field, namely in SQLModelMetaclass.__new__.

Example

Construct a table model with a foreign key explicitly constructed via the SQLAlchemy ForeignKey class:

fromsqlmodelimportField, SQLModel, create_enginefromsqlalchemy.sql.schemaimportForeignKeyclassUser(SQLModel, table=True):
id: int=Field(primary_key=True)
classPost(SQLModel, table=True):
id: int=Field(primary_key=True)
user_id: int=Field(
sa_column_args=(ForeignKey("user.id", ondelete="CASCADE"),)
)

Without the proposed changes, even just running these class definitions causes an error:

Traceback (most recent call last):
File "/home/daniil/coding/sqlmodel/experiments/__init__.py", line 12, in <module>
class Post(SQLModel, table=True):
File "/home/daniil/coding/sqlmodel/sqlmodel/main.py", line 329, in __init__
dict_used[field_name] = get_column_from_field(field_value)
File "/home/daniil/coding/sqlmodel/sqlmodel/main.py", line 457, in get_column_from_field
return Column(sa_type, *args, **kwargs) # type: ignore
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 1765, in __init__
self._init_items(*args)
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 144, in _init_items
spwd(self, **kw)
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 1047, in _set_parent_with_dispatch
self._set_parent(parent, **kw)
File "/home/daniil/.cache/pypoetry/virtualenvs/sqlmodel-TV15XYpK-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 2559, in _set_parent
raise exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: This ForeignKey already has a parent !

This is fixed with this PR.

Add the following to the example:

sqlite_url="sqlite:///:memory:"engine=create_engine(sqlite_url, echo=True)
SQLModel.metadata.create_all(engine)

There is no more error and the SQL statements are as expected:

CREATETABLEuser (
id INTEGERNOT NULL, PRIMARY KEY (id)
)
CREATETABLEpost (
id INTEGERNOT NULL, user_id INTEGERNOT NULL, PRIMARY KEY (id), FOREIGN KEY(user_id) REFERENCES user (id) ON DELETE CASCADE
)

PS

The codecov bot is weird... I really don't see how coverage could possibly have decreased. The statement below is also inconsistent with the report, if you click on it. Oh well.

have `get_column_from_field` be called once for each field (in `SQLModelMetaclass.__new__`)
@codecov

codecovBot commented Sep 9, 2022

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (ea79c47) 98.49% compared to head (5ad7c6a) 97.76%.
Report is 66 commits behind head on main.

❗ Current head 5ad7c6a differs from pull request most recent head 50a3f76. Consider uploading reports for the commit 50a3f76 to get more accurate results

Additional details and impacted files
@@ Coverage Diff @@## main #443 +/- ##
==========================================
- Coverage 98.49% 97.76% -0.74% 
==========================================
Files 185 187 +2 Lines 5856 6266 +410 ==========================================
+ Hits 5768 6126 +358 - Misses 88 140 +52 
FilesCoverage Δ
sqlmodel/main.py84.83% <ø> (ø)

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@github-actions

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit 5ad7c6a at: https://631b234f16d4cf27863512be--sqlmodel.netlify.app

@github-actions

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit eee8e80 at: https://639cdfc1a7c2ed00568a9bc6--sqlmodel.netlify.app

@Vanderhoof

Vanderhoof commented Mar 3, 2023

Copy link
Copy Markdown

Hey is there a workaround for cascade FK definition now while this fix is being investigated? 😊

UPD, probably something like this:

fromsqlmodelimportField, SQLModel, create_enginefromsqlalchemyimportColumn, Integerfromsqlalchemy.sql.schemaimportForeignKeyclassUser(SQLModel, table=True):
id: int=Field(primary_key=True)
classPost(SQLModel, table=True):
id: int=Field(primary_key=True)
user_id: int=Field(
sa_column=Column("user_id", Integer, ForeignKey("user.id", ondelete="CASCADE")),)
)

@tiangolotiangolo added the bug Something isn't working label Oct 22, 2023
@tiangolotiangolo changed the title 🐛 Remove repeated column construction from SQLModelMetaclass.__init__🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__Oct 23, 2023
@github-actions

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit 50a3f76 at: https://7b43a033.sqlmodel.pages.dev

@tiangolo

Copy link
Copy Markdown
Member

Awesome, thanks a lot for the thorough explanation @daniil-berg! 🚀

I tweaked it and updated the internals a bit.

This will be available in the next version, 0.0.9. 🎉

@tiangolotiangolo changed the title 🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__🐛 Fix allowing using a ForeignKey directly, remove repeated column construction from SQLModelMetaclass.__init__ and upgrade minimum SQLAlchemy to >=1.4.36Oct 23, 2023
@tiangolo
tiangolo merged commit 9809b5b into fastapi:mainOct 23, 2023
shabani1 added a commit to lexy-ai/lexy that referenced this pull request Nov 3, 2023
# What
This updates SQLModel to version 0.0.11, which solves an
[issue](fastapi/sqlmodel#443) with foreign key
declaration, and another
[issue](fastapi/sqlmodel#58) with AsyncSession
type annotations.
# Why
Prior to this update, attempting to delete a document with an associated
index record would throw a foreign key violation error.
# Test plan
Add a document which results in an index record being created. Delete
the same document and verify that
- The document is deleted without error
- The index records associated with the document are also deleted
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugSomething isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@daniil-berg@Vanderhoof@tiangolo