migration to sqlalchemy 2.0 - #563

Closed
farahats9 wants to merge 23 commits into
fastapi:mainfrom
farahats9:main
Closed

migration to sqlalchemy 2.0#563
farahats9 wants to merge 23 commits into
fastapi:mainfrom
farahats9:main

Conversation

@farahats9

@farahats9farahats9 commented Mar 7, 2023

Copy link
Copy Markdown
Contributor

made the required changes to pass the tests for sqlalchemy 2.0
python version had to be pumped to 3.7 instead of 3.6
removed sqlalchemy2-stubs since it is not compatible or required.

More testing in a real project scenario is welcome.

@github-actions

github-actionsBot commented Mar 7, 2023

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit 60318b4 at: https://6406825e11187c06ceaecaf0--sqlmodel.netlify.app

@mortendaehli

mortendaehli commented Mar 24, 2023

Copy link
Copy Markdown

@farahats9, great! Do you think you would be able to sort out the failing tests? :)

@farahats9

farahats9 commented Mar 24, 2023

Copy link
Copy Markdown
ContributorAuthor

I fixed some of the mypy linting errors but I am not good at these things so maybe someone can fix the rest.

@github-actions

github-actionsBot commented Mar 24, 2023

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit b48423f at: https://641d9f2736a7fa1c9568bdb7--sqlmodel.netlify.app

sbor23
sbor23 reviewed Mar 27, 2023
View reviewed changes
Comment threadsqlmodel/sql/expression.py Outdated
@honglei

honglei commented Mar 31, 2023

Copy link
Copy Markdown

I used this version in my project with env: Python3.11.2/sqlalchemy2.0.7/Win10/scheme="postgresql+asyncpg", it works all fine.

farahats9and others added 2 commits March 31, 2023 12:58
@github-actions

github-actionsBot commented Mar 31, 2023

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit 9c219d9 at: https://6426bd112dc67b078b6e5298--sqlmodel.netlify.app

@github-actions

github-actionsBot commented Mar 31, 2023

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit 0eee8e8 at: https://6426be4468302c086218a5bb--sqlmodel.netlify.app

sbor23
sbor23 reviewed Mar 31, 2023
View reviewed changes
Comment threaddocs/contributing.md Outdated
Comment threaddocs/tutorial/index.md
Comment thread.github/workflows/test.yml Outdated
@github-actions

github-actionsBot commented Mar 31, 2023

Copy link
Copy Markdown
Contributor

📝 Docs preview for commit 050cf02 at: https://6426c1ce9df2a80bfd37ae93--sqlmodel.netlify.app

@gabrielmbmb

gabrielmbmb commented Apr 12, 2023

Copy link
Copy Markdown

@farahats9 someone created a PR on your fork for fixing the mypy issues

@heyaco

heyaco commented Apr 20, 2023

Copy link
Copy Markdown

This seems good to go just needs merging

@8thgencore

8thgencore commented Apr 20, 2023

Copy link
Copy Markdown

We are very much waiting for this change

@antont

antont commented Apr 21, 2023

Copy link
Copy Markdown

We are very much waiting for this change

Why do you wait, instead of taking it to use? I guess feedback from users would be helpful info also for the maintainer to know when to merge.

@me-v2

me-v2 commented Apr 24, 2023

Copy link
Copy Markdown

🚀 👀👀

@synodriver

synodriver commented Apr 24, 2023

Copy link
Copy Markdown

Great! I would love to try this branch in my production environment.

@k-s-t-i

k-s-t-i commented Apr 24, 2023

Copy link
Copy Markdown

Would like to see this merged for asyncpg support. Asyncpg seems to greatly increase performance based on most simple benchmarks.

@farahats9

farahats9 commented Apr 25, 2023•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

Would like to see this merged for asyncpg support. Asyncpg seems to greatly increase performance based on most simple benchmarks.

@k-s-t-i You can currently use Asyncpg with sqlalchemy v1.4 and sqlmodel v0.0.8 with no issue in most cases. you will need to watch out for some problems like lazy loading and cascading deletes on the orm side.

@synodriver

synodriver commented Apr 25, 2023

Copy link
Copy Markdown

Great! I would love to try this branch in my production environment.

Update: This patch can work with asyncmy in a real project. Will try asyncpg later.

synodriver
synodriver reviewed Apr 25, 2023
View reviewed changes
Comment threadpyproject.toml Outdated
@synodriver

synodriver commented Apr 26, 2023

Copy link
Copy Markdown

Great! I would love to try this branch in my production environment.

Update: This patch can work with asyncmy in a real project. Will try asyncpg later.

Update: asyncpg works too. Tested with a real project on both windows and linux. Notice: to make relationship work, you may use Relationship(back_populates="xxx", sa_relationship_kwargs={"lazy": 'selectin'}) where sa_relationship_kwargs is important.

UnoYakshi
UnoYakshi reviewed Apr 30, 2023
View reviewed changes

@UnoYakshiUnoYakshi left a comment •
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really would like to use it in my new project! Thank you for the PR!

Comment threadpyproject.toml Outdated
Comment threadsqlmodel/engine/result.py
Comment threadsqlmodel/main.py
Comment threadsqlmodel/sql/expression.py
Comment threadsqlmodel/sql/expression.py
Comment threadsqlmodel/sql/expression.py
Comment threadsqlmodel/__init__.py
Comment threaddocs_src/tutorial/relationship_attributes/back_populates/tutorial003.py
@hongleihonglei mentioned this pull request Jun 7, 2023
Open
@honglei

honglei commented Jun 27, 2023

Copy link
Copy Markdown

Five months have been passed since the release of SQLalchemy2.0, https://www.sqlalchemy.org/blog/2023/01/26/sqlalchemy-2.0.0-released/

@antont

antont commented Jun 27, 2023

Copy link
Copy Markdown

Five months have been passed since the release of SQLalchemy2.0, https://www.sqlalchemy.org/blog/2023/01/26/sqlalchemy-2.0.0-released/

You can help by using this PR in your project(s) if you want, and reporting how well it works.

@50Bytes-dev

50Bytes-dev commented Jul 10, 2023

Copy link
Copy Markdown

Been working with this PR for almost a month now, works great. Thank you!

@FilipeMarch

FilipeMarch commented Jul 10, 2023•
edited
Loading

Copy link
Copy Markdown

Hello @FilipeMarch! I've just change order of classes definition and solve your error:

NameError: name 'Article' is not defined

Full example code (database.py):

fromtypingimportList, OptionalfromsqlmodelimportSQLModel, Field, Relationship, create_engineclassArticle(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
reviewer_links: List["ArticleReviewerLink"] =Relationship(
back_populates="article",
)
classArticleReviewerLink(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
article_id: Optional[int] =Field(default=None, foreign_key="article.id")
article: Article=Relationship(back_populates="reviewer_links")
sqlite_file_name="database.db"sqlite_url=f"sqlite:///{sqlite_file_name}"engine=create_engine(sqlite_url, echo=True)
defcreate_db_and_tables():
SQLModel.metadata.create_all(engine)
if__name__=="__main__":
create_db_and_tables()

Sorry but changing the order of classes definition does not solve the error for me. At this point, I don't know what to do, but this branch is useless for me. I already have done what you mentioned, but the error persists. It always says if raiseerr and "Mapped[" in raw_annotation: # type: ignore TypeError: argument of type 'ForwardRef' is not iterable.

My project has about 30 tables at this point, and it has been running for quite 1 year. My project does not run as is with this branch, and I already tried dozens of things (like switching the order of tables, removing Relationships, removing a bunch of tables) and I can't make it to work in any possible way. Errors keep coming non stop. I can deactivate the Relationships, move tables up or down, it does not matter, new errors come every time. I just feel wasting my time because obviously there is something wrong, but I can't stop the errors until I remove all tables from the code?

The error messages are not helpful at all. And they only happen in this specific PR.

I understand it may be working for a very simple case like you presented, but for me it is always showing these errors, no matter how much code I change:

py", line2479, inget_propertyreturnself._props[key]
KeyError: 'user'

or

annotation=eval(expression, base_globals, locals_)
File"<string>", line1, in<module>NameError: name'PostUserLike'isnotdefined

or

.py", line223, ineval_expressionraiseNameError(
NameError: Couldnotde-stringifyannotation'PostUserLike'

or

", line2342, in_extract_mapped_subtypeifraiseerrand"Mapped["inraw_annotation: # type: ignoreTypeError: argumentof type 'ForwardRef'isnotiterable

My current API is running perfectly with sqlmodel = "^0.0.8"

If someone wants to investigate the issue with me, I am willing to screenshare and show the project structure and try to make it to work; my Discord is filipemarch

But one thing I wonder, even if I were able to make this PR to work on my code base after countless changes, I don't think this is how library updates are supposed to work? Every time I type poetry update and more than 20 libraries update, I don't remember ever having to change anything in my code just to update any library.

This PR is certainly introducing breaking changes but I'm unable to tell what and why exactly, although I investigated a lot here I couldn't even make my project to run with it.

By the way I woud love to be using sqlalchemy 2.0 at this point, and it is very sad that sqlmodel seems to be abandoned since August 2022, not a single update.

@farahats9

farahats9 commented Jul 10, 2023

Copy link
Copy Markdown
ContributorAuthor

@FilipeMarch you are right there is an issue in ForwardRef handling and I am trying to investigate it when I have some free time. The issue appears more when you have large project with many relations between tables so that's why some people report it working fine and some don't.

I feel this is a priority to fix over the mypy linting, I will do my best to figure it out but any help is appreciated.

@50Bytes-dev

50Bytes-dev commented Jul 11, 2023•
edited
Loading

Copy link
Copy Markdown

I had an error Mapped[" in raw_annotation but I was able to solve it. Here is an example of my models, works fine. Maybe it will help you.

# models/__init__.py# need for alembicfrom . import (
company,
location,
client
)
# models/company.pyifTYPE_CHECKING:
frommodels.locationimportLocationfrommodels.clientimportClientclassCompany(BaseModel, CompanyBase, table=True):
locations: List["Location"] =Relationship(back_populates="company")
clients: List["Client"] =Relationship(back_populates="company")
# models/location.pyfrommodels.companyimportCompanyclassLocation(BaseModel, OrderedList, LocationBase, table=True):
company_id: int=Field(default=None, foreign_key="company.id")
company: Company=Relationship(back_populates="locations")
# models/client.pyfrommodels.companyimportCompanyclassClient(ClientBase, BaseModel, table=True):
company_id: int=Field(default=None, nullable=False, foreign_key="company.id")
company: Company=Relationship(back_populates="clients")

@matanper

matanper commented Jul 20, 2023

Copy link
Copy Markdown

Hey @farahats9,

I'm getting error when using sqlalchemy relationship:

classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
powers: List["Power"] =Relationship(
sa_relationship=relationship(back_populates='hero')
)
classPower(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
hero_id: int=Field(foreign_key="hero.id")
hero: "Hero"=Relationship(back_populates="powers")
NameError: name 'Power' is not defined
NameError: Could not de-stringify annotation 'Power'

any idea why?

@AntonDeMeester

AntonDeMeester commented Jul 24, 2023•
edited
Loading

Copy link
Copy Markdown
Contributor

@matanper I think the forward Relationship (with the Foreign Key) needs to be a proper class, and not a type string. Can you try this?

classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
powers: List["Power"] =Relationship(
sa_relationship=relationship(back_populates='hero')
)
classPower(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
hero_id: int=Field(foreign_key="hero.id")
hero: Hero=Relationship(back_populates="powers") # <== No quotes around Hero

@peterlandry

peterlandry commented Jul 27, 2023

Copy link
Copy Markdown
Contributor

I've looked at this PR a bit to try cleaning up the linting errors. I've reduced most of them, but execute, get, and query are a little complicated because of the overloads. One thing I noticed in the process though is that sqlalchemy 2.0 has enough typing info to break mypy in query code. In VS code, given a sqlalchemy ORM model, if I do:

q=User.name=="spongebob"

q becomes an instance of ColumnElement[bool], while if User is a SQLModel, q goes straight to a bool as far as mypy is concerned.

This shows up as an error, but is correct at runtime. At runtime, we see an InstrumentedAttribute for the attribute, but mypy thinks it's a string. I'm going to look further at that, and in the meantime submit a pull request here that fixes the majority of the linting errors.

@honglei

honglei commented Jul 28, 2023

Copy link
Copy Markdown

Environment:

pydantic version: 1.10.11
python version: 3.11.4 (main, Jun 27 2023, 15:03:25) [GCC 8.3.0]
platform: Linux-4.19.0-amd64-desktop-x86_64-with-glibc2.28
optional deps. installed: ['devtools', 'email-validator', 'typing-extensions']
sqlalchemy: 2.0.19

pytest errors:

================================================================================================================================================================================================================== short test summary info ==================================================================================================================================================================================================================
FAILED tests/test_tutorial/test_create_db_and_table/test_tutorial001.py::test_create_db_and_table - assert 'BEGIN' in "No module named 'docs_src'\n"
FAILED tests/test_tutorial/test_fastapi/test_delete/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_limit_and_offset/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...Read Hero'}}}} == {'components'...Read Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_multiple_models/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_multiple_models/test_tutorial002.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_read_one/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...Read Hero'}}}} == {'components'...Read Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_relationships/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_response_model/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_session_with_dependency/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_simple_hero_api/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_teams/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_update/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
===================================================================================================================================================================================

@AntonDeMeesterAntonDeMeester mentioned this pull request Aug 1, 2023
Closed
@tiangolotiangolo added the feature New feature or request label Oct 22, 2023
@tiangolotiangolo mentioned this pull request Nov 18, 2023
Merged
@tiangolo

tiangolo commented Nov 18, 2023

Copy link
Copy Markdown
Member

TL;DR: Good news, this is available in the just released SQLModel 0.0.12! 🎉

Awesome, thanks a lot for all the work @farahats9! 🚀

And thanks everyone for the comments, reviews, etc. ☕

@farahats9 I continued the work here with several things on top on PR #700. I took your commits so you're co-author of that PR (and contributor to SQLModel, check your badge here next to your username, or in https://github.com/tiangolo/sqlmodel/graphs/contributors 😎 ).

I did a few extra things:

  • I reverted a few things, e.g. altering the docs to workaround for a previous lack of support for forward references.
  • Added support/fix for forward reference errors.
  • Removed several sections/modules that are no longer necessary, as were mainly to add type annotations to SQLAlchemy things, but now with SQLAlchemy's new proper types that's no longer necessary and I can reuse SQLAlchemy's things directly.
  • Updated all needed tests, e.g. to support the latest FastAPI.
  • Updated all the generated selects to use the new SQLAlchemy types.
  • Updated the classes and type annotations with the new SQLAlchemy types, with some overloads to make sure they are compatible with SQLModel. E.g. select().where().
  • Redefine all functions that would take column operations to accept the types used by SQLModel, e.g. or_(Hero.name == "Deadpond") as SQLAlchemy's version would require Hero.name to be a Mapped[str] instead of a str.
  • I explored a lot if it was possible/sensible to use and encourage the new Mapped[X], but sadly that breaks type annotations for creating new instances and dataclass_transform, as then that would expect to receive values with Mapped[X], e.g. Mapped[str] instead of str. And it would also add complications to the Pydantic handling. So, this was a lot of work mostly exploratory, but not much ended in the PR.

It is now available in SQLModel 0.0.12. 🎉

As this is now included in the repo and released, I'll now close this PR. Thanks! 🍰

@gdoumenc

gdoumenc commented Nov 20, 2023•
edited
Loading

Copy link
Copy Markdown

Hope it can help : I had the same issue because the relationsship was always defined and I didn' decalre it Optional :
origin_plan: t.Optional["Plan"] <== Good
instead of :
origin_plan: "Plan" <== Wrong

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

4 more reviewers
@hongleihongleihonglei left review comments
@sbor23sbor23sbor23 left review comments
@UnoYakshiUnoYakshiUnoYakshi left review comments
@synodriversynodriversynodriver left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

featureNew feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

20 participants

@farahats9@mortendaehli@honglei@gabrielmbmb@heyaco@8thgencore@antont@me-v2@synodriver@k-s-t-i@FilipeMarch@podsvirov@50Bytes-dev@matanper@AntonDeMeester@peterlandry@tiangolo@gdoumenc@sbor23@UnoYakshi
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
migration to sqlalchemy 2.0 - #563
Closed
farahats9 wants to merge 23 commits into
fastapi:mainfrom 
farahats9:main
Closed
migration to sqlalchemy 2.0#563
farahats9 wants to merge 23 commits into
fastapi:mainfrom 
farahats9:main
Conversation
@farahats9
@farahats9farahats9
commented
Mar 7, 2023

Copy link


Copy Markdown

Contributor
made the required changes to pass the tests for sqlalchemy 2.0

python version had to be pumped to 3.7 instead of 3.6

removed sqlalchemy2-stubs since it is not compatible or required.
More testing in a real project scenario is welcome.
@github-actions
github-actionsBot
commented
Mar 7, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 60318b4 at: https://6406825e11187c06ceaecaf0--sqlmodel.netlify.app
@mortendaehli
mortendaehli
commented
Mar 24, 2023

Copy link


Copy Markdown

@farahats9, great! Do you think you would be able to sort out the failing tests? :)
@farahats9
farahats9
commented
Mar 24, 2023

Copy link


Copy Markdown

ContributorAuthor
I fixed some of the mypy linting errors but I am not good at these things so maybe someone can fix the rest.
@github-actions
github-actionsBot
commented
Mar 24, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit b48423f at: https://641d9f2736a7fa1c9568bdb7--sqlmodel.netlify.app
sbor23
sbor23
reviewed
Mar 27, 2023
View reviewed changes
Comment threadsqlmodel/sql/expression.py
Outdated

@honglei
honglei
commented
Mar 31, 2023

Copy link


Copy Markdown

I used this version in my project with env: Python3.11.2/sqlalchemy2.0.7/Win10/scheme="postgresql+asyncpg", it works all fine.
farahats9and others
added 2 commits
March 31, 2023 12:58
@github-actions
github-actionsBot
commented
Mar 31, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 9c219d9 at: https://6426bd112dc67b078b6e5298--sqlmodel.netlify.app
@github-actions
github-actionsBot
commented
Mar 31, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 0eee8e8 at: https://6426be4468302c086218a5bb--sqlmodel.netlify.app
sbor23
sbor23
reviewed
Mar 31, 2023
View reviewed changes
Comment threaddocs/contributing.md
Outdated

Comment threaddocs/tutorial/index.md
Comment thread.github/workflows/test.yml
Outdated

@github-actions
github-actionsBot
commented
Mar 31, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 050cf02 at: https://6426c1ce9df2a80bfd37ae93--sqlmodel.netlify.app
@gabrielmbmb
gabrielmbmb
commented
Apr 12, 2023

Copy link


Copy Markdown

@farahats9 someone created a PR on your fork for fixing the mypy issues
@heyaco
heyaco
commented
Apr 20, 2023

Copy link


Copy Markdown

This seems good to go just needs merging
@8thgencore
8thgencore
commented
Apr 20, 2023

Copy link


Copy Markdown

We are very much waiting for this change
@antont
antont
commented
Apr 21, 2023

Copy link


Copy Markdown

We are very much waiting for this change
Why do you wait, instead of taking it to use? I guess feedback from users would be helpful info also for the maintainer to know when to merge.
@me-v2
me-v2
commented
Apr 24, 2023

Copy link


Copy Markdown

🚀 👀👀
@synodriver
synodriver
commented
Apr 24, 2023

Copy link


Copy Markdown

Great! I would love to try this branch in my production environment.
@k-s-t-i
k-s-t-i
commented
Apr 24, 2023

Copy link


Copy Markdown

Would like to see this merged for asyncpg support. Asyncpg seems to greatly increase performance based on most simple benchmarks.
@farahats9
farahats9
commented
Apr 25, 2023•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Would like to see this merged for asyncpg support. Asyncpg seems to greatly increase performance based on most simple benchmarks.
@k-s-t-i You can currently use Asyncpg with sqlalchemy v1.4 and sqlmodel v0.0.8 with no issue in most cases. you will need to watch out for some problems like lazy loading and cascading deletes on the orm side.
@synodriver
synodriver
commented
Apr 25, 2023

Copy link


Copy Markdown

Great! I would love to try this branch in my production environment.
Update: This patch can work with asyncmy in a real project. Will try asyncpg later.
synodriver
synodriver
reviewed
Apr 25, 2023
View reviewed changes
Comment threadpyproject.toml
Outdated

@synodriver
synodriver
commented
Apr 26, 2023

Copy link


Copy Markdown

Great! I would love to try this branch in my production environment.
Update: This patch can work with asyncmy in a real project. Will try asyncpg later.
Update: asyncpg works too. Tested with a real project on both windows and linux. Notice: to make relationship work, you may use Relationship(back_populates="xxx", sa_relationship_kwargs={"lazy": 'selectin'}) where sa_relationship_kwargs is important.
UnoYakshi
UnoYakshi
reviewed
Apr 30, 2023
View reviewed changes
@UnoYakshiUnoYakshi
left a comment
•

edited

Loading

Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really would like to use it in my new project! Thank you for the PR!
Comment threadpyproject.toml
Outdated

Comment threadsqlmodel/engine/result.py
Comment threadsqlmodel/main.py
Comment threadsqlmodel/sql/expression.py
Comment threadsqlmodel/sql/expression.py
Comment threadsqlmodel/sql/expression.py
Comment threadsqlmodel/__init__.py
Comment threaddocs_src/tutorial/relationship_attributes/back_populates/tutorial003.py
@hongleihonglei
mentioned this pull request
Jun 7, 2023

Open

@honglei
honglei
commented
Jun 27, 2023

Copy link


Copy Markdown

Five months have been passed since the release of SQLalchemy2.0, https://www.sqlalchemy.org/blog/2023/01/26/sqlalchemy-2.0.0-released/
@antont
antont
commented
Jun 27, 2023

Copy link


Copy Markdown

Five months have been passed since the release of SQLalchemy2.0, https://www.sqlalchemy.org/blog/2023/01/26/sqlalchemy-2.0.0-released/
You can help by using this PR in your project(s) if you want, and reporting how well it works.
@50Bytes-dev
50Bytes-dev
commented
Jul 10, 2023

Copy link


Copy Markdown

Been working with this PR for almost a month now, works great. Thank you!
@FilipeMarch
FilipeMarch
commented
Jul 10, 2023•

edited

Loading

Copy link


Copy Markdown

Hello @FilipeMarch! I've just change order of classes definition and solve your error:
NameError: name 'Article' is not defined
Full example code (database.py):
fromtypingimportList, OptionalfromsqlmodelimportSQLModel, Field, Relationship, create_engineclassArticle(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
reviewer_links: List["ArticleReviewerLink"] =Relationship(
back_populates="article",
)
classArticleReviewerLink(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
article_id: Optional[int] =Field(default=None, foreign_key="article.id")
article: Article=Relationship(back_populates="reviewer_links")
sqlite_file_name="database.db"sqlite_url=f"sqlite:///{sqlite_file_name}"engine=create_engine(sqlite_url, echo=True)
defcreate_db_and_tables():
SQLModel.metadata.create_all(engine)
if__name__=="__main__":
create_db_and_tables()
Sorry but changing the order of classes definition does not solve the error for me. At this point, I don't know what to do, but this branch is useless for me. I already have done what you mentioned, but the error persists. It always says if raiseerr and "Mapped[" in raw_annotation: # type: ignore TypeError: argument of type 'ForwardRef' is not iterable.
My project has about 30 tables at this point, and it has been running for quite 1 year. My project does not run as is with this branch, and I already tried dozens of things (like switching the order of tables, removing Relationships, removing a bunch of tables) and I can't make it to work in any possible way. Errors keep coming non stop. I can deactivate the Relationships, move tables up or down, it does not matter, new errors come every time. I just feel wasting my time because obviously there is something wrong, but I can't stop the errors until I remove all tables from the code?
The error messages are not helpful at all. And they only happen in this specific PR.
I understand it may be working for a very simple case like you presented, but for me it is always showing these errors, no matter how much code I change:
py", line2479, inget_propertyreturnself._props[key]
KeyError: 'user'
or
annotation=eval(expression, base_globals, locals_)
File"<string>", line1, in<module>NameError: name'PostUserLike'isnotdefined
or
.py", line223, ineval_expressionraiseNameError(
NameError: Couldnotde-stringifyannotation'PostUserLike'
or
", line2342, in_extract_mapped_subtypeifraiseerrand"Mapped["inraw_annotation: # type: ignoreTypeError: argumentof type 'ForwardRef'isnotiterable
My current API is running perfectly with sqlmodel = "^0.0.8"
If someone wants to investigate the issue with me, I am willing to screenshare and show the project structure and try to make it to work; my Discord is filipemarch
But one thing I wonder, even if I were able to make this PR to work on my code base after countless changes, I don't think this is how library updates are supposed to work? Every time I type poetry update and more than 20 libraries update, I don't remember ever having to change anything in my code just to update any library.
This PR is certainly introducing breaking changes but I'm unable to tell what and why exactly, although I investigated a lot here I couldn't even make my project to run with it.
By the way I woud love to be using sqlalchemy 2.0 at this point, and it is very sad that sqlmodel seems to be abandoned since August 2022, not a single update.
@farahats9
farahats9
commented
Jul 10, 2023

Copy link


Copy Markdown

ContributorAuthor
@FilipeMarch you are right there is an issue in ForwardRef handling and I am trying to investigate it when I have some free time. The issue appears more when you have large project with many relations between tables so that's why some people report it working fine and some don't.
I feel this is a priority to fix over the mypy linting, I will do my best to figure it out but any help is appreciated.
@50Bytes-dev
50Bytes-dev
commented
Jul 11, 2023•

edited

Loading

Copy link


Copy Markdown

I had an error Mapped[" in raw_annotation but I was able to solve it. Here is an example of my models, works fine. Maybe it will help you.
# models/__init__.py# need for alembicfrom . import (
company,
location,
client
)
# models/company.pyifTYPE_CHECKING:
frommodels.locationimportLocationfrommodels.clientimportClientclassCompany(BaseModel, CompanyBase, table=True):
locations: List["Location"] =Relationship(back_populates="company")
clients: List["Client"] =Relationship(back_populates="company")
# models/location.pyfrommodels.companyimportCompanyclassLocation(BaseModel, OrderedList, LocationBase, table=True):
company_id: int=Field(default=None, foreign_key="company.id")
company: Company=Relationship(back_populates="locations")
# models/client.pyfrommodels.companyimportCompanyclassClient(ClientBase, BaseModel, table=True):
company_id: int=Field(default=None, nullable=False, foreign_key="company.id")
company: Company=Relationship(back_populates="clients")
@matanper
matanper
commented
Jul 20, 2023

Copy link


Copy Markdown

Hey @farahats9,
I'm getting error when using sqlalchemy relationship:
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
powers: List["Power"] =Relationship(
sa_relationship=relationship(back_populates='hero')
)
classPower(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
hero_id: int=Field(foreign_key="hero.id")
hero: "Hero"=Relationship(back_populates="powers")
NameError: name 'Power' is not defined
NameError: Could not de-stringify annotation 'Power'

any idea why?
@AntonDeMeester
AntonDeMeester
commented
Jul 24, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
@matanper I think the forward Relationship (with the Foreign Key) needs to be a proper class, and not a type string. Can you try this?
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
powers: List["Power"] =Relationship(
sa_relationship=relationship(back_populates='hero')
)
classPower(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
hero_id: int=Field(foreign_key="hero.id")
hero: Hero=Relationship(back_populates="powers") # <== No quotes around Hero
@peterlandry
peterlandry
commented
Jul 27, 2023

Copy link


Copy Markdown

Contributor
I've looked at this PR a bit to try cleaning up the linting errors. I've reduced most of them, but execute, get, and query are a little complicated because of the overloads. One thing I noticed in the process though is that sqlalchemy 2.0 has enough typing info to break mypy in query code. In VS code, given a sqlalchemy ORM model, if I do:
q=User.name=="spongebob"
q becomes an instance of ColumnElement[bool], while if User is a SQLModel, q goes straight to a bool as far as mypy is concerned.
This shows up as an error, but is correct at runtime. At runtime, we see an InstrumentedAttribute for the attribute, but mypy thinks it's a string. I'm going to look further at that, and in the meantime submit a pull request here that fixes the majority of the linting errors.
@honglei
honglei
commented
Jul 28, 2023

Copy link


Copy Markdown

Environment:
pydantic version: 1.10.11
python version: 3.11.4 (main, Jun 27 2023, 15:03:25) [GCC 8.3.0]
platform: Linux-4.19.0-amd64-desktop-x86_64-with-glibc2.28
optional deps. installed: ['devtools', 'email-validator', 'typing-extensions']
sqlalchemy: 2.0.19

pytest errors:
================================================================================================================================================================================================================== short test summary info ==================================================================================================================================================================================================================
FAILED tests/test_tutorial/test_create_db_and_table/test_tutorial001.py::test_create_db_and_table - assert 'BEGIN' in "No module named 'docs_src'\n"
FAILED tests/test_tutorial/test_fastapi/test_delete/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_limit_and_offset/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...Read Hero'}}}} == {'components'...Read Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_multiple_models/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_multiple_models/test_tutorial002.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_read_one/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...Read Hero'}}}} == {'components'...Read Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_relationships/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_response_model/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_session_with_dependency/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_simple_hero_api/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_teams/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_update/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
===================================================================================================================================================================================

@AntonDeMeesterAntonDeMeester
mentioned this pull request
Aug 1, 2023
 Closed

@tiangolotiangolo
added
the 
feature
New feature or request
label
Oct 22, 2023
@tiangolotiangolo
mentioned this pull request
Nov 18, 2023
 Merged

@tiangolo
tiangolo
commented
Nov 18, 2023

Copy link


Copy Markdown

Member
TL;DR: Good news, this is available in the just released SQLModel 0.0.12! 🎉
Awesome, thanks a lot for all the work @farahats9! 🚀
And thanks everyone for the comments, reviews, etc. ☕
@farahats9 I continued the work here with several things on top on PR #700. I took your commits so you're co-author of that PR (and contributor to SQLModel, check your badge here next to your username, or in https://github.com/tiangolo/sqlmodel/graphs/contributors 😎 ).
I did a few extra things:
  • I reverted a few things, e.g. altering the docs to workaround for a previous lack of support for forward references.
  • Added support/fix for forward reference errors.
  • Removed several sections/modules that are no longer necessary, as were mainly to add type annotations to SQLAlchemy things, but now with SQLAlchemy's new proper types that's no longer necessary and I can reuse SQLAlchemy's things directly.
  • Updated all needed tests, e.g. to support the latest FastAPI.
  • Updated all the generated selects to use the new SQLAlchemy types.
  • Updated the classes and type annotations with the new SQLAlchemy types, with some overloads to make sure they are compatible with SQLModel. E.g. select().where().
  • Redefine all functions that would take column operations to accept the types used by SQLModel, e.g. or_(Hero.name == "Deadpond") as SQLAlchemy's version would require Hero.name to be a Mapped[str] instead of a str.
  • I explored a lot if it was possible/sensible to use and encourage the new Mapped[X], but sadly that breaks type annotations for creating new instances and dataclass_transform, as then that would expect to receive values with Mapped[X], e.g. Mapped[str] instead of str. And it would also add complications to the Pydantic handling. So, this was a lot of work mostly exploratory, but not much ended in the PR.
It is now available in SQLModel 0.0.12. 🎉
As this is now included in the repo and released, I'll now close this PR. Thanks! 🍰
@gdoumenc
gdoumenc
commented
Nov 20, 2023•

edited

Loading

Copy link


Copy Markdown

Hope it can help : I had the same issue because the relationsship was always defined and I didn' decalre it Optional :
origin_plan: t.Optional["Plan"] <== Good

instead of :
origin_plan: "Plan" <== Wrong
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


4 more reviewers

@hongleihongleihonglei left review comments
@sbor23sbor23sbor23 left review comments
@UnoYakshiUnoYakshiUnoYakshi left review comments
@synodriversynodriversynodriver left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

featureNew feature or request

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

20 participants

@farahats9@mortendaehli@honglei@gabrielmbmb@heyaco@8thgencore@antont@me-v2@synodriver@k-s-t-i@FilipeMarch@podsvirov@50Bytes-dev@matanper@AntonDeMeester@peterlandry@tiangolo@gdoumenc@sbor23@UnoYakshi

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
migration to sqlalchemy 2.0 - #563
Closed
farahats9 wants to merge 23 commits into
fastapi:mainfrom 
farahats9:main
Closed
migration to sqlalchemy 2.0#563
farahats9 wants to merge 23 commits into
fastapi:mainfrom 
farahats9:main
Conversation
@farahats9
@farahats9farahats9
commented
Mar 7, 2023

Copy link


Copy Markdown

Contributor
made the required changes to pass the tests for sqlalchemy 2.0

python version had to be pumped to 3.7 instead of 3.6

removed sqlalchemy2-stubs since it is not compatible or required.
More testing in a real project scenario is welcome.
@github-actions
github-actionsBot
commented
Mar 7, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 60318b4 at: https://6406825e11187c06ceaecaf0--sqlmodel.netlify.app
@mortendaehli
mortendaehli
commented
Mar 24, 2023

Copy link


Copy Markdown

@farahats9, great! Do you think you would be able to sort out the failing tests? :)
@farahats9
farahats9
commented
Mar 24, 2023

Copy link


Copy Markdown

ContributorAuthor
I fixed some of the mypy linting errors but I am not good at these things so maybe someone can fix the rest.
@github-actions
github-actionsBot
commented
Mar 24, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit b48423f at: https://641d9f2736a7fa1c9568bdb7--sqlmodel.netlify.app
sbor23
sbor23
reviewed
Mar 27, 2023
View reviewed changes
Comment threadsqlmodel/sql/expression.py
Outdated

@honglei
honglei
commented
Mar 31, 2023

Copy link


Copy Markdown

I used this version in my project with env: Python3.11.2/sqlalchemy2.0.7/Win10/scheme="postgresql+asyncpg", it works all fine.
farahats9and others
added 2 commits
March 31, 2023 12:58
@github-actions
github-actionsBot
commented
Mar 31, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 9c219d9 at: https://6426bd112dc67b078b6e5298--sqlmodel.netlify.app
@github-actions
github-actionsBot
commented
Mar 31, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 0eee8e8 at: https://6426be4468302c086218a5bb--sqlmodel.netlify.app
sbor23
sbor23
reviewed
Mar 31, 2023
View reviewed changes
Comment threaddocs/contributing.md
Outdated

Comment threaddocs/tutorial/index.md
Comment thread.github/workflows/test.yml
Outdated

@github-actions
github-actionsBot
commented
Mar 31, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 050cf02 at: https://6426c1ce9df2a80bfd37ae93--sqlmodel.netlify.app
@gabrielmbmb
gabrielmbmb
commented
Apr 12, 2023

Copy link


Copy Markdown

@farahats9 someone created a PR on your fork for fixing the mypy issues
@heyaco
heyaco
commented
Apr 20, 2023

Copy link


Copy Markdown

This seems good to go just needs merging
@8thgencore
8thgencore
commented
Apr 20, 2023

Copy link


Copy Markdown

We are very much waiting for this change
@antont
antont
commented
Apr 21, 2023

Copy link


Copy Markdown

We are very much waiting for this change
Why do you wait, instead of taking it to use? I guess feedback from users would be helpful info also for the maintainer to know when to merge.
@me-v2
me-v2
commented
Apr 24, 2023

Copy link


Copy Markdown

🚀 👀👀
@synodriver
synodriver
commented
Apr 24, 2023

Copy link


Copy Markdown

Great! I would love to try this branch in my production environment.
@k-s-t-i
k-s-t-i
commented
Apr 24, 2023

Copy link


Copy Markdown

Would like to see this merged for asyncpg support. Asyncpg seems to greatly increase performance based on most simple benchmarks.
@farahats9
farahats9
commented
Apr 25, 2023•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Would like to see this merged for asyncpg support. Asyncpg seems to greatly increase performance based on most simple benchmarks.
@k-s-t-i You can currently use Asyncpg with sqlalchemy v1.4 and sqlmodel v0.0.8 with no issue in most cases. you will need to watch out for some problems like lazy loading and cascading deletes on the orm side.
@synodriver
synodriver
commented
Apr 25, 2023

Copy link


Copy Markdown

Great! I would love to try this branch in my production environment.
Update: This patch can work with asyncmy in a real project. Will try asyncpg later.
synodriver
synodriver
reviewed
Apr 25, 2023
View reviewed changes
Comment threadpyproject.toml
Outdated

@synodriver
synodriver
commented
Apr 26, 2023

Copy link


Copy Markdown

Great! I would love to try this branch in my production environment.
Update: This patch can work with asyncmy in a real project. Will try asyncpg later.
Update: asyncpg works too. Tested with a real project on both windows and linux. Notice: to make relationship work, you may use Relationship(back_populates="xxx", sa_relationship_kwargs={"lazy": 'selectin'}) where sa_relationship_kwargs is important.
UnoYakshi
UnoYakshi
reviewed
Apr 30, 2023
View reviewed changes
@UnoYakshiUnoYakshi
left a comment
•

edited

Loading

Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really would like to use it in my new project! Thank you for the PR!
Comment threadpyproject.toml
Outdated

Comment threadsqlmodel/engine/result.py
Comment threadsqlmodel/main.py
Comment threadsqlmodel/sql/expression.py
Comment threadsqlmodel/sql/expression.py
Comment threadsqlmodel/sql/expression.py
Comment threadsqlmodel/__init__.py
Comment threaddocs_src/tutorial/relationship_attributes/back_populates/tutorial003.py
@hongleihonglei
mentioned this pull request
Jun 7, 2023

Open

@honglei
honglei
commented
Jun 27, 2023

Copy link


Copy Markdown

Five months have been passed since the release of SQLalchemy2.0, https://www.sqlalchemy.org/blog/2023/01/26/sqlalchemy-2.0.0-released/
@antont
antont
commented
Jun 27, 2023

Copy link


Copy Markdown

Five months have been passed since the release of SQLalchemy2.0, https://www.sqlalchemy.org/blog/2023/01/26/sqlalchemy-2.0.0-released/
You can help by using this PR in your project(s) if you want, and reporting how well it works.
@50Bytes-dev
50Bytes-dev
commented
Jul 10, 2023

Copy link


Copy Markdown

Been working with this PR for almost a month now, works great. Thank you!
@FilipeMarch
FilipeMarch
commented
Jul 10, 2023•

edited

Loading

Copy link


Copy Markdown

Hello @FilipeMarch! I've just change order of classes definition and solve your error:
NameError: name 'Article' is not defined
Full example code (database.py):
fromtypingimportList, OptionalfromsqlmodelimportSQLModel, Field, Relationship, create_engineclassArticle(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
reviewer_links: List["ArticleReviewerLink"] =Relationship(
back_populates="article",
)
classArticleReviewerLink(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
article_id: Optional[int] =Field(default=None, foreign_key="article.id")
article: Article=Relationship(back_populates="reviewer_links")
sqlite_file_name="database.db"sqlite_url=f"sqlite:///{sqlite_file_name}"engine=create_engine(sqlite_url, echo=True)
defcreate_db_and_tables():
SQLModel.metadata.create_all(engine)
if__name__=="__main__":
create_db_and_tables()
Sorry but changing the order of classes definition does not solve the error for me. At this point, I don't know what to do, but this branch is useless for me. I already have done what you mentioned, but the error persists. It always says if raiseerr and "Mapped[" in raw_annotation: # type: ignore TypeError: argument of type 'ForwardRef' is not iterable.
My project has about 30 tables at this point, and it has been running for quite 1 year. My project does not run as is with this branch, and I already tried dozens of things (like switching the order of tables, removing Relationships, removing a bunch of tables) and I can't make it to work in any possible way. Errors keep coming non stop. I can deactivate the Relationships, move tables up or down, it does not matter, new errors come every time. I just feel wasting my time because obviously there is something wrong, but I can't stop the errors until I remove all tables from the code?
The error messages are not helpful at all. And they only happen in this specific PR.
I understand it may be working for a very simple case like you presented, but for me it is always showing these errors, no matter how much code I change:
py", line2479, inget_propertyreturnself._props[key]
KeyError: 'user'
or
annotation=eval(expression, base_globals, locals_)
File"<string>", line1, in<module>NameError: name'PostUserLike'isnotdefined
or
.py", line223, ineval_expressionraiseNameError(
NameError: Couldnotde-stringifyannotation'PostUserLike'
or
", line2342, in_extract_mapped_subtypeifraiseerrand"Mapped["inraw_annotation: # type: ignoreTypeError: argumentof type 'ForwardRef'isnotiterable
My current API is running perfectly with sqlmodel = "^0.0.8"
If someone wants to investigate the issue with me, I am willing to screenshare and show the project structure and try to make it to work; my Discord is filipemarch
But one thing I wonder, even if I were able to make this PR to work on my code base after countless changes, I don't think this is how library updates are supposed to work? Every time I type poetry update and more than 20 libraries update, I don't remember ever having to change anything in my code just to update any library.
This PR is certainly introducing breaking changes but I'm unable to tell what and why exactly, although I investigated a lot here I couldn't even make my project to run with it.
By the way I woud love to be using sqlalchemy 2.0 at this point, and it is very sad that sqlmodel seems to be abandoned since August 2022, not a single update.
@farahats9
farahats9
commented
Jul 10, 2023

Copy link


Copy Markdown

ContributorAuthor
@FilipeMarch you are right there is an issue in ForwardRef handling and I am trying to investigate it when I have some free time. The issue appears more when you have large project with many relations between tables so that's why some people report it working fine and some don't.
I feel this is a priority to fix over the mypy linting, I will do my best to figure it out but any help is appreciated.
@50Bytes-dev
50Bytes-dev
commented
Jul 11, 2023•

edited

Loading

Copy link


Copy Markdown

I had an error Mapped[" in raw_annotation but I was able to solve it. Here is an example of my models, works fine. Maybe it will help you.
# models/__init__.py# need for alembicfrom . import (
company,
location,
client
)
# models/company.pyifTYPE_CHECKING:
frommodels.locationimportLocationfrommodels.clientimportClientclassCompany(BaseModel, CompanyBase, table=True):
locations: List["Location"] =Relationship(back_populates="company")
clients: List["Client"] =Relationship(back_populates="company")
# models/location.pyfrommodels.companyimportCompanyclassLocation(BaseModel, OrderedList, LocationBase, table=True):
company_id: int=Field(default=None, foreign_key="company.id")
company: Company=Relationship(back_populates="locations")
# models/client.pyfrommodels.companyimportCompanyclassClient(ClientBase, BaseModel, table=True):
company_id: int=Field(default=None, nullable=False, foreign_key="company.id")
company: Company=Relationship(back_populates="clients")
@matanper
matanper
commented
Jul 20, 2023

Copy link


Copy Markdown

Hey @farahats9,
I'm getting error when using sqlalchemy relationship:
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
powers: List["Power"] =Relationship(
sa_relationship=relationship(back_populates='hero')
)
classPower(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
hero_id: int=Field(foreign_key="hero.id")
hero: "Hero"=Relationship(back_populates="powers")
NameError: name 'Power' is not defined
NameError: Could not de-stringify annotation 'Power'

any idea why?
@AntonDeMeester
AntonDeMeester
commented
Jul 24, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
@matanper I think the forward Relationship (with the Foreign Key) needs to be a proper class, and not a type string. Can you try this?
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
powers: List["Power"] =Relationship(
sa_relationship=relationship(back_populates='hero')
)
classPower(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
hero_id: int=Field(foreign_key="hero.id")
hero: Hero=Relationship(back_populates="powers") # <== No quotes around Hero
@peterlandry
peterlandry
commented
Jul 27, 2023

Copy link


Copy Markdown

Contributor
I've looked at this PR a bit to try cleaning up the linting errors. I've reduced most of them, but execute, get, and query are a little complicated because of the overloads. One thing I noticed in the process though is that sqlalchemy 2.0 has enough typing info to break mypy in query code. In VS code, given a sqlalchemy ORM model, if I do:
q=User.name=="spongebob"
q becomes an instance of ColumnElement[bool], while if User is a SQLModel, q goes straight to a bool as far as mypy is concerned.
This shows up as an error, but is correct at runtime. At runtime, we see an InstrumentedAttribute for the attribute, but mypy thinks it's a string. I'm going to look further at that, and in the meantime submit a pull request here that fixes the majority of the linting errors.
@honglei
honglei
commented
Jul 28, 2023

Copy link


Copy Markdown

Environment:
pydantic version: 1.10.11
python version: 3.11.4 (main, Jun 27 2023, 15:03:25) [GCC 8.3.0]
platform: Linux-4.19.0-amd64-desktop-x86_64-with-glibc2.28
optional deps. installed: ['devtools', 'email-validator', 'typing-extensions']
sqlalchemy: 2.0.19

pytest errors:
================================================================================================================================================================================================================== short test summary info ==================================================================================================================================================================================================================
FAILED tests/test_tutorial/test_create_db_and_table/test_tutorial001.py::test_create_db_and_table - assert 'BEGIN' in "No module named 'docs_src'\n"
FAILED tests/test_tutorial/test_fastapi/test_delete/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_limit_and_offset/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...Read Hero'}}}} == {'components'...Read Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_multiple_models/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_multiple_models/test_tutorial002.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_read_one/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...Read Hero'}}}} == {'components'...Read Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_relationships/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_response_model/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_session_with_dependency/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_simple_hero_api/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_teams/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_update/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
===================================================================================================================================================================================

@AntonDeMeesterAntonDeMeester
mentioned this pull request
Aug 1, 2023
 Closed

@tiangolotiangolo
added
the 
feature
New feature or request
label
Oct 22, 2023
@tiangolotiangolo
mentioned this pull request
Nov 18, 2023
 Merged

@tiangolo
tiangolo
commented
Nov 18, 2023

Copy link


Copy Markdown

Member
TL;DR: Good news, this is available in the just released SQLModel 0.0.12! 🎉
Awesome, thanks a lot for all the work @farahats9! 🚀
And thanks everyone for the comments, reviews, etc. ☕
@farahats9 I continued the work here with several things on top on PR #700. I took your commits so you're co-author of that PR (and contributor to SQLModel, check your badge here next to your username, or in https://github.com/tiangolo/sqlmodel/graphs/contributors 😎 ).
I did a few extra things:
  • I reverted a few things, e.g. altering the docs to workaround for a previous lack of support for forward references.
  • Added support/fix for forward reference errors.
  • Removed several sections/modules that are no longer necessary, as were mainly to add type annotations to SQLAlchemy things, but now with SQLAlchemy's new proper types that's no longer necessary and I can reuse SQLAlchemy's things directly.
  • Updated all needed tests, e.g. to support the latest FastAPI.
  • Updated all the generated selects to use the new SQLAlchemy types.
  • Updated the classes and type annotations with the new SQLAlchemy types, with some overloads to make sure they are compatible with SQLModel. E.g. select().where().
  • Redefine all functions that would take column operations to accept the types used by SQLModel, e.g. or_(Hero.name == "Deadpond") as SQLAlchemy's version would require Hero.name to be a Mapped[str] instead of a str.
  • I explored a lot if it was possible/sensible to use and encourage the new Mapped[X], but sadly that breaks type annotations for creating new instances and dataclass_transform, as then that would expect to receive values with Mapped[X], e.g. Mapped[str] instead of str. And it would also add complications to the Pydantic handling. So, this was a lot of work mostly exploratory, but not much ended in the PR.
It is now available in SQLModel 0.0.12. 🎉
As this is now included in the repo and released, I'll now close this PR. Thanks! 🍰
@gdoumenc
gdoumenc
commented
Nov 20, 2023•

edited

Loading

Copy link


Copy Markdown

Hope it can help : I had the same issue because the relationsship was always defined and I didn' decalre it Optional :
origin_plan: t.Optional["Plan"] <== Good

instead of :
origin_plan: "Plan" <== Wrong
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


4 more reviewers

@hongleihongleihonglei left review comments
@sbor23sbor23sbor23 left review comments
@UnoYakshiUnoYakshiUnoYakshi left review comments
@synodriversynodriversynodriver left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

featureNew feature or request

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

20 participants

@farahats9@mortendaehli@honglei@gabrielmbmb@heyaco@8thgencore@antont@me-v2@synodriver@k-s-t-i@FilipeMarch@podsvirov@50Bytes-dev@matanper@AntonDeMeester@peterlandry@tiangolo@gdoumenc@sbor23@UnoYakshi

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
migration to sqlalchemy 2.0 - #563
Closed
farahats9 wants to merge 23 commits into
fastapi:mainfrom 
farahats9:main
Closed
migration to sqlalchemy 2.0#563
farahats9 wants to merge 23 commits into
fastapi:mainfrom 
farahats9:main
Conversation
@farahats9
@farahats9farahats9
commented
Mar 7, 2023

Copy link


Copy Markdown

Contributor
made the required changes to pass the tests for sqlalchemy 2.0

python version had to be pumped to 3.7 instead of 3.6

removed sqlalchemy2-stubs since it is not compatible or required.
More testing in a real project scenario is welcome.
@github-actions
github-actionsBot
commented
Mar 7, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 60318b4 at: https://6406825e11187c06ceaecaf0--sqlmodel.netlify.app
@mortendaehli
mortendaehli
commented
Mar 24, 2023

Copy link


Copy Markdown

@farahats9, great! Do you think you would be able to sort out the failing tests? :)
@farahats9
farahats9
commented
Mar 24, 2023

Copy link


Copy Markdown

ContributorAuthor
I fixed some of the mypy linting errors but I am not good at these things so maybe someone can fix the rest.
@github-actions
github-actionsBot
commented
Mar 24, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit b48423f at: https://641d9f2736a7fa1c9568bdb7--sqlmodel.netlify.app
sbor23
sbor23
reviewed
Mar 27, 2023
View reviewed changes
Comment threadsqlmodel/sql/expression.py
Outdated

@honglei
honglei
commented
Mar 31, 2023

Copy link


Copy Markdown

I used this version in my project with env: Python3.11.2/sqlalchemy2.0.7/Win10/scheme="postgresql+asyncpg", it works all fine.
farahats9and others
added 2 commits
March 31, 2023 12:58
@github-actions
github-actionsBot
commented
Mar 31, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 9c219d9 at: https://6426bd112dc67b078b6e5298--sqlmodel.netlify.app
@github-actions
github-actionsBot
commented
Mar 31, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 0eee8e8 at: https://6426be4468302c086218a5bb--sqlmodel.netlify.app
sbor23
sbor23
reviewed
Mar 31, 2023
View reviewed changes
Comment threaddocs/contributing.md
Outdated

Comment threaddocs/tutorial/index.md
Comment thread.github/workflows/test.yml
Outdated

@github-actions
github-actionsBot
commented
Mar 31, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 050cf02 at: https://6426c1ce9df2a80bfd37ae93--sqlmodel.netlify.app
@gabrielmbmb
gabrielmbmb
commented
Apr 12, 2023

Copy link


Copy Markdown

@farahats9 someone created a PR on your fork for fixing the mypy issues
@heyaco
heyaco
commented
Apr 20, 2023

Copy link


Copy Markdown

This seems good to go just needs merging
@8thgencore
8thgencore
commented
Apr 20, 2023

Copy link


Copy Markdown

We are very much waiting for this change
@antont
antont
commented
Apr 21, 2023

Copy link


Copy Markdown

We are very much waiting for this change
Why do you wait, instead of taking it to use? I guess feedback from users would be helpful info also for the maintainer to know when to merge.
@me-v2
me-v2
commented
Apr 24, 2023

Copy link


Copy Markdown

🚀 👀👀
@synodriver
synodriver
commented
Apr 24, 2023

Copy link


Copy Markdown

Great! I would love to try this branch in my production environment.
@k-s-t-i
k-s-t-i
commented
Apr 24, 2023

Copy link


Copy Markdown

Would like to see this merged for asyncpg support. Asyncpg seems to greatly increase performance based on most simple benchmarks.
@farahats9
farahats9
commented
Apr 25, 2023•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Would like to see this merged for asyncpg support. Asyncpg seems to greatly increase performance based on most simple benchmarks.
@k-s-t-i You can currently use Asyncpg with sqlalchemy v1.4 and sqlmodel v0.0.8 with no issue in most cases. you will need to watch out for some problems like lazy loading and cascading deletes on the orm side.
@synodriver
synodriver
commented
Apr 25, 2023

Copy link


Copy Markdown

Great! I would love to try this branch in my production environment.
Update: This patch can work with asyncmy in a real project. Will try asyncpg later.
synodriver
synodriver
reviewed
Apr 25, 2023
View reviewed changes
Comment threadpyproject.toml
Outdated

@synodriver
synodriver
commented
Apr 26, 2023

Copy link


Copy Markdown

Great! I would love to try this branch in my production environment.
Update: This patch can work with asyncmy in a real project. Will try asyncpg later.
Update: asyncpg works too. Tested with a real project on both windows and linux. Notice: to make relationship work, you may use Relationship(back_populates="xxx", sa_relationship_kwargs={"lazy": 'selectin'}) where sa_relationship_kwargs is important.
UnoYakshi
UnoYakshi
reviewed
Apr 30, 2023
View reviewed changes
@UnoYakshiUnoYakshi
left a comment
•

edited

Loading

Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really would like to use it in my new project! Thank you for the PR!
Comment threadpyproject.toml
Outdated

Comment threadsqlmodel/engine/result.py
Comment threadsqlmodel/main.py
Comment threadsqlmodel/sql/expression.py
Comment threadsqlmodel/sql/expression.py
Comment threadsqlmodel/sql/expression.py
Comment threadsqlmodel/__init__.py
Comment threaddocs_src/tutorial/relationship_attributes/back_populates/tutorial003.py
@hongleihonglei
mentioned this pull request
Jun 7, 2023

Open

@honglei
honglei
commented
Jun 27, 2023

Copy link


Copy Markdown

Five months have been passed since the release of SQLalchemy2.0, https://www.sqlalchemy.org/blog/2023/01/26/sqlalchemy-2.0.0-released/
@antont
antont
commented
Jun 27, 2023

Copy link


Copy Markdown

Five months have been passed since the release of SQLalchemy2.0, https://www.sqlalchemy.org/blog/2023/01/26/sqlalchemy-2.0.0-released/
You can help by using this PR in your project(s) if you want, and reporting how well it works.
@50Bytes-dev
50Bytes-dev
commented
Jul 10, 2023

Copy link


Copy Markdown

Been working with this PR for almost a month now, works great. Thank you!
@FilipeMarch
FilipeMarch
commented
Jul 10, 2023•

edited

Loading

Copy link


Copy Markdown

Hello @FilipeMarch! I've just change order of classes definition and solve your error:
NameError: name 'Article' is not defined
Full example code (database.py):
fromtypingimportList, OptionalfromsqlmodelimportSQLModel, Field, Relationship, create_engineclassArticle(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
reviewer_links: List["ArticleReviewerLink"] =Relationship(
back_populates="article",
)
classArticleReviewerLink(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
article_id: Optional[int] =Field(default=None, foreign_key="article.id")
article: Article=Relationship(back_populates="reviewer_links")
sqlite_file_name="database.db"sqlite_url=f"sqlite:///{sqlite_file_name}"engine=create_engine(sqlite_url, echo=True)
defcreate_db_and_tables():
SQLModel.metadata.create_all(engine)
if__name__=="__main__":
create_db_and_tables()
Sorry but changing the order of classes definition does not solve the error for me. At this point, I don't know what to do, but this branch is useless for me. I already have done what you mentioned, but the error persists. It always says if raiseerr and "Mapped[" in raw_annotation: # type: ignore TypeError: argument of type 'ForwardRef' is not iterable.
My project has about 30 tables at this point, and it has been running for quite 1 year. My project does not run as is with this branch, and I already tried dozens of things (like switching the order of tables, removing Relationships, removing a bunch of tables) and I can't make it to work in any possible way. Errors keep coming non stop. I can deactivate the Relationships, move tables up or down, it does not matter, new errors come every time. I just feel wasting my time because obviously there is something wrong, but I can't stop the errors until I remove all tables from the code?
The error messages are not helpful at all. And they only happen in this specific PR.
I understand it may be working for a very simple case like you presented, but for me it is always showing these errors, no matter how much code I change:
py", line2479, inget_propertyreturnself._props[key]
KeyError: 'user'
or
annotation=eval(expression, base_globals, locals_)
File"<string>", line1, in<module>NameError: name'PostUserLike'isnotdefined
or
.py", line223, ineval_expressionraiseNameError(
NameError: Couldnotde-stringifyannotation'PostUserLike'
or
", line2342, in_extract_mapped_subtypeifraiseerrand"Mapped["inraw_annotation: # type: ignoreTypeError: argumentof type 'ForwardRef'isnotiterable
My current API is running perfectly with sqlmodel = "^0.0.8"
If someone wants to investigate the issue with me, I am willing to screenshare and show the project structure and try to make it to work; my Discord is filipemarch
But one thing I wonder, even if I were able to make this PR to work on my code base after countless changes, I don't think this is how library updates are supposed to work? Every time I type poetry update and more than 20 libraries update, I don't remember ever having to change anything in my code just to update any library.
This PR is certainly introducing breaking changes but I'm unable to tell what and why exactly, although I investigated a lot here I couldn't even make my project to run with it.
By the way I woud love to be using sqlalchemy 2.0 at this point, and it is very sad that sqlmodel seems to be abandoned since August 2022, not a single update.
@farahats9
farahats9
commented
Jul 10, 2023

Copy link


Copy Markdown

ContributorAuthor
@FilipeMarch you are right there is an issue in ForwardRef handling and I am trying to investigate it when I have some free time. The issue appears more when you have large project with many relations between tables so that's why some people report it working fine and some don't.
I feel this is a priority to fix over the mypy linting, I will do my best to figure it out but any help is appreciated.
@50Bytes-dev
50Bytes-dev
commented
Jul 11, 2023•

edited

Loading

Copy link


Copy Markdown

I had an error Mapped[" in raw_annotation but I was able to solve it. Here is an example of my models, works fine. Maybe it will help you.
# models/__init__.py# need for alembicfrom . import (
company,
location,
client
)
# models/company.pyifTYPE_CHECKING:
frommodels.locationimportLocationfrommodels.clientimportClientclassCompany(BaseModel, CompanyBase, table=True):
locations: List["Location"] =Relationship(back_populates="company")
clients: List["Client"] =Relationship(back_populates="company")
# models/location.pyfrommodels.companyimportCompanyclassLocation(BaseModel, OrderedList, LocationBase, table=True):
company_id: int=Field(default=None, foreign_key="company.id")
company: Company=Relationship(back_populates="locations")
# models/client.pyfrommodels.companyimportCompanyclassClient(ClientBase, BaseModel, table=True):
company_id: int=Field(default=None, nullable=False, foreign_key="company.id")
company: Company=Relationship(back_populates="clients")
@matanper
matanper
commented
Jul 20, 2023

Copy link


Copy Markdown

Hey @farahats9,
I'm getting error when using sqlalchemy relationship:
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
powers: List["Power"] =Relationship(
sa_relationship=relationship(back_populates='hero')
)
classPower(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
hero_id: int=Field(foreign_key="hero.id")
hero: "Hero"=Relationship(back_populates="powers")
NameError: name 'Power' is not defined
NameError: Could not de-stringify annotation 'Power'

any idea why?
@AntonDeMeester
AntonDeMeester
commented
Jul 24, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
@matanper I think the forward Relationship (with the Foreign Key) needs to be a proper class, and not a type string. Can you try this?
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
powers: List["Power"] =Relationship(
sa_relationship=relationship(back_populates='hero')
)
classPower(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
hero_id: int=Field(foreign_key="hero.id")
hero: Hero=Relationship(back_populates="powers") # <== No quotes around Hero
@peterlandry
peterlandry
commented
Jul 27, 2023

Copy link


Copy Markdown

Contributor
I've looked at this PR a bit to try cleaning up the linting errors. I've reduced most of them, but execute, get, and query are a little complicated because of the overloads. One thing I noticed in the process though is that sqlalchemy 2.0 has enough typing info to break mypy in query code. In VS code, given a sqlalchemy ORM model, if I do:
q=User.name=="spongebob"
q becomes an instance of ColumnElement[bool], while if User is a SQLModel, q goes straight to a bool as far as mypy is concerned.
This shows up as an error, but is correct at runtime. At runtime, we see an InstrumentedAttribute for the attribute, but mypy thinks it's a string. I'm going to look further at that, and in the meantime submit a pull request here that fixes the majority of the linting errors.
@honglei
honglei
commented
Jul 28, 2023

Copy link


Copy Markdown

Environment:
pydantic version: 1.10.11
python version: 3.11.4 (main, Jun 27 2023, 15:03:25) [GCC 8.3.0]
platform: Linux-4.19.0-amd64-desktop-x86_64-with-glibc2.28
optional deps. installed: ['devtools', 'email-validator', 'typing-extensions']
sqlalchemy: 2.0.19

pytest errors:
================================================================================================================================================================================================================== short test summary info ==================================================================================================================================================================================================================
FAILED tests/test_tutorial/test_create_db_and_table/test_tutorial001.py::test_create_db_and_table - assert 'BEGIN' in "No module named 'docs_src'\n"
FAILED tests/test_tutorial/test_fastapi/test_delete/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_limit_and_offset/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...Read Hero'}}}} == {'components'...Read Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_multiple_models/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_multiple_models/test_tutorial002.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_read_one/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...Read Hero'}}}} == {'components'...Read Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_relationships/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_response_model/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_session_with_dependency/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_simple_hero_api/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_teams/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_update/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
===================================================================================================================================================================================

@AntonDeMeesterAntonDeMeester
mentioned this pull request
Aug 1, 2023
 Closed

@tiangolotiangolo
added
the 
feature
New feature or request
label
Oct 22, 2023
@tiangolotiangolo
mentioned this pull request
Nov 18, 2023
 Merged

@tiangolo
tiangolo
commented
Nov 18, 2023

Copy link


Copy Markdown

Member
TL;DR: Good news, this is available in the just released SQLModel 0.0.12! 🎉
Awesome, thanks a lot for all the work @farahats9! 🚀
And thanks everyone for the comments, reviews, etc. ☕
@farahats9 I continued the work here with several things on top on PR #700. I took your commits so you're co-author of that PR (and contributor to SQLModel, check your badge here next to your username, or in https://github.com/tiangolo/sqlmodel/graphs/contributors 😎 ).
I did a few extra things:
  • I reverted a few things, e.g. altering the docs to workaround for a previous lack of support for forward references.
  • Added support/fix for forward reference errors.
  • Removed several sections/modules that are no longer necessary, as were mainly to add type annotations to SQLAlchemy things, but now with SQLAlchemy's new proper types that's no longer necessary and I can reuse SQLAlchemy's things directly.
  • Updated all needed tests, e.g. to support the latest FastAPI.
  • Updated all the generated selects to use the new SQLAlchemy types.
  • Updated the classes and type annotations with the new SQLAlchemy types, with some overloads to make sure they are compatible with SQLModel. E.g. select().where().
  • Redefine all functions that would take column operations to accept the types used by SQLModel, e.g. or_(Hero.name == "Deadpond") as SQLAlchemy's version would require Hero.name to be a Mapped[str] instead of a str.
  • I explored a lot if it was possible/sensible to use and encourage the new Mapped[X], but sadly that breaks type annotations for creating new instances and dataclass_transform, as then that would expect to receive values with Mapped[X], e.g. Mapped[str] instead of str. And it would also add complications to the Pydantic handling. So, this was a lot of work mostly exploratory, but not much ended in the PR.
It is now available in SQLModel 0.0.12. 🎉
As this is now included in the repo and released, I'll now close this PR. Thanks! 🍰
@gdoumenc
gdoumenc
commented
Nov 20, 2023•

edited

Loading

Copy link


Copy Markdown

Hope it can help : I had the same issue because the relationsship was always defined and I didn' decalre it Optional :
origin_plan: t.Optional["Plan"] <== Good

instead of :
origin_plan: "Plan" <== Wrong
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


4 more reviewers

@hongleihongleihonglei left review comments
@sbor23sbor23sbor23 left review comments
@UnoYakshiUnoYakshiUnoYakshi left review comments
@synodriversynodriversynodriver left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

featureNew feature or request

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

20 participants

@farahats9@mortendaehli@honglei@gabrielmbmb@heyaco@8thgencore@antont@me-v2@synodriver@k-s-t-i@FilipeMarch@podsvirov@50Bytes-dev@matanper@AntonDeMeester@peterlandry@tiangolo@gdoumenc@sbor23@UnoYakshi

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
migration to sqlalchemy 2.0 - #563
Closed
farahats9 wants to merge 23 commits into
fastapi:mainfrom 
farahats9:main
Closed
migration to sqlalchemy 2.0#563
farahats9 wants to merge 23 commits into
fastapi:mainfrom 
farahats9:main
Conversation
@farahats9
@farahats9farahats9
commented
Mar 7, 2023

Copy link


Copy Markdown

Contributor
made the required changes to pass the tests for sqlalchemy 2.0

python version had to be pumped to 3.7 instead of 3.6

removed sqlalchemy2-stubs since it is not compatible or required.
More testing in a real project scenario is welcome.
@github-actions
github-actionsBot
commented
Mar 7, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 60318b4 at: https://6406825e11187c06ceaecaf0--sqlmodel.netlify.app
@mortendaehli
mortendaehli
commented
Mar 24, 2023

Copy link


Copy Markdown

@farahats9, great! Do you think you would be able to sort out the failing tests? :)
@farahats9
farahats9
commented
Mar 24, 2023

Copy link


Copy Markdown

ContributorAuthor
I fixed some of the mypy linting errors but I am not good at these things so maybe someone can fix the rest.
@github-actions
github-actionsBot
commented
Mar 24, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit b48423f at: https://641d9f2736a7fa1c9568bdb7--sqlmodel.netlify.app
sbor23
sbor23
reviewed
Mar 27, 2023
View reviewed changes
Comment threadsqlmodel/sql/expression.py
Outdated

@honglei
honglei
commented
Mar 31, 2023

Copy link


Copy Markdown

I used this version in my project with env: Python3.11.2/sqlalchemy2.0.7/Win10/scheme="postgresql+asyncpg", it works all fine.
farahats9and others
added 2 commits
March 31, 2023 12:58
@github-actions
github-actionsBot
commented
Mar 31, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 9c219d9 at: https://6426bd112dc67b078b6e5298--sqlmodel.netlify.app
@github-actions
github-actionsBot
commented
Mar 31, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 0eee8e8 at: https://6426be4468302c086218a5bb--sqlmodel.netlify.app
sbor23
sbor23
reviewed
Mar 31, 2023
View reviewed changes
Comment threaddocs/contributing.md
Outdated

Comment threaddocs/tutorial/index.md
Comment thread.github/workflows/test.yml
Outdated

@github-actions
github-actionsBot
commented
Mar 31, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 050cf02 at: https://6426c1ce9df2a80bfd37ae93--sqlmodel.netlify.app
@gabrielmbmb
gabrielmbmb
commented
Apr 12, 2023

Copy link


Copy Markdown

@farahats9 someone created a PR on your fork for fixing the mypy issues
@heyaco
heyaco
commented
Apr 20, 2023

Copy link


Copy Markdown

This seems good to go just needs merging
@8thgencore
8thgencore
commented
Apr 20, 2023

Copy link


Copy Markdown

We are very much waiting for this change
@antont
antont
commented
Apr 21, 2023

Copy link


Copy Markdown

We are very much waiting for this change
Why do you wait, instead of taking it to use? I guess feedback from users would be helpful info also for the maintainer to know when to merge.
@me-v2
me-v2
commented
Apr 24, 2023

Copy link


Copy Markdown

🚀 👀👀
@synodriver
synodriver
commented
Apr 24, 2023

Copy link


Copy Markdown

Great! I would love to try this branch in my production environment.
@k-s-t-i
k-s-t-i
commented
Apr 24, 2023

Copy link


Copy Markdown

Would like to see this merged for asyncpg support. Asyncpg seems to greatly increase performance based on most simple benchmarks.
@farahats9
farahats9
commented
Apr 25, 2023•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Would like to see this merged for asyncpg support. Asyncpg seems to greatly increase performance based on most simple benchmarks.
@k-s-t-i You can currently use Asyncpg with sqlalchemy v1.4 and sqlmodel v0.0.8 with no issue in most cases. you will need to watch out for some problems like lazy loading and cascading deletes on the orm side.
@synodriver
synodriver
commented
Apr 25, 2023

Copy link


Copy Markdown

Great! I would love to try this branch in my production environment.
Update: This patch can work with asyncmy in a real project. Will try asyncpg later.
synodriver
synodriver
reviewed
Apr 25, 2023
View reviewed changes
Comment threadpyproject.toml
Outdated

@synodriver
synodriver
commented
Apr 26, 2023

Copy link


Copy Markdown

Great! I would love to try this branch in my production environment.
Update: This patch can work with asyncmy in a real project. Will try asyncpg later.
Update: asyncpg works too. Tested with a real project on both windows and linux. Notice: to make relationship work, you may use Relationship(back_populates="xxx", sa_relationship_kwargs={"lazy": 'selectin'}) where sa_relationship_kwargs is important.
UnoYakshi
UnoYakshi
reviewed
Apr 30, 2023
View reviewed changes
@UnoYakshiUnoYakshi
left a comment
•

edited

Loading

Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really would like to use it in my new project! Thank you for the PR!
Comment threadpyproject.toml
Outdated

Comment threadsqlmodel/engine/result.py
Comment threadsqlmodel/main.py
Comment threadsqlmodel/sql/expression.py
Comment threadsqlmodel/sql/expression.py
Comment threadsqlmodel/sql/expression.py
Comment threadsqlmodel/__init__.py
Comment threaddocs_src/tutorial/relationship_attributes/back_populates/tutorial003.py
@hongleihonglei
mentioned this pull request
Jun 7, 2023

Open

@honglei
honglei
commented
Jun 27, 2023

Copy link


Copy Markdown

Five months have been passed since the release of SQLalchemy2.0, https://www.sqlalchemy.org/blog/2023/01/26/sqlalchemy-2.0.0-released/
@antont
antont
commented
Jun 27, 2023

Copy link


Copy Markdown

Five months have been passed since the release of SQLalchemy2.0, https://www.sqlalchemy.org/blog/2023/01/26/sqlalchemy-2.0.0-released/
You can help by using this PR in your project(s) if you want, and reporting how well it works.
@50Bytes-dev
50Bytes-dev
commented
Jul 10, 2023

Copy link


Copy Markdown

Been working with this PR for almost a month now, works great. Thank you!
@FilipeMarch
FilipeMarch
commented
Jul 10, 2023•

edited

Loading

Copy link


Copy Markdown

Hello @FilipeMarch! I've just change order of classes definition and solve your error:
NameError: name 'Article' is not defined
Full example code (database.py):
fromtypingimportList, OptionalfromsqlmodelimportSQLModel, Field, Relationship, create_engineclassArticle(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
reviewer_links: List["ArticleReviewerLink"] =Relationship(
back_populates="article",
)
classArticleReviewerLink(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
article_id: Optional[int] =Field(default=None, foreign_key="article.id")
article: Article=Relationship(back_populates="reviewer_links")
sqlite_file_name="database.db"sqlite_url=f"sqlite:///{sqlite_file_name}"engine=create_engine(sqlite_url, echo=True)
defcreate_db_and_tables():
SQLModel.metadata.create_all(engine)
if__name__=="__main__":
create_db_and_tables()
Sorry but changing the order of classes definition does not solve the error for me. At this point, I don't know what to do, but this branch is useless for me. I already have done what you mentioned, but the error persists. It always says if raiseerr and "Mapped[" in raw_annotation: # type: ignore TypeError: argument of type 'ForwardRef' is not iterable.
My project has about 30 tables at this point, and it has been running for quite 1 year. My project does not run as is with this branch, and I already tried dozens of things (like switching the order of tables, removing Relationships, removing a bunch of tables) and I can't make it to work in any possible way. Errors keep coming non stop. I can deactivate the Relationships, move tables up or down, it does not matter, new errors come every time. I just feel wasting my time because obviously there is something wrong, but I can't stop the errors until I remove all tables from the code?
The error messages are not helpful at all. And they only happen in this specific PR.
I understand it may be working for a very simple case like you presented, but for me it is always showing these errors, no matter how much code I change:
py", line2479, inget_propertyreturnself._props[key]
KeyError: 'user'
or
annotation=eval(expression, base_globals, locals_)
File"<string>", line1, in<module>NameError: name'PostUserLike'isnotdefined
or
.py", line223, ineval_expressionraiseNameError(
NameError: Couldnotde-stringifyannotation'PostUserLike'
or
", line2342, in_extract_mapped_subtypeifraiseerrand"Mapped["inraw_annotation: # type: ignoreTypeError: argumentof type 'ForwardRef'isnotiterable
My current API is running perfectly with sqlmodel = "^0.0.8"
If someone wants to investigate the issue with me, I am willing to screenshare and show the project structure and try to make it to work; my Discord is filipemarch
But one thing I wonder, even if I were able to make this PR to work on my code base after countless changes, I don't think this is how library updates are supposed to work? Every time I type poetry update and more than 20 libraries update, I don't remember ever having to change anything in my code just to update any library.
This PR is certainly introducing breaking changes but I'm unable to tell what and why exactly, although I investigated a lot here I couldn't even make my project to run with it.
By the way I woud love to be using sqlalchemy 2.0 at this point, and it is very sad that sqlmodel seems to be abandoned since August 2022, not a single update.
@farahats9
farahats9
commented
Jul 10, 2023

Copy link


Copy Markdown

ContributorAuthor
@FilipeMarch you are right there is an issue in ForwardRef handling and I am trying to investigate it when I have some free time. The issue appears more when you have large project with many relations between tables so that's why some people report it working fine and some don't.
I feel this is a priority to fix over the mypy linting, I will do my best to figure it out but any help is appreciated.
@50Bytes-dev
50Bytes-dev
commented
Jul 11, 2023•

edited

Loading

Copy link


Copy Markdown

I had an error Mapped[" in raw_annotation but I was able to solve it. Here is an example of my models, works fine. Maybe it will help you.
# models/__init__.py# need for alembicfrom . import (
company,
location,
client
)
# models/company.pyifTYPE_CHECKING:
frommodels.locationimportLocationfrommodels.clientimportClientclassCompany(BaseModel, CompanyBase, table=True):
locations: List["Location"] =Relationship(back_populates="company")
clients: List["Client"] =Relationship(back_populates="company")
# models/location.pyfrommodels.companyimportCompanyclassLocation(BaseModel, OrderedList, LocationBase, table=True):
company_id: int=Field(default=None, foreign_key="company.id")
company: Company=Relationship(back_populates="locations")
# models/client.pyfrommodels.companyimportCompanyclassClient(ClientBase, BaseModel, table=True):
company_id: int=Field(default=None, nullable=False, foreign_key="company.id")
company: Company=Relationship(back_populates="clients")
@matanper
matanper
commented
Jul 20, 2023

Copy link


Copy Markdown

Hey @farahats9,
I'm getting error when using sqlalchemy relationship:
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
powers: List["Power"] =Relationship(
sa_relationship=relationship(back_populates='hero')
)
classPower(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
hero_id: int=Field(foreign_key="hero.id")
hero: "Hero"=Relationship(back_populates="powers")
NameError: name 'Power' is not defined
NameError: Could not de-stringify annotation 'Power'

any idea why?
@AntonDeMeester
AntonDeMeester
commented
Jul 24, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
@matanper I think the forward Relationship (with the Foreign Key) needs to be a proper class, and not a type string. Can you try this?
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
powers: List["Power"] =Relationship(
sa_relationship=relationship(back_populates='hero')
)
classPower(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
hero_id: int=Field(foreign_key="hero.id")
hero: Hero=Relationship(back_populates="powers") # <== No quotes around Hero
@peterlandry
peterlandry
commented
Jul 27, 2023

Copy link


Copy Markdown

Contributor
I've looked at this PR a bit to try cleaning up the linting errors. I've reduced most of them, but execute, get, and query are a little complicated because of the overloads. One thing I noticed in the process though is that sqlalchemy 2.0 has enough typing info to break mypy in query code. In VS code, given a sqlalchemy ORM model, if I do:
q=User.name=="spongebob"
q becomes an instance of ColumnElement[bool], while if User is a SQLModel, q goes straight to a bool as far as mypy is concerned.
This shows up as an error, but is correct at runtime. At runtime, we see an InstrumentedAttribute for the attribute, but mypy thinks it's a string. I'm going to look further at that, and in the meantime submit a pull request here that fixes the majority of the linting errors.
@honglei
honglei
commented
Jul 28, 2023

Copy link


Copy Markdown

Environment:
pydantic version: 1.10.11
python version: 3.11.4 (main, Jun 27 2023, 15:03:25) [GCC 8.3.0]
platform: Linux-4.19.0-amd64-desktop-x86_64-with-glibc2.28
optional deps. installed: ['devtools', 'email-validator', 'typing-extensions']
sqlalchemy: 2.0.19

pytest errors:
================================================================================================================================================================================================================== short test summary info ==================================================================================================================================================================================================================
FAILED tests/test_tutorial/test_create_db_and_table/test_tutorial001.py::test_create_db_and_table - assert 'BEGIN' in "No module named 'docs_src'\n"
FAILED tests/test_tutorial/test_fastapi/test_delete/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_limit_and_offset/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...Read Hero'}}}} == {'components'...Read Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_multiple_models/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_multiple_models/test_tutorial002.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_read_one/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...Read Hero'}}}} == {'components'...Read Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_relationships/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_response_model/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_session_with_dependency/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_simple_hero_api/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_teams/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_update/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
===================================================================================================================================================================================

@AntonDeMeesterAntonDeMeester
mentioned this pull request
Aug 1, 2023
 Closed

@tiangolotiangolo
added
the 
feature
New feature or request
label
Oct 22, 2023
@tiangolotiangolo
mentioned this pull request
Nov 18, 2023
 Merged

@tiangolo
tiangolo
commented
Nov 18, 2023

Copy link


Copy Markdown

Member
TL;DR: Good news, this is available in the just released SQLModel 0.0.12! 🎉
Awesome, thanks a lot for all the work @farahats9! 🚀
And thanks everyone for the comments, reviews, etc. ☕
@farahats9 I continued the work here with several things on top on PR #700. I took your commits so you're co-author of that PR (and contributor to SQLModel, check your badge here next to your username, or in https://github.com/tiangolo/sqlmodel/graphs/contributors 😎 ).
I did a few extra things:
  • I reverted a few things, e.g. altering the docs to workaround for a previous lack of support for forward references.
  • Added support/fix for forward reference errors.
  • Removed several sections/modules that are no longer necessary, as were mainly to add type annotations to SQLAlchemy things, but now with SQLAlchemy's new proper types that's no longer necessary and I can reuse SQLAlchemy's things directly.
  • Updated all needed tests, e.g. to support the latest FastAPI.
  • Updated all the generated selects to use the new SQLAlchemy types.
  • Updated the classes and type annotations with the new SQLAlchemy types, with some overloads to make sure they are compatible with SQLModel. E.g. select().where().
  • Redefine all functions that would take column operations to accept the types used by SQLModel, e.g. or_(Hero.name == "Deadpond") as SQLAlchemy's version would require Hero.name to be a Mapped[str] instead of a str.
  • I explored a lot if it was possible/sensible to use and encourage the new Mapped[X], but sadly that breaks type annotations for creating new instances and dataclass_transform, as then that would expect to receive values with Mapped[X], e.g. Mapped[str] instead of str. And it would also add complications to the Pydantic handling. So, this was a lot of work mostly exploratory, but not much ended in the PR.
It is now available in SQLModel 0.0.12. 🎉
As this is now included in the repo and released, I'll now close this PR. Thanks! 🍰
@gdoumenc
gdoumenc
commented
Nov 20, 2023•

edited

Loading

Copy link


Copy Markdown

Hope it can help : I had the same issue because the relationsship was always defined and I didn' decalre it Optional :
origin_plan: t.Optional["Plan"] <== Good

instead of :
origin_plan: "Plan" <== Wrong
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


4 more reviewers

@hongleihongleihonglei left review comments
@sbor23sbor23sbor23 left review comments
@UnoYakshiUnoYakshiUnoYakshi left review comments
@synodriversynodriversynodriver left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

featureNew feature or request

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

20 participants

@farahats9@mortendaehli@honglei@gabrielmbmb@heyaco@8thgencore@antont@me-v2@synodriver@k-s-t-i@FilipeMarch@podsvirov@50Bytes-dev@matanper@AntonDeMeester@peterlandry@tiangolo@gdoumenc@sbor23@UnoYakshi

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
migration to sqlalchemy 2.0 - #563
Closed
farahats9 wants to merge 23 commits into
fastapi:mainfrom 
farahats9:main
Closed
migration to sqlalchemy 2.0#563
farahats9 wants to merge 23 commits into
fastapi:mainfrom 
farahats9:main
Conversation
@farahats9
@farahats9farahats9
commented
Mar 7, 2023

Copy link


Copy Markdown

Contributor
made the required changes to pass the tests for sqlalchemy 2.0

python version had to be pumped to 3.7 instead of 3.6

removed sqlalchemy2-stubs since it is not compatible or required.
More testing in a real project scenario is welcome.
@github-actions
github-actionsBot
commented
Mar 7, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 60318b4 at: https://6406825e11187c06ceaecaf0--sqlmodel.netlify.app
@mortendaehli
mortendaehli
commented
Mar 24, 2023

Copy link


Copy Markdown

@farahats9, great! Do you think you would be able to sort out the failing tests? :)
@farahats9
farahats9
commented
Mar 24, 2023

Copy link


Copy Markdown

ContributorAuthor
I fixed some of the mypy linting errors but I am not good at these things so maybe someone can fix the rest.
@github-actions
github-actionsBot
commented
Mar 24, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit b48423f at: https://641d9f2736a7fa1c9568bdb7--sqlmodel.netlify.app
sbor23
sbor23
reviewed
Mar 27, 2023
View reviewed changes
Comment threadsqlmodel/sql/expression.py
Outdated

@honglei
honglei
commented
Mar 31, 2023

Copy link


Copy Markdown

I used this version in my project with env: Python3.11.2/sqlalchemy2.0.7/Win10/scheme="postgresql+asyncpg", it works all fine.
farahats9and others
added 2 commits
March 31, 2023 12:58
@github-actions
github-actionsBot
commented
Mar 31, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 9c219d9 at: https://6426bd112dc67b078b6e5298--sqlmodel.netlify.app
@github-actions
github-actionsBot
commented
Mar 31, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 0eee8e8 at: https://6426be4468302c086218a5bb--sqlmodel.netlify.app
sbor23
sbor23
reviewed
Mar 31, 2023
View reviewed changes
Comment threaddocs/contributing.md
Outdated

Comment threaddocs/tutorial/index.md
Comment thread.github/workflows/test.yml
Outdated

@github-actions
github-actionsBot
commented
Mar 31, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 050cf02 at: https://6426c1ce9df2a80bfd37ae93--sqlmodel.netlify.app
@gabrielmbmb
gabrielmbmb
commented
Apr 12, 2023

Copy link


Copy Markdown

@farahats9 someone created a PR on your fork for fixing the mypy issues
@heyaco
heyaco
commented
Apr 20, 2023

Copy link


Copy Markdown

This seems good to go just needs merging
@8thgencore
8thgencore
commented
Apr 20, 2023

Copy link


Copy Markdown

We are very much waiting for this change
@antont
antont
commented
Apr 21, 2023

Copy link


Copy Markdown

We are very much waiting for this change
Why do you wait, instead of taking it to use? I guess feedback from users would be helpful info also for the maintainer to know when to merge.
@me-v2
me-v2
commented
Apr 24, 2023

Copy link


Copy Markdown

🚀 👀👀
@synodriver
synodriver
commented
Apr 24, 2023

Copy link


Copy Markdown

Great! I would love to try this branch in my production environment.
@k-s-t-i
k-s-t-i
commented
Apr 24, 2023

Copy link


Copy Markdown

Would like to see this merged for asyncpg support. Asyncpg seems to greatly increase performance based on most simple benchmarks.
@farahats9
farahats9
commented
Apr 25, 2023•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Would like to see this merged for asyncpg support. Asyncpg seems to greatly increase performance based on most simple benchmarks.
@k-s-t-i You can currently use Asyncpg with sqlalchemy v1.4 and sqlmodel v0.0.8 with no issue in most cases. you will need to watch out for some problems like lazy loading and cascading deletes on the orm side.
@synodriver
synodriver
commented
Apr 25, 2023

Copy link


Copy Markdown

Great! I would love to try this branch in my production environment.
Update: This patch can work with asyncmy in a real project. Will try asyncpg later.
synodriver
synodriver
reviewed
Apr 25, 2023
View reviewed changes
Comment threadpyproject.toml
Outdated

@synodriver
synodriver
commented
Apr 26, 2023

Copy link


Copy Markdown

Great! I would love to try this branch in my production environment.
Update: This patch can work with asyncmy in a real project. Will try asyncpg later.
Update: asyncpg works too. Tested with a real project on both windows and linux. Notice: to make relationship work, you may use Relationship(back_populates="xxx", sa_relationship_kwargs={"lazy": 'selectin'}) where sa_relationship_kwargs is important.
UnoYakshi
UnoYakshi
reviewed
Apr 30, 2023
View reviewed changes
@UnoYakshiUnoYakshi
left a comment
•

edited

Loading

Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really would like to use it in my new project! Thank you for the PR!
Comment threadpyproject.toml
Outdated

Comment threadsqlmodel/engine/result.py
Comment threadsqlmodel/main.py
Comment threadsqlmodel/sql/expression.py
Comment threadsqlmodel/sql/expression.py
Comment threadsqlmodel/sql/expression.py
Comment threadsqlmodel/__init__.py
Comment threaddocs_src/tutorial/relationship_attributes/back_populates/tutorial003.py
@hongleihonglei
mentioned this pull request
Jun 7, 2023

Open

@honglei
honglei
commented
Jun 27, 2023

Copy link


Copy Markdown

Five months have been passed since the release of SQLalchemy2.0, https://www.sqlalchemy.org/blog/2023/01/26/sqlalchemy-2.0.0-released/
@antont
antont
commented
Jun 27, 2023

Copy link


Copy Markdown

Five months have been passed since the release of SQLalchemy2.0, https://www.sqlalchemy.org/blog/2023/01/26/sqlalchemy-2.0.0-released/
You can help by using this PR in your project(s) if you want, and reporting how well it works.
@50Bytes-dev
50Bytes-dev
commented
Jul 10, 2023

Copy link


Copy Markdown

Been working with this PR for almost a month now, works great. Thank you!
@FilipeMarch
FilipeMarch
commented
Jul 10, 2023•

edited

Loading

Copy link


Copy Markdown

Hello @FilipeMarch! I've just change order of classes definition and solve your error:
NameError: name 'Article' is not defined
Full example code (database.py):
fromtypingimportList, OptionalfromsqlmodelimportSQLModel, Field, Relationship, create_engineclassArticle(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
reviewer_links: List["ArticleReviewerLink"] =Relationship(
back_populates="article",
)
classArticleReviewerLink(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
article_id: Optional[int] =Field(default=None, foreign_key="article.id")
article: Article=Relationship(back_populates="reviewer_links")
sqlite_file_name="database.db"sqlite_url=f"sqlite:///{sqlite_file_name}"engine=create_engine(sqlite_url, echo=True)
defcreate_db_and_tables():
SQLModel.metadata.create_all(engine)
if__name__=="__main__":
create_db_and_tables()
Sorry but changing the order of classes definition does not solve the error for me. At this point, I don't know what to do, but this branch is useless for me. I already have done what you mentioned, but the error persists. It always says if raiseerr and "Mapped[" in raw_annotation: # type: ignore TypeError: argument of type 'ForwardRef' is not iterable.
My project has about 30 tables at this point, and it has been running for quite 1 year. My project does not run as is with this branch, and I already tried dozens of things (like switching the order of tables, removing Relationships, removing a bunch of tables) and I can't make it to work in any possible way. Errors keep coming non stop. I can deactivate the Relationships, move tables up or down, it does not matter, new errors come every time. I just feel wasting my time because obviously there is something wrong, but I can't stop the errors until I remove all tables from the code?
The error messages are not helpful at all. And they only happen in this specific PR.
I understand it may be working for a very simple case like you presented, but for me it is always showing these errors, no matter how much code I change:
py", line2479, inget_propertyreturnself._props[key]
KeyError: 'user'
or
annotation=eval(expression, base_globals, locals_)
File"<string>", line1, in<module>NameError: name'PostUserLike'isnotdefined
or
.py", line223, ineval_expressionraiseNameError(
NameError: Couldnotde-stringifyannotation'PostUserLike'
or
", line2342, in_extract_mapped_subtypeifraiseerrand"Mapped["inraw_annotation: # type: ignoreTypeError: argumentof type 'ForwardRef'isnotiterable
My current API is running perfectly with sqlmodel = "^0.0.8"
If someone wants to investigate the issue with me, I am willing to screenshare and show the project structure and try to make it to work; my Discord is filipemarch
But one thing I wonder, even if I were able to make this PR to work on my code base after countless changes, I don't think this is how library updates are supposed to work? Every time I type poetry update and more than 20 libraries update, I don't remember ever having to change anything in my code just to update any library.
This PR is certainly introducing breaking changes but I'm unable to tell what and why exactly, although I investigated a lot here I couldn't even make my project to run with it.
By the way I woud love to be using sqlalchemy 2.0 at this point, and it is very sad that sqlmodel seems to be abandoned since August 2022, not a single update.
@farahats9
farahats9
commented
Jul 10, 2023

Copy link


Copy Markdown

ContributorAuthor
@FilipeMarch you are right there is an issue in ForwardRef handling and I am trying to investigate it when I have some free time. The issue appears more when you have large project with many relations between tables so that's why some people report it working fine and some don't.
I feel this is a priority to fix over the mypy linting, I will do my best to figure it out but any help is appreciated.
@50Bytes-dev
50Bytes-dev
commented
Jul 11, 2023•

edited

Loading

Copy link


Copy Markdown

I had an error Mapped[" in raw_annotation but I was able to solve it. Here is an example of my models, works fine. Maybe it will help you.
# models/__init__.py# need for alembicfrom . import (
company,
location,
client
)
# models/company.pyifTYPE_CHECKING:
frommodels.locationimportLocationfrommodels.clientimportClientclassCompany(BaseModel, CompanyBase, table=True):
locations: List["Location"] =Relationship(back_populates="company")
clients: List["Client"] =Relationship(back_populates="company")
# models/location.pyfrommodels.companyimportCompanyclassLocation(BaseModel, OrderedList, LocationBase, table=True):
company_id: int=Field(default=None, foreign_key="company.id")
company: Company=Relationship(back_populates="locations")
# models/client.pyfrommodels.companyimportCompanyclassClient(ClientBase, BaseModel, table=True):
company_id: int=Field(default=None, nullable=False, foreign_key="company.id")
company: Company=Relationship(back_populates="clients")
@matanper
matanper
commented
Jul 20, 2023

Copy link


Copy Markdown

Hey @farahats9,
I'm getting error when using sqlalchemy relationship:
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
powers: List["Power"] =Relationship(
sa_relationship=relationship(back_populates='hero')
)
classPower(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
hero_id: int=Field(foreign_key="hero.id")
hero: "Hero"=Relationship(back_populates="powers")
NameError: name 'Power' is not defined
NameError: Could not de-stringify annotation 'Power'

any idea why?
@AntonDeMeester
AntonDeMeester
commented
Jul 24, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
@matanper I think the forward Relationship (with the Foreign Key) needs to be a proper class, and not a type string. Can you try this?
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
powers: List["Power"] =Relationship(
sa_relationship=relationship(back_populates='hero')
)
classPower(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
hero_id: int=Field(foreign_key="hero.id")
hero: Hero=Relationship(back_populates="powers") # <== No quotes around Hero
@peterlandry
peterlandry
commented
Jul 27, 2023

Copy link


Copy Markdown

Contributor
I've looked at this PR a bit to try cleaning up the linting errors. I've reduced most of them, but execute, get, and query are a little complicated because of the overloads. One thing I noticed in the process though is that sqlalchemy 2.0 has enough typing info to break mypy in query code. In VS code, given a sqlalchemy ORM model, if I do:
q=User.name=="spongebob"
q becomes an instance of ColumnElement[bool], while if User is a SQLModel, q goes straight to a bool as far as mypy is concerned.
This shows up as an error, but is correct at runtime. At runtime, we see an InstrumentedAttribute for the attribute, but mypy thinks it's a string. I'm going to look further at that, and in the meantime submit a pull request here that fixes the majority of the linting errors.
@honglei
honglei
commented
Jul 28, 2023

Copy link


Copy Markdown

Environment:
pydantic version: 1.10.11
python version: 3.11.4 (main, Jun 27 2023, 15:03:25) [GCC 8.3.0]
platform: Linux-4.19.0-amd64-desktop-x86_64-with-glibc2.28
optional deps. installed: ['devtools', 'email-validator', 'typing-extensions']
sqlalchemy: 2.0.19

pytest errors:
================================================================================================================================================================================================================== short test summary info ==================================================================================================================================================================================================================
FAILED tests/test_tutorial/test_create_db_and_table/test_tutorial001.py::test_create_db_and_table - assert 'BEGIN' in "No module named 'docs_src'\n"
FAILED tests/test_tutorial/test_fastapi/test_delete/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_limit_and_offset/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...Read Hero'}}}} == {'components'...Read Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_multiple_models/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_multiple_models/test_tutorial002.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_read_one/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...Read Hero'}}}} == {'components'...Read Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_relationships/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_response_model/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_session_with_dependency/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_simple_hero_api/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_teams/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_update/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
===================================================================================================================================================================================

@AntonDeMeesterAntonDeMeester
mentioned this pull request
Aug 1, 2023
 Closed

@tiangolotiangolo
added
the 
feature
New feature or request
label
Oct 22, 2023
@tiangolotiangolo
mentioned this pull request
Nov 18, 2023
 Merged

@tiangolo
tiangolo
commented
Nov 18, 2023

Copy link


Copy Markdown

Member
TL;DR: Good news, this is available in the just released SQLModel 0.0.12! 🎉
Awesome, thanks a lot for all the work @farahats9! 🚀
And thanks everyone for the comments, reviews, etc. ☕
@farahats9 I continued the work here with several things on top on PR #700. I took your commits so you're co-author of that PR (and contributor to SQLModel, check your badge here next to your username, or in https://github.com/tiangolo/sqlmodel/graphs/contributors 😎 ).
I did a few extra things:
  • I reverted a few things, e.g. altering the docs to workaround for a previous lack of support for forward references.
  • Added support/fix for forward reference errors.
  • Removed several sections/modules that are no longer necessary, as were mainly to add type annotations to SQLAlchemy things, but now with SQLAlchemy's new proper types that's no longer necessary and I can reuse SQLAlchemy's things directly.
  • Updated all needed tests, e.g. to support the latest FastAPI.
  • Updated all the generated selects to use the new SQLAlchemy types.
  • Updated the classes and type annotations with the new SQLAlchemy types, with some overloads to make sure they are compatible with SQLModel. E.g. select().where().
  • Redefine all functions that would take column operations to accept the types used by SQLModel, e.g. or_(Hero.name == "Deadpond") as SQLAlchemy's version would require Hero.name to be a Mapped[str] instead of a str.
  • I explored a lot if it was possible/sensible to use and encourage the new Mapped[X], but sadly that breaks type annotations for creating new instances and dataclass_transform, as then that would expect to receive values with Mapped[X], e.g. Mapped[str] instead of str. And it would also add complications to the Pydantic handling. So, this was a lot of work mostly exploratory, but not much ended in the PR.
It is now available in SQLModel 0.0.12. 🎉
As this is now included in the repo and released, I'll now close this PR. Thanks! 🍰
@gdoumenc
gdoumenc
commented
Nov 20, 2023•

edited

Loading

Copy link


Copy Markdown

Hope it can help : I had the same issue because the relationsship was always defined and I didn' decalre it Optional :
origin_plan: t.Optional["Plan"] <== Good

instead of :
origin_plan: "Plan" <== Wrong
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


4 more reviewers

@hongleihongleihonglei left review comments
@sbor23sbor23sbor23 left review comments
@UnoYakshiUnoYakshiUnoYakshi left review comments
@synodriversynodriversynodriver left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

featureNew feature or request

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

20 participants

@farahats9@mortendaehli@honglei@gabrielmbmb@heyaco@8thgencore@antont@me-v2@synodriver@k-s-t-i@FilipeMarch@podsvirov@50Bytes-dev@matanper@AntonDeMeester@peterlandry@tiangolo@gdoumenc@sbor23@UnoYakshi

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
migration to sqlalchemy 2.0 - #563
Closed
farahats9 wants to merge 23 commits into
fastapi:mainfrom 
farahats9:main
Closed
migration to sqlalchemy 2.0#563
farahats9 wants to merge 23 commits into
fastapi:mainfrom 
farahats9:main
Conversation
@farahats9
@farahats9farahats9
commented
Mar 7, 2023

Copy link


Copy Markdown

Contributor
made the required changes to pass the tests for sqlalchemy 2.0

python version had to be pumped to 3.7 instead of 3.6

removed sqlalchemy2-stubs since it is not compatible or required.
More testing in a real project scenario is welcome.
@github-actions
github-actionsBot
commented
Mar 7, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 60318b4 at: https://6406825e11187c06ceaecaf0--sqlmodel.netlify.app
@mortendaehli
mortendaehli
commented
Mar 24, 2023

Copy link


Copy Markdown

@farahats9, great! Do you think you would be able to sort out the failing tests? :)
@farahats9
farahats9
commented
Mar 24, 2023

Copy link


Copy Markdown

ContributorAuthor
I fixed some of the mypy linting errors but I am not good at these things so maybe someone can fix the rest.
@github-actions
github-actionsBot
commented
Mar 24, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit b48423f at: https://641d9f2736a7fa1c9568bdb7--sqlmodel.netlify.app
sbor23
sbor23
reviewed
Mar 27, 2023
View reviewed changes
Comment threadsqlmodel/sql/expression.py
Outdated

@honglei
honglei
commented
Mar 31, 2023

Copy link


Copy Markdown

I used this version in my project with env: Python3.11.2/sqlalchemy2.0.7/Win10/scheme="postgresql+asyncpg", it works all fine.
farahats9and others
added 2 commits
March 31, 2023 12:58
@github-actions
github-actionsBot
commented
Mar 31, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 9c219d9 at: https://6426bd112dc67b078b6e5298--sqlmodel.netlify.app
@github-actions
github-actionsBot
commented
Mar 31, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 0eee8e8 at: https://6426be4468302c086218a5bb--sqlmodel.netlify.app
sbor23
sbor23
reviewed
Mar 31, 2023
View reviewed changes
Comment threaddocs/contributing.md
Outdated

Comment threaddocs/tutorial/index.md
Comment thread.github/workflows/test.yml
Outdated

@github-actions
github-actionsBot
commented
Mar 31, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 050cf02 at: https://6426c1ce9df2a80bfd37ae93--sqlmodel.netlify.app
@gabrielmbmb
gabrielmbmb
commented
Apr 12, 2023

Copy link


Copy Markdown

@farahats9 someone created a PR on your fork for fixing the mypy issues
@heyaco
heyaco
commented
Apr 20, 2023

Copy link


Copy Markdown

This seems good to go just needs merging
@8thgencore
8thgencore
commented
Apr 20, 2023

Copy link


Copy Markdown

We are very much waiting for this change
@antont
antont
commented
Apr 21, 2023

Copy link


Copy Markdown

We are very much waiting for this change
Why do you wait, instead of taking it to use? I guess feedback from users would be helpful info also for the maintainer to know when to merge.
@me-v2
me-v2
commented
Apr 24, 2023

Copy link


Copy Markdown

🚀 👀👀
@synodriver
synodriver
commented
Apr 24, 2023

Copy link


Copy Markdown

Great! I would love to try this branch in my production environment.
@k-s-t-i
k-s-t-i
commented
Apr 24, 2023

Copy link


Copy Markdown

Would like to see this merged for asyncpg support. Asyncpg seems to greatly increase performance based on most simple benchmarks.
@farahats9
farahats9
commented
Apr 25, 2023•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Would like to see this merged for asyncpg support. Asyncpg seems to greatly increase performance based on most simple benchmarks.
@k-s-t-i You can currently use Asyncpg with sqlalchemy v1.4 and sqlmodel v0.0.8 with no issue in most cases. you will need to watch out for some problems like lazy loading and cascading deletes on the orm side.
@synodriver
synodriver
commented
Apr 25, 2023

Copy link


Copy Markdown

Great! I would love to try this branch in my production environment.
Update: This patch can work with asyncmy in a real project. Will try asyncpg later.
synodriver
synodriver
reviewed
Apr 25, 2023
View reviewed changes
Comment threadpyproject.toml
Outdated

@synodriver
synodriver
commented
Apr 26, 2023

Copy link


Copy Markdown

Great! I would love to try this branch in my production environment.
Update: This patch can work with asyncmy in a real project. Will try asyncpg later.
Update: asyncpg works too. Tested with a real project on both windows and linux. Notice: to make relationship work, you may use Relationship(back_populates="xxx", sa_relationship_kwargs={"lazy": 'selectin'}) where sa_relationship_kwargs is important.
UnoYakshi
UnoYakshi
reviewed
Apr 30, 2023
View reviewed changes
@UnoYakshiUnoYakshi
left a comment
•

edited

Loading

Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really would like to use it in my new project! Thank you for the PR!
Comment threadpyproject.toml
Outdated

Comment threadsqlmodel/engine/result.py
Comment threadsqlmodel/main.py
Comment threadsqlmodel/sql/expression.py
Comment threadsqlmodel/sql/expression.py
Comment threadsqlmodel/sql/expression.py
Comment threadsqlmodel/__init__.py
Comment threaddocs_src/tutorial/relationship_attributes/back_populates/tutorial003.py
@hongleihonglei
mentioned this pull request
Jun 7, 2023

Open

@honglei
honglei
commented
Jun 27, 2023

Copy link


Copy Markdown

Five months have been passed since the release of SQLalchemy2.0, https://www.sqlalchemy.org/blog/2023/01/26/sqlalchemy-2.0.0-released/
@antont
antont
commented
Jun 27, 2023

Copy link


Copy Markdown

Five months have been passed since the release of SQLalchemy2.0, https://www.sqlalchemy.org/blog/2023/01/26/sqlalchemy-2.0.0-released/
You can help by using this PR in your project(s) if you want, and reporting how well it works.
@50Bytes-dev
50Bytes-dev
commented
Jul 10, 2023

Copy link


Copy Markdown

Been working with this PR for almost a month now, works great. Thank you!
@FilipeMarch
FilipeMarch
commented
Jul 10, 2023•

edited

Loading

Copy link


Copy Markdown

Hello @FilipeMarch! I've just change order of classes definition and solve your error:
NameError: name 'Article' is not defined
Full example code (database.py):
fromtypingimportList, OptionalfromsqlmodelimportSQLModel, Field, Relationship, create_engineclassArticle(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
reviewer_links: List["ArticleReviewerLink"] =Relationship(
back_populates="article",
)
classArticleReviewerLink(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
article_id: Optional[int] =Field(default=None, foreign_key="article.id")
article: Article=Relationship(back_populates="reviewer_links")
sqlite_file_name="database.db"sqlite_url=f"sqlite:///{sqlite_file_name}"engine=create_engine(sqlite_url, echo=True)
defcreate_db_and_tables():
SQLModel.metadata.create_all(engine)
if__name__=="__main__":
create_db_and_tables()
Sorry but changing the order of classes definition does not solve the error for me. At this point, I don't know what to do, but this branch is useless for me. I already have done what you mentioned, but the error persists. It always says if raiseerr and "Mapped[" in raw_annotation: # type: ignore TypeError: argument of type 'ForwardRef' is not iterable.
My project has about 30 tables at this point, and it has been running for quite 1 year. My project does not run as is with this branch, and I already tried dozens of things (like switching the order of tables, removing Relationships, removing a bunch of tables) and I can't make it to work in any possible way. Errors keep coming non stop. I can deactivate the Relationships, move tables up or down, it does not matter, new errors come every time. I just feel wasting my time because obviously there is something wrong, but I can't stop the errors until I remove all tables from the code?
The error messages are not helpful at all. And they only happen in this specific PR.
I understand it may be working for a very simple case like you presented, but for me it is always showing these errors, no matter how much code I change:
py", line2479, inget_propertyreturnself._props[key]
KeyError: 'user'
or
annotation=eval(expression, base_globals, locals_)
File"<string>", line1, in<module>NameError: name'PostUserLike'isnotdefined
or
.py", line223, ineval_expressionraiseNameError(
NameError: Couldnotde-stringifyannotation'PostUserLike'
or
", line2342, in_extract_mapped_subtypeifraiseerrand"Mapped["inraw_annotation: # type: ignoreTypeError: argumentof type 'ForwardRef'isnotiterable
My current API is running perfectly with sqlmodel = "^0.0.8"
If someone wants to investigate the issue with me, I am willing to screenshare and show the project structure and try to make it to work; my Discord is filipemarch
But one thing I wonder, even if I were able to make this PR to work on my code base after countless changes, I don't think this is how library updates are supposed to work? Every time I type poetry update and more than 20 libraries update, I don't remember ever having to change anything in my code just to update any library.
This PR is certainly introducing breaking changes but I'm unable to tell what and why exactly, although I investigated a lot here I couldn't even make my project to run with it.
By the way I woud love to be using sqlalchemy 2.0 at this point, and it is very sad that sqlmodel seems to be abandoned since August 2022, not a single update.
@farahats9
farahats9
commented
Jul 10, 2023

Copy link


Copy Markdown

ContributorAuthor
@FilipeMarch you are right there is an issue in ForwardRef handling and I am trying to investigate it when I have some free time. The issue appears more when you have large project with many relations between tables so that's why some people report it working fine and some don't.
I feel this is a priority to fix over the mypy linting, I will do my best to figure it out but any help is appreciated.
@50Bytes-dev
50Bytes-dev
commented
Jul 11, 2023•

edited

Loading

Copy link


Copy Markdown

I had an error Mapped[" in raw_annotation but I was able to solve it. Here is an example of my models, works fine. Maybe it will help you.
# models/__init__.py# need for alembicfrom . import (
company,
location,
client
)
# models/company.pyifTYPE_CHECKING:
frommodels.locationimportLocationfrommodels.clientimportClientclassCompany(BaseModel, CompanyBase, table=True):
locations: List["Location"] =Relationship(back_populates="company")
clients: List["Client"] =Relationship(back_populates="company")
# models/location.pyfrommodels.companyimportCompanyclassLocation(BaseModel, OrderedList, LocationBase, table=True):
company_id: int=Field(default=None, foreign_key="company.id")
company: Company=Relationship(back_populates="locations")
# models/client.pyfrommodels.companyimportCompanyclassClient(ClientBase, BaseModel, table=True):
company_id: int=Field(default=None, nullable=False, foreign_key="company.id")
company: Company=Relationship(back_populates="clients")
@matanper
matanper
commented
Jul 20, 2023

Copy link


Copy Markdown

Hey @farahats9,
I'm getting error when using sqlalchemy relationship:
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
powers: List["Power"] =Relationship(
sa_relationship=relationship(back_populates='hero')
)
classPower(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
hero_id: int=Field(foreign_key="hero.id")
hero: "Hero"=Relationship(back_populates="powers")
NameError: name 'Power' is not defined
NameError: Could not de-stringify annotation 'Power'

any idea why?
@AntonDeMeester
AntonDeMeester
commented
Jul 24, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
@matanper I think the forward Relationship (with the Foreign Key) needs to be a proper class, and not a type string. Can you try this?
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
powers: List["Power"] =Relationship(
sa_relationship=relationship(back_populates='hero')
)
classPower(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
hero_id: int=Field(foreign_key="hero.id")
hero: Hero=Relationship(back_populates="powers") # <== No quotes around Hero
@peterlandry
peterlandry
commented
Jul 27, 2023

Copy link


Copy Markdown

Contributor
I've looked at this PR a bit to try cleaning up the linting errors. I've reduced most of them, but execute, get, and query are a little complicated because of the overloads. One thing I noticed in the process though is that sqlalchemy 2.0 has enough typing info to break mypy in query code. In VS code, given a sqlalchemy ORM model, if I do:
q=User.name=="spongebob"
q becomes an instance of ColumnElement[bool], while if User is a SQLModel, q goes straight to a bool as far as mypy is concerned.
This shows up as an error, but is correct at runtime. At runtime, we see an InstrumentedAttribute for the attribute, but mypy thinks it's a string. I'm going to look further at that, and in the meantime submit a pull request here that fixes the majority of the linting errors.
@honglei
honglei
commented
Jul 28, 2023

Copy link


Copy Markdown

Environment:
pydantic version: 1.10.11
python version: 3.11.4 (main, Jun 27 2023, 15:03:25) [GCC 8.3.0]
platform: Linux-4.19.0-amd64-desktop-x86_64-with-glibc2.28
optional deps. installed: ['devtools', 'email-validator', 'typing-extensions']
sqlalchemy: 2.0.19

pytest errors:
================================================================================================================================================================================================================== short test summary info ==================================================================================================================================================================================================================
FAILED tests/test_tutorial/test_create_db_and_table/test_tutorial001.py::test_create_db_and_table - assert 'BEGIN' in "No module named 'docs_src'\n"
FAILED tests/test_tutorial/test_fastapi/test_delete/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_limit_and_offset/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...Read Hero'}}}} == {'components'...Read Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_multiple_models/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_multiple_models/test_tutorial002.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_read_one/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...Read Hero'}}}} == {'components'...Read Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_relationships/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_response_model/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_session_with_dependency/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_simple_hero_api/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_teams/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_update/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
===================================================================================================================================================================================

@AntonDeMeesterAntonDeMeester
mentioned this pull request
Aug 1, 2023
 Closed

@tiangolotiangolo
added
the 
feature
New feature or request
label
Oct 22, 2023
@tiangolotiangolo
mentioned this pull request
Nov 18, 2023
 Merged

@tiangolo
tiangolo
commented
Nov 18, 2023

Copy link


Copy Markdown

Member
TL;DR: Good news, this is available in the just released SQLModel 0.0.12! 🎉
Awesome, thanks a lot for all the work @farahats9! 🚀
And thanks everyone for the comments, reviews, etc. ☕
@farahats9 I continued the work here with several things on top on PR #700. I took your commits so you're co-author of that PR (and contributor to SQLModel, check your badge here next to your username, or in https://github.com/tiangolo/sqlmodel/graphs/contributors 😎 ).
I did a few extra things:
  • I reverted a few things, e.g. altering the docs to workaround for a previous lack of support for forward references.
  • Added support/fix for forward reference errors.
  • Removed several sections/modules that are no longer necessary, as were mainly to add type annotations to SQLAlchemy things, but now with SQLAlchemy's new proper types that's no longer necessary and I can reuse SQLAlchemy's things directly.
  • Updated all needed tests, e.g. to support the latest FastAPI.
  • Updated all the generated selects to use the new SQLAlchemy types.
  • Updated the classes and type annotations with the new SQLAlchemy types, with some overloads to make sure they are compatible with SQLModel. E.g. select().where().
  • Redefine all functions that would take column operations to accept the types used by SQLModel, e.g. or_(Hero.name == "Deadpond") as SQLAlchemy's version would require Hero.name to be a Mapped[str] instead of a str.
  • I explored a lot if it was possible/sensible to use and encourage the new Mapped[X], but sadly that breaks type annotations for creating new instances and dataclass_transform, as then that would expect to receive values with Mapped[X], e.g. Mapped[str] instead of str. And it would also add complications to the Pydantic handling. So, this was a lot of work mostly exploratory, but not much ended in the PR.
It is now available in SQLModel 0.0.12. 🎉
As this is now included in the repo and released, I'll now close this PR. Thanks! 🍰
@gdoumenc
gdoumenc
commented
Nov 20, 2023•

edited

Loading

Copy link


Copy Markdown

Hope it can help : I had the same issue because the relationsship was always defined and I didn' decalre it Optional :
origin_plan: t.Optional["Plan"] <== Good

instead of :
origin_plan: "Plan" <== Wrong
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


4 more reviewers

@hongleihongleihonglei left review comments
@sbor23sbor23sbor23 left review comments
@UnoYakshiUnoYakshiUnoYakshi left review comments
@synodriversynodriversynodriver left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

featureNew feature or request

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

20 participants

@farahats9@mortendaehli@honglei@gabrielmbmb@heyaco@8thgencore@antont@me-v2@synodriver@k-s-t-i@FilipeMarch@podsvirov@50Bytes-dev@matanper@AntonDeMeester@peterlandry@tiangolo@gdoumenc@sbor23@UnoYakshi

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
migration to sqlalchemy 2.0 - #563
Closed
farahats9 wants to merge 23 commits into
fastapi:mainfrom 
farahats9:main
Closed
migration to sqlalchemy 2.0#563
farahats9 wants to merge 23 commits into
fastapi:mainfrom 
farahats9:main
Conversation
@farahats9
@farahats9farahats9
commented
Mar 7, 2023

Copy link


Copy Markdown

Contributor
made the required changes to pass the tests for sqlalchemy 2.0

python version had to be pumped to 3.7 instead of 3.6

removed sqlalchemy2-stubs since it is not compatible or required.
More testing in a real project scenario is welcome.
@github-actions
github-actionsBot
commented
Mar 7, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 60318b4 at: https://6406825e11187c06ceaecaf0--sqlmodel.netlify.app
@mortendaehli
mortendaehli
commented
Mar 24, 2023

Copy link


Copy Markdown

@farahats9, great! Do you think you would be able to sort out the failing tests? :)
@farahats9
farahats9
commented
Mar 24, 2023

Copy link


Copy Markdown

ContributorAuthor
I fixed some of the mypy linting errors but I am not good at these things so maybe someone can fix the rest.
@github-actions
github-actionsBot
commented
Mar 24, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit b48423f at: https://641d9f2736a7fa1c9568bdb7--sqlmodel.netlify.app
sbor23
sbor23
reviewed
Mar 27, 2023
View reviewed changes
Comment threadsqlmodel/sql/expression.py
Outdated

@honglei
honglei
commented
Mar 31, 2023

Copy link


Copy Markdown

I used this version in my project with env: Python3.11.2/sqlalchemy2.0.7/Win10/scheme="postgresql+asyncpg", it works all fine.
farahats9and others
added 2 commits
March 31, 2023 12:58
@github-actions
github-actionsBot
commented
Mar 31, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 9c219d9 at: https://6426bd112dc67b078b6e5298--sqlmodel.netlify.app
@github-actions
github-actionsBot
commented
Mar 31, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 0eee8e8 at: https://6426be4468302c086218a5bb--sqlmodel.netlify.app
sbor23
sbor23
reviewed
Mar 31, 2023
View reviewed changes
Comment threaddocs/contributing.md
Outdated

Comment threaddocs/tutorial/index.md
Comment thread.github/workflows/test.yml
Outdated

@github-actions
github-actionsBot
commented
Mar 31, 2023

Copy link


Copy Markdown

Contributor
📝 Docs preview for commit 050cf02 at: https://6426c1ce9df2a80bfd37ae93--sqlmodel.netlify.app
@gabrielmbmb
gabrielmbmb
commented
Apr 12, 2023

Copy link


Copy Markdown

@farahats9 someone created a PR on your fork for fixing the mypy issues
@heyaco
heyaco
commented
Apr 20, 2023

Copy link


Copy Markdown

This seems good to go just needs merging
@8thgencore
8thgencore
commented
Apr 20, 2023

Copy link


Copy Markdown

We are very much waiting for this change
@antont
antont
commented
Apr 21, 2023

Copy link


Copy Markdown

We are very much waiting for this change
Why do you wait, instead of taking it to use? I guess feedback from users would be helpful info also for the maintainer to know when to merge.
@me-v2
me-v2
commented
Apr 24, 2023

Copy link


Copy Markdown

🚀 👀👀
@synodriver
synodriver
commented
Apr 24, 2023

Copy link


Copy Markdown

Great! I would love to try this branch in my production environment.
@k-s-t-i
k-s-t-i
commented
Apr 24, 2023

Copy link


Copy Markdown

Would like to see this merged for asyncpg support. Asyncpg seems to greatly increase performance based on most simple benchmarks.
@farahats9
farahats9
commented
Apr 25, 2023•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Would like to see this merged for asyncpg support. Asyncpg seems to greatly increase performance based on most simple benchmarks.
@k-s-t-i You can currently use Asyncpg with sqlalchemy v1.4 and sqlmodel v0.0.8 with no issue in most cases. you will need to watch out for some problems like lazy loading and cascading deletes on the orm side.
@synodriver
synodriver
commented
Apr 25, 2023

Copy link


Copy Markdown

Great! I would love to try this branch in my production environment.
Update: This patch can work with asyncmy in a real project. Will try asyncpg later.
synodriver
synodriver
reviewed
Apr 25, 2023
View reviewed changes
Comment threadpyproject.toml
Outdated

@synodriver
synodriver
commented
Apr 26, 2023

Copy link


Copy Markdown

Great! I would love to try this branch in my production environment.
Update: This patch can work with asyncmy in a real project. Will try asyncpg later.
Update: asyncpg works too. Tested with a real project on both windows and linux. Notice: to make relationship work, you may use Relationship(back_populates="xxx", sa_relationship_kwargs={"lazy": 'selectin'}) where sa_relationship_kwargs is important.
UnoYakshi
UnoYakshi
reviewed
Apr 30, 2023
View reviewed changes
@UnoYakshiUnoYakshi
left a comment
•

edited

Loading

Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really would like to use it in my new project! Thank you for the PR!
Comment threadpyproject.toml
Outdated

Comment threadsqlmodel/engine/result.py
Comment threadsqlmodel/main.py
Comment threadsqlmodel/sql/expression.py
Comment threadsqlmodel/sql/expression.py
Comment threadsqlmodel/sql/expression.py
Comment threadsqlmodel/__init__.py
Comment threaddocs_src/tutorial/relationship_attributes/back_populates/tutorial003.py
@hongleihonglei
mentioned this pull request
Jun 7, 2023

Open

@honglei
honglei
commented
Jun 27, 2023

Copy link


Copy Markdown

Five months have been passed since the release of SQLalchemy2.0, https://www.sqlalchemy.org/blog/2023/01/26/sqlalchemy-2.0.0-released/
@antont
antont
commented
Jun 27, 2023

Copy link


Copy Markdown

Five months have been passed since the release of SQLalchemy2.0, https://www.sqlalchemy.org/blog/2023/01/26/sqlalchemy-2.0.0-released/
You can help by using this PR in your project(s) if you want, and reporting how well it works.
@50Bytes-dev
50Bytes-dev
commented
Jul 10, 2023

Copy link


Copy Markdown

Been working with this PR for almost a month now, works great. Thank you!
@FilipeMarch
FilipeMarch
commented
Jul 10, 2023•

edited

Loading

Copy link


Copy Markdown

Hello @FilipeMarch! I've just change order of classes definition and solve your error:
NameError: name 'Article' is not defined
Full example code (database.py):
fromtypingimportList, OptionalfromsqlmodelimportSQLModel, Field, Relationship, create_engineclassArticle(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
reviewer_links: List["ArticleReviewerLink"] =Relationship(
back_populates="article",
)
classArticleReviewerLink(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
article_id: Optional[int] =Field(default=None, foreign_key="article.id")
article: Article=Relationship(back_populates="reviewer_links")
sqlite_file_name="database.db"sqlite_url=f"sqlite:///{sqlite_file_name}"engine=create_engine(sqlite_url, echo=True)
defcreate_db_and_tables():
SQLModel.metadata.create_all(engine)
if__name__=="__main__":
create_db_and_tables()
Sorry but changing the order of classes definition does not solve the error for me. At this point, I don't know what to do, but this branch is useless for me. I already have done what you mentioned, but the error persists. It always says if raiseerr and "Mapped[" in raw_annotation: # type: ignore TypeError: argument of type 'ForwardRef' is not iterable.
My project has about 30 tables at this point, and it has been running for quite 1 year. My project does not run as is with this branch, and I already tried dozens of things (like switching the order of tables, removing Relationships, removing a bunch of tables) and I can't make it to work in any possible way. Errors keep coming non stop. I can deactivate the Relationships, move tables up or down, it does not matter, new errors come every time. I just feel wasting my time because obviously there is something wrong, but I can't stop the errors until I remove all tables from the code?
The error messages are not helpful at all. And they only happen in this specific PR.
I understand it may be working for a very simple case like you presented, but for me it is always showing these errors, no matter how much code I change:
py", line2479, inget_propertyreturnself._props[key]
KeyError: 'user'
or
annotation=eval(expression, base_globals, locals_)
File"<string>", line1, in<module>NameError: name'PostUserLike'isnotdefined
or
.py", line223, ineval_expressionraiseNameError(
NameError: Couldnotde-stringifyannotation'PostUserLike'
or
", line2342, in_extract_mapped_subtypeifraiseerrand"Mapped["inraw_annotation: # type: ignoreTypeError: argumentof type 'ForwardRef'isnotiterable
My current API is running perfectly with sqlmodel = "^0.0.8"
If someone wants to investigate the issue with me, I am willing to screenshare and show the project structure and try to make it to work; my Discord is filipemarch
But one thing I wonder, even if I were able to make this PR to work on my code base after countless changes, I don't think this is how library updates are supposed to work? Every time I type poetry update and more than 20 libraries update, I don't remember ever having to change anything in my code just to update any library.
This PR is certainly introducing breaking changes but I'm unable to tell what and why exactly, although I investigated a lot here I couldn't even make my project to run with it.
By the way I woud love to be using sqlalchemy 2.0 at this point, and it is very sad that sqlmodel seems to be abandoned since August 2022, not a single update.
@farahats9
farahats9
commented
Jul 10, 2023

Copy link


Copy Markdown

ContributorAuthor
@FilipeMarch you are right there is an issue in ForwardRef handling and I am trying to investigate it when I have some free time. The issue appears more when you have large project with many relations between tables so that's why some people report it working fine and some don't.
I feel this is a priority to fix over the mypy linting, I will do my best to figure it out but any help is appreciated.
@50Bytes-dev
50Bytes-dev
commented
Jul 11, 2023•

edited

Loading

Copy link


Copy Markdown

I had an error Mapped[" in raw_annotation but I was able to solve it. Here is an example of my models, works fine. Maybe it will help you.
# models/__init__.py# need for alembicfrom . import (
company,
location,
client
)
# models/company.pyifTYPE_CHECKING:
frommodels.locationimportLocationfrommodels.clientimportClientclassCompany(BaseModel, CompanyBase, table=True):
locations: List["Location"] =Relationship(back_populates="company")
clients: List["Client"] =Relationship(back_populates="company")
# models/location.pyfrommodels.companyimportCompanyclassLocation(BaseModel, OrderedList, LocationBase, table=True):
company_id: int=Field(default=None, foreign_key="company.id")
company: Company=Relationship(back_populates="locations")
# models/client.pyfrommodels.companyimportCompanyclassClient(ClientBase, BaseModel, table=True):
company_id: int=Field(default=None, nullable=False, foreign_key="company.id")
company: Company=Relationship(back_populates="clients")
@matanper
matanper
commented
Jul 20, 2023

Copy link


Copy Markdown

Hey @farahats9,
I'm getting error when using sqlalchemy relationship:
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
powers: List["Power"] =Relationship(
sa_relationship=relationship(back_populates='hero')
)
classPower(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
hero_id: int=Field(foreign_key="hero.id")
hero: "Hero"=Relationship(back_populates="powers")
NameError: name 'Power' is not defined
NameError: Could not de-stringify annotation 'Power'

any idea why?
@AntonDeMeester
AntonDeMeester
commented
Jul 24, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
@matanper I think the forward Relationship (with the Foreign Key) needs to be a proper class, and not a type string. Can you try this?
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
powers: List["Power"] =Relationship(
sa_relationship=relationship(back_populates='hero')
)
classPower(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
hero_id: int=Field(foreign_key="hero.id")
hero: Hero=Relationship(back_populates="powers") # <== No quotes around Hero
@peterlandry
peterlandry
commented
Jul 27, 2023

Copy link


Copy Markdown

Contributor
I've looked at this PR a bit to try cleaning up the linting errors. I've reduced most of them, but execute, get, and query are a little complicated because of the overloads. One thing I noticed in the process though is that sqlalchemy 2.0 has enough typing info to break mypy in query code. In VS code, given a sqlalchemy ORM model, if I do:
q=User.name=="spongebob"
q becomes an instance of ColumnElement[bool], while if User is a SQLModel, q goes straight to a bool as far as mypy is concerned.
This shows up as an error, but is correct at runtime. At runtime, we see an InstrumentedAttribute for the attribute, but mypy thinks it's a string. I'm going to look further at that, and in the meantime submit a pull request here that fixes the majority of the linting errors.
@honglei
honglei
commented
Jul 28, 2023

Copy link


Copy Markdown

Environment:
pydantic version: 1.10.11
python version: 3.11.4 (main, Jun 27 2023, 15:03:25) [GCC 8.3.0]
platform: Linux-4.19.0-amd64-desktop-x86_64-with-glibc2.28
optional deps. installed: ['devtools', 'email-validator', 'typing-extensions']
sqlalchemy: 2.0.19

pytest errors:
================================================================================================================================================================================================================== short test summary info ==================================================================================================================================================================================================================
FAILED tests/test_tutorial/test_create_db_and_table/test_tutorial001.py::test_create_db_and_table - assert 'BEGIN' in "No module named 'docs_src'\n"
FAILED tests/test_tutorial/test_fastapi/test_delete/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_limit_and_offset/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...Read Hero'}}}} == {'components'...Read Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_multiple_models/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_multiple_models/test_tutorial002.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_read_one/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...Read Hero'}}}} == {'components'...Read Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_relationships/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_response_model/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_session_with_dependency/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_simple_hero_api/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...eate Hero'}}}} == {'components'...eate Hero'}}}}
FAILED tests/test_tutorial/test_fastapi/test_teams/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
FAILED tests/test_tutorial/test_fastapi/test_update/test_tutorial001.py::test_tutorial - AssertionError: assert {'components'...or'}}, ...}}}} == {'components'...or'}}, ...}}}}
===================================================================================================================================================================================

@AntonDeMeesterAntonDeMeester
mentioned this pull request
Aug 1, 2023
 Closed

@tiangolotiangolo
added
the 
feature
New feature or request
label
Oct 22, 2023
@tiangolotiangolo
mentioned this pull request
Nov 18, 2023
 Merged

@tiangolo
tiangolo
commented
Nov 18, 2023

Copy link


Copy Markdown

Member
TL;DR: Good news, this is available in the just released SQLModel 0.0.12! 🎉
Awesome, thanks a lot for all the work @farahats9! 🚀
And thanks everyone for the comments, reviews, etc. ☕
@farahats9 I continued the work here with several things on top on PR #700. I took your commits so you're co-author of that PR (and contributor to SQLModel, check your badge here next to your username, or in https://github.com/tiangolo/sqlmodel/graphs/contributors 😎 ).
I did a few extra things:
  • I reverted a few things, e.g. altering the docs to workaround for a previous lack of support for forward references.
  • Added support/fix for forward reference errors.
  • Removed several sections/modules that are no longer necessary, as were mainly to add type annotations to SQLAlchemy things, but now with SQLAlchemy's new proper types that's no longer necessary and I can reuse SQLAlchemy's things directly.
  • Updated all needed tests, e.g. to support the latest FastAPI.
  • Updated all the generated selects to use the new SQLAlchemy types.
  • Updated the classes and type annotations with the new SQLAlchemy types, with some overloads to make sure they are compatible with SQLModel. E.g. select().where().
  • Redefine all functions that would take column operations to accept the types used by SQLModel, e.g. or_(Hero.name == "Deadpond") as SQLAlchemy's version would require Hero.name to be a Mapped[str] instead of a str.
  • I explored a lot if it was possible/sensible to use and encourage the new Mapped[X], but sadly that breaks type annotations for creating new instances and dataclass_transform, as then that would expect to receive values with Mapped[X], e.g. Mapped[str] instead of str. And it would also add complications to the Pydantic handling. So, this was a lot of work mostly exploratory, but not much ended in the PR.
It is now available in SQLModel 0.0.12. 🎉
As this is now included in the repo and released, I'll now close this PR. Thanks! 🍰
@gdoumenc
gdoumenc
commented
Nov 20, 2023•

edited

Loading

Copy link


Copy Markdown

Hope it can help : I had the same issue because the relationsship was always defined and I didn' decalre it Optional :
origin_plan: t.Optional["Plan"] <== Good

instead of :
origin_plan: "Plan" <== Wrong
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


4 more reviewers

@hongleihongleihonglei left review comments
@sbor23sbor23sbor23 left review comments
@UnoYakshiUnoYakshiUnoYakshi left review comments
@synodriversynodriversynodriver left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

featureNew feature or request

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

20 participants

@farahats9@mortendaehli@honglei@gabrielmbmb@heyaco@8thgencore@antont@me-v2@synodriver@k-s-t-i@FilipeMarch@podsvirov@50Bytes-dev@matanper@AntonDeMeester@peterlandry@tiangolo@gdoumenc@sbor23@UnoYakshi