[WIP] Upgrade to Pydantic 2

[AntonDeMeester]

Upgrades to Pydantic 2, using the new pydantic models. Switches for v1 and v2 can be built later

Builds on #563 to upgrade SQL Alchemy

Still WIP, did some uglier hacks which could provide some problems.

Most of it is just changing code with different naming of Pydantic 2. However, the validation of empty initialisations is broken because that now happens in rust and there is no static validate_model anymore. For now I set defaults to all provided Fields is there are none, so that it will always inititialise, but that screws a bit with the logic. This is also annoying for cls.model_validate as it screws up SQL Alchemy.

[luoshuijs]

Thank you very much for your PR. I can use it with SQLModel in the pydantic v2 environment, and I've found some issues and have some suggestions.

[luoshuijs]

NoneType has been moved to types in Python 3.10+.
https://docs.python.org/3/library/types.html#types.NoneType

[luoshuijs]

In the test tests\test_tutorial\test_delete, the following error occurred: AttributeError: 'Hero' object has no attribute '__pydantic_extra__'.

You can solve this problem by adding the following code to SQLmodel:

def__new__(cls, *args: Any, **kwargs: Any) ->Any:
new_object=super().__new__(cls)
object.__setattr__(new_object, "__pydantic_fields_set__", set())
returnnew_object

[AntonDeMeester]

I got almost everything working except for Open API. It provides optional for the OpenAPI if you use a table as a request or response type. While this isn't correct in the strict sense, it is correct for Pydantic as empty initialisations need to be supported. I tried a couple of other ways (creating a second class for the optional initialisations or trying to override the OpenAPI generation) but none of them worked out.

For now I commented the OpenAPI assertions, but @tiangolo will need to decide how to handle it. As he wrote FastAPI I'm sure he'll have better ideas 😄 I just hope I can give him some inspiration with this.

[honglei]

the following test not pass:

fromsqlmodelimportSQLModel, FieldclassArticle(SQLModel, table=True):
name : str|None=Field( description='the name of article')

Error:

 File "crud_test\test_sqlmodel.py", line 2, in <module>
class Article(SQLModel, table=True):
 File "crud_test\sqlmodel\main.py", line 318, in __new__
col = get_column_from_field(v)
 File "crud_test\sqlmodel\main.py", line 464, in get_column_from_field
sa_type = get_sqlalchemy_type(field)
 File "crud_test\sqlmodel\main.py", line 417, in get_sqlalchemy_type
if issubclass(type_, str):
builtins.TypeError: issubclass() arg 1 must be a class

[yallxe]

@honglei try Optional[str] instead of str | None

I think just one more case must be checked somewhere.

[honglei]

@honglei try Optional[str] instead of str | None

I think just one more case must be checked somewhere.

I made a pull request to support it. mbsantiago#1

[ma-ts]

Hi! Just checking if there's still something to be done for this PR, more than willing to help out!

[ogabrielluiz]

Hey! I'm also willing to help out.

[honglei]

@ma-ts@ogabrielluiz You can help by testing the PR in your projects, and reporting how well it works.
Right now my project based on https://github.com/mbsantiago/sqlmodel, with my version at: https://github.com/honglei/sqlmodel

[ogabrielluiz]

@honglei
After the normal pydantic migration which was pretty straightforward all Langflow's tests are passing. Will perform some more tests ASAP.

Installed with poetry using sqlmodel = { git = "https://github.com/honglei/sqlmodel.git", branch = "main" }

[50Bytes-dev]

@ogabrielluiz@honglei
.model_copy(...) raise error:

object has no attribute '__pydantic_extra__'

service_category: ServiceCategory=awaitself.get_object_or_404(id, company.id, async_db)
new_service_category=service_category.model_copy(update={'id': None})

Use this test:

deftest_model_copy(clear_sqlmodel):
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
name: strsecret_name: strage: Optional[int] =Nonehero=Hero(name="Deadpond", secret_name="Dive Wilson", age=25)
engine=create_engine("sqlite://")
SQLModel.metadata.create_all(engine)
withSession(engine) assession:
session.add(hero)
session.commit()
session.refresh(hero)
db_hero=session.get(Hero, hero.id)
copy=db_hero.model_copy(update={"name": "Deadpond Copy"})
assertcopy.name=="Deadpond Copy"and \
copy.secret_name=="Dive Wilson"and \
copy.age==25

[50Bytes-dev]

@ogabrielluiz@honglei .model_copy(...) raise error:

object has no attribute '__pydantic_extra__'

service_category: ServiceCategory=awaitself.get_object_or_404(id, company.id, async_db)
new_service_category=service_category.model_copy(update={'id': None})

Use this test:

deftest_model_copy(clear_sqlmodel):
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
name: strsecret_name: strage: Optional[int] =Nonehero=Hero(name="Deadpond", secret_name="Dive Wilson", age=25)
engine=create_engine("sqlite://")
SQLModel.metadata.create_all(engine)
withSession(engine) assession:
session.add(hero)
session.commit()
session.refresh(hero)
db_hero=session.get(Hero, hero.id)
copy=db_hero.model_copy(update={"name": "Deadpond Copy"})
assertcopy.name=="Deadpond Copy"and \
copy.secret_name=="Dive Wilson"and \
copy.age==25

fixed honglei#1

[8thgencore]

I wonder if @tiangolo will notice this sentence

[MattOates]

Not strictly a SQLModels problem, but if you try and use the current SQLModels 0.0.8 with FastAPI 0.103.0 where you've targeted fastapi[all] conflicts are created because of the pydantic version pins of SQLModel, as pydantic-settings requires a much later version than the max pinned. Just like to add a +1 for this PR being quite important to get merged.

[MatsiukMykola]

topic is very long, guys, possible to use with this PR?
@tiangolo can please comment? Your package is awesome, very needs an update.
Anyway big th! (hello from Ukraine)

[50Bytes-dev]

topic is very long, guys, possible to use with this PR? @tiangolo can please comment? Your package is awesome, very needs an update. Anyway big th! (hello from Ukraine)

@MatsiukMykola I'm using it in my project. Everything's great
Use this https://github.com/honglei/sqlmodel

[ikreb7]

The type EmailStr doesn't work. I get the error ValueError(f"The field {field.title} has no matching SQLAlchemy type").

[50Bytes-dev]

The type EmailStr doesn't work. I get the error ValueError(f"The field {field.title} has no matching SQLAlchemy type").

@ikreb7 use this

my_email: EmailStr=Field(sa_column=Column(String))

[honglei]

@ikreb7@50Bytes-dev support the following in my fork(https://github.com/honglei/sqlmodel), but how to change test.yml to do pip install pydantic[email]?

http: HttpUrl=Field(max_length=250)
email: EmailStrname_email : NameEmail=Field( max_length=50)
import_string:ImportString=Field(max_length=200, min_length=100)

[50Bytes-dev]

@ikreb7@50Bytes-dev support the following in my fork(https://github.com/honglei/sqlmodel), but how to change test.yml to do pip install pydantic[email]?

http: HttpUrl=Field(max_length=250)
email: EmailStrname_email : NameEmail=Field( max_length=50)
import_string:ImportString=Field(max_length=200, min_length=100)

@honglei
In pyproject.toml update line 36

pydantic = { version = "^2.1.1", extras = ["email"] }

[honglei]

@50Bytes-dev thanks!

[wojciech-mazur-mlg]

@honglei what's the status of your fork? Can we use it till a stable version of SQLmodel is released?

[honglei]

I used in my projects, it works fine. wojciech-mazur-mlg ***@***.***> 于 2023年9月18日周一 下午7:30写道：

…

@honglei <https://github.com/honglei> what's the status of your fork? Can we use it till a stable version of SQLmodel is released? — Reply to this email directly, view it on GitHub <#632 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHW5AVSOKK64F5EAV4P7LTX3AWDZANCNFSM6AAAAAA27PJTWM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[AntonDeMeester]

@honglei what's the status of your fork? Can we use it till a stable version of SQLmodel is released?

I have tried it out a bit and it seems to work. Some other people are also saying it works. But I would say it's not official until @tiangolo approves it or makes his own version. I also won't be trying everything out to be honest 😅

[BoManev]

After I updated, the "alias" stopped working

class UserBase(SQLModel):
password_hash: str = Field(alias="password")
def create_user(user: UserBase):
pass

The validation in create_user fails with "password ", but works with "password_hash"

[honglei]

@BoManev@AntonDeMeester
In pydantic, function pydantic.fields.Field will use alias to infer validation_alias,
but in sqlmodel.main.FieldInfo, validation_alias is not set, so alias not work under pydantic_core

https://github.com/pydantic/pydantic/blob/188018ccd0c26085e51e632ae4f909b3ee31a0f7/pydantic/fields.py#L819

[BoManev]

@honglei Thank you for the fast reply. In that case should I create a class using pydantic BaseModel for validation of my inputs and then map it with model_validate to my SQLModel.

I also found this #374

EDIT: Generally, how to handle data validation when field names change and data needs to be modified. In my case I receive "password", but internally I want my types to carry "password_hash". In Rust, I can specify Into/To/From traits and convert between types. Can I do something similar with pydantic/SQLModel, define the conversion of 1 data schema to other.

[JanBeelte]

Hey folks,
what is the current status and what are the next steps here?
Is there any ETA when this one will be merged to master?
I could also take over some tasks in case that helps, given I desperately need pydantic v2 support otherwise I cannot use SQLModel.
Best,
Jan

[lawrenceeldridge]

Really looking forward to seeing these changes go in - carrying out an early preview as I'm already using the latest versions. One thing I've observed, I've been struggling a little with using the async db setup in the context of pulling in additional relationships. All of the other routes work fine from this tutorial, but when I call read_hero I get the following error:

fastapi.exceptions.ResponseValidationError: 1 validation errors:
{'type': 'get_attribute_error', 'loc': ('response', 'team'), 'msg': "Error extracting attribute: MissingGreenlet: greenlet_spawn has not been called; can't call await_only() here. Was IO attempted in an unexpected place? (Background on this error at: https://sqlalche.me/e/20/xd2s)", 'input': Hero(secret_name='Batman', id=1, team_id=1, name='Bruce Wayne', age=42), 'ctx': {'error': "MissingGreenlet: greenlet_spawn has not been called; can't call await_only() here. Was IO attempted in an unexpected place? (Background on this error at: https://sqlalche.me/e/20/xd2s)"}, 'url': 'https://errors.pydantic.dev/2.4/v/get_attribute_error'}

This only happens when the response_model is set to HeroReadWithTeam, as the expectation is that we'd want to include teams with the hero. Using HeroRead works fine.

[lawrenceeldridge]

This is now resolved with the help of another post here. For relationships between models you may need to leverage the correct loading styles through the sa_relationship_kwargs property that is offered by SQLModel. For the curious this is how I've built up the tutorial with async in mind below.

In SQL Alchemy there's more reading here on the subject of lazy loading.

Really happy, so will crack on :)

fromcontextlibimportasynccontextmanagerfromtypingimportList, OptionalfromfastapiimportDepends, FastAPI, HTTPException, Queryfromsqlalchemy.ext.asyncioimportasync_sessionmaker, create_async_enginefromsqlmodelimportField, Relationship, SQLModelfromsqlalchemy.sql.expressionimportselectfromsqlmodel.ext.asyncio.sessionimportAsyncSessionfromconfigimportconfigclassTeamBase(SQLModel):
name: str=Field(index=True)
headquarters: strclassTeam(TeamBase, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
heroes: List["Hero"] =Relationship(back_populates="team", sa_relationship_kwargs={"lazy": "selectin"})
classTeamCreate(TeamBase):
passclassTeamRead(TeamBase):
id: intclassTeamUpdate(SQLModel):
id: Optional[int] =Nonename: Optional[str] =Noneheadquarters: Optional[str] =NoneclassHeroBase(SQLModel):
name: str=Field(index=True)
secret_name: strage: Optional[int] =Field(default=None, index=True)
team_id: Optional[int] =Field(default=None, foreign_key="team.id")
classHero(HeroBase, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
team: Optional[Team] =Relationship(back_populates="heroes", sa_relationship_kwargs={"lazy": "joined"})
classHeroRead(HeroBase):
id: intclassHeroCreate(HeroBase):
passclassHeroUpdate(SQLModel):
name: Optional[str] =Nonesecret_name: Optional[str] =Noneage: Optional[int] =Noneteam_id: Optional[int] =NoneclassHeroReadWithTeam(HeroRead):
team: Optional[TeamRead] =NoneclassTeamReadWithHeroes(TeamRead):
heroes: List[HeroRead] = []
engine=create_async_engine(config.DB_HOST, echo=True, future=True)
asyncdefinit_db():
asyncwithengine.begin() asconn:
awaitconn.run_sync(SQLModel.metadata.create_all)
asyncdefget_session() ->AsyncSession:
async_session=async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
asyncwithasync_session() assession:
yieldsession@asynccontextmanagerasyncdeflifespan(app: FastAPI):
# on startupawaitinit_db()
yield# on shutdownawaitengine.dispose()
app=FastAPI(lifespan=lifespan)
@app.post("/heroes", response_model=HeroRead)asyncdefcreate_hero(*, session: AsyncSession=Depends(get_session), hero: HeroCreate):
db_hero=Hero.model_validate(hero)
session.add(db_hero)
awaitsession.commit()
awaitsession.refresh(db_hero)
returndb_hero@app.get("/heroes", response_model=List[HeroRead])asyncdefread_heroes(
*,
session: AsyncSession=Depends(get_session),
offset: int=0,
limit: int=Query(default=100, lte=100),
):
statement=select(Hero).offset(offset).limit(limit)
heroes=awaitsession.execute(statement)
returnheroes.scalars().all()
@app.get("/heroes/{hero_id}", response_model=HeroReadWithTeam)asyncdefread_hero(*, session: AsyncSession=Depends(get_session), hero_id: int):
hero=awaitsession.get(Hero, hero_id)
ifnothero:
raiseHTTPException(status_code=404, detail="Hero not found")
returnhero@app.patch("/heroes/{hero_id}", response_model=HeroRead)asyncdefupdate_hero(
*, session: AsyncSession=Depends(get_session), hero_id: int, hero: HeroUpdate
):
db_hero=awaitsession.get(Hero, hero_id)
ifnotdb_hero:
raiseHTTPException(status_code=404, detail="Hero not found")
hero_data=hero.model_dump(exclude_unset=True)
forkey, valueinhero_data.items():
setattr(db_hero, key, value)
session.add(db_hero)
awaitsession.commit()
awaitsession.refresh(db_hero)
returndb_hero@app.delete("/heroes/{hero_id}")asyncdefdelete_hero(*, session: AsyncSession=Depends(get_session), hero_id: int):
hero=awaitsession.get(Hero, hero_id)
ifnothero:
raiseHTTPException(status_code=404, detail="Hero not found")
awaitsession.delete(hero)
awaitsession.commit()
return {"ok": True}
@app.post("/teams", response_model=TeamRead)asyncdefcreate_team(*, session: AsyncSession=Depends(get_session), team: TeamCreate):
db_team=Team.model_validate(team)
session.add(db_team)
awaitsession.commit()
awaitsession.refresh(db_team)
returndb_team@app.get("/teams", response_model=List[TeamRead])asyncdefread_teams(
*,
session: AsyncSession=Depends(get_session),
offset: int=0,
limit: int=Query(default=100, lte=100),
):
statement=select(Team).offset(offset).limit(limit)
teams=awaitsession.execute(statement)
returnteams.scalars().all()
@app.get("/teams/{team_id}", response_model=TeamReadWithHeroes)asyncdefread_team(*, team_id: int, session: AsyncSession=Depends(get_session)):
team=awaitsession.get(Team, team_id)
ifnotteam:
raiseHTTPException(status_code=404, detail="Team not found")
returnteam@app.patch("/teams/{team_id}", response_model=TeamRead)asyncdefupdate_team(
*,
session: AsyncSession=Depends(get_session),
team_id: int,
team: TeamUpdate,
):
db_team=awaitsession.get(Team, team_id)
ifnotdb_team:
raiseHTTPException(status_code=404, detail="Team not found")
team_data=team.model_dump(exclude_unset=True)
forkey, valueinteam_data.items():
setattr(db_team, key, value)
session.add(db_team)
awaitsession.commit()
awaitsession.refresh(db_team)
returndb_team@app.delete("/teams/{team_id}")asyncdefdelete_team(*, session: AsyncSession=Depends(get_session), team_id: int):
team=awaitsession.get(Team, team_id)
ifnotteam:
raiseHTTPException(status_code=404, detail="Team not found")
awaitsession.delete(team)
awaitsession.commit()
return {"ok": True}

[blakewatters]

Joining the party here trying to update a fairly large data model the fork.

One thing I am working on resolving and curious if anyone else has run into is an issue with model class inheritance producing SQLAlchemy errors regarding column reuse:

sqlalchemy.exc.ArgumentError: Column object 'created_at' already assigned to Table 'business_address_links'

My TimestampModel looks like:

classTimestampModel(BaseModel):
created_at: datetime.datetime=sqlmodel.Field(
sa_column=sqlalchemy.Column(
sqlalchemy.DateTime(timezone=True),
server_default=sqlalchemy.func.statement_timestamp(),
nullable=False
)
)
updated_at: typing.Optional[datetime.datetime] =sqlmodel.Field(
sa_column=sqlalchemy.Column(
sqlalchemy.DateTime(timezone=True),
onupdate=sqlalchemy.func.statement_timestamp(),
nullable=True
)
)

It looks like the Column definition is being reused across inherited subclasses

[MatsiukMykola]

always was interesting how pull to sa.Column - column comment based on Field Description?

[honglei]

@blakewatters try https://github.com/honglei/sqlmodel/blob/3005495a3ec6c8216b31cbd623f91c7bc8ba174f/tests/test_class_hierarchy.py#L24C17-L24C17

[norton120]

➕ on thanks for putting this PR up, my team is a few weeks into a new project and I'd hate to be stuck behind a major on 2 core libraries this early on, so this is huge. I'm running into a strange issue creating a one-to-many relationship (using this branch), any ideas would be appreciated. Here's the error:

sqlalchemy.exc.ArgumentError: relationship 'venues' expects a class or a mapper argument (received: <class 'list'>)

the related models:

classOrganization(SQLModel, table=True):
id: Optional[int] =sqlmodel.Field(default=None, primary_key=True)
venues: List["Venue"] =sqlmodel.Relationship(back_populates="organization")
@classmethodasyncdeflist(cls, session: AsyncSession):
query=select(cls)
result_from_db=awaitsession.execute(query)
collection=result_from_db.fetchall()
returncollectionclassVenue(SQLModel, table=True):
id: Optional[int] =sqlmodel.Field(default=None, primary_key=True)
organization_id: int=sqlmodel.Field(foreign_key="organization.id")
organization: Organization=sqlmodel.Relationship(back_populates="venues")

and the calling functionality:

classDatabaseConnection:
def__init__(self):
url=sqlalchemy.URL.create(*url_args)
self.async_engine=sqlalchemy.ext.asyncio.create_async_engine(
url, echo=True, future=True
)
asyncdefget_async_session(self) ->AsyncSession:
async_session=sessionmaker(
bind=self.async_engine, class_=AsyncSession, expire_on_commit=False
)
asyncwithasync_session() assession:
yieldsessiondb_session=DatabaseConnection().get_async_session@router.get("/")asyncdefindex(*, session=Depends(db_session)):
returnawaitOrganization.list(session=session)

I've tried changing annotation types, explicitly passing an sa.relationship to the Relationship, setting lazy loading... I'm not super familiar with the SQLModel code base so I'm not sure where to look next.

[nikdavis]

It looks like the both of the goals prior to migrating to sqlalchemy 2.0, and pydantic 2.0 have been reached (supporting the latest pre 2.0 versions of each). The next goal in the roadmap is migrating each to 2.0.

Any update here?

[github-actions]

📝 Docs preview for commit 7ecbc38 at: https://5165ba05.sqlmodel.pages.dev

[github-actions]

📝 Docs preview for commit ab07514 at: https://398280da.sqlmodel.pages.dev

[BoManev]

@AntonDeMeester I have been using an older commit from this repo, however today I tried to update my dependencies and got the following errors.

Because no versions of pydantic-settings match >2.0.0,<2.0.1 || >2.0.1,<2.0.2 || >2.0.2,<2.0.3 || >2.0.3,<2.1.0 || >2.1.0
and pydantic-settings (2.0.0) depends on pydantic (>=2.0b3), pydantic-settings (>=2.0.0,<2.0.1 || >2.0.1,<2.0.2 || >2.0.2,<2.0.3 || >2.0.3,<2.1.0 || >2.1.0) requires pydantic (>=2.0b3).
And because pydantic-settings (2.0.1) depends on pydantic (>=2.0.1), pydantic-settings (>=2.0.0,<2.0.2 || >2.0.2,<2.0.3 || >2.0.3,<2.1.0 || >2.1.0) requires pydantic (>=2.0b3).
And because pydantic-settings (2.0.2) depends on pydantic (>=2.0.1)
and pydantic-settings (2.0.3) depends on pydantic (>=2.0.1), pydantic-settings (>=2.0.0,<2.1.0 || >2.1.0) requires pydantic (>=2.0b3).
And because pydantic-settings (2.1.0) depends on pydantic (>=2.3.0)
and sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head depends on pydantic (^1.9.0), sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head is incompatible with pydantic-settings (>=2.0.0).
And because fastapi (0.103.2) depends on pydantic-settings (>=2.0.0)
and no versions of fastapi match >0.103.2,<0.104.0, sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head is incompatible with fastapi (>=0.103.2,<0.104.0).
So, because api depends on both fastapi (^0.103.2) and sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head, version solving failed.

Snippet from my pyproject.toml

fastapi = {extras = ["all"], version = "^0.103.2"}
sqlmodel = {git = "https://github.com/tiangolo/sqlmodel.git", rev = "refs/pull/632/head"}

The older commit from this PR is using pydantic = { version = ">=2.1.1,<=2.4.2", extras = ["email"] } this worked fine with pydantic-settings from fastapi extras

[AntonDeMeester]

@AntonDeMeester I have been using an older commit from this repo, however today I tried to update my dependencies and got the following errors.

The older commit from this PR is using pydantic = { version = ">=2.1.1,<=2.4.2", extras = ["email"] } this worked fine with pydantic-settings from fastapi extras

Ye I've been working on getting both Pydantic v1 and v2 to work at the same time, I might have broken some things. I'll change the PR to use the old commit again and push my changes to a new PR on top of this one until both work.

[github-actions]

📝 Docs preview for commit d0ae3e8 at: https://ff83edad.sqlmodel.pages.dev

[BoManev]

@AntonDeMeester This commit still doesn't work
The one Im using is this f926d9d

[honglei]

@AntonDeMeester sqlmodel has drop support for sqlalchemy<2.0, maybe next release will drop support for pydantic v1,
the author has no interests to fix some bugs in pydantic v1, which make me move my project to pydantic v2.

https://github.com/tiangolo/sqlmodel/blob/b1c2f822c9db036afe1b7ec81d335368e9546756/pyproject.toml#L34

[ogabrielluiz]

@AntonDeMeester how are you?

I've been using your fork internally for a while now and it seems stable.

Do you see this PR being merged this week?

Thanks in advance

[AntonDeMeester]

Closing in favour of @tiangolo PR here

[tiangolo]

Thank you for your work @AntonDeMeester! I included your commits in #722 (you can see your badge now says "Contributor" 😎 ), I changed a few extra things, more details in that PR. It's now released and available as SQLModel 0.0.14! 🎉 🌮

[samyxdev]

Not sure if replying to a closed PR is the way, but it seems that the issue raised by @50Bytes-dev about .model_copy raising an error is still hasn't been fixed with the new release 0.0.14, despite the fix he proposed (thanks for that !).
Am I missing something or was the matter moved to a different issue ?

Below is a quote of the mentionned error:

@ogabrielluiz@honglei .model_copy(...) raise error:

object has no attribute '__pydantic_extra__'

service_category: ServiceCategory=awaitself.get_object_or_404(id, company.id, async_db)
new_service_category=service_category.model_copy(update={'id': None})

Use this test:

deftest_model_copy(clear_sqlmodel):
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
name: strsecret_name: strage: Optional[int] =Nonehero=Hero(name="Deadpond", secret_name="Dive Wilson", age=25)
engine=create_engine("sqlite://")
SQLModel.metadata.create_all(engine)
withSession(engine) assession:
session.add(hero)
session.commit()
session.refresh(hero)
db_hero=session.get(Hero, hero.id)
copy=db_hero.model_copy(update={"name": "Deadpond Copy"})
assertcopy.name=="Deadpond Copy"and \
copy.secret_name=="Dive Wilson"and \
copy.age==25

fixed honglei#1

[ChrisNi888]

This error 'TypeError: issubclass() arg 1 must be a class' seems not fixed. ( I use Pydantic 2.7 and SQLModel 0.0.16. ) So I use Pydantic 1.0.11 and SQLModel 0.08. If anyone has same problem, hope it helps.

[AntonDeMeester]

Upgrades to Pydantic 2, using the new pydantic models. Switches for v1 and v2 can be built later

Builds on #563 to upgrade SQL Alchemy

Still WIP, did some uglier hacks which could provide some problems.

Most of it is just changing code with different naming of Pydantic 2. However, the validation of empty initialisations is broken because that now happens in rust and there is no static validate_model anymore. For now I set defaults to all provided Fields is there are none, so that it will always inititialise, but that screws a bit with the logic. This is also annoying for cls.model_validate as it screws up SQL Alchemy.

[luoshuijs]

Thank you very much for your PR. I can use it with SQLModel in the pydantic v2 environment, and I've found some issues and have some suggestions.

[luoshuijs]

NoneType has been moved to types in Python 3.10+.
https://docs.python.org/3/library/types.html#types.NoneType

[luoshuijs]

In the test tests\test_tutorial\test_delete, the following error occurred: AttributeError: 'Hero' object has no attribute '__pydantic_extra__'.

You can solve this problem by adding the following code to SQLmodel:

def__new__(cls, *args: Any, **kwargs: Any) ->Any:
new_object=super().__new__(cls)
object.__setattr__(new_object, "__pydantic_fields_set__", set())
returnnew_object

[AntonDeMeester]

I got almost everything working except for Open API. It provides optional for the OpenAPI if you use a table as a request or response type. While this isn't correct in the strict sense, it is correct for Pydantic as empty initialisations need to be supported. I tried a couple of other ways (creating a second class for the optional initialisations or trying to override the OpenAPI generation) but none of them worked out.

For now I commented the OpenAPI assertions, but @tiangolo will need to decide how to handle it. As he wrote FastAPI I'm sure he'll have better ideas 😄 I just hope I can give him some inspiration with this.

[honglei]

the following test not pass:

fromsqlmodelimportSQLModel, FieldclassArticle(SQLModel, table=True):
name : str|None=Field( description='the name of article')

Error:

 File "crud_test\test_sqlmodel.py", line 2, in <module>
class Article(SQLModel, table=True):
 File "crud_test\sqlmodel\main.py", line 318, in __new__
col = get_column_from_field(v)
 File "crud_test\sqlmodel\main.py", line 464, in get_column_from_field
sa_type = get_sqlalchemy_type(field)
 File "crud_test\sqlmodel\main.py", line 417, in get_sqlalchemy_type
if issubclass(type_, str):
builtins.TypeError: issubclass() arg 1 must be a class

[yallxe]

@honglei try Optional[str] instead of str | None

I think just one more case must be checked somewhere.

[honglei]

@honglei try Optional[str] instead of str | None

I think just one more case must be checked somewhere.

I made a pull request to support it. mbsantiago#1

[ma-ts]

Hi! Just checking if there's still something to be done for this PR, more than willing to help out!

[ogabrielluiz]

Hey! I'm also willing to help out.

[honglei]

@ma-ts@ogabrielluiz You can help by testing the PR in your projects, and reporting how well it works.
Right now my project based on https://github.com/mbsantiago/sqlmodel, with my version at: https://github.com/honglei/sqlmodel

[ogabrielluiz]

@honglei
After the normal pydantic migration which was pretty straightforward all Langflow's tests are passing. Will perform some more tests ASAP.

Installed with poetry using sqlmodel = { git = "https://github.com/honglei/sqlmodel.git", branch = "main" }

[50Bytes-dev]

@ogabrielluiz@honglei
.model_copy(...) raise error:

object has no attribute '__pydantic_extra__'

service_category: ServiceCategory=awaitself.get_object_or_404(id, company.id, async_db)
new_service_category=service_category.model_copy(update={'id': None})

Use this test:

deftest_model_copy(clear_sqlmodel):
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
name: strsecret_name: strage: Optional[int] =Nonehero=Hero(name="Deadpond", secret_name="Dive Wilson", age=25)
engine=create_engine("sqlite://")
SQLModel.metadata.create_all(engine)
withSession(engine) assession:
session.add(hero)
session.commit()
session.refresh(hero)
db_hero=session.get(Hero, hero.id)
copy=db_hero.model_copy(update={"name": "Deadpond Copy"})
assertcopy.name=="Deadpond Copy"and \
copy.secret_name=="Dive Wilson"and \
copy.age==25

[50Bytes-dev]

@ogabrielluiz@honglei .model_copy(...) raise error:

object has no attribute '__pydantic_extra__'

service_category: ServiceCategory=awaitself.get_object_or_404(id, company.id, async_db)
new_service_category=service_category.model_copy(update={'id': None})

Use this test:

deftest_model_copy(clear_sqlmodel):
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
name: strsecret_name: strage: Optional[int] =Nonehero=Hero(name="Deadpond", secret_name="Dive Wilson", age=25)
engine=create_engine("sqlite://")
SQLModel.metadata.create_all(engine)
withSession(engine) assession:
session.add(hero)
session.commit()
session.refresh(hero)
db_hero=session.get(Hero, hero.id)
copy=db_hero.model_copy(update={"name": "Deadpond Copy"})
assertcopy.name=="Deadpond Copy"and \
copy.secret_name=="Dive Wilson"and \
copy.age==25

fixed honglei#1

[8thgencore]

I wonder if @tiangolo will notice this sentence

[MattOates]

Not strictly a SQLModels problem, but if you try and use the current SQLModels 0.0.8 with FastAPI 0.103.0 where you've targeted fastapi[all] conflicts are created because of the pydantic version pins of SQLModel, as pydantic-settings requires a much later version than the max pinned. Just like to add a +1 for this PR being quite important to get merged.

[MatsiukMykola]

topic is very long, guys, possible to use with this PR?
@tiangolo can please comment? Your package is awesome, very needs an update.
Anyway big th! (hello from Ukraine)

[50Bytes-dev]

topic is very long, guys, possible to use with this PR? @tiangolo can please comment? Your package is awesome, very needs an update. Anyway big th! (hello from Ukraine)

@MatsiukMykola I'm using it in my project. Everything's great
Use this https://github.com/honglei/sqlmodel

[ikreb7]

The type EmailStr doesn't work. I get the error ValueError(f"The field {field.title} has no matching SQLAlchemy type").

[50Bytes-dev]

The type EmailStr doesn't work. I get the error ValueError(f"The field {field.title} has no matching SQLAlchemy type").

@ikreb7 use this

my_email: EmailStr=Field(sa_column=Column(String))

[honglei]

@ikreb7@50Bytes-dev support the following in my fork(https://github.com/honglei/sqlmodel), but how to change test.yml to do pip install pydantic[email]?

http: HttpUrl=Field(max_length=250)
email: EmailStrname_email : NameEmail=Field( max_length=50)
import_string:ImportString=Field(max_length=200, min_length=100)

[50Bytes-dev]

@ikreb7@50Bytes-dev support the following in my fork(https://github.com/honglei/sqlmodel), but how to change test.yml to do pip install pydantic[email]?

http: HttpUrl=Field(max_length=250)
email: EmailStrname_email : NameEmail=Field( max_length=50)
import_string:ImportString=Field(max_length=200, min_length=100)

@honglei
In pyproject.toml update line 36

pydantic = { version = "^2.1.1", extras = ["email"] }

[honglei]

@50Bytes-dev thanks!

[wojciech-mazur-mlg]

@honglei what's the status of your fork? Can we use it till a stable version of SQLmodel is released?

[honglei]

I used in my projects, it works fine. wojciech-mazur-mlg ***@***.***> 于 2023年9月18日周一 下午7:30写道：

…

@honglei <https://github.com/honglei> what's the status of your fork? Can we use it till a stable version of SQLmodel is released? — Reply to this email directly, view it on GitHub <#632 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHW5AVSOKK64F5EAV4P7LTX3AWDZANCNFSM6AAAAAA27PJTWM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[AntonDeMeester]

@honglei what's the status of your fork? Can we use it till a stable version of SQLmodel is released?

I have tried it out a bit and it seems to work. Some other people are also saying it works. But I would say it's not official until @tiangolo approves it or makes his own version. I also won't be trying everything out to be honest 😅

[BoManev]

After I updated, the "alias" stopped working

class UserBase(SQLModel):
password_hash: str = Field(alias="password")
def create_user(user: UserBase):
pass

The validation in create_user fails with "password ", but works with "password_hash"

[honglei]

@BoManev@AntonDeMeester
In pydantic, function pydantic.fields.Field will use alias to infer validation_alias,
but in sqlmodel.main.FieldInfo, validation_alias is not set, so alias not work under pydantic_core

https://github.com/pydantic/pydantic/blob/188018ccd0c26085e51e632ae4f909b3ee31a0f7/pydantic/fields.py#L819

[BoManev]

@honglei Thank you for the fast reply. In that case should I create a class using pydantic BaseModel for validation of my inputs and then map it with model_validate to my SQLModel.

I also found this #374

EDIT: Generally, how to handle data validation when field names change and data needs to be modified. In my case I receive "password", but internally I want my types to carry "password_hash". In Rust, I can specify Into/To/From traits and convert between types. Can I do something similar with pydantic/SQLModel, define the conversion of 1 data schema to other.

[JanBeelte]

Hey folks,
what is the current status and what are the next steps here?
Is there any ETA when this one will be merged to master?
I could also take over some tasks in case that helps, given I desperately need pydantic v2 support otherwise I cannot use SQLModel.
Best,
Jan

[lawrenceeldridge]

Really looking forward to seeing these changes go in - carrying out an early preview as I'm already using the latest versions. One thing I've observed, I've been struggling a little with using the async db setup in the context of pulling in additional relationships. All of the other routes work fine from this tutorial, but when I call read_hero I get the following error:

fastapi.exceptions.ResponseValidationError: 1 validation errors:
{'type': 'get_attribute_error', 'loc': ('response', 'team'), 'msg': "Error extracting attribute: MissingGreenlet: greenlet_spawn has not been called; can't call await_only() here. Was IO attempted in an unexpected place? (Background on this error at: https://sqlalche.me/e/20/xd2s)", 'input': Hero(secret_name='Batman', id=1, team_id=1, name='Bruce Wayne', age=42), 'ctx': {'error': "MissingGreenlet: greenlet_spawn has not been called; can't call await_only() here. Was IO attempted in an unexpected place? (Background on this error at: https://sqlalche.me/e/20/xd2s)"}, 'url': 'https://errors.pydantic.dev/2.4/v/get_attribute_error'}

This only happens when the response_model is set to HeroReadWithTeam, as the expectation is that we'd want to include teams with the hero. Using HeroRead works fine.

[lawrenceeldridge]

This is now resolved with the help of another post here. For relationships between models you may need to leverage the correct loading styles through the sa_relationship_kwargs property that is offered by SQLModel. For the curious this is how I've built up the tutorial with async in mind below.

In SQL Alchemy there's more reading here on the subject of lazy loading.

Really happy, so will crack on :)

fromcontextlibimportasynccontextmanagerfromtypingimportList, OptionalfromfastapiimportDepends, FastAPI, HTTPException, Queryfromsqlalchemy.ext.asyncioimportasync_sessionmaker, create_async_enginefromsqlmodelimportField, Relationship, SQLModelfromsqlalchemy.sql.expressionimportselectfromsqlmodel.ext.asyncio.sessionimportAsyncSessionfromconfigimportconfigclassTeamBase(SQLModel):
name: str=Field(index=True)
headquarters: strclassTeam(TeamBase, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
heroes: List["Hero"] =Relationship(back_populates="team", sa_relationship_kwargs={"lazy": "selectin"})
classTeamCreate(TeamBase):
passclassTeamRead(TeamBase):
id: intclassTeamUpdate(SQLModel):
id: Optional[int] =Nonename: Optional[str] =Noneheadquarters: Optional[str] =NoneclassHeroBase(SQLModel):
name: str=Field(index=True)
secret_name: strage: Optional[int] =Field(default=None, index=True)
team_id: Optional[int] =Field(default=None, foreign_key="team.id")
classHero(HeroBase, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
team: Optional[Team] =Relationship(back_populates="heroes", sa_relationship_kwargs={"lazy": "joined"})
classHeroRead(HeroBase):
id: intclassHeroCreate(HeroBase):
passclassHeroUpdate(SQLModel):
name: Optional[str] =Nonesecret_name: Optional[str] =Noneage: Optional[int] =Noneteam_id: Optional[int] =NoneclassHeroReadWithTeam(HeroRead):
team: Optional[TeamRead] =NoneclassTeamReadWithHeroes(TeamRead):
heroes: List[HeroRead] = []
engine=create_async_engine(config.DB_HOST, echo=True, future=True)
asyncdefinit_db():
asyncwithengine.begin() asconn:
awaitconn.run_sync(SQLModel.metadata.create_all)
asyncdefget_session() ->AsyncSession:
async_session=async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
asyncwithasync_session() assession:
yieldsession@asynccontextmanagerasyncdeflifespan(app: FastAPI):
# on startupawaitinit_db()
yield# on shutdownawaitengine.dispose()
app=FastAPI(lifespan=lifespan)
@app.post("/heroes", response_model=HeroRead)asyncdefcreate_hero(*, session: AsyncSession=Depends(get_session), hero: HeroCreate):
db_hero=Hero.model_validate(hero)
session.add(db_hero)
awaitsession.commit()
awaitsession.refresh(db_hero)
returndb_hero@app.get("/heroes", response_model=List[HeroRead])asyncdefread_heroes(
*,
session: AsyncSession=Depends(get_session),
offset: int=0,
limit: int=Query(default=100, lte=100),
):
statement=select(Hero).offset(offset).limit(limit)
heroes=awaitsession.execute(statement)
returnheroes.scalars().all()
@app.get("/heroes/{hero_id}", response_model=HeroReadWithTeam)asyncdefread_hero(*, session: AsyncSession=Depends(get_session), hero_id: int):
hero=awaitsession.get(Hero, hero_id)
ifnothero:
raiseHTTPException(status_code=404, detail="Hero not found")
returnhero@app.patch("/heroes/{hero_id}", response_model=HeroRead)asyncdefupdate_hero(
*, session: AsyncSession=Depends(get_session), hero_id: int, hero: HeroUpdate
):
db_hero=awaitsession.get(Hero, hero_id)
ifnotdb_hero:
raiseHTTPException(status_code=404, detail="Hero not found")
hero_data=hero.model_dump(exclude_unset=True)
forkey, valueinhero_data.items():
setattr(db_hero, key, value)
session.add(db_hero)
awaitsession.commit()
awaitsession.refresh(db_hero)
returndb_hero@app.delete("/heroes/{hero_id}")asyncdefdelete_hero(*, session: AsyncSession=Depends(get_session), hero_id: int):
hero=awaitsession.get(Hero, hero_id)
ifnothero:
raiseHTTPException(status_code=404, detail="Hero not found")
awaitsession.delete(hero)
awaitsession.commit()
return {"ok": True}
@app.post("/teams", response_model=TeamRead)asyncdefcreate_team(*, session: AsyncSession=Depends(get_session), team: TeamCreate):
db_team=Team.model_validate(team)
session.add(db_team)
awaitsession.commit()
awaitsession.refresh(db_team)
returndb_team@app.get("/teams", response_model=List[TeamRead])asyncdefread_teams(
*,
session: AsyncSession=Depends(get_session),
offset: int=0,
limit: int=Query(default=100, lte=100),
):
statement=select(Team).offset(offset).limit(limit)
teams=awaitsession.execute(statement)
returnteams.scalars().all()
@app.get("/teams/{team_id}", response_model=TeamReadWithHeroes)asyncdefread_team(*, team_id: int, session: AsyncSession=Depends(get_session)):
team=awaitsession.get(Team, team_id)
ifnotteam:
raiseHTTPException(status_code=404, detail="Team not found")
returnteam@app.patch("/teams/{team_id}", response_model=TeamRead)asyncdefupdate_team(
*,
session: AsyncSession=Depends(get_session),
team_id: int,
team: TeamUpdate,
):
db_team=awaitsession.get(Team, team_id)
ifnotdb_team:
raiseHTTPException(status_code=404, detail="Team not found")
team_data=team.model_dump(exclude_unset=True)
forkey, valueinteam_data.items():
setattr(db_team, key, value)
session.add(db_team)
awaitsession.commit()
awaitsession.refresh(db_team)
returndb_team@app.delete("/teams/{team_id}")asyncdefdelete_team(*, session: AsyncSession=Depends(get_session), team_id: int):
team=awaitsession.get(Team, team_id)
ifnotteam:
raiseHTTPException(status_code=404, detail="Team not found")
awaitsession.delete(team)
awaitsession.commit()
return {"ok": True}

[blakewatters]

Joining the party here trying to update a fairly large data model the fork.

One thing I am working on resolving and curious if anyone else has run into is an issue with model class inheritance producing SQLAlchemy errors regarding column reuse:

sqlalchemy.exc.ArgumentError: Column object 'created_at' already assigned to Table 'business_address_links'

My TimestampModel looks like:

classTimestampModel(BaseModel):
created_at: datetime.datetime=sqlmodel.Field(
sa_column=sqlalchemy.Column(
sqlalchemy.DateTime(timezone=True),
server_default=sqlalchemy.func.statement_timestamp(),
nullable=False
)
)
updated_at: typing.Optional[datetime.datetime] =sqlmodel.Field(
sa_column=sqlalchemy.Column(
sqlalchemy.DateTime(timezone=True),
onupdate=sqlalchemy.func.statement_timestamp(),
nullable=True
)
)

It looks like the Column definition is being reused across inherited subclasses

[MatsiukMykola]

always was interesting how pull to sa.Column - column comment based on Field Description?

[honglei]

@blakewatters try https://github.com/honglei/sqlmodel/blob/3005495a3ec6c8216b31cbd623f91c7bc8ba174f/tests/test_class_hierarchy.py#L24C17-L24C17

[norton120]

➕ on thanks for putting this PR up, my team is a few weeks into a new project and I'd hate to be stuck behind a major on 2 core libraries this early on, so this is huge. I'm running into a strange issue creating a one-to-many relationship (using this branch), any ideas would be appreciated. Here's the error:

sqlalchemy.exc.ArgumentError: relationship 'venues' expects a class or a mapper argument (received: <class 'list'>)

the related models:

classOrganization(SQLModel, table=True):
id: Optional[int] =sqlmodel.Field(default=None, primary_key=True)
venues: List["Venue"] =sqlmodel.Relationship(back_populates="organization")
@classmethodasyncdeflist(cls, session: AsyncSession):
query=select(cls)
result_from_db=awaitsession.execute(query)
collection=result_from_db.fetchall()
returncollectionclassVenue(SQLModel, table=True):
id: Optional[int] =sqlmodel.Field(default=None, primary_key=True)
organization_id: int=sqlmodel.Field(foreign_key="organization.id")
organization: Organization=sqlmodel.Relationship(back_populates="venues")

and the calling functionality:

classDatabaseConnection:
def__init__(self):
url=sqlalchemy.URL.create(*url_args)
self.async_engine=sqlalchemy.ext.asyncio.create_async_engine(
url, echo=True, future=True
)
asyncdefget_async_session(self) ->AsyncSession:
async_session=sessionmaker(
bind=self.async_engine, class_=AsyncSession, expire_on_commit=False
)
asyncwithasync_session() assession:
yieldsessiondb_session=DatabaseConnection().get_async_session@router.get("/")asyncdefindex(*, session=Depends(db_session)):
returnawaitOrganization.list(session=session)

I've tried changing annotation types, explicitly passing an sa.relationship to the Relationship, setting lazy loading... I'm not super familiar with the SQLModel code base so I'm not sure where to look next.

[nikdavis]

It looks like the both of the goals prior to migrating to sqlalchemy 2.0, and pydantic 2.0 have been reached (supporting the latest pre 2.0 versions of each). The next goal in the roadmap is migrating each to 2.0.

Any update here?

[github-actions]

📝 Docs preview for commit 7ecbc38 at: https://5165ba05.sqlmodel.pages.dev

[github-actions]

📝 Docs preview for commit ab07514 at: https://398280da.sqlmodel.pages.dev

[BoManev]

@AntonDeMeester I have been using an older commit from this repo, however today I tried to update my dependencies and got the following errors.

Because no versions of pydantic-settings match >2.0.0,<2.0.1 || >2.0.1,<2.0.2 || >2.0.2,<2.0.3 || >2.0.3,<2.1.0 || >2.1.0
and pydantic-settings (2.0.0) depends on pydantic (>=2.0b3), pydantic-settings (>=2.0.0,<2.0.1 || >2.0.1,<2.0.2 || >2.0.2,<2.0.3 || >2.0.3,<2.1.0 || >2.1.0) requires pydantic (>=2.0b3).
And because pydantic-settings (2.0.1) depends on pydantic (>=2.0.1), pydantic-settings (>=2.0.0,<2.0.2 || >2.0.2,<2.0.3 || >2.0.3,<2.1.0 || >2.1.0) requires pydantic (>=2.0b3).
And because pydantic-settings (2.0.2) depends on pydantic (>=2.0.1)
and pydantic-settings (2.0.3) depends on pydantic (>=2.0.1), pydantic-settings (>=2.0.0,<2.1.0 || >2.1.0) requires pydantic (>=2.0b3).
And because pydantic-settings (2.1.0) depends on pydantic (>=2.3.0)
and sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head depends on pydantic (^1.9.0), sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head is incompatible with pydantic-settings (>=2.0.0).
And because fastapi (0.103.2) depends on pydantic-settings (>=2.0.0)
and no versions of fastapi match >0.103.2,<0.104.0, sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head is incompatible with fastapi (>=0.103.2,<0.104.0).
So, because api depends on both fastapi (^0.103.2) and sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head, version solving failed.

Snippet from my pyproject.toml

fastapi = {extras = ["all"], version = "^0.103.2"}
sqlmodel = {git = "https://github.com/tiangolo/sqlmodel.git", rev = "refs/pull/632/head"}

The older commit from this PR is using pydantic = { version = ">=2.1.1,<=2.4.2", extras = ["email"] } this worked fine with pydantic-settings from fastapi extras

[AntonDeMeester]

@AntonDeMeester I have been using an older commit from this repo, however today I tried to update my dependencies and got the following errors.

The older commit from this PR is using pydantic = { version = ">=2.1.1,<=2.4.2", extras = ["email"] } this worked fine with pydantic-settings from fastapi extras

Ye I've been working on getting both Pydantic v1 and v2 to work at the same time, I might have broken some things. I'll change the PR to use the old commit again and push my changes to a new PR on top of this one until both work.

[github-actions]

📝 Docs preview for commit d0ae3e8 at: https://ff83edad.sqlmodel.pages.dev

[BoManev]

@AntonDeMeester This commit still doesn't work
The one Im using is this f926d9d

[honglei]

@AntonDeMeester sqlmodel has drop support for sqlalchemy<2.0, maybe next release will drop support for pydantic v1,
the author has no interests to fix some bugs in pydantic v1, which make me move my project to pydantic v2.

https://github.com/tiangolo/sqlmodel/blob/b1c2f822c9db036afe1b7ec81d335368e9546756/pyproject.toml#L34

[ogabrielluiz]

@AntonDeMeester how are you?

I've been using your fork internally for a while now and it seems stable.

Do you see this PR being merged this week?

Thanks in advance

[AntonDeMeester]

Closing in favour of @tiangolo PR here

[tiangolo]

Thank you for your work @AntonDeMeester! I included your commits in #722 (you can see your badge now says "Contributor" 😎 ), I changed a few extra things, more details in that PR. It's now released and available as SQLModel 0.0.14! 🎉 🌮

[samyxdev]

Not sure if replying to a closed PR is the way, but it seems that the issue raised by @50Bytes-dev about .model_copy raising an error is still hasn't been fixed with the new release 0.0.14, despite the fix he proposed (thanks for that !).
Am I missing something or was the matter moved to a different issue ?

Below is a quote of the mentionned error:

@ogabrielluiz@honglei .model_copy(...) raise error:

object has no attribute '__pydantic_extra__'

service_category: ServiceCategory=awaitself.get_object_or_404(id, company.id, async_db)
new_service_category=service_category.model_copy(update={'id': None})

Use this test:

deftest_model_copy(clear_sqlmodel):
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
name: strsecret_name: strage: Optional[int] =Nonehero=Hero(name="Deadpond", secret_name="Dive Wilson", age=25)
engine=create_engine("sqlite://")
SQLModel.metadata.create_all(engine)
withSession(engine) assession:
session.add(hero)
session.commit()
session.refresh(hero)
db_hero=session.get(Hero, hero.id)
copy=db_hero.model_copy(update={"name": "Deadpond Copy"})
assertcopy.name=="Deadpond Copy"and \
copy.secret_name=="Dive Wilson"and \
copy.age==25

fixed honglei#1

[ChrisNi888]

This error 'TypeError: issubclass() arg 1 must be a class' seems not fixed. ( I use Pydantic 2.7 and SQLModel 0.0.16. ) So I use Pydantic 1.0.11 and SQLModel 0.08. If anyone has same problem, hope it helps.

[AntonDeMeester]

Upgrades to Pydantic 2, using the new pydantic models. Switches for v1 and v2 can be built later

Builds on #563 to upgrade SQL Alchemy

Still WIP, did some uglier hacks which could provide some problems.

Most of it is just changing code with different naming of Pydantic 2. However, the validation of empty initialisations is broken because that now happens in rust and there is no static validate_model anymore. For now I set defaults to all provided Fields is there are none, so that it will always inititialise, but that screws a bit with the logic. This is also annoying for cls.model_validate as it screws up SQL Alchemy.

[luoshuijs]

Thank you very much for your PR. I can use it with SQLModel in the pydantic v2 environment, and I've found some issues and have some suggestions.

[luoshuijs]

NoneType has been moved to types in Python 3.10+.
https://docs.python.org/3/library/types.html#types.NoneType

[luoshuijs]

In the test tests\test_tutorial\test_delete, the following error occurred: AttributeError: 'Hero' object has no attribute '__pydantic_extra__'.

You can solve this problem by adding the following code to SQLmodel:

def__new__(cls, *args: Any, **kwargs: Any) ->Any:
new_object=super().__new__(cls)
object.__setattr__(new_object, "__pydantic_fields_set__", set())
returnnew_object

[AntonDeMeester]

I got almost everything working except for Open API. It provides optional for the OpenAPI if you use a table as a request or response type. While this isn't correct in the strict sense, it is correct for Pydantic as empty initialisations need to be supported. I tried a couple of other ways (creating a second class for the optional initialisations or trying to override the OpenAPI generation) but none of them worked out.

For now I commented the OpenAPI assertions, but @tiangolo will need to decide how to handle it. As he wrote FastAPI I'm sure he'll have better ideas 😄 I just hope I can give him some inspiration with this.

[honglei]

the following test not pass:

fromsqlmodelimportSQLModel, FieldclassArticle(SQLModel, table=True):
name : str|None=Field( description='the name of article')

Error:

 File "crud_test\test_sqlmodel.py", line 2, in <module>
class Article(SQLModel, table=True):
 File "crud_test\sqlmodel\main.py", line 318, in __new__
col = get_column_from_field(v)
 File "crud_test\sqlmodel\main.py", line 464, in get_column_from_field
sa_type = get_sqlalchemy_type(field)
 File "crud_test\sqlmodel\main.py", line 417, in get_sqlalchemy_type
if issubclass(type_, str):
builtins.TypeError: issubclass() arg 1 must be a class

[yallxe]

@honglei try Optional[str] instead of str | None

I think just one more case must be checked somewhere.

[honglei]

@honglei try Optional[str] instead of str | None

I think just one more case must be checked somewhere.

I made a pull request to support it. mbsantiago#1

[ma-ts]

Hi! Just checking if there's still something to be done for this PR, more than willing to help out!

[ogabrielluiz]

Hey! I'm also willing to help out.

[honglei]

@ma-ts@ogabrielluiz You can help by testing the PR in your projects, and reporting how well it works.
Right now my project based on https://github.com/mbsantiago/sqlmodel, with my version at: https://github.com/honglei/sqlmodel

[ogabrielluiz]

@honglei
After the normal pydantic migration which was pretty straightforward all Langflow's tests are passing. Will perform some more tests ASAP.

Installed with poetry using sqlmodel = { git = "https://github.com/honglei/sqlmodel.git", branch = "main" }

[50Bytes-dev]

@ogabrielluiz@honglei
.model_copy(...) raise error:

object has no attribute '__pydantic_extra__'

service_category: ServiceCategory=awaitself.get_object_or_404(id, company.id, async_db)
new_service_category=service_category.model_copy(update={'id': None})

Use this test:

deftest_model_copy(clear_sqlmodel):
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
name: strsecret_name: strage: Optional[int] =Nonehero=Hero(name="Deadpond", secret_name="Dive Wilson", age=25)
engine=create_engine("sqlite://")
SQLModel.metadata.create_all(engine)
withSession(engine) assession:
session.add(hero)
session.commit()
session.refresh(hero)
db_hero=session.get(Hero, hero.id)
copy=db_hero.model_copy(update={"name": "Deadpond Copy"})
assertcopy.name=="Deadpond Copy"and \
copy.secret_name=="Dive Wilson"and \
copy.age==25

[50Bytes-dev]

@ogabrielluiz@honglei .model_copy(...) raise error:

object has no attribute '__pydantic_extra__'

service_category: ServiceCategory=awaitself.get_object_or_404(id, company.id, async_db)
new_service_category=service_category.model_copy(update={'id': None})

Use this test:

deftest_model_copy(clear_sqlmodel):
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
name: strsecret_name: strage: Optional[int] =Nonehero=Hero(name="Deadpond", secret_name="Dive Wilson", age=25)
engine=create_engine("sqlite://")
SQLModel.metadata.create_all(engine)
withSession(engine) assession:
session.add(hero)
session.commit()
session.refresh(hero)
db_hero=session.get(Hero, hero.id)
copy=db_hero.model_copy(update={"name": "Deadpond Copy"})
assertcopy.name=="Deadpond Copy"and \
copy.secret_name=="Dive Wilson"and \
copy.age==25

fixed honglei#1

[8thgencore]

I wonder if @tiangolo will notice this sentence

[MattOates]

Not strictly a SQLModels problem, but if you try and use the current SQLModels 0.0.8 with FastAPI 0.103.0 where you've targeted fastapi[all] conflicts are created because of the pydantic version pins of SQLModel, as pydantic-settings requires a much later version than the max pinned. Just like to add a +1 for this PR being quite important to get merged.

[MatsiukMykola]

topic is very long, guys, possible to use with this PR?
@tiangolo can please comment? Your package is awesome, very needs an update.
Anyway big th! (hello from Ukraine)

[50Bytes-dev]

topic is very long, guys, possible to use with this PR? @tiangolo can please comment? Your package is awesome, very needs an update. Anyway big th! (hello from Ukraine)

@MatsiukMykola I'm using it in my project. Everything's great
Use this https://github.com/honglei/sqlmodel

[ikreb7]

The type EmailStr doesn't work. I get the error ValueError(f"The field {field.title} has no matching SQLAlchemy type").

[50Bytes-dev]

The type EmailStr doesn't work. I get the error ValueError(f"The field {field.title} has no matching SQLAlchemy type").

@ikreb7 use this

my_email: EmailStr=Field(sa_column=Column(String))

[honglei]

@ikreb7@50Bytes-dev support the following in my fork(https://github.com/honglei/sqlmodel), but how to change test.yml to do pip install pydantic[email]?

http: HttpUrl=Field(max_length=250)
email: EmailStrname_email : NameEmail=Field( max_length=50)
import_string:ImportString=Field(max_length=200, min_length=100)

[50Bytes-dev]

@ikreb7@50Bytes-dev support the following in my fork(https://github.com/honglei/sqlmodel), but how to change test.yml to do pip install pydantic[email]?

http: HttpUrl=Field(max_length=250)
email: EmailStrname_email : NameEmail=Field( max_length=50)
import_string:ImportString=Field(max_length=200, min_length=100)

@honglei
In pyproject.toml update line 36

pydantic = { version = "^2.1.1", extras = ["email"] }

[honglei]

@50Bytes-dev thanks!

[wojciech-mazur-mlg]

@honglei what's the status of your fork? Can we use it till a stable version of SQLmodel is released?

[honglei]

I used in my projects, it works fine. wojciech-mazur-mlg ***@***.***> 于 2023年9月18日周一 下午7:30写道：

…

@honglei <https://github.com/honglei> what's the status of your fork? Can we use it till a stable version of SQLmodel is released? — Reply to this email directly, view it on GitHub <#632 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHW5AVSOKK64F5EAV4P7LTX3AWDZANCNFSM6AAAAAA27PJTWM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[AntonDeMeester]

@honglei what's the status of your fork? Can we use it till a stable version of SQLmodel is released?

I have tried it out a bit and it seems to work. Some other people are also saying it works. But I would say it's not official until @tiangolo approves it or makes his own version. I also won't be trying everything out to be honest 😅

[BoManev]

After I updated, the "alias" stopped working

class UserBase(SQLModel):
password_hash: str = Field(alias="password")
def create_user(user: UserBase):
pass

The validation in create_user fails with "password ", but works with "password_hash"

[honglei]

@BoManev@AntonDeMeester
In pydantic, function pydantic.fields.Field will use alias to infer validation_alias,
but in sqlmodel.main.FieldInfo, validation_alias is not set, so alias not work under pydantic_core

https://github.com/pydantic/pydantic/blob/188018ccd0c26085e51e632ae4f909b3ee31a0f7/pydantic/fields.py#L819

[BoManev]

@honglei Thank you for the fast reply. In that case should I create a class using pydantic BaseModel for validation of my inputs and then map it with model_validate to my SQLModel.

I also found this #374

EDIT: Generally, how to handle data validation when field names change and data needs to be modified. In my case I receive "password", but internally I want my types to carry "password_hash". In Rust, I can specify Into/To/From traits and convert between types. Can I do something similar with pydantic/SQLModel, define the conversion of 1 data schema to other.

[JanBeelte]

Hey folks,
what is the current status and what are the next steps here?
Is there any ETA when this one will be merged to master?
I could also take over some tasks in case that helps, given I desperately need pydantic v2 support otherwise I cannot use SQLModel.
Best,
Jan

[lawrenceeldridge]

Really looking forward to seeing these changes go in - carrying out an early preview as I'm already using the latest versions. One thing I've observed, I've been struggling a little with using the async db setup in the context of pulling in additional relationships. All of the other routes work fine from this tutorial, but when I call read_hero I get the following error:

fastapi.exceptions.ResponseValidationError: 1 validation errors:
{'type': 'get_attribute_error', 'loc': ('response', 'team'), 'msg': "Error extracting attribute: MissingGreenlet: greenlet_spawn has not been called; can't call await_only() here. Was IO attempted in an unexpected place? (Background on this error at: https://sqlalche.me/e/20/xd2s)", 'input': Hero(secret_name='Batman', id=1, team_id=1, name='Bruce Wayne', age=42), 'ctx': {'error': "MissingGreenlet: greenlet_spawn has not been called; can't call await_only() here. Was IO attempted in an unexpected place? (Background on this error at: https://sqlalche.me/e/20/xd2s)"}, 'url': 'https://errors.pydantic.dev/2.4/v/get_attribute_error'}

This only happens when the response_model is set to HeroReadWithTeam, as the expectation is that we'd want to include teams with the hero. Using HeroRead works fine.

[lawrenceeldridge]

This is now resolved with the help of another post here. For relationships between models you may need to leverage the correct loading styles through the sa_relationship_kwargs property that is offered by SQLModel. For the curious this is how I've built up the tutorial with async in mind below.

In SQL Alchemy there's more reading here on the subject of lazy loading.

Really happy, so will crack on :)

fromcontextlibimportasynccontextmanagerfromtypingimportList, OptionalfromfastapiimportDepends, FastAPI, HTTPException, Queryfromsqlalchemy.ext.asyncioimportasync_sessionmaker, create_async_enginefromsqlmodelimportField, Relationship, SQLModelfromsqlalchemy.sql.expressionimportselectfromsqlmodel.ext.asyncio.sessionimportAsyncSessionfromconfigimportconfigclassTeamBase(SQLModel):
name: str=Field(index=True)
headquarters: strclassTeam(TeamBase, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
heroes: List["Hero"] =Relationship(back_populates="team", sa_relationship_kwargs={"lazy": "selectin"})
classTeamCreate(TeamBase):
passclassTeamRead(TeamBase):
id: intclassTeamUpdate(SQLModel):
id: Optional[int] =Nonename: Optional[str] =Noneheadquarters: Optional[str] =NoneclassHeroBase(SQLModel):
name: str=Field(index=True)
secret_name: strage: Optional[int] =Field(default=None, index=True)
team_id: Optional[int] =Field(default=None, foreign_key="team.id")
classHero(HeroBase, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
team: Optional[Team] =Relationship(back_populates="heroes", sa_relationship_kwargs={"lazy": "joined"})
classHeroRead(HeroBase):
id: intclassHeroCreate(HeroBase):
passclassHeroUpdate(SQLModel):
name: Optional[str] =Nonesecret_name: Optional[str] =Noneage: Optional[int] =Noneteam_id: Optional[int] =NoneclassHeroReadWithTeam(HeroRead):
team: Optional[TeamRead] =NoneclassTeamReadWithHeroes(TeamRead):
heroes: List[HeroRead] = []
engine=create_async_engine(config.DB_HOST, echo=True, future=True)
asyncdefinit_db():
asyncwithengine.begin() asconn:
awaitconn.run_sync(SQLModel.metadata.create_all)
asyncdefget_session() ->AsyncSession:
async_session=async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
asyncwithasync_session() assession:
yieldsession@asynccontextmanagerasyncdeflifespan(app: FastAPI):
# on startupawaitinit_db()
yield# on shutdownawaitengine.dispose()
app=FastAPI(lifespan=lifespan)
@app.post("/heroes", response_model=HeroRead)asyncdefcreate_hero(*, session: AsyncSession=Depends(get_session), hero: HeroCreate):
db_hero=Hero.model_validate(hero)
session.add(db_hero)
awaitsession.commit()
awaitsession.refresh(db_hero)
returndb_hero@app.get("/heroes", response_model=List[HeroRead])asyncdefread_heroes(
*,
session: AsyncSession=Depends(get_session),
offset: int=0,
limit: int=Query(default=100, lte=100),
):
statement=select(Hero).offset(offset).limit(limit)
heroes=awaitsession.execute(statement)
returnheroes.scalars().all()
@app.get("/heroes/{hero_id}", response_model=HeroReadWithTeam)asyncdefread_hero(*, session: AsyncSession=Depends(get_session), hero_id: int):
hero=awaitsession.get(Hero, hero_id)
ifnothero:
raiseHTTPException(status_code=404, detail="Hero not found")
returnhero@app.patch("/heroes/{hero_id}", response_model=HeroRead)asyncdefupdate_hero(
*, session: AsyncSession=Depends(get_session), hero_id: int, hero: HeroUpdate
):
db_hero=awaitsession.get(Hero, hero_id)
ifnotdb_hero:
raiseHTTPException(status_code=404, detail="Hero not found")
hero_data=hero.model_dump(exclude_unset=True)
forkey, valueinhero_data.items():
setattr(db_hero, key, value)
session.add(db_hero)
awaitsession.commit()
awaitsession.refresh(db_hero)
returndb_hero@app.delete("/heroes/{hero_id}")asyncdefdelete_hero(*, session: AsyncSession=Depends(get_session), hero_id: int):
hero=awaitsession.get(Hero, hero_id)
ifnothero:
raiseHTTPException(status_code=404, detail="Hero not found")
awaitsession.delete(hero)
awaitsession.commit()
return {"ok": True}
@app.post("/teams", response_model=TeamRead)asyncdefcreate_team(*, session: AsyncSession=Depends(get_session), team: TeamCreate):
db_team=Team.model_validate(team)
session.add(db_team)
awaitsession.commit()
awaitsession.refresh(db_team)
returndb_team@app.get("/teams", response_model=List[TeamRead])asyncdefread_teams(
*,
session: AsyncSession=Depends(get_session),
offset: int=0,
limit: int=Query(default=100, lte=100),
):
statement=select(Team).offset(offset).limit(limit)
teams=awaitsession.execute(statement)
returnteams.scalars().all()
@app.get("/teams/{team_id}", response_model=TeamReadWithHeroes)asyncdefread_team(*, team_id: int, session: AsyncSession=Depends(get_session)):
team=awaitsession.get(Team, team_id)
ifnotteam:
raiseHTTPException(status_code=404, detail="Team not found")
returnteam@app.patch("/teams/{team_id}", response_model=TeamRead)asyncdefupdate_team(
*,
session: AsyncSession=Depends(get_session),
team_id: int,
team: TeamUpdate,
):
db_team=awaitsession.get(Team, team_id)
ifnotdb_team:
raiseHTTPException(status_code=404, detail="Team not found")
team_data=team.model_dump(exclude_unset=True)
forkey, valueinteam_data.items():
setattr(db_team, key, value)
session.add(db_team)
awaitsession.commit()
awaitsession.refresh(db_team)
returndb_team@app.delete("/teams/{team_id}")asyncdefdelete_team(*, session: AsyncSession=Depends(get_session), team_id: int):
team=awaitsession.get(Team, team_id)
ifnotteam:
raiseHTTPException(status_code=404, detail="Team not found")
awaitsession.delete(team)
awaitsession.commit()
return {"ok": True}

[blakewatters]

Joining the party here trying to update a fairly large data model the fork.

One thing I am working on resolving and curious if anyone else has run into is an issue with model class inheritance producing SQLAlchemy errors regarding column reuse:

sqlalchemy.exc.ArgumentError: Column object 'created_at' already assigned to Table 'business_address_links'

My TimestampModel looks like:

classTimestampModel(BaseModel):
created_at: datetime.datetime=sqlmodel.Field(
sa_column=sqlalchemy.Column(
sqlalchemy.DateTime(timezone=True),
server_default=sqlalchemy.func.statement_timestamp(),
nullable=False
)
)
updated_at: typing.Optional[datetime.datetime] =sqlmodel.Field(
sa_column=sqlalchemy.Column(
sqlalchemy.DateTime(timezone=True),
onupdate=sqlalchemy.func.statement_timestamp(),
nullable=True
)
)

It looks like the Column definition is being reused across inherited subclasses

[MatsiukMykola]

always was interesting how pull to sa.Column - column comment based on Field Description?

[honglei]

@blakewatters try https://github.com/honglei/sqlmodel/blob/3005495a3ec6c8216b31cbd623f91c7bc8ba174f/tests/test_class_hierarchy.py#L24C17-L24C17

[norton120]

➕ on thanks for putting this PR up, my team is a few weeks into a new project and I'd hate to be stuck behind a major on 2 core libraries this early on, so this is huge. I'm running into a strange issue creating a one-to-many relationship (using this branch), any ideas would be appreciated. Here's the error:

sqlalchemy.exc.ArgumentError: relationship 'venues' expects a class or a mapper argument (received: <class 'list'>)

the related models:

classOrganization(SQLModel, table=True):
id: Optional[int] =sqlmodel.Field(default=None, primary_key=True)
venues: List["Venue"] =sqlmodel.Relationship(back_populates="organization")
@classmethodasyncdeflist(cls, session: AsyncSession):
query=select(cls)
result_from_db=awaitsession.execute(query)
collection=result_from_db.fetchall()
returncollectionclassVenue(SQLModel, table=True):
id: Optional[int] =sqlmodel.Field(default=None, primary_key=True)
organization_id: int=sqlmodel.Field(foreign_key="organization.id")
organization: Organization=sqlmodel.Relationship(back_populates="venues")

and the calling functionality:

classDatabaseConnection:
def__init__(self):
url=sqlalchemy.URL.create(*url_args)
self.async_engine=sqlalchemy.ext.asyncio.create_async_engine(
url, echo=True, future=True
)
asyncdefget_async_session(self) ->AsyncSession:
async_session=sessionmaker(
bind=self.async_engine, class_=AsyncSession, expire_on_commit=False
)
asyncwithasync_session() assession:
yieldsessiondb_session=DatabaseConnection().get_async_session@router.get("/")asyncdefindex(*, session=Depends(db_session)):
returnawaitOrganization.list(session=session)

I've tried changing annotation types, explicitly passing an sa.relationship to the Relationship, setting lazy loading... I'm not super familiar with the SQLModel code base so I'm not sure where to look next.

[nikdavis]

It looks like the both of the goals prior to migrating to sqlalchemy 2.0, and pydantic 2.0 have been reached (supporting the latest pre 2.0 versions of each). The next goal in the roadmap is migrating each to 2.0.

Any update here?

[github-actions]

📝 Docs preview for commit 7ecbc38 at: https://5165ba05.sqlmodel.pages.dev

[github-actions]

📝 Docs preview for commit ab07514 at: https://398280da.sqlmodel.pages.dev

[BoManev]

@AntonDeMeester I have been using an older commit from this repo, however today I tried to update my dependencies and got the following errors.

Because no versions of pydantic-settings match >2.0.0,<2.0.1 || >2.0.1,<2.0.2 || >2.0.2,<2.0.3 || >2.0.3,<2.1.0 || >2.1.0
and pydantic-settings (2.0.0) depends on pydantic (>=2.0b3), pydantic-settings (>=2.0.0,<2.0.1 || >2.0.1,<2.0.2 || >2.0.2,<2.0.3 || >2.0.3,<2.1.0 || >2.1.0) requires pydantic (>=2.0b3).
And because pydantic-settings (2.0.1) depends on pydantic (>=2.0.1), pydantic-settings (>=2.0.0,<2.0.2 || >2.0.2,<2.0.3 || >2.0.3,<2.1.0 || >2.1.0) requires pydantic (>=2.0b3).
And because pydantic-settings (2.0.2) depends on pydantic (>=2.0.1)
and pydantic-settings (2.0.3) depends on pydantic (>=2.0.1), pydantic-settings (>=2.0.0,<2.1.0 || >2.1.0) requires pydantic (>=2.0b3).
And because pydantic-settings (2.1.0) depends on pydantic (>=2.3.0)
and sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head depends on pydantic (^1.9.0), sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head is incompatible with pydantic-settings (>=2.0.0).
And because fastapi (0.103.2) depends on pydantic-settings (>=2.0.0)
and no versions of fastapi match >0.103.2,<0.104.0, sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head is incompatible with fastapi (>=0.103.2,<0.104.0).
So, because api depends on both fastapi (^0.103.2) and sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head, version solving failed.

Snippet from my pyproject.toml

fastapi = {extras = ["all"], version = "^0.103.2"}
sqlmodel = {git = "https://github.com/tiangolo/sqlmodel.git", rev = "refs/pull/632/head"}

The older commit from this PR is using pydantic = { version = ">=2.1.1,<=2.4.2", extras = ["email"] } this worked fine with pydantic-settings from fastapi extras

[AntonDeMeester]

@AntonDeMeester I have been using an older commit from this repo, however today I tried to update my dependencies and got the following errors.

The older commit from this PR is using pydantic = { version = ">=2.1.1,<=2.4.2", extras = ["email"] } this worked fine with pydantic-settings from fastapi extras

Ye I've been working on getting both Pydantic v1 and v2 to work at the same time, I might have broken some things. I'll change the PR to use the old commit again and push my changes to a new PR on top of this one until both work.

[github-actions]

📝 Docs preview for commit d0ae3e8 at: https://ff83edad.sqlmodel.pages.dev

[BoManev]

@AntonDeMeester This commit still doesn't work
The one Im using is this f926d9d

[honglei]

@AntonDeMeester sqlmodel has drop support for sqlalchemy<2.0, maybe next release will drop support for pydantic v1,
the author has no interests to fix some bugs in pydantic v1, which make me move my project to pydantic v2.

https://github.com/tiangolo/sqlmodel/blob/b1c2f822c9db036afe1b7ec81d335368e9546756/pyproject.toml#L34

[ogabrielluiz]

@AntonDeMeester how are you?

I've been using your fork internally for a while now and it seems stable.

Do you see this PR being merged this week?

Thanks in advance

[AntonDeMeester]

Closing in favour of @tiangolo PR here

[tiangolo]

Thank you for your work @AntonDeMeester! I included your commits in #722 (you can see your badge now says "Contributor" 😎 ), I changed a few extra things, more details in that PR. It's now released and available as SQLModel 0.0.14! 🎉 🌮

[samyxdev]

Not sure if replying to a closed PR is the way, but it seems that the issue raised by @50Bytes-dev about .model_copy raising an error is still hasn't been fixed with the new release 0.0.14, despite the fix he proposed (thanks for that !).
Am I missing something or was the matter moved to a different issue ?

Below is a quote of the mentionned error:

@ogabrielluiz@honglei .model_copy(...) raise error:

object has no attribute '__pydantic_extra__'

service_category: ServiceCategory=awaitself.get_object_or_404(id, company.id, async_db)
new_service_category=service_category.model_copy(update={'id': None})

Use this test:

deftest_model_copy(clear_sqlmodel):
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
name: strsecret_name: strage: Optional[int] =Nonehero=Hero(name="Deadpond", secret_name="Dive Wilson", age=25)
engine=create_engine("sqlite://")
SQLModel.metadata.create_all(engine)
withSession(engine) assession:
session.add(hero)
session.commit()
session.refresh(hero)
db_hero=session.get(Hero, hero.id)
copy=db_hero.model_copy(update={"name": "Deadpond Copy"})
assertcopy.name=="Deadpond Copy"and \
copy.secret_name=="Dive Wilson"and \
copy.age==25

fixed honglei#1

[ChrisNi888]

This error 'TypeError: issubclass() arg 1 must be a class' seems not fixed. ( I use Pydantic 2.7 and SQLModel 0.0.16. ) So I use Pydantic 1.0.11 and SQLModel 0.08. If anyone has same problem, hope it helps.

[AntonDeMeester]

Upgrades to Pydantic 2, using the new pydantic models. Switches for v1 and v2 can be built later

Builds on #563 to upgrade SQL Alchemy

Still WIP, did some uglier hacks which could provide some problems.

Most of it is just changing code with different naming of Pydantic 2. However, the validation of empty initialisations is broken because that now happens in rust and there is no static validate_model anymore. For now I set defaults to all provided Fields is there are none, so that it will always inititialise, but that screws a bit with the logic. This is also annoying for cls.model_validate as it screws up SQL Alchemy.

[luoshuijs]

Thank you very much for your PR. I can use it with SQLModel in the pydantic v2 environment, and I've found some issues and have some suggestions.

[luoshuijs]

NoneType has been moved to types in Python 3.10+.
https://docs.python.org/3/library/types.html#types.NoneType

[luoshuijs]

In the test tests\test_tutorial\test_delete, the following error occurred: AttributeError: 'Hero' object has no attribute '__pydantic_extra__'.

You can solve this problem by adding the following code to SQLmodel:

def__new__(cls, *args: Any, **kwargs: Any) ->Any:
new_object=super().__new__(cls)
object.__setattr__(new_object, "__pydantic_fields_set__", set())
returnnew_object

[AntonDeMeester]

I got almost everything working except for Open API. It provides optional for the OpenAPI if you use a table as a request or response type. While this isn't correct in the strict sense, it is correct for Pydantic as empty initialisations need to be supported. I tried a couple of other ways (creating a second class for the optional initialisations or trying to override the OpenAPI generation) but none of them worked out.

For now I commented the OpenAPI assertions, but @tiangolo will need to decide how to handle it. As he wrote FastAPI I'm sure he'll have better ideas 😄 I just hope I can give him some inspiration with this.

[honglei]

the following test not pass:

fromsqlmodelimportSQLModel, FieldclassArticle(SQLModel, table=True):
name : str|None=Field( description='the name of article')

Error:

 File "crud_test\test_sqlmodel.py", line 2, in <module>
class Article(SQLModel, table=True):
 File "crud_test\sqlmodel\main.py", line 318, in __new__
col = get_column_from_field(v)
 File "crud_test\sqlmodel\main.py", line 464, in get_column_from_field
sa_type = get_sqlalchemy_type(field)
 File "crud_test\sqlmodel\main.py", line 417, in get_sqlalchemy_type
if issubclass(type_, str):
builtins.TypeError: issubclass() arg 1 must be a class

[yallxe]

@honglei try Optional[str] instead of str | None

I think just one more case must be checked somewhere.

[honglei]

@honglei try Optional[str] instead of str | None

I think just one more case must be checked somewhere.

I made a pull request to support it. mbsantiago#1

[ma-ts]

Hi! Just checking if there's still something to be done for this PR, more than willing to help out!

[ogabrielluiz]

Hey! I'm also willing to help out.

[honglei]

@ma-ts@ogabrielluiz You can help by testing the PR in your projects, and reporting how well it works.
Right now my project based on https://github.com/mbsantiago/sqlmodel, with my version at: https://github.com/honglei/sqlmodel

[ogabrielluiz]

@honglei
After the normal pydantic migration which was pretty straightforward all Langflow's tests are passing. Will perform some more tests ASAP.

Installed with poetry using sqlmodel = { git = "https://github.com/honglei/sqlmodel.git", branch = "main" }

[50Bytes-dev]

@ogabrielluiz@honglei
.model_copy(...) raise error:

object has no attribute '__pydantic_extra__'

service_category: ServiceCategory=awaitself.get_object_or_404(id, company.id, async_db)
new_service_category=service_category.model_copy(update={'id': None})

Use this test:

deftest_model_copy(clear_sqlmodel):
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
name: strsecret_name: strage: Optional[int] =Nonehero=Hero(name="Deadpond", secret_name="Dive Wilson", age=25)
engine=create_engine("sqlite://")
SQLModel.metadata.create_all(engine)
withSession(engine) assession:
session.add(hero)
session.commit()
session.refresh(hero)
db_hero=session.get(Hero, hero.id)
copy=db_hero.model_copy(update={"name": "Deadpond Copy"})
assertcopy.name=="Deadpond Copy"and \
copy.secret_name=="Dive Wilson"and \
copy.age==25

[50Bytes-dev]

@ogabrielluiz@honglei .model_copy(...) raise error:

object has no attribute '__pydantic_extra__'

service_category: ServiceCategory=awaitself.get_object_or_404(id, company.id, async_db)
new_service_category=service_category.model_copy(update={'id': None})

Use this test:

deftest_model_copy(clear_sqlmodel):
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
name: strsecret_name: strage: Optional[int] =Nonehero=Hero(name="Deadpond", secret_name="Dive Wilson", age=25)
engine=create_engine("sqlite://")
SQLModel.metadata.create_all(engine)
withSession(engine) assession:
session.add(hero)
session.commit()
session.refresh(hero)
db_hero=session.get(Hero, hero.id)
copy=db_hero.model_copy(update={"name": "Deadpond Copy"})
assertcopy.name=="Deadpond Copy"and \
copy.secret_name=="Dive Wilson"and \
copy.age==25

fixed honglei#1

[8thgencore]

I wonder if @tiangolo will notice this sentence

[MattOates]

Not strictly a SQLModels problem, but if you try and use the current SQLModels 0.0.8 with FastAPI 0.103.0 where you've targeted fastapi[all] conflicts are created because of the pydantic version pins of SQLModel, as pydantic-settings requires a much later version than the max pinned. Just like to add a +1 for this PR being quite important to get merged.

[MatsiukMykola]

topic is very long, guys, possible to use with this PR?
@tiangolo can please comment? Your package is awesome, very needs an update.
Anyway big th! (hello from Ukraine)

[50Bytes-dev]

topic is very long, guys, possible to use with this PR? @tiangolo can please comment? Your package is awesome, very needs an update. Anyway big th! (hello from Ukraine)

@MatsiukMykola I'm using it in my project. Everything's great
Use this https://github.com/honglei/sqlmodel

[ikreb7]

The type EmailStr doesn't work. I get the error ValueError(f"The field {field.title} has no matching SQLAlchemy type").

[50Bytes-dev]

The type EmailStr doesn't work. I get the error ValueError(f"The field {field.title} has no matching SQLAlchemy type").

@ikreb7 use this

my_email: EmailStr=Field(sa_column=Column(String))

[honglei]

@ikreb7@50Bytes-dev support the following in my fork(https://github.com/honglei/sqlmodel), but how to change test.yml to do pip install pydantic[email]?

http: HttpUrl=Field(max_length=250)
email: EmailStrname_email : NameEmail=Field( max_length=50)
import_string:ImportString=Field(max_length=200, min_length=100)

[50Bytes-dev]

@ikreb7@50Bytes-dev support the following in my fork(https://github.com/honglei/sqlmodel), but how to change test.yml to do pip install pydantic[email]?

http: HttpUrl=Field(max_length=250)
email: EmailStrname_email : NameEmail=Field( max_length=50)
import_string:ImportString=Field(max_length=200, min_length=100)

@honglei
In pyproject.toml update line 36

pydantic = { version = "^2.1.1", extras = ["email"] }

[honglei]

@50Bytes-dev thanks!

[wojciech-mazur-mlg]

@honglei what's the status of your fork? Can we use it till a stable version of SQLmodel is released?

[honglei]

I used in my projects, it works fine. wojciech-mazur-mlg ***@***.***> 于 2023年9月18日周一 下午7:30写道：

…

@honglei <https://github.com/honglei> what's the status of your fork? Can we use it till a stable version of SQLmodel is released? — Reply to this email directly, view it on GitHub <#632 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHW5AVSOKK64F5EAV4P7LTX3AWDZANCNFSM6AAAAAA27PJTWM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[AntonDeMeester]

@honglei what's the status of your fork? Can we use it till a stable version of SQLmodel is released?

I have tried it out a bit and it seems to work. Some other people are also saying it works. But I would say it's not official until @tiangolo approves it or makes his own version. I also won't be trying everything out to be honest 😅

[BoManev]

After I updated, the "alias" stopped working

class UserBase(SQLModel):
password_hash: str = Field(alias="password")
def create_user(user: UserBase):
pass

The validation in create_user fails with "password ", but works with "password_hash"

[honglei]

@BoManev@AntonDeMeester
In pydantic, function pydantic.fields.Field will use alias to infer validation_alias,
but in sqlmodel.main.FieldInfo, validation_alias is not set, so alias not work under pydantic_core

https://github.com/pydantic/pydantic/blob/188018ccd0c26085e51e632ae4f909b3ee31a0f7/pydantic/fields.py#L819

[BoManev]

@honglei Thank you for the fast reply. In that case should I create a class using pydantic BaseModel for validation of my inputs and then map it with model_validate to my SQLModel.

I also found this #374

EDIT: Generally, how to handle data validation when field names change and data needs to be modified. In my case I receive "password", but internally I want my types to carry "password_hash". In Rust, I can specify Into/To/From traits and convert between types. Can I do something similar with pydantic/SQLModel, define the conversion of 1 data schema to other.

[JanBeelte]

Hey folks,
what is the current status and what are the next steps here?
Is there any ETA when this one will be merged to master?
I could also take over some tasks in case that helps, given I desperately need pydantic v2 support otherwise I cannot use SQLModel.
Best,
Jan

[lawrenceeldridge]

Really looking forward to seeing these changes go in - carrying out an early preview as I'm already using the latest versions. One thing I've observed, I've been struggling a little with using the async db setup in the context of pulling in additional relationships. All of the other routes work fine from this tutorial, but when I call read_hero I get the following error:

fastapi.exceptions.ResponseValidationError: 1 validation errors:
{'type': 'get_attribute_error', 'loc': ('response', 'team'), 'msg': "Error extracting attribute: MissingGreenlet: greenlet_spawn has not been called; can't call await_only() here. Was IO attempted in an unexpected place? (Background on this error at: https://sqlalche.me/e/20/xd2s)", 'input': Hero(secret_name='Batman', id=1, team_id=1, name='Bruce Wayne', age=42), 'ctx': {'error': "MissingGreenlet: greenlet_spawn has not been called; can't call await_only() here. Was IO attempted in an unexpected place? (Background on this error at: https://sqlalche.me/e/20/xd2s)"}, 'url': 'https://errors.pydantic.dev/2.4/v/get_attribute_error'}

This only happens when the response_model is set to HeroReadWithTeam, as the expectation is that we'd want to include teams with the hero. Using HeroRead works fine.

[lawrenceeldridge]

This is now resolved with the help of another post here. For relationships between models you may need to leverage the correct loading styles through the sa_relationship_kwargs property that is offered by SQLModel. For the curious this is how I've built up the tutorial with async in mind below.

In SQL Alchemy there's more reading here on the subject of lazy loading.

Really happy, so will crack on :)

fromcontextlibimportasynccontextmanagerfromtypingimportList, OptionalfromfastapiimportDepends, FastAPI, HTTPException, Queryfromsqlalchemy.ext.asyncioimportasync_sessionmaker, create_async_enginefromsqlmodelimportField, Relationship, SQLModelfromsqlalchemy.sql.expressionimportselectfromsqlmodel.ext.asyncio.sessionimportAsyncSessionfromconfigimportconfigclassTeamBase(SQLModel):
name: str=Field(index=True)
headquarters: strclassTeam(TeamBase, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
heroes: List["Hero"] =Relationship(back_populates="team", sa_relationship_kwargs={"lazy": "selectin"})
classTeamCreate(TeamBase):
passclassTeamRead(TeamBase):
id: intclassTeamUpdate(SQLModel):
id: Optional[int] =Nonename: Optional[str] =Noneheadquarters: Optional[str] =NoneclassHeroBase(SQLModel):
name: str=Field(index=True)
secret_name: strage: Optional[int] =Field(default=None, index=True)
team_id: Optional[int] =Field(default=None, foreign_key="team.id")
classHero(HeroBase, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
team: Optional[Team] =Relationship(back_populates="heroes", sa_relationship_kwargs={"lazy": "joined"})
classHeroRead(HeroBase):
id: intclassHeroCreate(HeroBase):
passclassHeroUpdate(SQLModel):
name: Optional[str] =Nonesecret_name: Optional[str] =Noneage: Optional[int] =Noneteam_id: Optional[int] =NoneclassHeroReadWithTeam(HeroRead):
team: Optional[TeamRead] =NoneclassTeamReadWithHeroes(TeamRead):
heroes: List[HeroRead] = []
engine=create_async_engine(config.DB_HOST, echo=True, future=True)
asyncdefinit_db():
asyncwithengine.begin() asconn:
awaitconn.run_sync(SQLModel.metadata.create_all)
asyncdefget_session() ->AsyncSession:
async_session=async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
asyncwithasync_session() assession:
yieldsession@asynccontextmanagerasyncdeflifespan(app: FastAPI):
# on startupawaitinit_db()
yield# on shutdownawaitengine.dispose()
app=FastAPI(lifespan=lifespan)
@app.post("/heroes", response_model=HeroRead)asyncdefcreate_hero(*, session: AsyncSession=Depends(get_session), hero: HeroCreate):
db_hero=Hero.model_validate(hero)
session.add(db_hero)
awaitsession.commit()
awaitsession.refresh(db_hero)
returndb_hero@app.get("/heroes", response_model=List[HeroRead])asyncdefread_heroes(
*,
session: AsyncSession=Depends(get_session),
offset: int=0,
limit: int=Query(default=100, lte=100),
):
statement=select(Hero).offset(offset).limit(limit)
heroes=awaitsession.execute(statement)
returnheroes.scalars().all()
@app.get("/heroes/{hero_id}", response_model=HeroReadWithTeam)asyncdefread_hero(*, session: AsyncSession=Depends(get_session), hero_id: int):
hero=awaitsession.get(Hero, hero_id)
ifnothero:
raiseHTTPException(status_code=404, detail="Hero not found")
returnhero@app.patch("/heroes/{hero_id}", response_model=HeroRead)asyncdefupdate_hero(
*, session: AsyncSession=Depends(get_session), hero_id: int, hero: HeroUpdate
):
db_hero=awaitsession.get(Hero, hero_id)
ifnotdb_hero:
raiseHTTPException(status_code=404, detail="Hero not found")
hero_data=hero.model_dump(exclude_unset=True)
forkey, valueinhero_data.items():
setattr(db_hero, key, value)
session.add(db_hero)
awaitsession.commit()
awaitsession.refresh(db_hero)
returndb_hero@app.delete("/heroes/{hero_id}")asyncdefdelete_hero(*, session: AsyncSession=Depends(get_session), hero_id: int):
hero=awaitsession.get(Hero, hero_id)
ifnothero:
raiseHTTPException(status_code=404, detail="Hero not found")
awaitsession.delete(hero)
awaitsession.commit()
return {"ok": True}
@app.post("/teams", response_model=TeamRead)asyncdefcreate_team(*, session: AsyncSession=Depends(get_session), team: TeamCreate):
db_team=Team.model_validate(team)
session.add(db_team)
awaitsession.commit()
awaitsession.refresh(db_team)
returndb_team@app.get("/teams", response_model=List[TeamRead])asyncdefread_teams(
*,
session: AsyncSession=Depends(get_session),
offset: int=0,
limit: int=Query(default=100, lte=100),
):
statement=select(Team).offset(offset).limit(limit)
teams=awaitsession.execute(statement)
returnteams.scalars().all()
@app.get("/teams/{team_id}", response_model=TeamReadWithHeroes)asyncdefread_team(*, team_id: int, session: AsyncSession=Depends(get_session)):
team=awaitsession.get(Team, team_id)
ifnotteam:
raiseHTTPException(status_code=404, detail="Team not found")
returnteam@app.patch("/teams/{team_id}", response_model=TeamRead)asyncdefupdate_team(
*,
session: AsyncSession=Depends(get_session),
team_id: int,
team: TeamUpdate,
):
db_team=awaitsession.get(Team, team_id)
ifnotdb_team:
raiseHTTPException(status_code=404, detail="Team not found")
team_data=team.model_dump(exclude_unset=True)
forkey, valueinteam_data.items():
setattr(db_team, key, value)
session.add(db_team)
awaitsession.commit()
awaitsession.refresh(db_team)
returndb_team@app.delete("/teams/{team_id}")asyncdefdelete_team(*, session: AsyncSession=Depends(get_session), team_id: int):
team=awaitsession.get(Team, team_id)
ifnotteam:
raiseHTTPException(status_code=404, detail="Team not found")
awaitsession.delete(team)
awaitsession.commit()
return {"ok": True}

[blakewatters]

Joining the party here trying to update a fairly large data model the fork.

One thing I am working on resolving and curious if anyone else has run into is an issue with model class inheritance producing SQLAlchemy errors regarding column reuse:

sqlalchemy.exc.ArgumentError: Column object 'created_at' already assigned to Table 'business_address_links'

My TimestampModel looks like:

classTimestampModel(BaseModel):
created_at: datetime.datetime=sqlmodel.Field(
sa_column=sqlalchemy.Column(
sqlalchemy.DateTime(timezone=True),
server_default=sqlalchemy.func.statement_timestamp(),
nullable=False
)
)
updated_at: typing.Optional[datetime.datetime] =sqlmodel.Field(
sa_column=sqlalchemy.Column(
sqlalchemy.DateTime(timezone=True),
onupdate=sqlalchemy.func.statement_timestamp(),
nullable=True
)
)

It looks like the Column definition is being reused across inherited subclasses

[MatsiukMykola]

always was interesting how pull to sa.Column - column comment based on Field Description?

[honglei]

@blakewatters try https://github.com/honglei/sqlmodel/blob/3005495a3ec6c8216b31cbd623f91c7bc8ba174f/tests/test_class_hierarchy.py#L24C17-L24C17

[norton120]

➕ on thanks for putting this PR up, my team is a few weeks into a new project and I'd hate to be stuck behind a major on 2 core libraries this early on, so this is huge. I'm running into a strange issue creating a one-to-many relationship (using this branch), any ideas would be appreciated. Here's the error:

sqlalchemy.exc.ArgumentError: relationship 'venues' expects a class or a mapper argument (received: <class 'list'>)

the related models:

classOrganization(SQLModel, table=True):
id: Optional[int] =sqlmodel.Field(default=None, primary_key=True)
venues: List["Venue"] =sqlmodel.Relationship(back_populates="organization")
@classmethodasyncdeflist(cls, session: AsyncSession):
query=select(cls)
result_from_db=awaitsession.execute(query)
collection=result_from_db.fetchall()
returncollectionclassVenue(SQLModel, table=True):
id: Optional[int] =sqlmodel.Field(default=None, primary_key=True)
organization_id: int=sqlmodel.Field(foreign_key="organization.id")
organization: Organization=sqlmodel.Relationship(back_populates="venues")

and the calling functionality:

classDatabaseConnection:
def__init__(self):
url=sqlalchemy.URL.create(*url_args)
self.async_engine=sqlalchemy.ext.asyncio.create_async_engine(
url, echo=True, future=True
)
asyncdefget_async_session(self) ->AsyncSession:
async_session=sessionmaker(
bind=self.async_engine, class_=AsyncSession, expire_on_commit=False
)
asyncwithasync_session() assession:
yieldsessiondb_session=DatabaseConnection().get_async_session@router.get("/")asyncdefindex(*, session=Depends(db_session)):
returnawaitOrganization.list(session=session)

I've tried changing annotation types, explicitly passing an sa.relationship to the Relationship, setting lazy loading... I'm not super familiar with the SQLModel code base so I'm not sure where to look next.

[nikdavis]

It looks like the both of the goals prior to migrating to sqlalchemy 2.0, and pydantic 2.0 have been reached (supporting the latest pre 2.0 versions of each). The next goal in the roadmap is migrating each to 2.0.

Any update here?

[github-actions]

📝 Docs preview for commit 7ecbc38 at: https://5165ba05.sqlmodel.pages.dev

[github-actions]

📝 Docs preview for commit ab07514 at: https://398280da.sqlmodel.pages.dev

[BoManev]

@AntonDeMeester I have been using an older commit from this repo, however today I tried to update my dependencies and got the following errors.

Because no versions of pydantic-settings match >2.0.0,<2.0.1 || >2.0.1,<2.0.2 || >2.0.2,<2.0.3 || >2.0.3,<2.1.0 || >2.1.0
and pydantic-settings (2.0.0) depends on pydantic (>=2.0b3), pydantic-settings (>=2.0.0,<2.0.1 || >2.0.1,<2.0.2 || >2.0.2,<2.0.3 || >2.0.3,<2.1.0 || >2.1.0) requires pydantic (>=2.0b3).
And because pydantic-settings (2.0.1) depends on pydantic (>=2.0.1), pydantic-settings (>=2.0.0,<2.0.2 || >2.0.2,<2.0.3 || >2.0.3,<2.1.0 || >2.1.0) requires pydantic (>=2.0b3).
And because pydantic-settings (2.0.2) depends on pydantic (>=2.0.1)
and pydantic-settings (2.0.3) depends on pydantic (>=2.0.1), pydantic-settings (>=2.0.0,<2.1.0 || >2.1.0) requires pydantic (>=2.0b3).
And because pydantic-settings (2.1.0) depends on pydantic (>=2.3.0)
and sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head depends on pydantic (^1.9.0), sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head is incompatible with pydantic-settings (>=2.0.0).
And because fastapi (0.103.2) depends on pydantic-settings (>=2.0.0)
and no versions of fastapi match >0.103.2,<0.104.0, sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head is incompatible with fastapi (>=0.103.2,<0.104.0).
So, because api depends on both fastapi (^0.103.2) and sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head, version solving failed.

Snippet from my pyproject.toml

fastapi = {extras = ["all"], version = "^0.103.2"}
sqlmodel = {git = "https://github.com/tiangolo/sqlmodel.git", rev = "refs/pull/632/head"}

The older commit from this PR is using pydantic = { version = ">=2.1.1,<=2.4.2", extras = ["email"] } this worked fine with pydantic-settings from fastapi extras

[AntonDeMeester]

@AntonDeMeester I have been using an older commit from this repo, however today I tried to update my dependencies and got the following errors.

The older commit from this PR is using pydantic = { version = ">=2.1.1,<=2.4.2", extras = ["email"] } this worked fine with pydantic-settings from fastapi extras

Ye I've been working on getting both Pydantic v1 and v2 to work at the same time, I might have broken some things. I'll change the PR to use the old commit again and push my changes to a new PR on top of this one until both work.

[github-actions]

📝 Docs preview for commit d0ae3e8 at: https://ff83edad.sqlmodel.pages.dev

[BoManev]

@AntonDeMeester This commit still doesn't work
The one Im using is this f926d9d

[honglei]

@AntonDeMeester sqlmodel has drop support for sqlalchemy<2.0, maybe next release will drop support for pydantic v1,
the author has no interests to fix some bugs in pydantic v1, which make me move my project to pydantic v2.

https://github.com/tiangolo/sqlmodel/blob/b1c2f822c9db036afe1b7ec81d335368e9546756/pyproject.toml#L34

[ogabrielluiz]

@AntonDeMeester how are you?

I've been using your fork internally for a while now and it seems stable.

Do you see this PR being merged this week?

Thanks in advance

[AntonDeMeester]

Closing in favour of @tiangolo PR here

[tiangolo]

Thank you for your work @AntonDeMeester! I included your commits in #722 (you can see your badge now says "Contributor" 😎 ), I changed a few extra things, more details in that PR. It's now released and available as SQLModel 0.0.14! 🎉 🌮

[samyxdev]

Not sure if replying to a closed PR is the way, but it seems that the issue raised by @50Bytes-dev about .model_copy raising an error is still hasn't been fixed with the new release 0.0.14, despite the fix he proposed (thanks for that !).
Am I missing something or was the matter moved to a different issue ?

Below is a quote of the mentionned error:

@ogabrielluiz@honglei .model_copy(...) raise error:

object has no attribute '__pydantic_extra__'

service_category: ServiceCategory=awaitself.get_object_or_404(id, company.id, async_db)
new_service_category=service_category.model_copy(update={'id': None})

Use this test:

deftest_model_copy(clear_sqlmodel):
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
name: strsecret_name: strage: Optional[int] =Nonehero=Hero(name="Deadpond", secret_name="Dive Wilson", age=25)
engine=create_engine("sqlite://")
SQLModel.metadata.create_all(engine)
withSession(engine) assession:
session.add(hero)
session.commit()
session.refresh(hero)
db_hero=session.get(Hero, hero.id)
copy=db_hero.model_copy(update={"name": "Deadpond Copy"})
assertcopy.name=="Deadpond Copy"and \
copy.secret_name=="Dive Wilson"and \
copy.age==25

fixed honglei#1

[ChrisNi888]

This error 'TypeError: issubclass() arg 1 must be a class' seems not fixed. ( I use Pydantic 2.7 and SQLModel 0.0.16. ) So I use Pydantic 1.0.11 and SQLModel 0.08. If anyone has same problem, hope it helps.

[AntonDeMeester]

Upgrades to Pydantic 2, using the new pydantic models. Switches for v1 and v2 can be built later

Builds on #563 to upgrade SQL Alchemy

Still WIP, did some uglier hacks which could provide some problems.

Most of it is just changing code with different naming of Pydantic 2. However, the validation of empty initialisations is broken because that now happens in rust and there is no static validate_model anymore. For now I set defaults to all provided Fields is there are none, so that it will always inititialise, but that screws a bit with the logic. This is also annoying for cls.model_validate as it screws up SQL Alchemy.

[luoshuijs]

Thank you very much for your PR. I can use it with SQLModel in the pydantic v2 environment, and I've found some issues and have some suggestions.

[luoshuijs]

NoneType has been moved to types in Python 3.10+.
https://docs.python.org/3/library/types.html#types.NoneType

[luoshuijs]

In the test tests\test_tutorial\test_delete, the following error occurred: AttributeError: 'Hero' object has no attribute '__pydantic_extra__'.

You can solve this problem by adding the following code to SQLmodel:

def__new__(cls, *args: Any, **kwargs: Any) ->Any:
new_object=super().__new__(cls)
object.__setattr__(new_object, "__pydantic_fields_set__", set())
returnnew_object

[AntonDeMeester]

I got almost everything working except for Open API. It provides optional for the OpenAPI if you use a table as a request or response type. While this isn't correct in the strict sense, it is correct for Pydantic as empty initialisations need to be supported. I tried a couple of other ways (creating a second class for the optional initialisations or trying to override the OpenAPI generation) but none of them worked out.

For now I commented the OpenAPI assertions, but @tiangolo will need to decide how to handle it. As he wrote FastAPI I'm sure he'll have better ideas 😄 I just hope I can give him some inspiration with this.

[honglei]

the following test not pass:

fromsqlmodelimportSQLModel, FieldclassArticle(SQLModel, table=True):
name : str|None=Field( description='the name of article')

Error:

 File "crud_test\test_sqlmodel.py", line 2, in <module>
class Article(SQLModel, table=True):
 File "crud_test\sqlmodel\main.py", line 318, in __new__
col = get_column_from_field(v)
 File "crud_test\sqlmodel\main.py", line 464, in get_column_from_field
sa_type = get_sqlalchemy_type(field)
 File "crud_test\sqlmodel\main.py", line 417, in get_sqlalchemy_type
if issubclass(type_, str):
builtins.TypeError: issubclass() arg 1 must be a class

[yallxe]

@honglei try Optional[str] instead of str | None

I think just one more case must be checked somewhere.

[honglei]

@honglei try Optional[str] instead of str | None

I think just one more case must be checked somewhere.

I made a pull request to support it. mbsantiago#1

[ma-ts]

Hi! Just checking if there's still something to be done for this PR, more than willing to help out!

[ogabrielluiz]

Hey! I'm also willing to help out.

[honglei]

@ma-ts@ogabrielluiz You can help by testing the PR in your projects, and reporting how well it works.
Right now my project based on https://github.com/mbsantiago/sqlmodel, with my version at: https://github.com/honglei/sqlmodel

[ogabrielluiz]

@honglei
After the normal pydantic migration which was pretty straightforward all Langflow's tests are passing. Will perform some more tests ASAP.

Installed with poetry using sqlmodel = { git = "https://github.com/honglei/sqlmodel.git", branch = "main" }

[50Bytes-dev]

@ogabrielluiz@honglei
.model_copy(...) raise error:

object has no attribute '__pydantic_extra__'

service_category: ServiceCategory=awaitself.get_object_or_404(id, company.id, async_db)
new_service_category=service_category.model_copy(update={'id': None})

Use this test:

deftest_model_copy(clear_sqlmodel):
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
name: strsecret_name: strage: Optional[int] =Nonehero=Hero(name="Deadpond", secret_name="Dive Wilson", age=25)
engine=create_engine("sqlite://")
SQLModel.metadata.create_all(engine)
withSession(engine) assession:
session.add(hero)
session.commit()
session.refresh(hero)
db_hero=session.get(Hero, hero.id)
copy=db_hero.model_copy(update={"name": "Deadpond Copy"})
assertcopy.name=="Deadpond Copy"and \
copy.secret_name=="Dive Wilson"and \
copy.age==25

[50Bytes-dev]

@ogabrielluiz@honglei .model_copy(...) raise error:

object has no attribute '__pydantic_extra__'

service_category: ServiceCategory=awaitself.get_object_or_404(id, company.id, async_db)
new_service_category=service_category.model_copy(update={'id': None})

Use this test:

deftest_model_copy(clear_sqlmodel):
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
name: strsecret_name: strage: Optional[int] =Nonehero=Hero(name="Deadpond", secret_name="Dive Wilson", age=25)
engine=create_engine("sqlite://")
SQLModel.metadata.create_all(engine)
withSession(engine) assession:
session.add(hero)
session.commit()
session.refresh(hero)
db_hero=session.get(Hero, hero.id)
copy=db_hero.model_copy(update={"name": "Deadpond Copy"})
assertcopy.name=="Deadpond Copy"and \
copy.secret_name=="Dive Wilson"and \
copy.age==25

fixed honglei#1

[8thgencore]

I wonder if @tiangolo will notice this sentence

[MattOates]

Not strictly a SQLModels problem, but if you try and use the current SQLModels 0.0.8 with FastAPI 0.103.0 where you've targeted fastapi[all] conflicts are created because of the pydantic version pins of SQLModel, as pydantic-settings requires a much later version than the max pinned. Just like to add a +1 for this PR being quite important to get merged.

[MatsiukMykola]

topic is very long, guys, possible to use with this PR?
@tiangolo can please comment? Your package is awesome, very needs an update.
Anyway big th! (hello from Ukraine)

[50Bytes-dev]

topic is very long, guys, possible to use with this PR? @tiangolo can please comment? Your package is awesome, very needs an update. Anyway big th! (hello from Ukraine)

@MatsiukMykola I'm using it in my project. Everything's great
Use this https://github.com/honglei/sqlmodel

[ikreb7]

The type EmailStr doesn't work. I get the error ValueError(f"The field {field.title} has no matching SQLAlchemy type").

[50Bytes-dev]

The type EmailStr doesn't work. I get the error ValueError(f"The field {field.title} has no matching SQLAlchemy type").

@ikreb7 use this

my_email: EmailStr=Field(sa_column=Column(String))

[honglei]

@ikreb7@50Bytes-dev support the following in my fork(https://github.com/honglei/sqlmodel), but how to change test.yml to do pip install pydantic[email]?

http: HttpUrl=Field(max_length=250)
email: EmailStrname_email : NameEmail=Field( max_length=50)
import_string:ImportString=Field(max_length=200, min_length=100)

[50Bytes-dev]

@ikreb7@50Bytes-dev support the following in my fork(https://github.com/honglei/sqlmodel), but how to change test.yml to do pip install pydantic[email]?

http: HttpUrl=Field(max_length=250)
email: EmailStrname_email : NameEmail=Field( max_length=50)
import_string:ImportString=Field(max_length=200, min_length=100)

@honglei
In pyproject.toml update line 36

pydantic = { version = "^2.1.1", extras = ["email"] }

[honglei]

@50Bytes-dev thanks!

[wojciech-mazur-mlg]

@honglei what's the status of your fork? Can we use it till a stable version of SQLmodel is released?

[honglei]

I used in my projects, it works fine. wojciech-mazur-mlg ***@***.***> 于 2023年9月18日周一 下午7:30写道：

…

@honglei <https://github.com/honglei> what's the status of your fork? Can we use it till a stable version of SQLmodel is released? — Reply to this email directly, view it on GitHub <#632 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHW5AVSOKK64F5EAV4P7LTX3AWDZANCNFSM6AAAAAA27PJTWM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[AntonDeMeester]

@honglei what's the status of your fork? Can we use it till a stable version of SQLmodel is released?

I have tried it out a bit and it seems to work. Some other people are also saying it works. But I would say it's not official until @tiangolo approves it or makes his own version. I also won't be trying everything out to be honest 😅

[BoManev]

After I updated, the "alias" stopped working

class UserBase(SQLModel):
password_hash: str = Field(alias="password")
def create_user(user: UserBase):
pass

The validation in create_user fails with "password ", but works with "password_hash"

[honglei]

@BoManev@AntonDeMeester
In pydantic, function pydantic.fields.Field will use alias to infer validation_alias,
but in sqlmodel.main.FieldInfo, validation_alias is not set, so alias not work under pydantic_core

https://github.com/pydantic/pydantic/blob/188018ccd0c26085e51e632ae4f909b3ee31a0f7/pydantic/fields.py#L819

[BoManev]

@honglei Thank you for the fast reply. In that case should I create a class using pydantic BaseModel for validation of my inputs and then map it with model_validate to my SQLModel.

I also found this #374

EDIT: Generally, how to handle data validation when field names change and data needs to be modified. In my case I receive "password", but internally I want my types to carry "password_hash". In Rust, I can specify Into/To/From traits and convert between types. Can I do something similar with pydantic/SQLModel, define the conversion of 1 data schema to other.

[JanBeelte]

Hey folks,
what is the current status and what are the next steps here?
Is there any ETA when this one will be merged to master?
I could also take over some tasks in case that helps, given I desperately need pydantic v2 support otherwise I cannot use SQLModel.
Best,
Jan

[lawrenceeldridge]

Really looking forward to seeing these changes go in - carrying out an early preview as I'm already using the latest versions. One thing I've observed, I've been struggling a little with using the async db setup in the context of pulling in additional relationships. All of the other routes work fine from this tutorial, but when I call read_hero I get the following error:

fastapi.exceptions.ResponseValidationError: 1 validation errors:
{'type': 'get_attribute_error', 'loc': ('response', 'team'), 'msg': "Error extracting attribute: MissingGreenlet: greenlet_spawn has not been called; can't call await_only() here. Was IO attempted in an unexpected place? (Background on this error at: https://sqlalche.me/e/20/xd2s)", 'input': Hero(secret_name='Batman', id=1, team_id=1, name='Bruce Wayne', age=42), 'ctx': {'error': "MissingGreenlet: greenlet_spawn has not been called; can't call await_only() here. Was IO attempted in an unexpected place? (Background on this error at: https://sqlalche.me/e/20/xd2s)"}, 'url': 'https://errors.pydantic.dev/2.4/v/get_attribute_error'}

This only happens when the response_model is set to HeroReadWithTeam, as the expectation is that we'd want to include teams with the hero. Using HeroRead works fine.

[lawrenceeldridge]

This is now resolved with the help of another post here. For relationships between models you may need to leverage the correct loading styles through the sa_relationship_kwargs property that is offered by SQLModel. For the curious this is how I've built up the tutorial with async in mind below.

In SQL Alchemy there's more reading here on the subject of lazy loading.

Really happy, so will crack on :)

fromcontextlibimportasynccontextmanagerfromtypingimportList, OptionalfromfastapiimportDepends, FastAPI, HTTPException, Queryfromsqlalchemy.ext.asyncioimportasync_sessionmaker, create_async_enginefromsqlmodelimportField, Relationship, SQLModelfromsqlalchemy.sql.expressionimportselectfromsqlmodel.ext.asyncio.sessionimportAsyncSessionfromconfigimportconfigclassTeamBase(SQLModel):
name: str=Field(index=True)
headquarters: strclassTeam(TeamBase, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
heroes: List["Hero"] =Relationship(back_populates="team", sa_relationship_kwargs={"lazy": "selectin"})
classTeamCreate(TeamBase):
passclassTeamRead(TeamBase):
id: intclassTeamUpdate(SQLModel):
id: Optional[int] =Nonename: Optional[str] =Noneheadquarters: Optional[str] =NoneclassHeroBase(SQLModel):
name: str=Field(index=True)
secret_name: strage: Optional[int] =Field(default=None, index=True)
team_id: Optional[int] =Field(default=None, foreign_key="team.id")
classHero(HeroBase, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
team: Optional[Team] =Relationship(back_populates="heroes", sa_relationship_kwargs={"lazy": "joined"})
classHeroRead(HeroBase):
id: intclassHeroCreate(HeroBase):
passclassHeroUpdate(SQLModel):
name: Optional[str] =Nonesecret_name: Optional[str] =Noneage: Optional[int] =Noneteam_id: Optional[int] =NoneclassHeroReadWithTeam(HeroRead):
team: Optional[TeamRead] =NoneclassTeamReadWithHeroes(TeamRead):
heroes: List[HeroRead] = []
engine=create_async_engine(config.DB_HOST, echo=True, future=True)
asyncdefinit_db():
asyncwithengine.begin() asconn:
awaitconn.run_sync(SQLModel.metadata.create_all)
asyncdefget_session() ->AsyncSession:
async_session=async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
asyncwithasync_session() assession:
yieldsession@asynccontextmanagerasyncdeflifespan(app: FastAPI):
# on startupawaitinit_db()
yield# on shutdownawaitengine.dispose()
app=FastAPI(lifespan=lifespan)
@app.post("/heroes", response_model=HeroRead)asyncdefcreate_hero(*, session: AsyncSession=Depends(get_session), hero: HeroCreate):
db_hero=Hero.model_validate(hero)
session.add(db_hero)
awaitsession.commit()
awaitsession.refresh(db_hero)
returndb_hero@app.get("/heroes", response_model=List[HeroRead])asyncdefread_heroes(
*,
session: AsyncSession=Depends(get_session),
offset: int=0,
limit: int=Query(default=100, lte=100),
):
statement=select(Hero).offset(offset).limit(limit)
heroes=awaitsession.execute(statement)
returnheroes.scalars().all()
@app.get("/heroes/{hero_id}", response_model=HeroReadWithTeam)asyncdefread_hero(*, session: AsyncSession=Depends(get_session), hero_id: int):
hero=awaitsession.get(Hero, hero_id)
ifnothero:
raiseHTTPException(status_code=404, detail="Hero not found")
returnhero@app.patch("/heroes/{hero_id}", response_model=HeroRead)asyncdefupdate_hero(
*, session: AsyncSession=Depends(get_session), hero_id: int, hero: HeroUpdate
):
db_hero=awaitsession.get(Hero, hero_id)
ifnotdb_hero:
raiseHTTPException(status_code=404, detail="Hero not found")
hero_data=hero.model_dump(exclude_unset=True)
forkey, valueinhero_data.items():
setattr(db_hero, key, value)
session.add(db_hero)
awaitsession.commit()
awaitsession.refresh(db_hero)
returndb_hero@app.delete("/heroes/{hero_id}")asyncdefdelete_hero(*, session: AsyncSession=Depends(get_session), hero_id: int):
hero=awaitsession.get(Hero, hero_id)
ifnothero:
raiseHTTPException(status_code=404, detail="Hero not found")
awaitsession.delete(hero)
awaitsession.commit()
return {"ok": True}
@app.post("/teams", response_model=TeamRead)asyncdefcreate_team(*, session: AsyncSession=Depends(get_session), team: TeamCreate):
db_team=Team.model_validate(team)
session.add(db_team)
awaitsession.commit()
awaitsession.refresh(db_team)
returndb_team@app.get("/teams", response_model=List[TeamRead])asyncdefread_teams(
*,
session: AsyncSession=Depends(get_session),
offset: int=0,
limit: int=Query(default=100, lte=100),
):
statement=select(Team).offset(offset).limit(limit)
teams=awaitsession.execute(statement)
returnteams.scalars().all()
@app.get("/teams/{team_id}", response_model=TeamReadWithHeroes)asyncdefread_team(*, team_id: int, session: AsyncSession=Depends(get_session)):
team=awaitsession.get(Team, team_id)
ifnotteam:
raiseHTTPException(status_code=404, detail="Team not found")
returnteam@app.patch("/teams/{team_id}", response_model=TeamRead)asyncdefupdate_team(
*,
session: AsyncSession=Depends(get_session),
team_id: int,
team: TeamUpdate,
):
db_team=awaitsession.get(Team, team_id)
ifnotdb_team:
raiseHTTPException(status_code=404, detail="Team not found")
team_data=team.model_dump(exclude_unset=True)
forkey, valueinteam_data.items():
setattr(db_team, key, value)
session.add(db_team)
awaitsession.commit()
awaitsession.refresh(db_team)
returndb_team@app.delete("/teams/{team_id}")asyncdefdelete_team(*, session: AsyncSession=Depends(get_session), team_id: int):
team=awaitsession.get(Team, team_id)
ifnotteam:
raiseHTTPException(status_code=404, detail="Team not found")
awaitsession.delete(team)
awaitsession.commit()
return {"ok": True}

[blakewatters]

Joining the party here trying to update a fairly large data model the fork.

One thing I am working on resolving and curious if anyone else has run into is an issue with model class inheritance producing SQLAlchemy errors regarding column reuse:

sqlalchemy.exc.ArgumentError: Column object 'created_at' already assigned to Table 'business_address_links'

My TimestampModel looks like:

classTimestampModel(BaseModel):
created_at: datetime.datetime=sqlmodel.Field(
sa_column=sqlalchemy.Column(
sqlalchemy.DateTime(timezone=True),
server_default=sqlalchemy.func.statement_timestamp(),
nullable=False
)
)
updated_at: typing.Optional[datetime.datetime] =sqlmodel.Field(
sa_column=sqlalchemy.Column(
sqlalchemy.DateTime(timezone=True),
onupdate=sqlalchemy.func.statement_timestamp(),
nullable=True
)
)

It looks like the Column definition is being reused across inherited subclasses

[MatsiukMykola]

always was interesting how pull to sa.Column - column comment based on Field Description?

[honglei]

@blakewatters try https://github.com/honglei/sqlmodel/blob/3005495a3ec6c8216b31cbd623f91c7bc8ba174f/tests/test_class_hierarchy.py#L24C17-L24C17

[norton120]

➕ on thanks for putting this PR up, my team is a few weeks into a new project and I'd hate to be stuck behind a major on 2 core libraries this early on, so this is huge. I'm running into a strange issue creating a one-to-many relationship (using this branch), any ideas would be appreciated. Here's the error:

sqlalchemy.exc.ArgumentError: relationship 'venues' expects a class or a mapper argument (received: <class 'list'>)

the related models:

classOrganization(SQLModel, table=True):
id: Optional[int] =sqlmodel.Field(default=None, primary_key=True)
venues: List["Venue"] =sqlmodel.Relationship(back_populates="organization")
@classmethodasyncdeflist(cls, session: AsyncSession):
query=select(cls)
result_from_db=awaitsession.execute(query)
collection=result_from_db.fetchall()
returncollectionclassVenue(SQLModel, table=True):
id: Optional[int] =sqlmodel.Field(default=None, primary_key=True)
organization_id: int=sqlmodel.Field(foreign_key="organization.id")
organization: Organization=sqlmodel.Relationship(back_populates="venues")

and the calling functionality:

classDatabaseConnection:
def__init__(self):
url=sqlalchemy.URL.create(*url_args)
self.async_engine=sqlalchemy.ext.asyncio.create_async_engine(
url, echo=True, future=True
)
asyncdefget_async_session(self) ->AsyncSession:
async_session=sessionmaker(
bind=self.async_engine, class_=AsyncSession, expire_on_commit=False
)
asyncwithasync_session() assession:
yieldsessiondb_session=DatabaseConnection().get_async_session@router.get("/")asyncdefindex(*, session=Depends(db_session)):
returnawaitOrganization.list(session=session)

I've tried changing annotation types, explicitly passing an sa.relationship to the Relationship, setting lazy loading... I'm not super familiar with the SQLModel code base so I'm not sure where to look next.

[nikdavis]

It looks like the both of the goals prior to migrating to sqlalchemy 2.0, and pydantic 2.0 have been reached (supporting the latest pre 2.0 versions of each). The next goal in the roadmap is migrating each to 2.0.

Any update here?

[github-actions]

📝 Docs preview for commit 7ecbc38 at: https://5165ba05.sqlmodel.pages.dev

[github-actions]

📝 Docs preview for commit ab07514 at: https://398280da.sqlmodel.pages.dev

[BoManev]

@AntonDeMeester I have been using an older commit from this repo, however today I tried to update my dependencies and got the following errors.

Because no versions of pydantic-settings match >2.0.0,<2.0.1 || >2.0.1,<2.0.2 || >2.0.2,<2.0.3 || >2.0.3,<2.1.0 || >2.1.0
and pydantic-settings (2.0.0) depends on pydantic (>=2.0b3), pydantic-settings (>=2.0.0,<2.0.1 || >2.0.1,<2.0.2 || >2.0.2,<2.0.3 || >2.0.3,<2.1.0 || >2.1.0) requires pydantic (>=2.0b3).
And because pydantic-settings (2.0.1) depends on pydantic (>=2.0.1), pydantic-settings (>=2.0.0,<2.0.2 || >2.0.2,<2.0.3 || >2.0.3,<2.1.0 || >2.1.0) requires pydantic (>=2.0b3).
And because pydantic-settings (2.0.2) depends on pydantic (>=2.0.1)
and pydantic-settings (2.0.3) depends on pydantic (>=2.0.1), pydantic-settings (>=2.0.0,<2.1.0 || >2.1.0) requires pydantic (>=2.0b3).
And because pydantic-settings (2.1.0) depends on pydantic (>=2.3.0)
and sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head depends on pydantic (^1.9.0), sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head is incompatible with pydantic-settings (>=2.0.0).
And because fastapi (0.103.2) depends on pydantic-settings (>=2.0.0)
and no versions of fastapi match >0.103.2,<0.104.0, sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head is incompatible with fastapi (>=0.103.2,<0.104.0).
So, because api depends on both fastapi (^0.103.2) and sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head, version solving failed.

Snippet from my pyproject.toml

fastapi = {extras = ["all"], version = "^0.103.2"}
sqlmodel = {git = "https://github.com/tiangolo/sqlmodel.git", rev = "refs/pull/632/head"}

The older commit from this PR is using pydantic = { version = ">=2.1.1,<=2.4.2", extras = ["email"] } this worked fine with pydantic-settings from fastapi extras

[AntonDeMeester]

@AntonDeMeester I have been using an older commit from this repo, however today I tried to update my dependencies and got the following errors.

The older commit from this PR is using pydantic = { version = ">=2.1.1,<=2.4.2", extras = ["email"] } this worked fine with pydantic-settings from fastapi extras

Ye I've been working on getting both Pydantic v1 and v2 to work at the same time, I might have broken some things. I'll change the PR to use the old commit again and push my changes to a new PR on top of this one until both work.

[github-actions]

📝 Docs preview for commit d0ae3e8 at: https://ff83edad.sqlmodel.pages.dev

[BoManev]

@AntonDeMeester This commit still doesn't work
The one Im using is this f926d9d

[honglei]

@AntonDeMeester sqlmodel has drop support for sqlalchemy<2.0, maybe next release will drop support for pydantic v1,
the author has no interests to fix some bugs in pydantic v1, which make me move my project to pydantic v2.

https://github.com/tiangolo/sqlmodel/blob/b1c2f822c9db036afe1b7ec81d335368e9546756/pyproject.toml#L34

[ogabrielluiz]

@AntonDeMeester how are you?

I've been using your fork internally for a while now and it seems stable.

Do you see this PR being merged this week?

Thanks in advance

[AntonDeMeester]

Closing in favour of @tiangolo PR here

[tiangolo]

Thank you for your work @AntonDeMeester! I included your commits in #722 (you can see your badge now says "Contributor" 😎 ), I changed a few extra things, more details in that PR. It's now released and available as SQLModel 0.0.14! 🎉 🌮

[samyxdev]

Not sure if replying to a closed PR is the way, but it seems that the issue raised by @50Bytes-dev about .model_copy raising an error is still hasn't been fixed with the new release 0.0.14, despite the fix he proposed (thanks for that !).
Am I missing something or was the matter moved to a different issue ?

Below is a quote of the mentionned error:

@ogabrielluiz@honglei .model_copy(...) raise error:

object has no attribute '__pydantic_extra__'

service_category: ServiceCategory=awaitself.get_object_or_404(id, company.id, async_db)
new_service_category=service_category.model_copy(update={'id': None})

Use this test:

deftest_model_copy(clear_sqlmodel):
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
name: strsecret_name: strage: Optional[int] =Nonehero=Hero(name="Deadpond", secret_name="Dive Wilson", age=25)
engine=create_engine("sqlite://")
SQLModel.metadata.create_all(engine)
withSession(engine) assession:
session.add(hero)
session.commit()
session.refresh(hero)
db_hero=session.get(Hero, hero.id)
copy=db_hero.model_copy(update={"name": "Deadpond Copy"})
assertcopy.name=="Deadpond Copy"and \
copy.secret_name=="Dive Wilson"and \
copy.age==25

fixed honglei#1

[ChrisNi888]

This error 'TypeError: issubclass() arg 1 must be a class' seems not fixed. ( I use Pydantic 2.7 and SQLModel 0.0.16. ) So I use Pydantic 1.0.11 and SQLModel 0.08. If anyone has same problem, hope it helps.

[AntonDeMeester]

Upgrades to Pydantic 2, using the new pydantic models. Switches for v1 and v2 can be built later

Builds on #563 to upgrade SQL Alchemy

Still WIP, did some uglier hacks which could provide some problems.

Most of it is just changing code with different naming of Pydantic 2. However, the validation of empty initialisations is broken because that now happens in rust and there is no static validate_model anymore. For now I set defaults to all provided Fields is there are none, so that it will always inititialise, but that screws a bit with the logic. This is also annoying for cls.model_validate as it screws up SQL Alchemy.

[luoshuijs]

Thank you very much for your PR. I can use it with SQLModel in the pydantic v2 environment, and I've found some issues and have some suggestions.

[luoshuijs]

NoneType has been moved to types in Python 3.10+.
https://docs.python.org/3/library/types.html#types.NoneType

[luoshuijs]

In the test tests\test_tutorial\test_delete, the following error occurred: AttributeError: 'Hero' object has no attribute '__pydantic_extra__'.

You can solve this problem by adding the following code to SQLmodel:

def__new__(cls, *args: Any, **kwargs: Any) ->Any:
new_object=super().__new__(cls)
object.__setattr__(new_object, "__pydantic_fields_set__", set())
returnnew_object

[AntonDeMeester]

I got almost everything working except for Open API. It provides optional for the OpenAPI if you use a table as a request or response type. While this isn't correct in the strict sense, it is correct for Pydantic as empty initialisations need to be supported. I tried a couple of other ways (creating a second class for the optional initialisations or trying to override the OpenAPI generation) but none of them worked out.

For now I commented the OpenAPI assertions, but @tiangolo will need to decide how to handle it. As he wrote FastAPI I'm sure he'll have better ideas 😄 I just hope I can give him some inspiration with this.

[honglei]

the following test not pass:

fromsqlmodelimportSQLModel, FieldclassArticle(SQLModel, table=True):
name : str|None=Field( description='the name of article')

Error:

 File "crud_test\test_sqlmodel.py", line 2, in <module>
class Article(SQLModel, table=True):
 File "crud_test\sqlmodel\main.py", line 318, in __new__
col = get_column_from_field(v)
 File "crud_test\sqlmodel\main.py", line 464, in get_column_from_field
sa_type = get_sqlalchemy_type(field)
 File "crud_test\sqlmodel\main.py", line 417, in get_sqlalchemy_type
if issubclass(type_, str):
builtins.TypeError: issubclass() arg 1 must be a class

[yallxe]

@honglei try Optional[str] instead of str | None

I think just one more case must be checked somewhere.

[honglei]

@honglei try Optional[str] instead of str | None

I think just one more case must be checked somewhere.

I made a pull request to support it. mbsantiago#1

[ma-ts]

Hi! Just checking if there's still something to be done for this PR, more than willing to help out!

[ogabrielluiz]

Hey! I'm also willing to help out.

[honglei]

@ma-ts@ogabrielluiz You can help by testing the PR in your projects, and reporting how well it works.
Right now my project based on https://github.com/mbsantiago/sqlmodel, with my version at: https://github.com/honglei/sqlmodel

[ogabrielluiz]

@honglei
After the normal pydantic migration which was pretty straightforward all Langflow's tests are passing. Will perform some more tests ASAP.

Installed with poetry using sqlmodel = { git = "https://github.com/honglei/sqlmodel.git", branch = "main" }

[50Bytes-dev]

@ogabrielluiz@honglei
.model_copy(...) raise error:

object has no attribute '__pydantic_extra__'

service_category: ServiceCategory=awaitself.get_object_or_404(id, company.id, async_db)
new_service_category=service_category.model_copy(update={'id': None})

Use this test:

deftest_model_copy(clear_sqlmodel):
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
name: strsecret_name: strage: Optional[int] =Nonehero=Hero(name="Deadpond", secret_name="Dive Wilson", age=25)
engine=create_engine("sqlite://")
SQLModel.metadata.create_all(engine)
withSession(engine) assession:
session.add(hero)
session.commit()
session.refresh(hero)
db_hero=session.get(Hero, hero.id)
copy=db_hero.model_copy(update={"name": "Deadpond Copy"})
assertcopy.name=="Deadpond Copy"and \
copy.secret_name=="Dive Wilson"and \
copy.age==25

[50Bytes-dev]

@ogabrielluiz@honglei .model_copy(...) raise error:

object has no attribute '__pydantic_extra__'

service_category: ServiceCategory=awaitself.get_object_or_404(id, company.id, async_db)
new_service_category=service_category.model_copy(update={'id': None})

Use this test:

deftest_model_copy(clear_sqlmodel):
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
name: strsecret_name: strage: Optional[int] =Nonehero=Hero(name="Deadpond", secret_name="Dive Wilson", age=25)
engine=create_engine("sqlite://")
SQLModel.metadata.create_all(engine)
withSession(engine) assession:
session.add(hero)
session.commit()
session.refresh(hero)
db_hero=session.get(Hero, hero.id)
copy=db_hero.model_copy(update={"name": "Deadpond Copy"})
assertcopy.name=="Deadpond Copy"and \
copy.secret_name=="Dive Wilson"and \
copy.age==25

fixed honglei#1

[8thgencore]

I wonder if @tiangolo will notice this sentence

[MattOates]

Not strictly a SQLModels problem, but if you try and use the current SQLModels 0.0.8 with FastAPI 0.103.0 where you've targeted fastapi[all] conflicts are created because of the pydantic version pins of SQLModel, as pydantic-settings requires a much later version than the max pinned. Just like to add a +1 for this PR being quite important to get merged.

[MatsiukMykola]

topic is very long, guys, possible to use with this PR?
@tiangolo can please comment? Your package is awesome, very needs an update.
Anyway big th! (hello from Ukraine)

[50Bytes-dev]

topic is very long, guys, possible to use with this PR? @tiangolo can please comment? Your package is awesome, very needs an update. Anyway big th! (hello from Ukraine)

@MatsiukMykola I'm using it in my project. Everything's great
Use this https://github.com/honglei/sqlmodel

[ikreb7]

The type EmailStr doesn't work. I get the error ValueError(f"The field {field.title} has no matching SQLAlchemy type").

[50Bytes-dev]

The type EmailStr doesn't work. I get the error ValueError(f"The field {field.title} has no matching SQLAlchemy type").

@ikreb7 use this

my_email: EmailStr=Field(sa_column=Column(String))

[honglei]

@ikreb7@50Bytes-dev support the following in my fork(https://github.com/honglei/sqlmodel), but how to change test.yml to do pip install pydantic[email]?

http: HttpUrl=Field(max_length=250)
email: EmailStrname_email : NameEmail=Field( max_length=50)
import_string:ImportString=Field(max_length=200, min_length=100)

[50Bytes-dev]

@ikreb7@50Bytes-dev support the following in my fork(https://github.com/honglei/sqlmodel), but how to change test.yml to do pip install pydantic[email]?

http: HttpUrl=Field(max_length=250)
email: EmailStrname_email : NameEmail=Field( max_length=50)
import_string:ImportString=Field(max_length=200, min_length=100)

@honglei
In pyproject.toml update line 36

pydantic = { version = "^2.1.1", extras = ["email"] }

[honglei]

@50Bytes-dev thanks!

[wojciech-mazur-mlg]

@honglei what's the status of your fork? Can we use it till a stable version of SQLmodel is released?

[honglei]

I used in my projects, it works fine. wojciech-mazur-mlg ***@***.***> 于 2023年9月18日周一 下午7:30写道：

…

@honglei <https://github.com/honglei> what's the status of your fork? Can we use it till a stable version of SQLmodel is released? — Reply to this email directly, view it on GitHub <#632 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHW5AVSOKK64F5EAV4P7LTX3AWDZANCNFSM6AAAAAA27PJTWM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[AntonDeMeester]

@honglei what's the status of your fork? Can we use it till a stable version of SQLmodel is released?

I have tried it out a bit and it seems to work. Some other people are also saying it works. But I would say it's not official until @tiangolo approves it or makes his own version. I also won't be trying everything out to be honest 😅

[BoManev]

After I updated, the "alias" stopped working

class UserBase(SQLModel):
password_hash: str = Field(alias="password")
def create_user(user: UserBase):
pass

The validation in create_user fails with "password ", but works with "password_hash"

[honglei]

@BoManev@AntonDeMeester
In pydantic, function pydantic.fields.Field will use alias to infer validation_alias,
but in sqlmodel.main.FieldInfo, validation_alias is not set, so alias not work under pydantic_core

https://github.com/pydantic/pydantic/blob/188018ccd0c26085e51e632ae4f909b3ee31a0f7/pydantic/fields.py#L819

[BoManev]

@honglei Thank you for the fast reply. In that case should I create a class using pydantic BaseModel for validation of my inputs and then map it with model_validate to my SQLModel.

I also found this #374

EDIT: Generally, how to handle data validation when field names change and data needs to be modified. In my case I receive "password", but internally I want my types to carry "password_hash". In Rust, I can specify Into/To/From traits and convert between types. Can I do something similar with pydantic/SQLModel, define the conversion of 1 data schema to other.

[JanBeelte]

Hey folks,
what is the current status and what are the next steps here?
Is there any ETA when this one will be merged to master?
I could also take over some tasks in case that helps, given I desperately need pydantic v2 support otherwise I cannot use SQLModel.
Best,
Jan

[lawrenceeldridge]

Really looking forward to seeing these changes go in - carrying out an early preview as I'm already using the latest versions. One thing I've observed, I've been struggling a little with using the async db setup in the context of pulling in additional relationships. All of the other routes work fine from this tutorial, but when I call read_hero I get the following error:

fastapi.exceptions.ResponseValidationError: 1 validation errors:
{'type': 'get_attribute_error', 'loc': ('response', 'team'), 'msg': "Error extracting attribute: MissingGreenlet: greenlet_spawn has not been called; can't call await_only() here. Was IO attempted in an unexpected place? (Background on this error at: https://sqlalche.me/e/20/xd2s)", 'input': Hero(secret_name='Batman', id=1, team_id=1, name='Bruce Wayne', age=42), 'ctx': {'error': "MissingGreenlet: greenlet_spawn has not been called; can't call await_only() here. Was IO attempted in an unexpected place? (Background on this error at: https://sqlalche.me/e/20/xd2s)"}, 'url': 'https://errors.pydantic.dev/2.4/v/get_attribute_error'}

This only happens when the response_model is set to HeroReadWithTeam, as the expectation is that we'd want to include teams with the hero. Using HeroRead works fine.

[lawrenceeldridge]

This is now resolved with the help of another post here. For relationships between models you may need to leverage the correct loading styles through the sa_relationship_kwargs property that is offered by SQLModel. For the curious this is how I've built up the tutorial with async in mind below.

In SQL Alchemy there's more reading here on the subject of lazy loading.

Really happy, so will crack on :)

fromcontextlibimportasynccontextmanagerfromtypingimportList, OptionalfromfastapiimportDepends, FastAPI, HTTPException, Queryfromsqlalchemy.ext.asyncioimportasync_sessionmaker, create_async_enginefromsqlmodelimportField, Relationship, SQLModelfromsqlalchemy.sql.expressionimportselectfromsqlmodel.ext.asyncio.sessionimportAsyncSessionfromconfigimportconfigclassTeamBase(SQLModel):
name: str=Field(index=True)
headquarters: strclassTeam(TeamBase, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
heroes: List["Hero"] =Relationship(back_populates="team", sa_relationship_kwargs={"lazy": "selectin"})
classTeamCreate(TeamBase):
passclassTeamRead(TeamBase):
id: intclassTeamUpdate(SQLModel):
id: Optional[int] =Nonename: Optional[str] =Noneheadquarters: Optional[str] =NoneclassHeroBase(SQLModel):
name: str=Field(index=True)
secret_name: strage: Optional[int] =Field(default=None, index=True)
team_id: Optional[int] =Field(default=None, foreign_key="team.id")
classHero(HeroBase, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
team: Optional[Team] =Relationship(back_populates="heroes", sa_relationship_kwargs={"lazy": "joined"})
classHeroRead(HeroBase):
id: intclassHeroCreate(HeroBase):
passclassHeroUpdate(SQLModel):
name: Optional[str] =Nonesecret_name: Optional[str] =Noneage: Optional[int] =Noneteam_id: Optional[int] =NoneclassHeroReadWithTeam(HeroRead):
team: Optional[TeamRead] =NoneclassTeamReadWithHeroes(TeamRead):
heroes: List[HeroRead] = []
engine=create_async_engine(config.DB_HOST, echo=True, future=True)
asyncdefinit_db():
asyncwithengine.begin() asconn:
awaitconn.run_sync(SQLModel.metadata.create_all)
asyncdefget_session() ->AsyncSession:
async_session=async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
asyncwithasync_session() assession:
yieldsession@asynccontextmanagerasyncdeflifespan(app: FastAPI):
# on startupawaitinit_db()
yield# on shutdownawaitengine.dispose()
app=FastAPI(lifespan=lifespan)
@app.post("/heroes", response_model=HeroRead)asyncdefcreate_hero(*, session: AsyncSession=Depends(get_session), hero: HeroCreate):
db_hero=Hero.model_validate(hero)
session.add(db_hero)
awaitsession.commit()
awaitsession.refresh(db_hero)
returndb_hero@app.get("/heroes", response_model=List[HeroRead])asyncdefread_heroes(
*,
session: AsyncSession=Depends(get_session),
offset: int=0,
limit: int=Query(default=100, lte=100),
):
statement=select(Hero).offset(offset).limit(limit)
heroes=awaitsession.execute(statement)
returnheroes.scalars().all()
@app.get("/heroes/{hero_id}", response_model=HeroReadWithTeam)asyncdefread_hero(*, session: AsyncSession=Depends(get_session), hero_id: int):
hero=awaitsession.get(Hero, hero_id)
ifnothero:
raiseHTTPException(status_code=404, detail="Hero not found")
returnhero@app.patch("/heroes/{hero_id}", response_model=HeroRead)asyncdefupdate_hero(
*, session: AsyncSession=Depends(get_session), hero_id: int, hero: HeroUpdate
):
db_hero=awaitsession.get(Hero, hero_id)
ifnotdb_hero:
raiseHTTPException(status_code=404, detail="Hero not found")
hero_data=hero.model_dump(exclude_unset=True)
forkey, valueinhero_data.items():
setattr(db_hero, key, value)
session.add(db_hero)
awaitsession.commit()
awaitsession.refresh(db_hero)
returndb_hero@app.delete("/heroes/{hero_id}")asyncdefdelete_hero(*, session: AsyncSession=Depends(get_session), hero_id: int):
hero=awaitsession.get(Hero, hero_id)
ifnothero:
raiseHTTPException(status_code=404, detail="Hero not found")
awaitsession.delete(hero)
awaitsession.commit()
return {"ok": True}
@app.post("/teams", response_model=TeamRead)asyncdefcreate_team(*, session: AsyncSession=Depends(get_session), team: TeamCreate):
db_team=Team.model_validate(team)
session.add(db_team)
awaitsession.commit()
awaitsession.refresh(db_team)
returndb_team@app.get("/teams", response_model=List[TeamRead])asyncdefread_teams(
*,
session: AsyncSession=Depends(get_session),
offset: int=0,
limit: int=Query(default=100, lte=100),
):
statement=select(Team).offset(offset).limit(limit)
teams=awaitsession.execute(statement)
returnteams.scalars().all()
@app.get("/teams/{team_id}", response_model=TeamReadWithHeroes)asyncdefread_team(*, team_id: int, session: AsyncSession=Depends(get_session)):
team=awaitsession.get(Team, team_id)
ifnotteam:
raiseHTTPException(status_code=404, detail="Team not found")
returnteam@app.patch("/teams/{team_id}", response_model=TeamRead)asyncdefupdate_team(
*,
session: AsyncSession=Depends(get_session),
team_id: int,
team: TeamUpdate,
):
db_team=awaitsession.get(Team, team_id)
ifnotdb_team:
raiseHTTPException(status_code=404, detail="Team not found")
team_data=team.model_dump(exclude_unset=True)
forkey, valueinteam_data.items():
setattr(db_team, key, value)
session.add(db_team)
awaitsession.commit()
awaitsession.refresh(db_team)
returndb_team@app.delete("/teams/{team_id}")asyncdefdelete_team(*, session: AsyncSession=Depends(get_session), team_id: int):
team=awaitsession.get(Team, team_id)
ifnotteam:
raiseHTTPException(status_code=404, detail="Team not found")
awaitsession.delete(team)
awaitsession.commit()
return {"ok": True}

[blakewatters]

Joining the party here trying to update a fairly large data model the fork.

One thing I am working on resolving and curious if anyone else has run into is an issue with model class inheritance producing SQLAlchemy errors regarding column reuse:

sqlalchemy.exc.ArgumentError: Column object 'created_at' already assigned to Table 'business_address_links'

My TimestampModel looks like:

classTimestampModel(BaseModel):
created_at: datetime.datetime=sqlmodel.Field(
sa_column=sqlalchemy.Column(
sqlalchemy.DateTime(timezone=True),
server_default=sqlalchemy.func.statement_timestamp(),
nullable=False
)
)
updated_at: typing.Optional[datetime.datetime] =sqlmodel.Field(
sa_column=sqlalchemy.Column(
sqlalchemy.DateTime(timezone=True),
onupdate=sqlalchemy.func.statement_timestamp(),
nullable=True
)
)

It looks like the Column definition is being reused across inherited subclasses

[MatsiukMykola]

always was interesting how pull to sa.Column - column comment based on Field Description?

[honglei]

@blakewatters try https://github.com/honglei/sqlmodel/blob/3005495a3ec6c8216b31cbd623f91c7bc8ba174f/tests/test_class_hierarchy.py#L24C17-L24C17

[norton120]

➕ on thanks for putting this PR up, my team is a few weeks into a new project and I'd hate to be stuck behind a major on 2 core libraries this early on, so this is huge. I'm running into a strange issue creating a one-to-many relationship (using this branch), any ideas would be appreciated. Here's the error:

sqlalchemy.exc.ArgumentError: relationship 'venues' expects a class or a mapper argument (received: <class 'list'>)

the related models:

classOrganization(SQLModel, table=True):
id: Optional[int] =sqlmodel.Field(default=None, primary_key=True)
venues: List["Venue"] =sqlmodel.Relationship(back_populates="organization")
@classmethodasyncdeflist(cls, session: AsyncSession):
query=select(cls)
result_from_db=awaitsession.execute(query)
collection=result_from_db.fetchall()
returncollectionclassVenue(SQLModel, table=True):
id: Optional[int] =sqlmodel.Field(default=None, primary_key=True)
organization_id: int=sqlmodel.Field(foreign_key="organization.id")
organization: Organization=sqlmodel.Relationship(back_populates="venues")

and the calling functionality:

classDatabaseConnection:
def__init__(self):
url=sqlalchemy.URL.create(*url_args)
self.async_engine=sqlalchemy.ext.asyncio.create_async_engine(
url, echo=True, future=True
)
asyncdefget_async_session(self) ->AsyncSession:
async_session=sessionmaker(
bind=self.async_engine, class_=AsyncSession, expire_on_commit=False
)
asyncwithasync_session() assession:
yieldsessiondb_session=DatabaseConnection().get_async_session@router.get("/")asyncdefindex(*, session=Depends(db_session)):
returnawaitOrganization.list(session=session)

I've tried changing annotation types, explicitly passing an sa.relationship to the Relationship, setting lazy loading... I'm not super familiar with the SQLModel code base so I'm not sure where to look next.

[nikdavis]

It looks like the both of the goals prior to migrating to sqlalchemy 2.0, and pydantic 2.0 have been reached (supporting the latest pre 2.0 versions of each). The next goal in the roadmap is migrating each to 2.0.

Any update here?

[github-actions]

📝 Docs preview for commit 7ecbc38 at: https://5165ba05.sqlmodel.pages.dev

[github-actions]

📝 Docs preview for commit ab07514 at: https://398280da.sqlmodel.pages.dev

[BoManev]

@AntonDeMeester I have been using an older commit from this repo, however today I tried to update my dependencies and got the following errors.

Because no versions of pydantic-settings match >2.0.0,<2.0.1 || >2.0.1,<2.0.2 || >2.0.2,<2.0.3 || >2.0.3,<2.1.0 || >2.1.0
and pydantic-settings (2.0.0) depends on pydantic (>=2.0b3), pydantic-settings (>=2.0.0,<2.0.1 || >2.0.1,<2.0.2 || >2.0.2,<2.0.3 || >2.0.3,<2.1.0 || >2.1.0) requires pydantic (>=2.0b3).
And because pydantic-settings (2.0.1) depends on pydantic (>=2.0.1), pydantic-settings (>=2.0.0,<2.0.2 || >2.0.2,<2.0.3 || >2.0.3,<2.1.0 || >2.1.0) requires pydantic (>=2.0b3).
And because pydantic-settings (2.0.2) depends on pydantic (>=2.0.1)
and pydantic-settings (2.0.3) depends on pydantic (>=2.0.1), pydantic-settings (>=2.0.0,<2.1.0 || >2.1.0) requires pydantic (>=2.0b3).
And because pydantic-settings (2.1.0) depends on pydantic (>=2.3.0)
and sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head depends on pydantic (^1.9.0), sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head is incompatible with pydantic-settings (>=2.0.0).
And because fastapi (0.103.2) depends on pydantic-settings (>=2.0.0)
and no versions of fastapi match >0.103.2,<0.104.0, sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head is incompatible with fastapi (>=0.103.2,<0.104.0).
So, because api depends on both fastapi (^0.103.2) and sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head, version solving failed.

Snippet from my pyproject.toml

fastapi = {extras = ["all"], version = "^0.103.2"}
sqlmodel = {git = "https://github.com/tiangolo/sqlmodel.git", rev = "refs/pull/632/head"}

The older commit from this PR is using pydantic = { version = ">=2.1.1,<=2.4.2", extras = ["email"] } this worked fine with pydantic-settings from fastapi extras

[AntonDeMeester]

@AntonDeMeester I have been using an older commit from this repo, however today I tried to update my dependencies and got the following errors.

The older commit from this PR is using pydantic = { version = ">=2.1.1,<=2.4.2", extras = ["email"] } this worked fine with pydantic-settings from fastapi extras

Ye I've been working on getting both Pydantic v1 and v2 to work at the same time, I might have broken some things. I'll change the PR to use the old commit again and push my changes to a new PR on top of this one until both work.

[github-actions]

📝 Docs preview for commit d0ae3e8 at: https://ff83edad.sqlmodel.pages.dev

[BoManev]

@AntonDeMeester This commit still doesn't work
The one Im using is this f926d9d

[honglei]

@AntonDeMeester sqlmodel has drop support for sqlalchemy<2.0, maybe next release will drop support for pydantic v1,
the author has no interests to fix some bugs in pydantic v1, which make me move my project to pydantic v2.

https://github.com/tiangolo/sqlmodel/blob/b1c2f822c9db036afe1b7ec81d335368e9546756/pyproject.toml#L34

[ogabrielluiz]

@AntonDeMeester how are you?

I've been using your fork internally for a while now and it seems stable.

Do you see this PR being merged this week?

Thanks in advance

[AntonDeMeester]

Closing in favour of @tiangolo PR here

[tiangolo]

Thank you for your work @AntonDeMeester! I included your commits in #722 (you can see your badge now says "Contributor" 😎 ), I changed a few extra things, more details in that PR. It's now released and available as SQLModel 0.0.14! 🎉 🌮

[samyxdev]

Not sure if replying to a closed PR is the way, but it seems that the issue raised by @50Bytes-dev about .model_copy raising an error is still hasn't been fixed with the new release 0.0.14, despite the fix he proposed (thanks for that !).
Am I missing something or was the matter moved to a different issue ?

Below is a quote of the mentionned error:

@ogabrielluiz@honglei .model_copy(...) raise error:

object has no attribute '__pydantic_extra__'

service_category: ServiceCategory=awaitself.get_object_or_404(id, company.id, async_db)
new_service_category=service_category.model_copy(update={'id': None})

Use this test:

deftest_model_copy(clear_sqlmodel):
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
name: strsecret_name: strage: Optional[int] =Nonehero=Hero(name="Deadpond", secret_name="Dive Wilson", age=25)
engine=create_engine("sqlite://")
SQLModel.metadata.create_all(engine)
withSession(engine) assession:
session.add(hero)
session.commit()
session.refresh(hero)
db_hero=session.get(Hero, hero.id)
copy=db_hero.model_copy(update={"name": "Deadpond Copy"})
assertcopy.name=="Deadpond Copy"and \
copy.secret_name=="Dive Wilson"and \
copy.age==25

fixed honglei#1

[ChrisNi888]

This error 'TypeError: issubclass() arg 1 must be a class' seems not fixed. ( I use Pydantic 2.7 and SQLModel 0.0.16. ) So I use Pydantic 1.0.11 and SQLModel 0.08. If anyone has same problem, hope it helps.

[AntonDeMeester]

Upgrades to Pydantic 2, using the new pydantic models. Switches for v1 and v2 can be built later

Builds on #563 to upgrade SQL Alchemy

Still WIP, did some uglier hacks which could provide some problems.

Most of it is just changing code with different naming of Pydantic 2. However, the validation of empty initialisations is broken because that now happens in rust and there is no static validate_model anymore. For now I set defaults to all provided Fields is there are none, so that it will always inititialise, but that screws a bit with the logic. This is also annoying for cls.model_validate as it screws up SQL Alchemy.

[luoshuijs]

Thank you very much for your PR. I can use it with SQLModel in the pydantic v2 environment, and I've found some issues and have some suggestions.

[luoshuijs]

NoneType has been moved to types in Python 3.10+.
https://docs.python.org/3/library/types.html#types.NoneType

[luoshuijs]

In the test tests\test_tutorial\test_delete, the following error occurred: AttributeError: 'Hero' object has no attribute '__pydantic_extra__'.

You can solve this problem by adding the following code to SQLmodel:

def__new__(cls, *args: Any, **kwargs: Any) ->Any:
new_object=super().__new__(cls)
object.__setattr__(new_object, "__pydantic_fields_set__", set())
returnnew_object

[AntonDeMeester]

I got almost everything working except for Open API. It provides optional for the OpenAPI if you use a table as a request or response type. While this isn't correct in the strict sense, it is correct for Pydantic as empty initialisations need to be supported. I tried a couple of other ways (creating a second class for the optional initialisations or trying to override the OpenAPI generation) but none of them worked out.

For now I commented the OpenAPI assertions, but @tiangolo will need to decide how to handle it. As he wrote FastAPI I'm sure he'll have better ideas 😄 I just hope I can give him some inspiration with this.

[honglei]

the following test not pass:

fromsqlmodelimportSQLModel, FieldclassArticle(SQLModel, table=True):
name : str|None=Field( description='the name of article')

Error:

 File "crud_test\test_sqlmodel.py", line 2, in <module>
class Article(SQLModel, table=True):
 File "crud_test\sqlmodel\main.py", line 318, in __new__
col = get_column_from_field(v)
 File "crud_test\sqlmodel\main.py", line 464, in get_column_from_field
sa_type = get_sqlalchemy_type(field)
 File "crud_test\sqlmodel\main.py", line 417, in get_sqlalchemy_type
if issubclass(type_, str):
builtins.TypeError: issubclass() arg 1 must be a class

[yallxe]

@honglei try Optional[str] instead of str | None

I think just one more case must be checked somewhere.

[honglei]

@honglei try Optional[str] instead of str | None

I think just one more case must be checked somewhere.

I made a pull request to support it. mbsantiago#1

[ma-ts]

Hi! Just checking if there's still something to be done for this PR, more than willing to help out!

[ogabrielluiz]

Hey! I'm also willing to help out.

[honglei]

@ma-ts@ogabrielluiz You can help by testing the PR in your projects, and reporting how well it works.
Right now my project based on https://github.com/mbsantiago/sqlmodel, with my version at: https://github.com/honglei/sqlmodel

[ogabrielluiz]

@honglei
After the normal pydantic migration which was pretty straightforward all Langflow's tests are passing. Will perform some more tests ASAP.

Installed with poetry using sqlmodel = { git = "https://github.com/honglei/sqlmodel.git", branch = "main" }

[50Bytes-dev]

@ogabrielluiz@honglei
.model_copy(...) raise error:

object has no attribute '__pydantic_extra__'

service_category: ServiceCategory=awaitself.get_object_or_404(id, company.id, async_db)
new_service_category=service_category.model_copy(update={'id': None})

Use this test:

deftest_model_copy(clear_sqlmodel):
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
name: strsecret_name: strage: Optional[int] =Nonehero=Hero(name="Deadpond", secret_name="Dive Wilson", age=25)
engine=create_engine("sqlite://")
SQLModel.metadata.create_all(engine)
withSession(engine) assession:
session.add(hero)
session.commit()
session.refresh(hero)
db_hero=session.get(Hero, hero.id)
copy=db_hero.model_copy(update={"name": "Deadpond Copy"})
assertcopy.name=="Deadpond Copy"and \
copy.secret_name=="Dive Wilson"and \
copy.age==25

[50Bytes-dev]

@ogabrielluiz@honglei .model_copy(...) raise error:

object has no attribute '__pydantic_extra__'

service_category: ServiceCategory=awaitself.get_object_or_404(id, company.id, async_db)
new_service_category=service_category.model_copy(update={'id': None})

Use this test:

deftest_model_copy(clear_sqlmodel):
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
name: strsecret_name: strage: Optional[int] =Nonehero=Hero(name="Deadpond", secret_name="Dive Wilson", age=25)
engine=create_engine("sqlite://")
SQLModel.metadata.create_all(engine)
withSession(engine) assession:
session.add(hero)
session.commit()
session.refresh(hero)
db_hero=session.get(Hero, hero.id)
copy=db_hero.model_copy(update={"name": "Deadpond Copy"})
assertcopy.name=="Deadpond Copy"and \
copy.secret_name=="Dive Wilson"and \
copy.age==25

fixed honglei#1

[8thgencore]

I wonder if @tiangolo will notice this sentence

[MattOates]

Not strictly a SQLModels problem, but if you try and use the current SQLModels 0.0.8 with FastAPI 0.103.0 where you've targeted fastapi[all] conflicts are created because of the pydantic version pins of SQLModel, as pydantic-settings requires a much later version than the max pinned. Just like to add a +1 for this PR being quite important to get merged.

[MatsiukMykola]

topic is very long, guys, possible to use with this PR?
@tiangolo can please comment? Your package is awesome, very needs an update.
Anyway big th! (hello from Ukraine)

[50Bytes-dev]

topic is very long, guys, possible to use with this PR? @tiangolo can please comment? Your package is awesome, very needs an update. Anyway big th! (hello from Ukraine)

@MatsiukMykola I'm using it in my project. Everything's great
Use this https://github.com/honglei/sqlmodel

[ikreb7]

The type EmailStr doesn't work. I get the error ValueError(f"The field {field.title} has no matching SQLAlchemy type").

[50Bytes-dev]

The type EmailStr doesn't work. I get the error ValueError(f"The field {field.title} has no matching SQLAlchemy type").

@ikreb7 use this

my_email: EmailStr=Field(sa_column=Column(String))

[honglei]

@ikreb7@50Bytes-dev support the following in my fork(https://github.com/honglei/sqlmodel), but how to change test.yml to do pip install pydantic[email]?

http: HttpUrl=Field(max_length=250)
email: EmailStrname_email : NameEmail=Field( max_length=50)
import_string:ImportString=Field(max_length=200, min_length=100)

[50Bytes-dev]

@ikreb7@50Bytes-dev support the following in my fork(https://github.com/honglei/sqlmodel), but how to change test.yml to do pip install pydantic[email]?

http: HttpUrl=Field(max_length=250)
email: EmailStrname_email : NameEmail=Field( max_length=50)
import_string:ImportString=Field(max_length=200, min_length=100)

@honglei
In pyproject.toml update line 36

pydantic = { version = "^2.1.1", extras = ["email"] }

[honglei]

@50Bytes-dev thanks!

[wojciech-mazur-mlg]

@honglei what's the status of your fork? Can we use it till a stable version of SQLmodel is released?

[honglei]

I used in my projects, it works fine. wojciech-mazur-mlg ***@***.***> 于 2023年9月18日周一 下午7:30写道：

…

@honglei <https://github.com/honglei> what's the status of your fork? Can we use it till a stable version of SQLmodel is released? — Reply to this email directly, view it on GitHub <#632 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHW5AVSOKK64F5EAV4P7LTX3AWDZANCNFSM6AAAAAA27PJTWM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[AntonDeMeester]

@honglei what's the status of your fork? Can we use it till a stable version of SQLmodel is released?

I have tried it out a bit and it seems to work. Some other people are also saying it works. But I would say it's not official until @tiangolo approves it or makes his own version. I also won't be trying everything out to be honest 😅

[BoManev]

After I updated, the "alias" stopped working

class UserBase(SQLModel):
password_hash: str = Field(alias="password")
def create_user(user: UserBase):
pass

The validation in create_user fails with "password ", but works with "password_hash"

[honglei]

@BoManev@AntonDeMeester
In pydantic, function pydantic.fields.Field will use alias to infer validation_alias,
but in sqlmodel.main.FieldInfo, validation_alias is not set, so alias not work under pydantic_core

https://github.com/pydantic/pydantic/blob/188018ccd0c26085e51e632ae4f909b3ee31a0f7/pydantic/fields.py#L819

[BoManev]

@honglei Thank you for the fast reply. In that case should I create a class using pydantic BaseModel for validation of my inputs and then map it with model_validate to my SQLModel.

I also found this #374

EDIT: Generally, how to handle data validation when field names change and data needs to be modified. In my case I receive "password", but internally I want my types to carry "password_hash". In Rust, I can specify Into/To/From traits and convert between types. Can I do something similar with pydantic/SQLModel, define the conversion of 1 data schema to other.

[JanBeelte]

Hey folks,
what is the current status and what are the next steps here?
Is there any ETA when this one will be merged to master?
I could also take over some tasks in case that helps, given I desperately need pydantic v2 support otherwise I cannot use SQLModel.
Best,
Jan

[lawrenceeldridge]

Really looking forward to seeing these changes go in - carrying out an early preview as I'm already using the latest versions. One thing I've observed, I've been struggling a little with using the async db setup in the context of pulling in additional relationships. All of the other routes work fine from this tutorial, but when I call read_hero I get the following error:

fastapi.exceptions.ResponseValidationError: 1 validation errors:
{'type': 'get_attribute_error', 'loc': ('response', 'team'), 'msg': "Error extracting attribute: MissingGreenlet: greenlet_spawn has not been called; can't call await_only() here. Was IO attempted in an unexpected place? (Background on this error at: https://sqlalche.me/e/20/xd2s)", 'input': Hero(secret_name='Batman', id=1, team_id=1, name='Bruce Wayne', age=42), 'ctx': {'error': "MissingGreenlet: greenlet_spawn has not been called; can't call await_only() here. Was IO attempted in an unexpected place? (Background on this error at: https://sqlalche.me/e/20/xd2s)"}, 'url': 'https://errors.pydantic.dev/2.4/v/get_attribute_error'}

This only happens when the response_model is set to HeroReadWithTeam, as the expectation is that we'd want to include teams with the hero. Using HeroRead works fine.

[lawrenceeldridge]

This is now resolved with the help of another post here. For relationships between models you may need to leverage the correct loading styles through the sa_relationship_kwargs property that is offered by SQLModel. For the curious this is how I've built up the tutorial with async in mind below.

In SQL Alchemy there's more reading here on the subject of lazy loading.

Really happy, so will crack on :)

fromcontextlibimportasynccontextmanagerfromtypingimportList, OptionalfromfastapiimportDepends, FastAPI, HTTPException, Queryfromsqlalchemy.ext.asyncioimportasync_sessionmaker, create_async_enginefromsqlmodelimportField, Relationship, SQLModelfromsqlalchemy.sql.expressionimportselectfromsqlmodel.ext.asyncio.sessionimportAsyncSessionfromconfigimportconfigclassTeamBase(SQLModel):
name: str=Field(index=True)
headquarters: strclassTeam(TeamBase, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
heroes: List["Hero"] =Relationship(back_populates="team", sa_relationship_kwargs={"lazy": "selectin"})
classTeamCreate(TeamBase):
passclassTeamRead(TeamBase):
id: intclassTeamUpdate(SQLModel):
id: Optional[int] =Nonename: Optional[str] =Noneheadquarters: Optional[str] =NoneclassHeroBase(SQLModel):
name: str=Field(index=True)
secret_name: strage: Optional[int] =Field(default=None, index=True)
team_id: Optional[int] =Field(default=None, foreign_key="team.id")
classHero(HeroBase, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
team: Optional[Team] =Relationship(back_populates="heroes", sa_relationship_kwargs={"lazy": "joined"})
classHeroRead(HeroBase):
id: intclassHeroCreate(HeroBase):
passclassHeroUpdate(SQLModel):
name: Optional[str] =Nonesecret_name: Optional[str] =Noneage: Optional[int] =Noneteam_id: Optional[int] =NoneclassHeroReadWithTeam(HeroRead):
team: Optional[TeamRead] =NoneclassTeamReadWithHeroes(TeamRead):
heroes: List[HeroRead] = []
engine=create_async_engine(config.DB_HOST, echo=True, future=True)
asyncdefinit_db():
asyncwithengine.begin() asconn:
awaitconn.run_sync(SQLModel.metadata.create_all)
asyncdefget_session() ->AsyncSession:
async_session=async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
asyncwithasync_session() assession:
yieldsession@asynccontextmanagerasyncdeflifespan(app: FastAPI):
# on startupawaitinit_db()
yield# on shutdownawaitengine.dispose()
app=FastAPI(lifespan=lifespan)
@app.post("/heroes", response_model=HeroRead)asyncdefcreate_hero(*, session: AsyncSession=Depends(get_session), hero: HeroCreate):
db_hero=Hero.model_validate(hero)
session.add(db_hero)
awaitsession.commit()
awaitsession.refresh(db_hero)
returndb_hero@app.get("/heroes", response_model=List[HeroRead])asyncdefread_heroes(
*,
session: AsyncSession=Depends(get_session),
offset: int=0,
limit: int=Query(default=100, lte=100),
):
statement=select(Hero).offset(offset).limit(limit)
heroes=awaitsession.execute(statement)
returnheroes.scalars().all()
@app.get("/heroes/{hero_id}", response_model=HeroReadWithTeam)asyncdefread_hero(*, session: AsyncSession=Depends(get_session), hero_id: int):
hero=awaitsession.get(Hero, hero_id)
ifnothero:
raiseHTTPException(status_code=404, detail="Hero not found")
returnhero@app.patch("/heroes/{hero_id}", response_model=HeroRead)asyncdefupdate_hero(
*, session: AsyncSession=Depends(get_session), hero_id: int, hero: HeroUpdate
):
db_hero=awaitsession.get(Hero, hero_id)
ifnotdb_hero:
raiseHTTPException(status_code=404, detail="Hero not found")
hero_data=hero.model_dump(exclude_unset=True)
forkey, valueinhero_data.items():
setattr(db_hero, key, value)
session.add(db_hero)
awaitsession.commit()
awaitsession.refresh(db_hero)
returndb_hero@app.delete("/heroes/{hero_id}")asyncdefdelete_hero(*, session: AsyncSession=Depends(get_session), hero_id: int):
hero=awaitsession.get(Hero, hero_id)
ifnothero:
raiseHTTPException(status_code=404, detail="Hero not found")
awaitsession.delete(hero)
awaitsession.commit()
return {"ok": True}
@app.post("/teams", response_model=TeamRead)asyncdefcreate_team(*, session: AsyncSession=Depends(get_session), team: TeamCreate):
db_team=Team.model_validate(team)
session.add(db_team)
awaitsession.commit()
awaitsession.refresh(db_team)
returndb_team@app.get("/teams", response_model=List[TeamRead])asyncdefread_teams(
*,
session: AsyncSession=Depends(get_session),
offset: int=0,
limit: int=Query(default=100, lte=100),
):
statement=select(Team).offset(offset).limit(limit)
teams=awaitsession.execute(statement)
returnteams.scalars().all()
@app.get("/teams/{team_id}", response_model=TeamReadWithHeroes)asyncdefread_team(*, team_id: int, session: AsyncSession=Depends(get_session)):
team=awaitsession.get(Team, team_id)
ifnotteam:
raiseHTTPException(status_code=404, detail="Team not found")
returnteam@app.patch("/teams/{team_id}", response_model=TeamRead)asyncdefupdate_team(
*,
session: AsyncSession=Depends(get_session),
team_id: int,
team: TeamUpdate,
):
db_team=awaitsession.get(Team, team_id)
ifnotdb_team:
raiseHTTPException(status_code=404, detail="Team not found")
team_data=team.model_dump(exclude_unset=True)
forkey, valueinteam_data.items():
setattr(db_team, key, value)
session.add(db_team)
awaitsession.commit()
awaitsession.refresh(db_team)
returndb_team@app.delete("/teams/{team_id}")asyncdefdelete_team(*, session: AsyncSession=Depends(get_session), team_id: int):
team=awaitsession.get(Team, team_id)
ifnotteam:
raiseHTTPException(status_code=404, detail="Team not found")
awaitsession.delete(team)
awaitsession.commit()
return {"ok": True}

[blakewatters]

Joining the party here trying to update a fairly large data model the fork.

One thing I am working on resolving and curious if anyone else has run into is an issue with model class inheritance producing SQLAlchemy errors regarding column reuse:

sqlalchemy.exc.ArgumentError: Column object 'created_at' already assigned to Table 'business_address_links'

My TimestampModel looks like:

classTimestampModel(BaseModel):
created_at: datetime.datetime=sqlmodel.Field(
sa_column=sqlalchemy.Column(
sqlalchemy.DateTime(timezone=True),
server_default=sqlalchemy.func.statement_timestamp(),
nullable=False
)
)
updated_at: typing.Optional[datetime.datetime] =sqlmodel.Field(
sa_column=sqlalchemy.Column(
sqlalchemy.DateTime(timezone=True),
onupdate=sqlalchemy.func.statement_timestamp(),
nullable=True
)
)

It looks like the Column definition is being reused across inherited subclasses

[MatsiukMykola]

always was interesting how pull to sa.Column - column comment based on Field Description?

[honglei]

@blakewatters try https://github.com/honglei/sqlmodel/blob/3005495a3ec6c8216b31cbd623f91c7bc8ba174f/tests/test_class_hierarchy.py#L24C17-L24C17

[norton120]

➕ on thanks for putting this PR up, my team is a few weeks into a new project and I'd hate to be stuck behind a major on 2 core libraries this early on, so this is huge. I'm running into a strange issue creating a one-to-many relationship (using this branch), any ideas would be appreciated. Here's the error:

sqlalchemy.exc.ArgumentError: relationship 'venues' expects a class or a mapper argument (received: <class 'list'>)

the related models:

classOrganization(SQLModel, table=True):
id: Optional[int] =sqlmodel.Field(default=None, primary_key=True)
venues: List["Venue"] =sqlmodel.Relationship(back_populates="organization")
@classmethodasyncdeflist(cls, session: AsyncSession):
query=select(cls)
result_from_db=awaitsession.execute(query)
collection=result_from_db.fetchall()
returncollectionclassVenue(SQLModel, table=True):
id: Optional[int] =sqlmodel.Field(default=None, primary_key=True)
organization_id: int=sqlmodel.Field(foreign_key="organization.id")
organization: Organization=sqlmodel.Relationship(back_populates="venues")

and the calling functionality:

classDatabaseConnection:
def__init__(self):
url=sqlalchemy.URL.create(*url_args)
self.async_engine=sqlalchemy.ext.asyncio.create_async_engine(
url, echo=True, future=True
)
asyncdefget_async_session(self) ->AsyncSession:
async_session=sessionmaker(
bind=self.async_engine, class_=AsyncSession, expire_on_commit=False
)
asyncwithasync_session() assession:
yieldsessiondb_session=DatabaseConnection().get_async_session@router.get("/")asyncdefindex(*, session=Depends(db_session)):
returnawaitOrganization.list(session=session)

I've tried changing annotation types, explicitly passing an sa.relationship to the Relationship, setting lazy loading... I'm not super familiar with the SQLModel code base so I'm not sure where to look next.

[nikdavis]

It looks like the both of the goals prior to migrating to sqlalchemy 2.0, and pydantic 2.0 have been reached (supporting the latest pre 2.0 versions of each). The next goal in the roadmap is migrating each to 2.0.

Any update here?

[github-actions]

📝 Docs preview for commit 7ecbc38 at: https://5165ba05.sqlmodel.pages.dev

[github-actions]

📝 Docs preview for commit ab07514 at: https://398280da.sqlmodel.pages.dev

[BoManev]

@AntonDeMeester I have been using an older commit from this repo, however today I tried to update my dependencies and got the following errors.

Because no versions of pydantic-settings match >2.0.0,<2.0.1 || >2.0.1,<2.0.2 || >2.0.2,<2.0.3 || >2.0.3,<2.1.0 || >2.1.0
and pydantic-settings (2.0.0) depends on pydantic (>=2.0b3), pydantic-settings (>=2.0.0,<2.0.1 || >2.0.1,<2.0.2 || >2.0.2,<2.0.3 || >2.0.3,<2.1.0 || >2.1.0) requires pydantic (>=2.0b3).
And because pydantic-settings (2.0.1) depends on pydantic (>=2.0.1), pydantic-settings (>=2.0.0,<2.0.2 || >2.0.2,<2.0.3 || >2.0.3,<2.1.0 || >2.1.0) requires pydantic (>=2.0b3).
And because pydantic-settings (2.0.2) depends on pydantic (>=2.0.1)
and pydantic-settings (2.0.3) depends on pydantic (>=2.0.1), pydantic-settings (>=2.0.0,<2.1.0 || >2.1.0) requires pydantic (>=2.0b3).
And because pydantic-settings (2.1.0) depends on pydantic (>=2.3.0)
and sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head depends on pydantic (^1.9.0), sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head is incompatible with pydantic-settings (>=2.0.0).
And because fastapi (0.103.2) depends on pydantic-settings (>=2.0.0)
and no versions of fastapi match >0.103.2,<0.104.0, sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head is incompatible with fastapi (>=0.103.2,<0.104.0).
So, because api depends on both fastapi (^0.103.2) and sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head, version solving failed.

Snippet from my pyproject.toml

fastapi = {extras = ["all"], version = "^0.103.2"}
sqlmodel = {git = "https://github.com/tiangolo/sqlmodel.git", rev = "refs/pull/632/head"}

The older commit from this PR is using pydantic = { version = ">=2.1.1,<=2.4.2", extras = ["email"] } this worked fine with pydantic-settings from fastapi extras

[AntonDeMeester]

@AntonDeMeester I have been using an older commit from this repo, however today I tried to update my dependencies and got the following errors.

The older commit from this PR is using pydantic = { version = ">=2.1.1,<=2.4.2", extras = ["email"] } this worked fine with pydantic-settings from fastapi extras

Ye I've been working on getting both Pydantic v1 and v2 to work at the same time, I might have broken some things. I'll change the PR to use the old commit again and push my changes to a new PR on top of this one until both work.

[github-actions]

📝 Docs preview for commit d0ae3e8 at: https://ff83edad.sqlmodel.pages.dev

[BoManev]

@AntonDeMeester This commit still doesn't work
The one Im using is this f926d9d

[honglei]

@AntonDeMeester sqlmodel has drop support for sqlalchemy<2.0, maybe next release will drop support for pydantic v1,
the author has no interests to fix some bugs in pydantic v1, which make me move my project to pydantic v2.

https://github.com/tiangolo/sqlmodel/blob/b1c2f822c9db036afe1b7ec81d335368e9546756/pyproject.toml#L34

[ogabrielluiz]

@AntonDeMeester how are you?

I've been using your fork internally for a while now and it seems stable.

Do you see this PR being merged this week?

Thanks in advance

[AntonDeMeester]

Closing in favour of @tiangolo PR here

[tiangolo]

Thank you for your work @AntonDeMeester! I included your commits in #722 (you can see your badge now says "Contributor" 😎 ), I changed a few extra things, more details in that PR. It's now released and available as SQLModel 0.0.14! 🎉 🌮

[samyxdev]

Not sure if replying to a closed PR is the way, but it seems that the issue raised by @50Bytes-dev about .model_copy raising an error is still hasn't been fixed with the new release 0.0.14, despite the fix he proposed (thanks for that !).
Am I missing something or was the matter moved to a different issue ?

Below is a quote of the mentionned error:

@ogabrielluiz@honglei .model_copy(...) raise error:

object has no attribute '__pydantic_extra__'

service_category: ServiceCategory=awaitself.get_object_or_404(id, company.id, async_db)
new_service_category=service_category.model_copy(update={'id': None})

Use this test:

deftest_model_copy(clear_sqlmodel):
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
name: strsecret_name: strage: Optional[int] =Nonehero=Hero(name="Deadpond", secret_name="Dive Wilson", age=25)
engine=create_engine("sqlite://")
SQLModel.metadata.create_all(engine)
withSession(engine) assession:
session.add(hero)
session.commit()
session.refresh(hero)
db_hero=session.get(Hero, hero.id)
copy=db_hero.model_copy(update={"name": "Deadpond Copy"})
assertcopy.name=="Deadpond Copy"and \
copy.secret_name=="Dive Wilson"and \
copy.age==25

fixed honglei#1

[ChrisNi888]

This error 'TypeError: issubclass() arg 1 must be a class' seems not fixed. ( I use Pydantic 2.7 and SQLModel 0.0.16. ) So I use Pydantic 1.0.11 and SQLModel 0.08. If anyone has same problem, hope it helps.

[AntonDeMeester]

Upgrades to Pydantic 2, using the new pydantic models. Switches for v1 and v2 can be built later

Builds on #563 to upgrade SQL Alchemy

Still WIP, did some uglier hacks which could provide some problems.

Most of it is just changing code with different naming of Pydantic 2. However, the validation of empty initialisations is broken because that now happens in rust and there is no static validate_model anymore. For now I set defaults to all provided Fields is there are none, so that it will always inititialise, but that screws a bit with the logic. This is also annoying for cls.model_validate as it screws up SQL Alchemy.

[luoshuijs]

Thank you very much for your PR. I can use it with SQLModel in the pydantic v2 environment, and I've found some issues and have some suggestions.

[luoshuijs]

NoneType has been moved to types in Python 3.10+.
https://docs.python.org/3/library/types.html#types.NoneType

[luoshuijs]

In the test tests\test_tutorial\test_delete, the following error occurred: AttributeError: 'Hero' object has no attribute '__pydantic_extra__'.

You can solve this problem by adding the following code to SQLmodel:

def__new__(cls, *args: Any, **kwargs: Any) ->Any:
new_object=super().__new__(cls)
object.__setattr__(new_object, "__pydantic_fields_set__", set())
returnnew_object

[AntonDeMeester]

I got almost everything working except for Open API. It provides optional for the OpenAPI if you use a table as a request or response type. While this isn't correct in the strict sense, it is correct for Pydantic as empty initialisations need to be supported. I tried a couple of other ways (creating a second class for the optional initialisations or trying to override the OpenAPI generation) but none of them worked out.

For now I commented the OpenAPI assertions, but @tiangolo will need to decide how to handle it. As he wrote FastAPI I'm sure he'll have better ideas 😄 I just hope I can give him some inspiration with this.

[honglei]

the following test not pass:

fromsqlmodelimportSQLModel, FieldclassArticle(SQLModel, table=True):
name : str|None=Field( description='the name of article')

Error:

 File "crud_test\test_sqlmodel.py", line 2, in <module>
class Article(SQLModel, table=True):
 File "crud_test\sqlmodel\main.py", line 318, in __new__
col = get_column_from_field(v)
 File "crud_test\sqlmodel\main.py", line 464, in get_column_from_field
sa_type = get_sqlalchemy_type(field)
 File "crud_test\sqlmodel\main.py", line 417, in get_sqlalchemy_type
if issubclass(type_, str):
builtins.TypeError: issubclass() arg 1 must be a class

[yallxe]

@honglei try Optional[str] instead of str | None

I think just one more case must be checked somewhere.

[honglei]

@honglei try Optional[str] instead of str | None

I think just one more case must be checked somewhere.

I made a pull request to support it. mbsantiago#1

[ma-ts]

Hi! Just checking if there's still something to be done for this PR, more than willing to help out!

[ogabrielluiz]

Hey! I'm also willing to help out.

[honglei]

@ma-ts@ogabrielluiz You can help by testing the PR in your projects, and reporting how well it works.
Right now my project based on https://github.com/mbsantiago/sqlmodel, with my version at: https://github.com/honglei/sqlmodel

[ogabrielluiz]

@honglei
After the normal pydantic migration which was pretty straightforward all Langflow's tests are passing. Will perform some more tests ASAP.

Installed with poetry using sqlmodel = { git = "https://github.com/honglei/sqlmodel.git", branch = "main" }

[50Bytes-dev]

@ogabrielluiz@honglei
.model_copy(...) raise error:

object has no attribute '__pydantic_extra__'

service_category: ServiceCategory=awaitself.get_object_or_404(id, company.id, async_db)
new_service_category=service_category.model_copy(update={'id': None})

Use this test:

deftest_model_copy(clear_sqlmodel):
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
name: strsecret_name: strage: Optional[int] =Nonehero=Hero(name="Deadpond", secret_name="Dive Wilson", age=25)
engine=create_engine("sqlite://")
SQLModel.metadata.create_all(engine)
withSession(engine) assession:
session.add(hero)
session.commit()
session.refresh(hero)
db_hero=session.get(Hero, hero.id)
copy=db_hero.model_copy(update={"name": "Deadpond Copy"})
assertcopy.name=="Deadpond Copy"and \
copy.secret_name=="Dive Wilson"and \
copy.age==25

[50Bytes-dev]

@ogabrielluiz@honglei .model_copy(...) raise error:

object has no attribute '__pydantic_extra__'

service_category: ServiceCategory=awaitself.get_object_or_404(id, company.id, async_db)
new_service_category=service_category.model_copy(update={'id': None})

Use this test:

deftest_model_copy(clear_sqlmodel):
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
name: strsecret_name: strage: Optional[int] =Nonehero=Hero(name="Deadpond", secret_name="Dive Wilson", age=25)
engine=create_engine("sqlite://")
SQLModel.metadata.create_all(engine)
withSession(engine) assession:
session.add(hero)
session.commit()
session.refresh(hero)
db_hero=session.get(Hero, hero.id)
copy=db_hero.model_copy(update={"name": "Deadpond Copy"})
assertcopy.name=="Deadpond Copy"and \
copy.secret_name=="Dive Wilson"and \
copy.age==25

fixed honglei#1

[8thgencore]

I wonder if @tiangolo will notice this sentence

[MattOates]

Not strictly a SQLModels problem, but if you try and use the current SQLModels 0.0.8 with FastAPI 0.103.0 where you've targeted fastapi[all] conflicts are created because of the pydantic version pins of SQLModel, as pydantic-settings requires a much later version than the max pinned. Just like to add a +1 for this PR being quite important to get merged.

[MatsiukMykola]

topic is very long, guys, possible to use with this PR?
@tiangolo can please comment? Your package is awesome, very needs an update.
Anyway big th! (hello from Ukraine)

[50Bytes-dev]

topic is very long, guys, possible to use with this PR? @tiangolo can please comment? Your package is awesome, very needs an update. Anyway big th! (hello from Ukraine)

@MatsiukMykola I'm using it in my project. Everything's great
Use this https://github.com/honglei/sqlmodel

[ikreb7]

The type EmailStr doesn't work. I get the error ValueError(f"The field {field.title} has no matching SQLAlchemy type").

[50Bytes-dev]

The type EmailStr doesn't work. I get the error ValueError(f"The field {field.title} has no matching SQLAlchemy type").

@ikreb7 use this

my_email: EmailStr=Field(sa_column=Column(String))

[honglei]

@ikreb7@50Bytes-dev support the following in my fork(https://github.com/honglei/sqlmodel), but how to change test.yml to do pip install pydantic[email]?

http: HttpUrl=Field(max_length=250)
email: EmailStrname_email : NameEmail=Field( max_length=50)
import_string:ImportString=Field(max_length=200, min_length=100)

[50Bytes-dev]

@ikreb7@50Bytes-dev support the following in my fork(https://github.com/honglei/sqlmodel), but how to change test.yml to do pip install pydantic[email]?

http: HttpUrl=Field(max_length=250)
email: EmailStrname_email : NameEmail=Field( max_length=50)
import_string:ImportString=Field(max_length=200, min_length=100)

@honglei
In pyproject.toml update line 36

pydantic = { version = "^2.1.1", extras = ["email"] }

[honglei]

@50Bytes-dev thanks!

[wojciech-mazur-mlg]

@honglei what's the status of your fork? Can we use it till a stable version of SQLmodel is released?

[honglei]

I used in my projects, it works fine. wojciech-mazur-mlg ***@***.***> 于 2023年9月18日周一 下午7:30写道：

…

@honglei <https://github.com/honglei> what's the status of your fork? Can we use it till a stable version of SQLmodel is released? — Reply to this email directly, view it on GitHub <#632 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHW5AVSOKK64F5EAV4P7LTX3AWDZANCNFSM6AAAAAA27PJTWM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[AntonDeMeester]

@honglei what's the status of your fork? Can we use it till a stable version of SQLmodel is released?

I have tried it out a bit and it seems to work. Some other people are also saying it works. But I would say it's not official until @tiangolo approves it or makes his own version. I also won't be trying everything out to be honest 😅

[BoManev]

After I updated, the "alias" stopped working

class UserBase(SQLModel):
password_hash: str = Field(alias="password")
def create_user(user: UserBase):
pass

The validation in create_user fails with "password ", but works with "password_hash"

[honglei]

@BoManev@AntonDeMeester
In pydantic, function pydantic.fields.Field will use alias to infer validation_alias,
but in sqlmodel.main.FieldInfo, validation_alias is not set, so alias not work under pydantic_core

https://github.com/pydantic/pydantic/blob/188018ccd0c26085e51e632ae4f909b3ee31a0f7/pydantic/fields.py#L819

[BoManev]

@honglei Thank you for the fast reply. In that case should I create a class using pydantic BaseModel for validation of my inputs and then map it with model_validate to my SQLModel.

I also found this #374

EDIT: Generally, how to handle data validation when field names change and data needs to be modified. In my case I receive "password", but internally I want my types to carry "password_hash". In Rust, I can specify Into/To/From traits and convert between types. Can I do something similar with pydantic/SQLModel, define the conversion of 1 data schema to other.

[JanBeelte]

Hey folks,
what is the current status and what are the next steps here?
Is there any ETA when this one will be merged to master?
I could also take over some tasks in case that helps, given I desperately need pydantic v2 support otherwise I cannot use SQLModel.
Best,
Jan

[lawrenceeldridge]

Really looking forward to seeing these changes go in - carrying out an early preview as I'm already using the latest versions. One thing I've observed, I've been struggling a little with using the async db setup in the context of pulling in additional relationships. All of the other routes work fine from this tutorial, but when I call read_hero I get the following error:

fastapi.exceptions.ResponseValidationError: 1 validation errors:
{'type': 'get_attribute_error', 'loc': ('response', 'team'), 'msg': "Error extracting attribute: MissingGreenlet: greenlet_spawn has not been called; can't call await_only() here. Was IO attempted in an unexpected place? (Background on this error at: https://sqlalche.me/e/20/xd2s)", 'input': Hero(secret_name='Batman', id=1, team_id=1, name='Bruce Wayne', age=42), 'ctx': {'error': "MissingGreenlet: greenlet_spawn has not been called; can't call await_only() here. Was IO attempted in an unexpected place? (Background on this error at: https://sqlalche.me/e/20/xd2s)"}, 'url': 'https://errors.pydantic.dev/2.4/v/get_attribute_error'}

This only happens when the response_model is set to HeroReadWithTeam, as the expectation is that we'd want to include teams with the hero. Using HeroRead works fine.

[lawrenceeldridge]

This is now resolved with the help of another post here. For relationships between models you may need to leverage the correct loading styles through the sa_relationship_kwargs property that is offered by SQLModel. For the curious this is how I've built up the tutorial with async in mind below.

In SQL Alchemy there's more reading here on the subject of lazy loading.

Really happy, so will crack on :)

fromcontextlibimportasynccontextmanagerfromtypingimportList, OptionalfromfastapiimportDepends, FastAPI, HTTPException, Queryfromsqlalchemy.ext.asyncioimportasync_sessionmaker, create_async_enginefromsqlmodelimportField, Relationship, SQLModelfromsqlalchemy.sql.expressionimportselectfromsqlmodel.ext.asyncio.sessionimportAsyncSessionfromconfigimportconfigclassTeamBase(SQLModel):
name: str=Field(index=True)
headquarters: strclassTeam(TeamBase, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
heroes: List["Hero"] =Relationship(back_populates="team", sa_relationship_kwargs={"lazy": "selectin"})
classTeamCreate(TeamBase):
passclassTeamRead(TeamBase):
id: intclassTeamUpdate(SQLModel):
id: Optional[int] =Nonename: Optional[str] =Noneheadquarters: Optional[str] =NoneclassHeroBase(SQLModel):
name: str=Field(index=True)
secret_name: strage: Optional[int] =Field(default=None, index=True)
team_id: Optional[int] =Field(default=None, foreign_key="team.id")
classHero(HeroBase, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
team: Optional[Team] =Relationship(back_populates="heroes", sa_relationship_kwargs={"lazy": "joined"})
classHeroRead(HeroBase):
id: intclassHeroCreate(HeroBase):
passclassHeroUpdate(SQLModel):
name: Optional[str] =Nonesecret_name: Optional[str] =Noneage: Optional[int] =Noneteam_id: Optional[int] =NoneclassHeroReadWithTeam(HeroRead):
team: Optional[TeamRead] =NoneclassTeamReadWithHeroes(TeamRead):
heroes: List[HeroRead] = []
engine=create_async_engine(config.DB_HOST, echo=True, future=True)
asyncdefinit_db():
asyncwithengine.begin() asconn:
awaitconn.run_sync(SQLModel.metadata.create_all)
asyncdefget_session() ->AsyncSession:
async_session=async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
asyncwithasync_session() assession:
yieldsession@asynccontextmanagerasyncdeflifespan(app: FastAPI):
# on startupawaitinit_db()
yield# on shutdownawaitengine.dispose()
app=FastAPI(lifespan=lifespan)
@app.post("/heroes", response_model=HeroRead)asyncdefcreate_hero(*, session: AsyncSession=Depends(get_session), hero: HeroCreate):
db_hero=Hero.model_validate(hero)
session.add(db_hero)
awaitsession.commit()
awaitsession.refresh(db_hero)
returndb_hero@app.get("/heroes", response_model=List[HeroRead])asyncdefread_heroes(
*,
session: AsyncSession=Depends(get_session),
offset: int=0,
limit: int=Query(default=100, lte=100),
):
statement=select(Hero).offset(offset).limit(limit)
heroes=awaitsession.execute(statement)
returnheroes.scalars().all()
@app.get("/heroes/{hero_id}", response_model=HeroReadWithTeam)asyncdefread_hero(*, session: AsyncSession=Depends(get_session), hero_id: int):
hero=awaitsession.get(Hero, hero_id)
ifnothero:
raiseHTTPException(status_code=404, detail="Hero not found")
returnhero@app.patch("/heroes/{hero_id}", response_model=HeroRead)asyncdefupdate_hero(
*, session: AsyncSession=Depends(get_session), hero_id: int, hero: HeroUpdate
):
db_hero=awaitsession.get(Hero, hero_id)
ifnotdb_hero:
raiseHTTPException(status_code=404, detail="Hero not found")
hero_data=hero.model_dump(exclude_unset=True)
forkey, valueinhero_data.items():
setattr(db_hero, key, value)
session.add(db_hero)
awaitsession.commit()
awaitsession.refresh(db_hero)
returndb_hero@app.delete("/heroes/{hero_id}")asyncdefdelete_hero(*, session: AsyncSession=Depends(get_session), hero_id: int):
hero=awaitsession.get(Hero, hero_id)
ifnothero:
raiseHTTPException(status_code=404, detail="Hero not found")
awaitsession.delete(hero)
awaitsession.commit()
return {"ok": True}
@app.post("/teams", response_model=TeamRead)asyncdefcreate_team(*, session: AsyncSession=Depends(get_session), team: TeamCreate):
db_team=Team.model_validate(team)
session.add(db_team)
awaitsession.commit()
awaitsession.refresh(db_team)
returndb_team@app.get("/teams", response_model=List[TeamRead])asyncdefread_teams(
*,
session: AsyncSession=Depends(get_session),
offset: int=0,
limit: int=Query(default=100, lte=100),
):
statement=select(Team).offset(offset).limit(limit)
teams=awaitsession.execute(statement)
returnteams.scalars().all()
@app.get("/teams/{team_id}", response_model=TeamReadWithHeroes)asyncdefread_team(*, team_id: int, session: AsyncSession=Depends(get_session)):
team=awaitsession.get(Team, team_id)
ifnotteam:
raiseHTTPException(status_code=404, detail="Team not found")
returnteam@app.patch("/teams/{team_id}", response_model=TeamRead)asyncdefupdate_team(
*,
session: AsyncSession=Depends(get_session),
team_id: int,
team: TeamUpdate,
):
db_team=awaitsession.get(Team, team_id)
ifnotdb_team:
raiseHTTPException(status_code=404, detail="Team not found")
team_data=team.model_dump(exclude_unset=True)
forkey, valueinteam_data.items():
setattr(db_team, key, value)
session.add(db_team)
awaitsession.commit()
awaitsession.refresh(db_team)
returndb_team@app.delete("/teams/{team_id}")asyncdefdelete_team(*, session: AsyncSession=Depends(get_session), team_id: int):
team=awaitsession.get(Team, team_id)
ifnotteam:
raiseHTTPException(status_code=404, detail="Team not found")
awaitsession.delete(team)
awaitsession.commit()
return {"ok": True}

[blakewatters]

Joining the party here trying to update a fairly large data model the fork.

One thing I am working on resolving and curious if anyone else has run into is an issue with model class inheritance producing SQLAlchemy errors regarding column reuse:

sqlalchemy.exc.ArgumentError: Column object 'created_at' already assigned to Table 'business_address_links'

My TimestampModel looks like:

classTimestampModel(BaseModel):
created_at: datetime.datetime=sqlmodel.Field(
sa_column=sqlalchemy.Column(
sqlalchemy.DateTime(timezone=True),
server_default=sqlalchemy.func.statement_timestamp(),
nullable=False
)
)
updated_at: typing.Optional[datetime.datetime] =sqlmodel.Field(
sa_column=sqlalchemy.Column(
sqlalchemy.DateTime(timezone=True),
onupdate=sqlalchemy.func.statement_timestamp(),
nullable=True
)
)

It looks like the Column definition is being reused across inherited subclasses

[MatsiukMykola]

always was interesting how pull to sa.Column - column comment based on Field Description?

[honglei]

@blakewatters try https://github.com/honglei/sqlmodel/blob/3005495a3ec6c8216b31cbd623f91c7bc8ba174f/tests/test_class_hierarchy.py#L24C17-L24C17

[norton120]

➕ on thanks for putting this PR up, my team is a few weeks into a new project and I'd hate to be stuck behind a major on 2 core libraries this early on, so this is huge. I'm running into a strange issue creating a one-to-many relationship (using this branch), any ideas would be appreciated. Here's the error:

sqlalchemy.exc.ArgumentError: relationship 'venues' expects a class or a mapper argument (received: <class 'list'>)

the related models:

classOrganization(SQLModel, table=True):
id: Optional[int] =sqlmodel.Field(default=None, primary_key=True)
venues: List["Venue"] =sqlmodel.Relationship(back_populates="organization")
@classmethodasyncdeflist(cls, session: AsyncSession):
query=select(cls)
result_from_db=awaitsession.execute(query)
collection=result_from_db.fetchall()
returncollectionclassVenue(SQLModel, table=True):
id: Optional[int] =sqlmodel.Field(default=None, primary_key=True)
organization_id: int=sqlmodel.Field(foreign_key="organization.id")
organization: Organization=sqlmodel.Relationship(back_populates="venues")

and the calling functionality:

classDatabaseConnection:
def__init__(self):
url=sqlalchemy.URL.create(*url_args)
self.async_engine=sqlalchemy.ext.asyncio.create_async_engine(
url, echo=True, future=True
)
asyncdefget_async_session(self) ->AsyncSession:
async_session=sessionmaker(
bind=self.async_engine, class_=AsyncSession, expire_on_commit=False
)
asyncwithasync_session() assession:
yieldsessiondb_session=DatabaseConnection().get_async_session@router.get("/")asyncdefindex(*, session=Depends(db_session)):
returnawaitOrganization.list(session=session)

I've tried changing annotation types, explicitly passing an sa.relationship to the Relationship, setting lazy loading... I'm not super familiar with the SQLModel code base so I'm not sure where to look next.

[nikdavis]

It looks like the both of the goals prior to migrating to sqlalchemy 2.0, and pydantic 2.0 have been reached (supporting the latest pre 2.0 versions of each). The next goal in the roadmap is migrating each to 2.0.

Any update here?

[github-actions]

📝 Docs preview for commit 7ecbc38 at: https://5165ba05.sqlmodel.pages.dev

[github-actions]

📝 Docs preview for commit ab07514 at: https://398280da.sqlmodel.pages.dev

[BoManev]

@AntonDeMeester I have been using an older commit from this repo, however today I tried to update my dependencies and got the following errors.

Because no versions of pydantic-settings match >2.0.0,<2.0.1 || >2.0.1,<2.0.2 || >2.0.2,<2.0.3 || >2.0.3,<2.1.0 || >2.1.0
and pydantic-settings (2.0.0) depends on pydantic (>=2.0b3), pydantic-settings (>=2.0.0,<2.0.1 || >2.0.1,<2.0.2 || >2.0.2,<2.0.3 || >2.0.3,<2.1.0 || >2.1.0) requires pydantic (>=2.0b3).
And because pydantic-settings (2.0.1) depends on pydantic (>=2.0.1), pydantic-settings (>=2.0.0,<2.0.2 || >2.0.2,<2.0.3 || >2.0.3,<2.1.0 || >2.1.0) requires pydantic (>=2.0b3).
And because pydantic-settings (2.0.2) depends on pydantic (>=2.0.1)
and pydantic-settings (2.0.3) depends on pydantic (>=2.0.1), pydantic-settings (>=2.0.0,<2.1.0 || >2.1.0) requires pydantic (>=2.0b3).
And because pydantic-settings (2.1.0) depends on pydantic (>=2.3.0)
and sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head depends on pydantic (^1.9.0), sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head is incompatible with pydantic-settings (>=2.0.0).
And because fastapi (0.103.2) depends on pydantic-settings (>=2.0.0)
and no versions of fastapi match >0.103.2,<0.104.0, sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head is incompatible with fastapi (>=0.103.2,<0.104.0).
So, because api depends on both fastapi (^0.103.2) and sqlmodel (0) @ git+https://github.com/tiangolo/sqlmodel.git@refs/pull/632/head, version solving failed.

Snippet from my pyproject.toml

fastapi = {extras = ["all"], version = "^0.103.2"}
sqlmodel = {git = "https://github.com/tiangolo/sqlmodel.git", rev = "refs/pull/632/head"}

The older commit from this PR is using pydantic = { version = ">=2.1.1,<=2.4.2", extras = ["email"] } this worked fine with pydantic-settings from fastapi extras

[AntonDeMeester]

@AntonDeMeester I have been using an older commit from this repo, however today I tried to update my dependencies and got the following errors.

The older commit from this PR is using pydantic = { version = ">=2.1.1,<=2.4.2", extras = ["email"] } this worked fine with pydantic-settings from fastapi extras

Ye I've been working on getting both Pydantic v1 and v2 to work at the same time, I might have broken some things. I'll change the PR to use the old commit again and push my changes to a new PR on top of this one until both work.

[github-actions]

📝 Docs preview for commit d0ae3e8 at: https://ff83edad.sqlmodel.pages.dev

[BoManev]

@AntonDeMeester This commit still doesn't work
The one Im using is this f926d9d

[honglei]

@AntonDeMeester sqlmodel has drop support for sqlalchemy<2.0, maybe next release will drop support for pydantic v1,
the author has no interests to fix some bugs in pydantic v1, which make me move my project to pydantic v2.

https://github.com/tiangolo/sqlmodel/blob/b1c2f822c9db036afe1b7ec81d335368e9546756/pyproject.toml#L34

[ogabrielluiz]

@AntonDeMeester how are you?

I've been using your fork internally for a while now and it seems stable.

Do you see this PR being merged this week?

Thanks in advance

[AntonDeMeester]

Closing in favour of @tiangolo PR here

[tiangolo]

Thank you for your work @AntonDeMeester! I included your commits in #722 (you can see your badge now says "Contributor" 😎 ), I changed a few extra things, more details in that PR. It's now released and available as SQLModel 0.0.14! 🎉 🌮

[samyxdev]

Not sure if replying to a closed PR is the way, but it seems that the issue raised by @50Bytes-dev about .model_copy raising an error is still hasn't been fixed with the new release 0.0.14, despite the fix he proposed (thanks for that !).
Am I missing something or was the matter moved to a different issue ?

Below is a quote of the mentionned error:

@ogabrielluiz@honglei .model_copy(...) raise error:

object has no attribute '__pydantic_extra__'

service_category: ServiceCategory=awaitself.get_object_or_404(id, company.id, async_db)
new_service_category=service_category.model_copy(update={'id': None})

Use this test:

deftest_model_copy(clear_sqlmodel):
classHero(SQLModel, table=True):
id: Optional[int] =Field(default=None, primary_key=True)
name: strsecret_name: strage: Optional[int] =Nonehero=Hero(name="Deadpond", secret_name="Dive Wilson", age=25)
engine=create_engine("sqlite://")
SQLModel.metadata.create_all(engine)
withSession(engine) assession:
session.add(hero)
session.commit()
session.refresh(hero)
db_hero=session.get(Hero, hero.id)
copy=db_hero.model_copy(update={"name": "Deadpond Copy"})
assertcopy.name=="Deadpond Copy"and \
copy.secret_name=="Dive Wilson"and \
copy.age==25

fixed honglei#1

[ChrisNi888]

This error 'TypeError: issubclass() arg 1 must be a class' seems not fixed. ( I use Pydantic 2.7 and SQLModel 0.0.16. ) So I use Pydantic 1.0.11 and SQLModel 0.08. If anyone has same problem, hope it helps.