Improper UUID serialisation dropping leading zeros ▶️ "ValueError: badly formed hexadecimal UUID string"

[andrewbolster]

First Check

• I added a very descriptive title to this issue.
• I used the GitHub search to find a similar issue and didn't find it.
• I searched the SQLModel documentation, with the integrated search.
• I already searched in Google "How to X in SQLModel" and didn't find any information.
• I already read and followed all the tutorial in the docs and didn't find an answer.
• I already checked if it is not related to SQLModel but to Pydantic.
• I already checked if it is not related to SQLModel but to SQLAlchemy.

Commit to Help

• I commit to help with one of those options 👆

Example Code

import uuid
from typing import Optional

import sqlalchemy.exc
from sqlmodel import SQLModel, Field
from sqlmodel import create_engine, Session, select


class Hero(SQLModel, table=True):
 id: Optional[uuid.UUID] = Field(primary_key=True,
 default_factory=uuid.uuid4)
 name: str
 secret_name: str
 age: Optional[int] = None

 def __str__(self):
 return f'{self.name} (AKA {self.secret_name}) [{self.age if self.age is not None else "Unknown"}]'


sqlite_file_name = "database.db"
sqlite_url = f"sqlite:///{sqlite_file_name}"

engine = create_engine(sqlite_url, echo=True) # remove echo for prod


def create_db_and_tables():
 try:
 Hero.__table__.drop(engine)
 except sqlalchemy.exc.OperationalError: # table doesn't exist yet
 pass
 SQLModel.metadata.create_all(engine)


def create_heroes():
 heros = [
 Hero(name="Deadpond", secret_name="Dive Wilson"),
 Hero(name="Spider-Boy", secret_name="Pedro Parqueador"),
 Hero(name="Rusty-Man", secret_name="Tommy Sharp", age=48),
 Hero(name="Tarantula", secret_name="Natalia Roman-on", age=32),
 Hero(name="Black Lion", secret_name="Trevor Challa", age=35),
 Hero(name="Dr. Weird", secret_name="Steve Weird", age=36),
 Hero(name="Captain North America", secret_name="Esteban Rogelios", age=93),
 ]

 with Session(engine) as session:
 for hero in heros:
 session.add(hero)
 session.commit()


def select_heroes():
 with Session(engine) as session:
 for hero in session.exec(select(Hero)):
 print(hero.id)


if __name__ == "__main__":
 while True:
 try:
 create_db_and_tables()
 create_heroes()
 select_heroes()
 except ValueError:
 raise

Cleansed Traceback

Traceback (most recent call last):
 File “/…/sqlmodel_tutorial/minimal_example.py", line 89, in <module>
 select_heroes()
 File “/…/sqlmodel_tutorial/minimal_example.py", line 80, in select_heroes
 for hero in session.exec(select(Hero)):
 File "/.../lib/python3.8/site-packages/sqlalchemy/engine/result.py", line 381, in iterrows
 for row in self._fetchiter_impl():
 File "/.../lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 147, in chunks
 fetch = cursor._raw_all_rows()
 File "/.../lib/python3.8/site-packages/sqlalchemy/engine/result.py", line 392, in _raw_all_rows
 return [make_row(row) for row in rows]
 File "/.../lib/python3.8/site-packages/sqlalchemy/engine/result.py", line 392, in <listcomp>
 return [make_row(row) for row in rows]
 File "/.../lib/python3.8/site-packages/sqlalchemy/sql/type_api.py", line 1430, in process
 return process_value(value, dialect)
 File "/.../lib/python3.8/site-packages/sqlmodel/sql/sqltypes.py", line 61, in process_result_value
 value = uuid.UUID(value)
 File “/…/lib/python3.8/uuid.py", line 171, in __init__
 raise ValueError('badly formed hexadecimal UUID string')
ValueError: badly formed hexadecimal UUID string

Description

Running through the tutorial up until around this point, I tried swapping out the int id key for a UUID key.

This works except whenever the uuid.uuid4 default factory creates a UUID with a hexstring that starts with a zero.

It appears that here in sqltypes.py that, rather than using UUID().hex, sqltypes.py carries on sqlalchemy's integer stringification apprach, however, doesn't include any padding declaration.

IMO there are two options;

1. faithfully fix the replication of SQLAlchemy's UUID ▶️ int ▶️ str approach with the padding fixed (f"{v.int:032x}")
2. switch to using the stdlib UUID.hex implementation for serialisation

Personally I'm a fan of the latter so will spin up a PR

Operating System

macOS

Operating System Details

No response

SQLModel Version

0.0.4

Python Version

3.8.8

Additional Context

No response

[alucarddelta]

I am also hitting this issue with UUIDs being "badly formed".

[dit7ya]

I am also getting this issue. A new release with the PR merged will be very helpful.

[chriswhite199]

For those awaiting a fix for this, here's a method to ensure the generated UUIDs don't start with leading zeros:

def new_uuid() -> uuid.UUID:
 # Note: Work around UUIDs with leading zeros: https://github.com/tiangolo/sqlmodel/issues/25
 # by making sure uuid str does not start with a leading 0
 val = uuid.uuid4()
 while val.hex[0] == '0':
 val = uuid.uuid4()

 return val


class UuidMixin(SQLModel):
 # amend index / primary_key as needed
 id: uuid.UUID = Field(index=False, primary_key=True, default_factory=new_uuid)

[frankie567]

Other solution: installing the version of @andrewbolster's PR:

pip install git+https://github.com/andrewbolster/sqlmodel.git@patch-1

[junekiwi]

Another (temporary) solution, with SQLAlchemy-Utils:

from sqlalchemy_utils import UUIDType
from sqlmodel import Column, Field, SQLModel
from uuid import UUID, uuid4

class MyModel(SQLModel, table=True):
 uuid: UUID = Field(default_factory=uuid4, sa_column=Column(UUIDType(), primary_key=True))