Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions docs/advanced/sa-column.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
# SQLAlchemy Columns

In some cases you may need more control over the columns generated by SQLModel, this can be done by using the `sa_column`, `sa_column_args`, and `sa_column_kwargs` arguments when creating the `Field` object.

There are many use cases for this, but ones where this is particularity useful is when you want more advanced defaults for values than what is easy to implement with Pydantic, such `created_at` or `update_at` timestamps for rows.

## Columns for Timestamps

Two ways of implementing `created_at` timestamps with Pydantic are [default factories](https://pydantic-docs.helpmanual.io/usage/models/#field-with-dynamic-default-value) and [validators](https://pydantic-docs.helpmanual.io/usage/validators/#validate-always), however there's no straightforward way to have an `update_at` timestamp.
Comment thread
RobertRosca marked this conversation as resolved.

The SQLAlchemy docs describe how `created_at` timestamps can be automatically set with either [default](https://docs.sqlalchemy.org/en/14/core/defaults.html#python-executed-functions) or [server-default](https://docs.sqlalchemy.org/en/14/core/defaults.html#server-invoked-ddl-explicit-default-expressions) functions, by using `sa_column=Column(...)` as described in the SQLAlchemy documentation we can achieve the same behaviour:

{* ./docs_src/advanced/sa_column/tutorial001.py ln[9:21] hl[16,20] *}

Above we are saying that the `registered_at` column should have a `server_default` value of `func.now()` (see full code for imports), which means that if there is no provided value then the current time will be the recorded value for that row.

As there is a value there now, then it will not be changed automatically in the future.

The `updated_at` column has an `onupdate` value of `func.now()`, this means that each time an `UPDATE` is performed, the function will be executed, meaning that the timestamp changes whenever a change is made to the row.

/// warning

The difference between client-side python functions, server-side ddl expressions, and server-side implicit defaults is important in some situations but too in-depth to go into here. Check the SQL and SQLAlchemy docs for more information.

///

### Pydantic Implementation

Implementing these timestamps on the DB side with SQLAlchemy works very well as the database itself is what will create and update the fields whenever a relevant database interaction occurs.

It's possible to achieve similar behaviour with Pydantic, for the `created_at` timestamp by using a Pydantic `Field` with a `default_factory`:

```python
from datetime import datetime

from pydantic import BaseModel, Field


class Model(BaseModel):
created_at: datetime = Field(default_factory=datetime.utcnow)


m1 = Model()
m2 = Model()
print(f'{m1.created_at} != {m2.created_at}')
#> 2022-05-19 10:49:22.053624 != 2022-05-19 10:49:22.053641
```

Another approach is to use a Pydantic `validator`:

```python
from datetime import datetime

from pydantic import BaseModel, validator

class Model(BaseModel):
created_at: datetime = None

@validator('ts', pre=True, always=True)
def set_created_at_now(cls, v):
return v or datetime.now()
```
Comment on lines +49 to +62

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would remove this code example with validator


Both of these approaches come with the major caveat that default fields are set during the **Pydantic model instantiation**, as opposed to during **interactions with the database**, instead of the SQLModel approach which sets it with `server_default` which means that the timestamp will be exactly when the row is created in the database.

The real issue starts when looking at the `updated_at` timestamp - SQLAlchemy has the `onupdate` default which runs a function when the row is updated in the database, but there is no easy way to do this in Pydantic as it has no concept of 'about to be saved'.

So the pure Pydantic approach would require some additional logic to always change the `updated_at` timestamp before doing a write to the database, which adds some more complexity to the code and does not have benefits over the SQLAlchemy approach.
Empty file.
74 changes: 74 additions & 0 deletions docs_src/advanced/sa_column/tutorial001.py
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
from datetime import datetime
from time import sleep
from typing import Optional

from sqlalchemy import Column, DateTime, func
from sqlmodel import Field, Session, SQLModel, create_engine, select


class Hero(SQLModel, table=True):
id: Optional[int] = Field(default=None, primary_key=True)
name: str
secret_name: str
age: Optional[int] = None

registered_at: datetime = Field(
sa_column=Column(DateTime(timezone=False), server_default=func.now())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add nullable=False?

)

updated_at: Optional[datetime] = Field(
sa_column=Column(DateTime(timezone=False), onupdate=func.now())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add server_default=func.now() and nullable=False?

)


sqlite_file_name = "database.db"
sqlite_url = f"sqlite:///{sqlite_file_name}"

engine = create_engine(sqlite_url, echo=True)


def create_db_and_tables():
SQLModel.metadata.create_all(engine)


def create_heroes():
hero_1 = Hero(name="Deadpond", secret_name="Dive Wilson")
hero_2 = Hero(name="Spider-Boy", secret_name="Pedro Parqueador")
hero_3 = Hero(name="Rusty-Man", secret_name="Tommy Sharp", age=48)
Comment on lines +36 to +37

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only update hero1, I would remove hero2 and hero3 to simplify the example


session = Session(engine)

session.add(hero_1)
session.add(hero_2)
session.add(hero_3)

session.commit()

session.close()


def update_hero_age(new_secret_name):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
defupdate_hero_age(new_secret_name):
defupdate_hero_secret_name(new_secret_name):

with Session(engine) as session:
statement = select(Hero).where(Hero.name == "Spider-Boy")
results = session.exec(statement)
hero = results.one()
print("Hero:", hero)

hero.secret_name = new_secret_name
session.add(hero)
session.commit()
session.refresh(hero)
print("Updated hero:", hero)


def main():
create_db_and_tables()
create_heroes()
sleep(1)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to find way to avoid sleeping as it will make test run slower

update_hero_age("Arachnid-Lad")
sleep(1)
update_hero_age("The Wallclimber")


if __name__ == "__main__":
main()
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line numberDiff line numberDiff line change
Expand Up@@ -128,6 +128,7 @@ nav:
- advanced/index.md
- advanced/decimal.md
- advanced/uuid.md
- advanced/sa-column.md
- Resources:
- resources/index.md
- help.md
Expand Down
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions docs/advanced/sa-column.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
# SQLAlchemy Columns

In some cases you may need more control over the columns generated by SQLModel, this can be done by using the `sa_column`, `sa_column_args`, and `sa_column_kwargs` arguments when creating the `Field` object.

There are many use cases for this, but ones where this is particularity useful is when you want more advanced defaults for values than what is easy to implement with Pydantic, such `created_at` or `update_at` timestamps for rows.

## Columns for Timestamps

Two ways of implementing `created_at` timestamps with Pydantic are [default factories](https://pydantic-docs.helpmanual.io/usage/models/#field-with-dynamic-default-value) and [validators](https://pydantic-docs.helpmanual.io/usage/validators/#validate-always), however there's no straightforward way to have an `update_at` timestamp.
Comment thread
RobertRosca marked this conversation as resolved.

The SQLAlchemy docs describe how `created_at` timestamps can be automatically set with either [default](https://docs.sqlalchemy.org/en/14/core/defaults.html#python-executed-functions) or [server-default](https://docs.sqlalchemy.org/en/14/core/defaults.html#server-invoked-ddl-explicit-default-expressions) functions, by using `sa_column=Column(...)` as described in the SQLAlchemy documentation we can achieve the same behaviour:

{* ./docs_src/advanced/sa_column/tutorial001.py ln[9:21] hl[16,20] *}

Above we are saying that the `registered_at` column should have a `server_default` value of `func.now()` (see full code for imports), which means that if there is no provided value then the current time will be the recorded value for that row.

As there is a value there now, then it will not be changed automatically in the future.

The `updated_at` column has an `onupdate` value of `func.now()`, this means that each time an `UPDATE` is performed, the function will be executed, meaning that the timestamp changes whenever a change is made to the row.

/// warning

The difference between client-side python functions, server-side ddl expressions, and server-side implicit defaults is important in some situations but too in-depth to go into here. Check the SQL and SQLAlchemy docs for more information.

///

### Pydantic Implementation

Implementing these timestamps on the DB side with SQLAlchemy works very well as the database itself is what will create and update the fields whenever a relevant database interaction occurs.

It's possible to achieve similar behaviour with Pydantic, for the `created_at` timestamp by using a Pydantic `Field` with a `default_factory`:

```python
from datetime import datetime

from pydantic import BaseModel, Field


class Model(BaseModel):
created_at: datetime = Field(default_factory=datetime.utcnow)


m1 = Model()
m2 = Model()
print(f'{m1.created_at} != {m2.created_at}')
#> 2022-05-19 10:49:22.053624 != 2022-05-19 10:49:22.053641
```

Another approach is to use a Pydantic `validator`:

```python
from datetime import datetime

from pydantic import BaseModel, validator

class Model(BaseModel):
created_at: datetime = None

@validator('ts', pre=True, always=True)
def set_created_at_now(cls, v):
return v or datetime.now()
```
Comment on lines +49 to +62

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would remove this code example with validator


Both of these approaches come with the major caveat that default fields are set during the **Pydantic model instantiation**, as opposed to during **interactions with the database**, instead of the SQLModel approach which sets it with `server_default` which means that the timestamp will be exactly when the row is created in the database.

The real issue starts when looking at the `updated_at` timestamp - SQLAlchemy has the `onupdate` default which runs a function when the row is updated in the database, but there is no easy way to do this in Pydantic as it has no concept of 'about to be saved'.

So the pure Pydantic approach would require some additional logic to always change the `updated_at` timestamp before doing a write to the database, which adds some more complexity to the code and does not have benefits over the SQLAlchemy approach.
Empty file.
74 changes: 74 additions & 0 deletions docs_src/advanced/sa_column/tutorial001.py
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
from datetime import datetime
from time import sleep
from typing import Optional

from sqlalchemy import Column, DateTime, func
from sqlmodel import Field, Session, SQLModel, create_engine, select


class Hero(SQLModel, table=True):
id: Optional[int] = Field(default=None, primary_key=True)
name: str
secret_name: str
age: Optional[int] = None

registered_at: datetime = Field(
sa_column=Column(DateTime(timezone=False), server_default=func.now())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add nullable=False?

)

updated_at: Optional[datetime] = Field(
sa_column=Column(DateTime(timezone=False), onupdate=func.now())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add server_default=func.now() and nullable=False?

)


sqlite_file_name = "database.db"
sqlite_url = f"sqlite:///{sqlite_file_name}"

engine = create_engine(sqlite_url, echo=True)


def create_db_and_tables():
SQLModel.metadata.create_all(engine)


def create_heroes():
hero_1 = Hero(name="Deadpond", secret_name="Dive Wilson")
hero_2 = Hero(name="Spider-Boy", secret_name="Pedro Parqueador")
hero_3 = Hero(name="Rusty-Man", secret_name="Tommy Sharp", age=48)
Comment on lines +36 to +37

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only update hero1, I would remove hero2 and hero3 to simplify the example


session = Session(engine)

session.add(hero_1)
session.add(hero_2)
session.add(hero_3)

session.commit()

session.close()


def update_hero_age(new_secret_name):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
defupdate_hero_age(new_secret_name):
defupdate_hero_secret_name(new_secret_name):

with Session(engine) as session:
statement = select(Hero).where(Hero.name == "Spider-Boy")
results = session.exec(statement)
hero = results.one()
print("Hero:", hero)

hero.secret_name = new_secret_name
session.add(hero)
session.commit()
session.refresh(hero)
print("Updated hero:", hero)


def main():
create_db_and_tables()
create_heroes()
sleep(1)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to find way to avoid sleeping as it will make test run slower

update_hero_age("Arachnid-Lad")
sleep(1)
update_hero_age("The Wallclimber")


if __name__ == "__main__":
main()
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line numberDiff line numberDiff line change
Expand Up@@ -128,6 +128,7 @@ nav:
- advanced/index.md
- advanced/decimal.md
- advanced/uuid.md
- advanced/sa-column.md
- Resources:
- resources/index.md
- help.md
Expand Down
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions docs/advanced/sa-column.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
# SQLAlchemy Columns

In some cases you may need more control over the columns generated by SQLModel, this can be done by using the `sa_column`, `sa_column_args`, and `sa_column_kwargs` arguments when creating the `Field` object.

There are many use cases for this, but ones where this is particularity useful is when you want more advanced defaults for values than what is easy to implement with Pydantic, such `created_at` or `update_at` timestamps for rows.

## Columns for Timestamps

Two ways of implementing `created_at` timestamps with Pydantic are [default factories](https://pydantic-docs.helpmanual.io/usage/models/#field-with-dynamic-default-value) and [validators](https://pydantic-docs.helpmanual.io/usage/validators/#validate-always), however there's no straightforward way to have an `update_at` timestamp.
Comment thread
RobertRosca marked this conversation as resolved.

The SQLAlchemy docs describe how `created_at` timestamps can be automatically set with either [default](https://docs.sqlalchemy.org/en/14/core/defaults.html#python-executed-functions) or [server-default](https://docs.sqlalchemy.org/en/14/core/defaults.html#server-invoked-ddl-explicit-default-expressions) functions, by using `sa_column=Column(...)` as described in the SQLAlchemy documentation we can achieve the same behaviour:

{* ./docs_src/advanced/sa_column/tutorial001.py ln[9:21] hl[16,20] *}

Above we are saying that the `registered_at` column should have a `server_default` value of `func.now()` (see full code for imports), which means that if there is no provided value then the current time will be the recorded value for that row.

As there is a value there now, then it will not be changed automatically in the future.

The `updated_at` column has an `onupdate` value of `func.now()`, this means that each time an `UPDATE` is performed, the function will be executed, meaning that the timestamp changes whenever a change is made to the row.

/// warning

The difference between client-side python functions, server-side ddl expressions, and server-side implicit defaults is important in some situations but too in-depth to go into here. Check the SQL and SQLAlchemy docs for more information.

///

### Pydantic Implementation

Implementing these timestamps on the DB side with SQLAlchemy works very well as the database itself is what will create and update the fields whenever a relevant database interaction occurs.

It's possible to achieve similar behaviour with Pydantic, for the `created_at` timestamp by using a Pydantic `Field` with a `default_factory`:

```python
from datetime import datetime

from pydantic import BaseModel, Field


class Model(BaseModel):
created_at: datetime = Field(default_factory=datetime.utcnow)


m1 = Model()
m2 = Model()
print(f'{m1.created_at} != {m2.created_at}')
#> 2022-05-19 10:49:22.053624 != 2022-05-19 10:49:22.053641
```

Another approach is to use a Pydantic `validator`:

```python
from datetime import datetime

from pydantic import BaseModel, validator

class Model(BaseModel):
created_at: datetime = None

@validator('ts', pre=True, always=True)
def set_created_at_now(cls, v):
return v or datetime.now()
```
Comment on lines +49 to +62

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would remove this code example with validator


Both of these approaches come with the major caveat that default fields are set during the **Pydantic model instantiation**, as opposed to during **interactions with the database**, instead of the SQLModel approach which sets it with `server_default` which means that the timestamp will be exactly when the row is created in the database.

The real issue starts when looking at the `updated_at` timestamp - SQLAlchemy has the `onupdate` default which runs a function when the row is updated in the database, but there is no easy way to do this in Pydantic as it has no concept of 'about to be saved'.

So the pure Pydantic approach would require some additional logic to always change the `updated_at` timestamp before doing a write to the database, which adds some more complexity to the code and does not have benefits over the SQLAlchemy approach.
Empty file.
74 changes: 74 additions & 0 deletions docs_src/advanced/sa_column/tutorial001.py
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
from datetime import datetime
from time import sleep
from typing import Optional

from sqlalchemy import Column, DateTime, func
from sqlmodel import Field, Session, SQLModel, create_engine, select


class Hero(SQLModel, table=True):
id: Optional[int] = Field(default=None, primary_key=True)
name: str
secret_name: str
age: Optional[int] = None

registered_at: datetime = Field(
sa_column=Column(DateTime(timezone=False), server_default=func.now())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add nullable=False?

)

updated_at: Optional[datetime] = Field(
sa_column=Column(DateTime(timezone=False), onupdate=func.now())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add server_default=func.now() and nullable=False?

)


sqlite_file_name = "database.db"
sqlite_url = f"sqlite:///{sqlite_file_name}"

engine = create_engine(sqlite_url, echo=True)


def create_db_and_tables():
SQLModel.metadata.create_all(engine)


def create_heroes():
hero_1 = Hero(name="Deadpond", secret_name="Dive Wilson")
hero_2 = Hero(name="Spider-Boy", secret_name="Pedro Parqueador")
hero_3 = Hero(name="Rusty-Man", secret_name="Tommy Sharp", age=48)
Comment on lines +36 to +37

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only update hero1, I would remove hero2 and hero3 to simplify the example


session = Session(engine)

session.add(hero_1)
session.add(hero_2)
session.add(hero_3)

session.commit()

session.close()


def update_hero_age(new_secret_name):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
defupdate_hero_age(new_secret_name):
defupdate_hero_secret_name(new_secret_name):

with Session(engine) as session:
statement = select(Hero).where(Hero.name == "Spider-Boy")
results = session.exec(statement)
hero = results.one()
print("Hero:", hero)

hero.secret_name = new_secret_name
session.add(hero)
session.commit()
session.refresh(hero)
print("Updated hero:", hero)


def main():
create_db_and_tables()
create_heroes()
sleep(1)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to find way to avoid sleeping as it will make test run slower

update_hero_age("Arachnid-Lad")
sleep(1)
update_hero_age("The Wallclimber")


if __name__ == "__main__":
main()
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line numberDiff line numberDiff line change
Expand Up@@ -128,6 +128,7 @@ nav:
- advanced/index.md
- advanced/decimal.md
- advanced/uuid.md
- advanced/sa-column.md
- Resources:
- resources/index.md
- help.md
Expand Down
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions docs/advanced/sa-column.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
# SQLAlchemy Columns

In some cases you may need more control over the columns generated by SQLModel, this can be done by using the `sa_column`, `sa_column_args`, and `sa_column_kwargs` arguments when creating the `Field` object.

There are many use cases for this, but ones where this is particularity useful is when you want more advanced defaults for values than what is easy to implement with Pydantic, such `created_at` or `update_at` timestamps for rows.

## Columns for Timestamps

Two ways of implementing `created_at` timestamps with Pydantic are [default factories](https://pydantic-docs.helpmanual.io/usage/models/#field-with-dynamic-default-value) and [validators](https://pydantic-docs.helpmanual.io/usage/validators/#validate-always), however there's no straightforward way to have an `update_at` timestamp.
Comment thread
RobertRosca marked this conversation as resolved.

The SQLAlchemy docs describe how `created_at` timestamps can be automatically set with either [default](https://docs.sqlalchemy.org/en/14/core/defaults.html#python-executed-functions) or [server-default](https://docs.sqlalchemy.org/en/14/core/defaults.html#server-invoked-ddl-explicit-default-expressions) functions, by using `sa_column=Column(...)` as described in the SQLAlchemy documentation we can achieve the same behaviour:

{* ./docs_src/advanced/sa_column/tutorial001.py ln[9:21] hl[16,20] *}

Above we are saying that the `registered_at` column should have a `server_default` value of `func.now()` (see full code for imports), which means that if there is no provided value then the current time will be the recorded value for that row.

As there is a value there now, then it will not be changed automatically in the future.

The `updated_at` column has an `onupdate` value of `func.now()`, this means that each time an `UPDATE` is performed, the function will be executed, meaning that the timestamp changes whenever a change is made to the row.

/// warning

The difference between client-side python functions, server-side ddl expressions, and server-side implicit defaults is important in some situations but too in-depth to go into here. Check the SQL and SQLAlchemy docs for more information.

///

### Pydantic Implementation

Implementing these timestamps on the DB side with SQLAlchemy works very well as the database itself is what will create and update the fields whenever a relevant database interaction occurs.

It's possible to achieve similar behaviour with Pydantic, for the `created_at` timestamp by using a Pydantic `Field` with a `default_factory`:

```python
from datetime import datetime

from pydantic import BaseModel, Field


class Model(BaseModel):
created_at: datetime = Field(default_factory=datetime.utcnow)


m1 = Model()
m2 = Model()
print(f'{m1.created_at} != {m2.created_at}')
#> 2022-05-19 10:49:22.053624 != 2022-05-19 10:49:22.053641
```

Another approach is to use a Pydantic `validator`:

```python
from datetime import datetime

from pydantic import BaseModel, validator

class Model(BaseModel):
created_at: datetime = None

@validator('ts', pre=True, always=True)
def set_created_at_now(cls, v):
return v or datetime.now()
```
Comment on lines +49 to +62

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would remove this code example with validator


Both of these approaches come with the major caveat that default fields are set during the **Pydantic model instantiation**, as opposed to during **interactions with the database**, instead of the SQLModel approach which sets it with `server_default` which means that the timestamp will be exactly when the row is created in the database.

The real issue starts when looking at the `updated_at` timestamp - SQLAlchemy has the `onupdate` default which runs a function when the row is updated in the database, but there is no easy way to do this in Pydantic as it has no concept of 'about to be saved'.

So the pure Pydantic approach would require some additional logic to always change the `updated_at` timestamp before doing a write to the database, which adds some more complexity to the code and does not have benefits over the SQLAlchemy approach.
Empty file.
74 changes: 74 additions & 0 deletions docs_src/advanced/sa_column/tutorial001.py
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
from datetime import datetime
from time import sleep
from typing import Optional

from sqlalchemy import Column, DateTime, func
from sqlmodel import Field, Session, SQLModel, create_engine, select


class Hero(SQLModel, table=True):
id: Optional[int] = Field(default=None, primary_key=True)
name: str
secret_name: str
age: Optional[int] = None

registered_at: datetime = Field(
sa_column=Column(DateTime(timezone=False), server_default=func.now())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add nullable=False?

)

updated_at: Optional[datetime] = Field(
sa_column=Column(DateTime(timezone=False), onupdate=func.now())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add server_default=func.now() and nullable=False?

)


sqlite_file_name = "database.db"
sqlite_url = f"sqlite:///{sqlite_file_name}"

engine = create_engine(sqlite_url, echo=True)


def create_db_and_tables():
SQLModel.metadata.create_all(engine)


def create_heroes():
hero_1 = Hero(name="Deadpond", secret_name="Dive Wilson")
hero_2 = Hero(name="Spider-Boy", secret_name="Pedro Parqueador")
hero_3 = Hero(name="Rusty-Man", secret_name="Tommy Sharp", age=48)
Comment on lines +36 to +37

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only update hero1, I would remove hero2 and hero3 to simplify the example


session = Session(engine)

session.add(hero_1)
session.add(hero_2)
session.add(hero_3)

session.commit()

session.close()


def update_hero_age(new_secret_name):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
defupdate_hero_age(new_secret_name):
defupdate_hero_secret_name(new_secret_name):

with Session(engine) as session:
statement = select(Hero).where(Hero.name == "Spider-Boy")
results = session.exec(statement)
hero = results.one()
print("Hero:", hero)

hero.secret_name = new_secret_name
session.add(hero)
session.commit()
session.refresh(hero)
print("Updated hero:", hero)


def main():
create_db_and_tables()
create_heroes()
sleep(1)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to find way to avoid sleeping as it will make test run slower

update_hero_age("Arachnid-Lad")
sleep(1)
update_hero_age("The Wallclimber")


if __name__ == "__main__":
main()
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line numberDiff line numberDiff line change
Expand Up@@ -128,6 +128,7 @@ nav:
- advanced/index.md
- advanced/decimal.md
- advanced/uuid.md
- advanced/sa-column.md
- Resources:
- resources/index.md
- help.md
Expand Down
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions docs/advanced/sa-column.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
# SQLAlchemy Columns

In some cases you may need more control over the columns generated by SQLModel, this can be done by using the `sa_column`, `sa_column_args`, and `sa_column_kwargs` arguments when creating the `Field` object.

There are many use cases for this, but ones where this is particularity useful is when you want more advanced defaults for values than what is easy to implement with Pydantic, such `created_at` or `update_at` timestamps for rows.

## Columns for Timestamps

Two ways of implementing `created_at` timestamps with Pydantic are [default factories](https://pydantic-docs.helpmanual.io/usage/models/#field-with-dynamic-default-value) and [validators](https://pydantic-docs.helpmanual.io/usage/validators/#validate-always), however there's no straightforward way to have an `update_at` timestamp.
Comment thread
RobertRosca marked this conversation as resolved.

The SQLAlchemy docs describe how `created_at` timestamps can be automatically set with either [default](https://docs.sqlalchemy.org/en/14/core/defaults.html#python-executed-functions) or [server-default](https://docs.sqlalchemy.org/en/14/core/defaults.html#server-invoked-ddl-explicit-default-expressions) functions, by using `sa_column=Column(...)` as described in the SQLAlchemy documentation we can achieve the same behaviour:

{* ./docs_src/advanced/sa_column/tutorial001.py ln[9:21] hl[16,20] *}

Above we are saying that the `registered_at` column should have a `server_default` value of `func.now()` (see full code for imports), which means that if there is no provided value then the current time will be the recorded value for that row.

As there is a value there now, then it will not be changed automatically in the future.

The `updated_at` column has an `onupdate` value of `func.now()`, this means that each time an `UPDATE` is performed, the function will be executed, meaning that the timestamp changes whenever a change is made to the row.

/// warning

The difference between client-side python functions, server-side ddl expressions, and server-side implicit defaults is important in some situations but too in-depth to go into here. Check the SQL and SQLAlchemy docs for more information.

///

### Pydantic Implementation

Implementing these timestamps on the DB side with SQLAlchemy works very well as the database itself is what will create and update the fields whenever a relevant database interaction occurs.

It's possible to achieve similar behaviour with Pydantic, for the `created_at` timestamp by using a Pydantic `Field` with a `default_factory`:

```python
from datetime import datetime

from pydantic import BaseModel, Field


class Model(BaseModel):
created_at: datetime = Field(default_factory=datetime.utcnow)


m1 = Model()
m2 = Model()
print(f'{m1.created_at} != {m2.created_at}')
#> 2022-05-19 10:49:22.053624 != 2022-05-19 10:49:22.053641
```

Another approach is to use a Pydantic `validator`:

```python
from datetime import datetime

from pydantic import BaseModel, validator

class Model(BaseModel):
created_at: datetime = None

@validator('ts', pre=True, always=True)
def set_created_at_now(cls, v):
return v or datetime.now()
```
Comment on lines +49 to +62

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would remove this code example with validator


Both of these approaches come with the major caveat that default fields are set during the **Pydantic model instantiation**, as opposed to during **interactions with the database**, instead of the SQLModel approach which sets it with `server_default` which means that the timestamp will be exactly when the row is created in the database.

The real issue starts when looking at the `updated_at` timestamp - SQLAlchemy has the `onupdate` default which runs a function when the row is updated in the database, but there is no easy way to do this in Pydantic as it has no concept of 'about to be saved'.

So the pure Pydantic approach would require some additional logic to always change the `updated_at` timestamp before doing a write to the database, which adds some more complexity to the code and does not have benefits over the SQLAlchemy approach.
Empty file.
74 changes: 74 additions & 0 deletions docs_src/advanced/sa_column/tutorial001.py
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
from datetime import datetime
from time import sleep
from typing import Optional

from sqlalchemy import Column, DateTime, func
from sqlmodel import Field, Session, SQLModel, create_engine, select


class Hero(SQLModel, table=True):
id: Optional[int] = Field(default=None, primary_key=True)
name: str
secret_name: str
age: Optional[int] = None

registered_at: datetime = Field(
sa_column=Column(DateTime(timezone=False), server_default=func.now())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add nullable=False?

)

updated_at: Optional[datetime] = Field(
sa_column=Column(DateTime(timezone=False), onupdate=func.now())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add server_default=func.now() and nullable=False?

)


sqlite_file_name = "database.db"
sqlite_url = f"sqlite:///{sqlite_file_name}"

engine = create_engine(sqlite_url, echo=True)


def create_db_and_tables():
SQLModel.metadata.create_all(engine)


def create_heroes():
hero_1 = Hero(name="Deadpond", secret_name="Dive Wilson")
hero_2 = Hero(name="Spider-Boy", secret_name="Pedro Parqueador")
hero_3 = Hero(name="Rusty-Man", secret_name="Tommy Sharp", age=48)
Comment on lines +36 to +37

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only update hero1, I would remove hero2 and hero3 to simplify the example


session = Session(engine)

session.add(hero_1)
session.add(hero_2)
session.add(hero_3)

session.commit()

session.close()


def update_hero_age(new_secret_name):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
defupdate_hero_age(new_secret_name):
defupdate_hero_secret_name(new_secret_name):

with Session(engine) as session:
statement = select(Hero).where(Hero.name == "Spider-Boy")
results = session.exec(statement)
hero = results.one()
print("Hero:", hero)

hero.secret_name = new_secret_name
session.add(hero)
session.commit()
session.refresh(hero)
print("Updated hero:", hero)


def main():
create_db_and_tables()
create_heroes()
sleep(1)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to find way to avoid sleeping as it will make test run slower

update_hero_age("Arachnid-Lad")
sleep(1)
update_hero_age("The Wallclimber")


if __name__ == "__main__":
main()
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line numberDiff line numberDiff line change
Expand Up@@ -128,6 +128,7 @@ nav:
- advanced/index.md
- advanced/decimal.md
- advanced/uuid.md
- advanced/sa-column.md
- Resources:
- resources/index.md
- help.md
Expand Down
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions docs/advanced/sa-column.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
# SQLAlchemy Columns

In some cases you may need more control over the columns generated by SQLModel, this can be done by using the `sa_column`, `sa_column_args`, and `sa_column_kwargs` arguments when creating the `Field` object.

There are many use cases for this, but ones where this is particularity useful is when you want more advanced defaults for values than what is easy to implement with Pydantic, such `created_at` or `update_at` timestamps for rows.

## Columns for Timestamps

Two ways of implementing `created_at` timestamps with Pydantic are [default factories](https://pydantic-docs.helpmanual.io/usage/models/#field-with-dynamic-default-value) and [validators](https://pydantic-docs.helpmanual.io/usage/validators/#validate-always), however there's no straightforward way to have an `update_at` timestamp.
Comment thread
RobertRosca marked this conversation as resolved.

The SQLAlchemy docs describe how `created_at` timestamps can be automatically set with either [default](https://docs.sqlalchemy.org/en/14/core/defaults.html#python-executed-functions) or [server-default](https://docs.sqlalchemy.org/en/14/core/defaults.html#server-invoked-ddl-explicit-default-expressions) functions, by using `sa_column=Column(...)` as described in the SQLAlchemy documentation we can achieve the same behaviour:

{* ./docs_src/advanced/sa_column/tutorial001.py ln[9:21] hl[16,20] *}

Above we are saying that the `registered_at` column should have a `server_default` value of `func.now()` (see full code for imports), which means that if there is no provided value then the current time will be the recorded value for that row.

As there is a value there now, then it will not be changed automatically in the future.

The `updated_at` column has an `onupdate` value of `func.now()`, this means that each time an `UPDATE` is performed, the function will be executed, meaning that the timestamp changes whenever a change is made to the row.

/// warning

The difference between client-side python functions, server-side ddl expressions, and server-side implicit defaults is important in some situations but too in-depth to go into here. Check the SQL and SQLAlchemy docs for more information.

///

### Pydantic Implementation

Implementing these timestamps on the DB side with SQLAlchemy works very well as the database itself is what will create and update the fields whenever a relevant database interaction occurs.

It's possible to achieve similar behaviour with Pydantic, for the `created_at` timestamp by using a Pydantic `Field` with a `default_factory`:

```python
from datetime import datetime

from pydantic import BaseModel, Field


class Model(BaseModel):
created_at: datetime = Field(default_factory=datetime.utcnow)


m1 = Model()
m2 = Model()
print(f'{m1.created_at} != {m2.created_at}')
#> 2022-05-19 10:49:22.053624 != 2022-05-19 10:49:22.053641
```

Another approach is to use a Pydantic `validator`:

```python
from datetime import datetime

from pydantic import BaseModel, validator

class Model(BaseModel):
created_at: datetime = None

@validator('ts', pre=True, always=True)
def set_created_at_now(cls, v):
return v or datetime.now()
```
Comment on lines +49 to +62

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would remove this code example with validator


Both of these approaches come with the major caveat that default fields are set during the **Pydantic model instantiation**, as opposed to during **interactions with the database**, instead of the SQLModel approach which sets it with `server_default` which means that the timestamp will be exactly when the row is created in the database.

The real issue starts when looking at the `updated_at` timestamp - SQLAlchemy has the `onupdate` default which runs a function when the row is updated in the database, but there is no easy way to do this in Pydantic as it has no concept of 'about to be saved'.

So the pure Pydantic approach would require some additional logic to always change the `updated_at` timestamp before doing a write to the database, which adds some more complexity to the code and does not have benefits over the SQLAlchemy approach.
Empty file.
74 changes: 74 additions & 0 deletions docs_src/advanced/sa_column/tutorial001.py
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
from datetime import datetime
from time import sleep
from typing import Optional

from sqlalchemy import Column, DateTime, func
from sqlmodel import Field, Session, SQLModel, create_engine, select


class Hero(SQLModel, table=True):
id: Optional[int] = Field(default=None, primary_key=True)
name: str
secret_name: str
age: Optional[int] = None

registered_at: datetime = Field(
sa_column=Column(DateTime(timezone=False), server_default=func.now())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add nullable=False?

)

updated_at: Optional[datetime] = Field(
sa_column=Column(DateTime(timezone=False), onupdate=func.now())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add server_default=func.now() and nullable=False?

)


sqlite_file_name = "database.db"
sqlite_url = f"sqlite:///{sqlite_file_name}"

engine = create_engine(sqlite_url, echo=True)


def create_db_and_tables():
SQLModel.metadata.create_all(engine)


def create_heroes():
hero_1 = Hero(name="Deadpond", secret_name="Dive Wilson")
hero_2 = Hero(name="Spider-Boy", secret_name="Pedro Parqueador")
hero_3 = Hero(name="Rusty-Man", secret_name="Tommy Sharp", age=48)
Comment on lines +36 to +37

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only update hero1, I would remove hero2 and hero3 to simplify the example


session = Session(engine)

session.add(hero_1)
session.add(hero_2)
session.add(hero_3)

session.commit()

session.close()


def update_hero_age(new_secret_name):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
defupdate_hero_age(new_secret_name):
defupdate_hero_secret_name(new_secret_name):

with Session(engine) as session:
statement = select(Hero).where(Hero.name == "Spider-Boy")
results = session.exec(statement)
hero = results.one()
print("Hero:", hero)

hero.secret_name = new_secret_name
session.add(hero)
session.commit()
session.refresh(hero)
print("Updated hero:", hero)


def main():
create_db_and_tables()
create_heroes()
sleep(1)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to find way to avoid sleeping as it will make test run slower

update_hero_age("Arachnid-Lad")
sleep(1)
update_hero_age("The Wallclimber")


if __name__ == "__main__":
main()
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line numberDiff line numberDiff line change
Expand Up@@ -128,6 +128,7 @@ nav:
- advanced/index.md
- advanced/decimal.md
- advanced/uuid.md
- advanced/sa-column.md
- Resources:
- resources/index.md
- help.md
Expand Down
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions docs/advanced/sa-column.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
# SQLAlchemy Columns

In some cases you may need more control over the columns generated by SQLModel, this can be done by using the `sa_column`, `sa_column_args`, and `sa_column_kwargs` arguments when creating the `Field` object.

There are many use cases for this, but ones where this is particularity useful is when you want more advanced defaults for values than what is easy to implement with Pydantic, such `created_at` or `update_at` timestamps for rows.

## Columns for Timestamps

Two ways of implementing `created_at` timestamps with Pydantic are [default factories](https://pydantic-docs.helpmanual.io/usage/models/#field-with-dynamic-default-value) and [validators](https://pydantic-docs.helpmanual.io/usage/validators/#validate-always), however there's no straightforward way to have an `update_at` timestamp.
Comment thread
RobertRosca marked this conversation as resolved.

The SQLAlchemy docs describe how `created_at` timestamps can be automatically set with either [default](https://docs.sqlalchemy.org/en/14/core/defaults.html#python-executed-functions) or [server-default](https://docs.sqlalchemy.org/en/14/core/defaults.html#server-invoked-ddl-explicit-default-expressions) functions, by using `sa_column=Column(...)` as described in the SQLAlchemy documentation we can achieve the same behaviour:

{* ./docs_src/advanced/sa_column/tutorial001.py ln[9:21] hl[16,20] *}

Above we are saying that the `registered_at` column should have a `server_default` value of `func.now()` (see full code for imports), which means that if there is no provided value then the current time will be the recorded value for that row.

As there is a value there now, then it will not be changed automatically in the future.

The `updated_at` column has an `onupdate` value of `func.now()`, this means that each time an `UPDATE` is performed, the function will be executed, meaning that the timestamp changes whenever a change is made to the row.

/// warning

The difference between client-side python functions, server-side ddl expressions, and server-side implicit defaults is important in some situations but too in-depth to go into here. Check the SQL and SQLAlchemy docs for more information.

///

### Pydantic Implementation

Implementing these timestamps on the DB side with SQLAlchemy works very well as the database itself is what will create and update the fields whenever a relevant database interaction occurs.

It's possible to achieve similar behaviour with Pydantic, for the `created_at` timestamp by using a Pydantic `Field` with a `default_factory`:

```python
from datetime import datetime

from pydantic import BaseModel, Field


class Model(BaseModel):
created_at: datetime = Field(default_factory=datetime.utcnow)


m1 = Model()
m2 = Model()
print(f'{m1.created_at} != {m2.created_at}')
#> 2022-05-19 10:49:22.053624 != 2022-05-19 10:49:22.053641
```

Another approach is to use a Pydantic `validator`:

```python
from datetime import datetime

from pydantic import BaseModel, validator

class Model(BaseModel):
created_at: datetime = None

@validator('ts', pre=True, always=True)
def set_created_at_now(cls, v):
return v or datetime.now()
```
Comment on lines +49 to +62

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would remove this code example with validator


Both of these approaches come with the major caveat that default fields are set during the **Pydantic model instantiation**, as opposed to during **interactions with the database**, instead of the SQLModel approach which sets it with `server_default` which means that the timestamp will be exactly when the row is created in the database.

The real issue starts when looking at the `updated_at` timestamp - SQLAlchemy has the `onupdate` default which runs a function when the row is updated in the database, but there is no easy way to do this in Pydantic as it has no concept of 'about to be saved'.

So the pure Pydantic approach would require some additional logic to always change the `updated_at` timestamp before doing a write to the database, which adds some more complexity to the code and does not have benefits over the SQLAlchemy approach.
Empty file.
74 changes: 74 additions & 0 deletions docs_src/advanced/sa_column/tutorial001.py
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
from datetime import datetime
from time import sleep
from typing import Optional

from sqlalchemy import Column, DateTime, func
from sqlmodel import Field, Session, SQLModel, create_engine, select


class Hero(SQLModel, table=True):
id: Optional[int] = Field(default=None, primary_key=True)
name: str
secret_name: str
age: Optional[int] = None

registered_at: datetime = Field(
sa_column=Column(DateTime(timezone=False), server_default=func.now())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add nullable=False?

)

updated_at: Optional[datetime] = Field(
sa_column=Column(DateTime(timezone=False), onupdate=func.now())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add server_default=func.now() and nullable=False?

)


sqlite_file_name = "database.db"
sqlite_url = f"sqlite:///{sqlite_file_name}"

engine = create_engine(sqlite_url, echo=True)


def create_db_and_tables():
SQLModel.metadata.create_all(engine)


def create_heroes():
hero_1 = Hero(name="Deadpond", secret_name="Dive Wilson")
hero_2 = Hero(name="Spider-Boy", secret_name="Pedro Parqueador")
hero_3 = Hero(name="Rusty-Man", secret_name="Tommy Sharp", age=48)
Comment on lines +36 to +37

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only update hero1, I would remove hero2 and hero3 to simplify the example


session = Session(engine)

session.add(hero_1)
session.add(hero_2)
session.add(hero_3)

session.commit()

session.close()


def update_hero_age(new_secret_name):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
defupdate_hero_age(new_secret_name):
defupdate_hero_secret_name(new_secret_name):

with Session(engine) as session:
statement = select(Hero).where(Hero.name == "Spider-Boy")
results = session.exec(statement)
hero = results.one()
print("Hero:", hero)

hero.secret_name = new_secret_name
session.add(hero)
session.commit()
session.refresh(hero)
print("Updated hero:", hero)


def main():
create_db_and_tables()
create_heroes()
sleep(1)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to find way to avoid sleeping as it will make test run slower

update_hero_age("Arachnid-Lad")
sleep(1)
update_hero_age("The Wallclimber")


if __name__ == "__main__":
main()
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line numberDiff line numberDiff line change
Expand Up@@ -128,6 +128,7 @@ nav:
- advanced/index.md
- advanced/decimal.md
- advanced/uuid.md
- advanced/sa-column.md
- Resources:
- resources/index.md
- help.md
Expand Down
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions docs/advanced/sa-column.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
# SQLAlchemy Columns

In some cases you may need more control over the columns generated by SQLModel, this can be done by using the `sa_column`, `sa_column_args`, and `sa_column_kwargs` arguments when creating the `Field` object.

There are many use cases for this, but ones where this is particularity useful is when you want more advanced defaults for values than what is easy to implement with Pydantic, such `created_at` or `update_at` timestamps for rows.

## Columns for Timestamps

Two ways of implementing `created_at` timestamps with Pydantic are [default factories](https://pydantic-docs.helpmanual.io/usage/models/#field-with-dynamic-default-value) and [validators](https://pydantic-docs.helpmanual.io/usage/validators/#validate-always), however there's no straightforward way to have an `update_at` timestamp.
Comment thread
RobertRosca marked this conversation as resolved.

The SQLAlchemy docs describe how `created_at` timestamps can be automatically set with either [default](https://docs.sqlalchemy.org/en/14/core/defaults.html#python-executed-functions) or [server-default](https://docs.sqlalchemy.org/en/14/core/defaults.html#server-invoked-ddl-explicit-default-expressions) functions, by using `sa_column=Column(...)` as described in the SQLAlchemy documentation we can achieve the same behaviour:

{* ./docs_src/advanced/sa_column/tutorial001.py ln[9:21] hl[16,20] *}

Above we are saying that the `registered_at` column should have a `server_default` value of `func.now()` (see full code for imports), which means that if there is no provided value then the current time will be the recorded value for that row.

As there is a value there now, then it will not be changed automatically in the future.

The `updated_at` column has an `onupdate` value of `func.now()`, this means that each time an `UPDATE` is performed, the function will be executed, meaning that the timestamp changes whenever a change is made to the row.

/// warning

The difference between client-side python functions, server-side ddl expressions, and server-side implicit defaults is important in some situations but too in-depth to go into here. Check the SQL and SQLAlchemy docs for more information.

///

### Pydantic Implementation

Implementing these timestamps on the DB side with SQLAlchemy works very well as the database itself is what will create and update the fields whenever a relevant database interaction occurs.

It's possible to achieve similar behaviour with Pydantic, for the `created_at` timestamp by using a Pydantic `Field` with a `default_factory`:

```python
from datetime import datetime

from pydantic import BaseModel, Field


class Model(BaseModel):
created_at: datetime = Field(default_factory=datetime.utcnow)


m1 = Model()
m2 = Model()
print(f'{m1.created_at} != {m2.created_at}')
#> 2022-05-19 10:49:22.053624 != 2022-05-19 10:49:22.053641
```

Another approach is to use a Pydantic `validator`:

```python
from datetime import datetime

from pydantic import BaseModel, validator

class Model(BaseModel):
created_at: datetime = None

@validator('ts', pre=True, always=True)
def set_created_at_now(cls, v):
return v or datetime.now()
```
Comment on lines +49 to +62

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would remove this code example with validator


Both of these approaches come with the major caveat that default fields are set during the **Pydantic model instantiation**, as opposed to during **interactions with the database**, instead of the SQLModel approach which sets it with `server_default` which means that the timestamp will be exactly when the row is created in the database.

The real issue starts when looking at the `updated_at` timestamp - SQLAlchemy has the `onupdate` default which runs a function when the row is updated in the database, but there is no easy way to do this in Pydantic as it has no concept of 'about to be saved'.

So the pure Pydantic approach would require some additional logic to always change the `updated_at` timestamp before doing a write to the database, which adds some more complexity to the code and does not have benefits over the SQLAlchemy approach.
Empty file.
74 changes: 74 additions & 0 deletions docs_src/advanced/sa_column/tutorial001.py
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
from datetime import datetime
from time import sleep
from typing import Optional

from sqlalchemy import Column, DateTime, func
from sqlmodel import Field, Session, SQLModel, create_engine, select


class Hero(SQLModel, table=True):
id: Optional[int] = Field(default=None, primary_key=True)
name: str
secret_name: str
age: Optional[int] = None

registered_at: datetime = Field(
sa_column=Column(DateTime(timezone=False), server_default=func.now())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add nullable=False?

)

updated_at: Optional[datetime] = Field(
sa_column=Column(DateTime(timezone=False), onupdate=func.now())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add server_default=func.now() and nullable=False?

)


sqlite_file_name = "database.db"
sqlite_url = f"sqlite:///{sqlite_file_name}"

engine = create_engine(sqlite_url, echo=True)


def create_db_and_tables():
SQLModel.metadata.create_all(engine)


def create_heroes():
hero_1 = Hero(name="Deadpond", secret_name="Dive Wilson")
hero_2 = Hero(name="Spider-Boy", secret_name="Pedro Parqueador")
hero_3 = Hero(name="Rusty-Man", secret_name="Tommy Sharp", age=48)
Comment on lines +36 to +37

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only update hero1, I would remove hero2 and hero3 to simplify the example


session = Session(engine)

session.add(hero_1)
session.add(hero_2)
session.add(hero_3)

session.commit()

session.close()


def update_hero_age(new_secret_name):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
defupdate_hero_age(new_secret_name):
defupdate_hero_secret_name(new_secret_name):

with Session(engine) as session:
statement = select(Hero).where(Hero.name == "Spider-Boy")
results = session.exec(statement)
hero = results.one()
print("Hero:", hero)

hero.secret_name = new_secret_name
session.add(hero)
session.commit()
session.refresh(hero)
print("Updated hero:", hero)


def main():
create_db_and_tables()
create_heroes()
sleep(1)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to find way to avoid sleeping as it will make test run slower

update_hero_age("Arachnid-Lad")
sleep(1)
update_hero_age("The Wallclimber")


if __name__ == "__main__":
main()
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line numberDiff line numberDiff line change
Expand Up@@ -128,6 +128,7 @@ nav:
- advanced/index.md
- advanced/decimal.md
- advanced/uuid.md
- advanced/sa-column.md
- Resources:
- resources/index.md
- help.md
Expand Down