Fix operation error

[ebreton]

Details of the bug can be found in issue #31

The solution makes use of the 'pessimistic' approach as described in https://docs.sqlalchemy.org/en/13/core/pooling.html#pool-disconnects

The approach adds a small bit of overhead to the connection checkout process, however is otherwise the most simple and reliable approach to completely eliminating database errors due to stale pooled connections. The calling application does not need to be concerned about organizing operations to be able to recover from stale connections checked out from the pool

[tiangolo]

Thanks! That looks like a small change with a long time debugging and investigating 🤓

Were you able to test it? Did it solve the problem?

[ebreton]

Hi @tiangolo , I do not have a reproductible test, yet here is what I have done:

• before fix, restart the db container, and query the API backend -> 500 errors come at once, in numbers

• after fix, restart the db container with same conditions -> there is a few-seconds lag in the first query (probably timeout and/or retrials from the connection pool), then a couple of 500 errors before the DB is up again

I have also checked the logs, and there are no more OperationalErrors after applying the fix (I will check again during the week)

From my point of view, the issue is fixed, but not automatically tested. Which does not really concern me, since the management of the connection cool is on the SQLAlchemy level, and therefore tested by them

[tiangolo]

Cool!

Thanks for the explanation and the local tests you did. That's good enough ✔️

Thanks for your work! 🎉 🚀 🍰

[ebreton]

Details of the bug can be found in issue #31

The solution makes use of the 'pessimistic' approach as described in https://docs.sqlalchemy.org/en/13/core/pooling.html#pool-disconnects

The approach adds a small bit of overhead to the connection checkout process, however is otherwise the most simple and reliable approach to completely eliminating database errors due to stale pooled connections. The calling application does not need to be concerned about organizing operations to be able to recover from stale connections checked out from the pool

[tiangolo]

Thanks! That looks like a small change with a long time debugging and investigating 🤓

Were you able to test it? Did it solve the problem?

[ebreton]

Hi @tiangolo , I do not have a reproductible test, yet here is what I have done:

• before fix, restart the db container, and query the API backend -> 500 errors come at once, in numbers

• after fix, restart the db container with same conditions -> there is a few-seconds lag in the first query (probably timeout and/or retrials from the connection pool), then a couple of 500 errors before the DB is up again

I have also checked the logs, and there are no more OperationalErrors after applying the fix (I will check again during the week)

From my point of view, the issue is fixed, but not automatically tested. Which does not really concern me, since the management of the connection cool is on the SQLAlchemy level, and therefore tested by them

[tiangolo]

Cool!

Thanks for the explanation and the local tests you did. That's good enough ✔️

Thanks for your work! 🎉 🚀 🍰

[ebreton]

Details of the bug can be found in issue #31

The solution makes use of the 'pessimistic' approach as described in https://docs.sqlalchemy.org/en/13/core/pooling.html#pool-disconnects

The approach adds a small bit of overhead to the connection checkout process, however is otherwise the most simple and reliable approach to completely eliminating database errors due to stale pooled connections. The calling application does not need to be concerned about organizing operations to be able to recover from stale connections checked out from the pool

[tiangolo]

Thanks! That looks like a small change with a long time debugging and investigating 🤓

Were you able to test it? Did it solve the problem?

[ebreton]

Hi @tiangolo , I do not have a reproductible test, yet here is what I have done:

• before fix, restart the db container, and query the API backend -> 500 errors come at once, in numbers

• after fix, restart the db container with same conditions -> there is a few-seconds lag in the first query (probably timeout and/or retrials from the connection pool), then a couple of 500 errors before the DB is up again

I have also checked the logs, and there are no more OperationalErrors after applying the fix (I will check again during the week)

From my point of view, the issue is fixed, but not automatically tested. Which does not really concern me, since the management of the connection cool is on the SQLAlchemy level, and therefore tested by them

[tiangolo]

Cool!

Thanks for the explanation and the local tests you did. That's good enough ✔️

Thanks for your work! 🎉 🚀 🍰

[ebreton]

Details of the bug can be found in issue #31

The solution makes use of the 'pessimistic' approach as described in https://docs.sqlalchemy.org/en/13/core/pooling.html#pool-disconnects

The approach adds a small bit of overhead to the connection checkout process, however is otherwise the most simple and reliable approach to completely eliminating database errors due to stale pooled connections. The calling application does not need to be concerned about organizing operations to be able to recover from stale connections checked out from the pool

[tiangolo]

Thanks! That looks like a small change with a long time debugging and investigating 🤓

Were you able to test it? Did it solve the problem?

[ebreton]

Hi @tiangolo , I do not have a reproductible test, yet here is what I have done:

• before fix, restart the db container, and query the API backend -> 500 errors come at once, in numbers

• after fix, restart the db container with same conditions -> there is a few-seconds lag in the first query (probably timeout and/or retrials from the connection pool), then a couple of 500 errors before the DB is up again

I have also checked the logs, and there are no more OperationalErrors after applying the fix (I will check again during the week)

From my point of view, the issue is fixed, but not automatically tested. Which does not really concern me, since the management of the connection cool is on the SQLAlchemy level, and therefore tested by them

[tiangolo]

Cool!

Thanks for the explanation and the local tests you did. That's good enough ✔️

Thanks for your work! 🎉 🚀 🍰

[ebreton]

Details of the bug can be found in issue #31

The solution makes use of the 'pessimistic' approach as described in https://docs.sqlalchemy.org/en/13/core/pooling.html#pool-disconnects

The approach adds a small bit of overhead to the connection checkout process, however is otherwise the most simple and reliable approach to completely eliminating database errors due to stale pooled connections. The calling application does not need to be concerned about organizing operations to be able to recover from stale connections checked out from the pool

[tiangolo]

Thanks! That looks like a small change with a long time debugging and investigating 🤓

Were you able to test it? Did it solve the problem?

[ebreton]

Hi @tiangolo , I do not have a reproductible test, yet here is what I have done:

• before fix, restart the db container, and query the API backend -> 500 errors come at once, in numbers

• after fix, restart the db container with same conditions -> there is a few-seconds lag in the first query (probably timeout and/or retrials from the connection pool), then a couple of 500 errors before the DB is up again

I have also checked the logs, and there are no more OperationalErrors after applying the fix (I will check again during the week)

From my point of view, the issue is fixed, but not automatically tested. Which does not really concern me, since the management of the connection cool is on the SQLAlchemy level, and therefore tested by them

[tiangolo]

Cool!

Thanks for the explanation and the local tests you did. That's good enough ✔️

Thanks for your work! 🎉 🚀 🍰

[ebreton]

Details of the bug can be found in issue #31

The solution makes use of the 'pessimistic' approach as described in https://docs.sqlalchemy.org/en/13/core/pooling.html#pool-disconnects

The approach adds a small bit of overhead to the connection checkout process, however is otherwise the most simple and reliable approach to completely eliminating database errors due to stale pooled connections. The calling application does not need to be concerned about organizing operations to be able to recover from stale connections checked out from the pool

[tiangolo]

Thanks! That looks like a small change with a long time debugging and investigating 🤓

Were you able to test it? Did it solve the problem?

[ebreton]

Hi @tiangolo , I do not have a reproductible test, yet here is what I have done:

• before fix, restart the db container, and query the API backend -> 500 errors come at once, in numbers

• after fix, restart the db container with same conditions -> there is a few-seconds lag in the first query (probably timeout and/or retrials from the connection pool), then a couple of 500 errors before the DB is up again

I have also checked the logs, and there are no more OperationalErrors after applying the fix (I will check again during the week)

From my point of view, the issue is fixed, but not automatically tested. Which does not really concern me, since the management of the connection cool is on the SQLAlchemy level, and therefore tested by them

[tiangolo]

Cool!

Thanks for the explanation and the local tests you did. That's good enough ✔️

Thanks for your work! 🎉 🚀 🍰

[ebreton]

Details of the bug can be found in issue #31

The solution makes use of the 'pessimistic' approach as described in https://docs.sqlalchemy.org/en/13/core/pooling.html#pool-disconnects

The approach adds a small bit of overhead to the connection checkout process, however is otherwise the most simple and reliable approach to completely eliminating database errors due to stale pooled connections. The calling application does not need to be concerned about organizing operations to be able to recover from stale connections checked out from the pool

[tiangolo]

Thanks! That looks like a small change with a long time debugging and investigating 🤓

Were you able to test it? Did it solve the problem?

[ebreton]

Hi @tiangolo , I do not have a reproductible test, yet here is what I have done:

• before fix, restart the db container, and query the API backend -> 500 errors come at once, in numbers

• after fix, restart the db container with same conditions -> there is a few-seconds lag in the first query (probably timeout and/or retrials from the connection pool), then a couple of 500 errors before the DB is up again

I have also checked the logs, and there are no more OperationalErrors after applying the fix (I will check again during the week)

From my point of view, the issue is fixed, but not automatically tested. Which does not really concern me, since the management of the connection cool is on the SQLAlchemy level, and therefore tested by them

[tiangolo]

Cool!

Thanks for the explanation and the local tests you did. That's good enough ✔️

Thanks for your work! 🎉 🚀 🍰

[ebreton]

Details of the bug can be found in issue #31

The solution makes use of the 'pessimistic' approach as described in https://docs.sqlalchemy.org/en/13/core/pooling.html#pool-disconnects

The approach adds a small bit of overhead to the connection checkout process, however is otherwise the most simple and reliable approach to completely eliminating database errors due to stale pooled connections. The calling application does not need to be concerned about organizing operations to be able to recover from stale connections checked out from the pool

[tiangolo]

Thanks! That looks like a small change with a long time debugging and investigating 🤓

Were you able to test it? Did it solve the problem?

[ebreton]

Hi @tiangolo , I do not have a reproductible test, yet here is what I have done:

• before fix, restart the db container, and query the API backend -> 500 errors come at once, in numbers

• after fix, restart the db container with same conditions -> there is a few-seconds lag in the first query (probably timeout and/or retrials from the connection pool), then a couple of 500 errors before the DB is up again

I have also checked the logs, and there are no more OperationalErrors after applying the fix (I will check again during the week)

From my point of view, the issue is fixed, but not automatically tested. Which does not really concern me, since the management of the connection cool is on the SQLAlchemy level, and therefore tested by them

[tiangolo]

Cool!

Thanks for the explanation and the local tests you did. That's good enough ✔️

Thanks for your work! 🎉 🚀 🍰