Pool API improve slots validation

[pierrejeambrun]

Add backend validation to pool API on slots attribute. It is not possible anymore to create or update pools slots to have null or negative values.

[jason810496]

It seems -1 is still supported for pools, as it is treated as infinite. So maybe we should allow positive integers and -1 for the slots field.

airflow/airflow-core/src/airflow/models/pool.py

Lines 90 to 91 in bb0ff04

	# -1 for infinite
	slots: Mapped[int] =mapped_column(Integer, default=0)

airflow/airflow-core/src/airflow/models/pool.py

Lines 216 to 222 in bb0ff04

	pool_rows=session.execute(query)
	forpool_name, total_slots_in, include_deferredinpool_rows:
	total_slots=float("inf") iftotal_slots_in==-1elsetotal_slots_in
	pools[pool_name] =PoolStats(
	total=total_slots, running=0, queued=0, open=0, deferred=0, scheduled=0
	)
	pool_includes_deferred[pool_name] =include_deferred

[pierrejeambrun]

Arf, yes there is that

[pierrejeambrun]

Ok, after more thoughts, lets merge this for now, which is better than no validation at all.

Currently -1 is not possible to use via the API, because it will be returned as infinity to the server which will 500. (and the front does not currently handle infinity as available pool slots).

I'll create a separate issue for that.

[github-actions]

Backport successfully created: v3-1-test

Status	Branch	Result
✅	v3-1-test

[pierrejeambrun]

Issue here #61115

[jhohrath1]

In our airflow maintenance procedures, we have been setting pools to 0 in order to stop tasks being scheduled so we can perform maintenance without losing task runs.

Since this change makes that not possible, was that practice not recommended?

[pierrejeambrun]

@jhohrath1 I didn't know that the '0' was actually being used as a 'feature'. There was a follow up PR there #61140 that allows back -1 and 0. Will come out in 3.1.8 that we are about to release.

[pierrejeambrun]

Add backend validation to pool API on slots attribute. It is not possible anymore to create or update pools slots to have null or negative values.

[jason810496]

It seems -1 is still supported for pools, as it is treated as infinite. So maybe we should allow positive integers and -1 for the slots field.

airflow/airflow-core/src/airflow/models/pool.py

Lines 90 to 91 in bb0ff04

	# -1 for infinite
	slots: Mapped[int] =mapped_column(Integer, default=0)

airflow/airflow-core/src/airflow/models/pool.py

Lines 216 to 222 in bb0ff04

	pool_rows=session.execute(query)
	forpool_name, total_slots_in, include_deferredinpool_rows:
	total_slots=float("inf") iftotal_slots_in==-1elsetotal_slots_in
	pools[pool_name] =PoolStats(
	total=total_slots, running=0, queued=0, open=0, deferred=0, scheduled=0
	)
	pool_includes_deferred[pool_name] =include_deferred

[pierrejeambrun]

Arf, yes there is that

[pierrejeambrun]

Ok, after more thoughts, lets merge this for now, which is better than no validation at all.

Currently -1 is not possible to use via the API, because it will be returned as infinity to the server which will 500. (and the front does not currently handle infinity as available pool slots).

I'll create a separate issue for that.

[github-actions]

Backport successfully created: v3-1-test

Status	Branch	Result
✅	v3-1-test

[pierrejeambrun]

Issue here #61115

[jhohrath1]

In our airflow maintenance procedures, we have been setting pools to 0 in order to stop tasks being scheduled so we can perform maintenance without losing task runs.

Since this change makes that not possible, was that practice not recommended?

[pierrejeambrun]

@jhohrath1 I didn't know that the '0' was actually being used as a 'feature'. There was a follow up PR there #61140 that allows back -1 and 0. Will come out in 3.1.8 that we are about to release.

[pierrejeambrun]

Add backend validation to pool API on slots attribute. It is not possible anymore to create or update pools slots to have null or negative values.

[jason810496]

It seems -1 is still supported for pools, as it is treated as infinite. So maybe we should allow positive integers and -1 for the slots field.

airflow/airflow-core/src/airflow/models/pool.py

Lines 90 to 91 in bb0ff04

	# -1 for infinite
	slots: Mapped[int] =mapped_column(Integer, default=0)

airflow/airflow-core/src/airflow/models/pool.py

Lines 216 to 222 in bb0ff04

	pool_rows=session.execute(query)
	forpool_name, total_slots_in, include_deferredinpool_rows:
	total_slots=float("inf") iftotal_slots_in==-1elsetotal_slots_in
	pools[pool_name] =PoolStats(
	total=total_slots, running=0, queued=0, open=0, deferred=0, scheduled=0
	)
	pool_includes_deferred[pool_name] =include_deferred

[pierrejeambrun]

Arf, yes there is that

[pierrejeambrun]

Ok, after more thoughts, lets merge this for now, which is better than no validation at all.

Currently -1 is not possible to use via the API, because it will be returned as infinity to the server which will 500. (and the front does not currently handle infinity as available pool slots).

I'll create a separate issue for that.

[github-actions]

Backport successfully created: v3-1-test

Status	Branch	Result
✅	v3-1-test

[pierrejeambrun]

Issue here #61115

[jhohrath1]

In our airflow maintenance procedures, we have been setting pools to 0 in order to stop tasks being scheduled so we can perform maintenance without losing task runs.

Since this change makes that not possible, was that practice not recommended?

[pierrejeambrun]

@jhohrath1 I didn't know that the '0' was actually being used as a 'feature'. There was a follow up PR there #61140 that allows back -1 and 0. Will come out in 3.1.8 that we are about to release.

[pierrejeambrun]

Add backend validation to pool API on slots attribute. It is not possible anymore to create or update pools slots to have null or negative values.

[jason810496]

It seems -1 is still supported for pools, as it is treated as infinite. So maybe we should allow positive integers and -1 for the slots field.

airflow/airflow-core/src/airflow/models/pool.py

Lines 90 to 91 in bb0ff04

	# -1 for infinite
	slots: Mapped[int] =mapped_column(Integer, default=0)

airflow/airflow-core/src/airflow/models/pool.py

Lines 216 to 222 in bb0ff04

	pool_rows=session.execute(query)
	forpool_name, total_slots_in, include_deferredinpool_rows:
	total_slots=float("inf") iftotal_slots_in==-1elsetotal_slots_in
	pools[pool_name] =PoolStats(
	total=total_slots, running=0, queued=0, open=0, deferred=0, scheduled=0
	)
	pool_includes_deferred[pool_name] =include_deferred

[pierrejeambrun]

Arf, yes there is that

[pierrejeambrun]

Ok, after more thoughts, lets merge this for now, which is better than no validation at all.

Currently -1 is not possible to use via the API, because it will be returned as infinity to the server which will 500. (and the front does not currently handle infinity as available pool slots).

I'll create a separate issue for that.

[github-actions]

Backport successfully created: v3-1-test

Status	Branch	Result
✅	v3-1-test

[pierrejeambrun]

Issue here #61115

[jhohrath1]

In our airflow maintenance procedures, we have been setting pools to 0 in order to stop tasks being scheduled so we can perform maintenance without losing task runs.

Since this change makes that not possible, was that practice not recommended?

[pierrejeambrun]

@jhohrath1 I didn't know that the '0' was actually being used as a 'feature'. There was a follow up PR there #61140 that allows back -1 and 0. Will come out in 3.1.8 that we are about to release.

[pierrejeambrun]

Add backend validation to pool API on slots attribute. It is not possible anymore to create or update pools slots to have null or negative values.

[jason810496]

It seems -1 is still supported for pools, as it is treated as infinite. So maybe we should allow positive integers and -1 for the slots field.

airflow/airflow-core/src/airflow/models/pool.py

Lines 90 to 91 in bb0ff04

	# -1 for infinite
	slots: Mapped[int] =mapped_column(Integer, default=0)

airflow/airflow-core/src/airflow/models/pool.py

Lines 216 to 222 in bb0ff04

	pool_rows=session.execute(query)
	forpool_name, total_slots_in, include_deferredinpool_rows:
	total_slots=float("inf") iftotal_slots_in==-1elsetotal_slots_in
	pools[pool_name] =PoolStats(
	total=total_slots, running=0, queued=0, open=0, deferred=0, scheduled=0
	)
	pool_includes_deferred[pool_name] =include_deferred

[pierrejeambrun]

Arf, yes there is that

[pierrejeambrun]

Ok, after more thoughts, lets merge this for now, which is better than no validation at all.

Currently -1 is not possible to use via the API, because it will be returned as infinity to the server which will 500. (and the front does not currently handle infinity as available pool slots).

I'll create a separate issue for that.

[github-actions]

Backport successfully created: v3-1-test

Status	Branch	Result
✅	v3-1-test

[pierrejeambrun]

Issue here #61115

[jhohrath1]

In our airflow maintenance procedures, we have been setting pools to 0 in order to stop tasks being scheduled so we can perform maintenance without losing task runs.

Since this change makes that not possible, was that practice not recommended?

[pierrejeambrun]

@jhohrath1 I didn't know that the '0' was actually being used as a 'feature'. There was a follow up PR there #61140 that allows back -1 and 0. Will come out in 3.1.8 that we are about to release.

[pierrejeambrun]

Add backend validation to pool API on slots attribute. It is not possible anymore to create or update pools slots to have null or negative values.

[jason810496]

It seems -1 is still supported for pools, as it is treated as infinite. So maybe we should allow positive integers and -1 for the slots field.

airflow/airflow-core/src/airflow/models/pool.py

Lines 90 to 91 in bb0ff04

	# -1 for infinite
	slots: Mapped[int] =mapped_column(Integer, default=0)

airflow/airflow-core/src/airflow/models/pool.py

Lines 216 to 222 in bb0ff04

	pool_rows=session.execute(query)
	forpool_name, total_slots_in, include_deferredinpool_rows:
	total_slots=float("inf") iftotal_slots_in==-1elsetotal_slots_in
	pools[pool_name] =PoolStats(
	total=total_slots, running=0, queued=0, open=0, deferred=0, scheduled=0
	)
	pool_includes_deferred[pool_name] =include_deferred

[pierrejeambrun]

Arf, yes there is that

[pierrejeambrun]

Ok, after more thoughts, lets merge this for now, which is better than no validation at all.

Currently -1 is not possible to use via the API, because it will be returned as infinity to the server which will 500. (and the front does not currently handle infinity as available pool slots).

I'll create a separate issue for that.

[github-actions]

Backport successfully created: v3-1-test

Status	Branch	Result
✅	v3-1-test

[pierrejeambrun]

Issue here #61115

[jhohrath1]

In our airflow maintenance procedures, we have been setting pools to 0 in order to stop tasks being scheduled so we can perform maintenance without losing task runs.

Since this change makes that not possible, was that practice not recommended?

[pierrejeambrun]

@jhohrath1 I didn't know that the '0' was actually being used as a 'feature'. There was a follow up PR there #61140 that allows back -1 and 0. Will come out in 3.1.8 that we are about to release.

[pierrejeambrun]

Add backend validation to pool API on slots attribute. It is not possible anymore to create or update pools slots to have null or negative values.

[jason810496]

It seems -1 is still supported for pools, as it is treated as infinite. So maybe we should allow positive integers and -1 for the slots field.

airflow/airflow-core/src/airflow/models/pool.py

Lines 90 to 91 in bb0ff04

	# -1 for infinite
	slots: Mapped[int] =mapped_column(Integer, default=0)

airflow/airflow-core/src/airflow/models/pool.py

Lines 216 to 222 in bb0ff04

	pool_rows=session.execute(query)
	forpool_name, total_slots_in, include_deferredinpool_rows:
	total_slots=float("inf") iftotal_slots_in==-1elsetotal_slots_in
	pools[pool_name] =PoolStats(
	total=total_slots, running=0, queued=0, open=0, deferred=0, scheduled=0
	)
	pool_includes_deferred[pool_name] =include_deferred

[pierrejeambrun]

Arf, yes there is that

[pierrejeambrun]

Ok, after more thoughts, lets merge this for now, which is better than no validation at all.

Currently -1 is not possible to use via the API, because it will be returned as infinity to the server which will 500. (and the front does not currently handle infinity as available pool slots).

I'll create a separate issue for that.

[github-actions]

Backport successfully created: v3-1-test

Status	Branch	Result
✅	v3-1-test

[pierrejeambrun]

Issue here #61115

[jhohrath1]

In our airflow maintenance procedures, we have been setting pools to 0 in order to stop tasks being scheduled so we can perform maintenance without losing task runs.

Since this change makes that not possible, was that practice not recommended?

[pierrejeambrun]

@jhohrath1 I didn't know that the '0' was actually being used as a 'feature'. There was a follow up PR there #61140 that allows back -1 and 0. Will come out in 3.1.8 that we are about to release.

[pierrejeambrun]

Add backend validation to pool API on slots attribute. It is not possible anymore to create or update pools slots to have null or negative values.

[jason810496]

It seems -1 is still supported for pools, as it is treated as infinite. So maybe we should allow positive integers and -1 for the slots field.

airflow/airflow-core/src/airflow/models/pool.py

Lines 90 to 91 in bb0ff04

	# -1 for infinite
	slots: Mapped[int] =mapped_column(Integer, default=0)

airflow/airflow-core/src/airflow/models/pool.py

Lines 216 to 222 in bb0ff04

	pool_rows=session.execute(query)
	forpool_name, total_slots_in, include_deferredinpool_rows:
	total_slots=float("inf") iftotal_slots_in==-1elsetotal_slots_in
	pools[pool_name] =PoolStats(
	total=total_slots, running=0, queued=0, open=0, deferred=0, scheduled=0
	)
	pool_includes_deferred[pool_name] =include_deferred

[pierrejeambrun]

Arf, yes there is that

[pierrejeambrun]

Ok, after more thoughts, lets merge this for now, which is better than no validation at all.

Currently -1 is not possible to use via the API, because it will be returned as infinity to the server which will 500. (and the front does not currently handle infinity as available pool slots).

I'll create a separate issue for that.

[github-actions]

Backport successfully created: v3-1-test

Status	Branch	Result
✅	v3-1-test

[pierrejeambrun]

Issue here #61115

[jhohrath1]

In our airflow maintenance procedures, we have been setting pools to 0 in order to stop tasks being scheduled so we can perform maintenance without losing task runs.

Since this change makes that not possible, was that practice not recommended?

[pierrejeambrun]

@jhohrath1 I didn't know that the '0' was actually being used as a 'feature'. There was a follow up PR there #61140 that allows back -1 and 0. Will come out in 3.1.8 that we are about to release.