Add button to download task logs as a text file

[aarochuk]

Added ability to download task instance logs in airflow 3 ui as described in issue #47689
closes: #47689

[tirkarthi]

This looks the filename for the downloaded log file. This should include dagId and taskInstance details in the name like Airflow 2 else this might keep overwriting the file on disk or create something like taskInstanceLogs(1).txt

[bbovenzi]

+1 on ${dagId}-${taskId}-${runId}-${mapIndex}-${tryNumber}.txt

[tirkarthi]

I guess you can pass text/plain in the accept here and get the raw value of the log file in the storage which is the value users usually want to reduce the code for formatting.

accept?: "application/json" | "text/plain" | "*/*";

airflow/airflow-core/src/airflow/api_fastapi/core_api/routes/public/log.py

Lines 136 to 153 in 96c6daa

	ifaccept==Mimetype.JSONoraccept==Mimetype.ANY: # default
	logs, metadata=task_log_reader.read_log_chunks(ti, try_number, metadata)
	encoded_token=None
	ifnotmetadata.get("end_of_log", False):
	encoded_token=URLSafeSerializer(request.app.state.secret_key).dumps(metadata)
	returnTaskInstancesLogResponse.model_construct(continuation_token=encoded_token, content=logs)
	# text/plain, or something else we don't understand. Return raw log content
	# We need to exhaust the iterator before we can generate the continuation token.
	# We could improve this by making it a streaming/async response, and by then setting the header using
	# HTTP Trailers
	logs="".join(task_log_reader.read_log_stream(ti, try_number, metadata))
	headers=None
	ifnotmetadata.get("end_of_log", False):
	headers= {
	"Airflow-Continuation-Token": URLSafeSerializer(request.app.state.secret_key).dumps(metadata)
	}
	returnResponse(media_type="application/x-ndjson", content=logs, headers=headers)

[bbovenzi]

@aarochuk mind rebasing and addressing PR comments?

[aarochuk]

@bbovenzi I will make sure to make another push by the end of the week sorry it has taken so long

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

[bbovenzi]

@aarochuk are you still able to finish this PR?

[phanikumv]

@aarochuk just checking in to see if you need help with this PR?

[uranusjr]

Removing from 3.1.0 due to lack of response and the PR not in a working state. Please comment if anyone is interested in picking this up, we can add it back if possible.

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

[aarochuk]

Added ability to download task instance logs in airflow 3 ui as described in issue #47689
closes: #47689

[tirkarthi]

This looks the filename for the downloaded log file. This should include dagId and taskInstance details in the name like Airflow 2 else this might keep overwriting the file on disk or create something like taskInstanceLogs(1).txt

[bbovenzi]

+1 on ${dagId}-${taskId}-${runId}-${mapIndex}-${tryNumber}.txt

[tirkarthi]

I guess you can pass text/plain in the accept here and get the raw value of the log file in the storage which is the value users usually want to reduce the code for formatting.

accept?: "application/json" | "text/plain" | "*/*";

airflow/airflow-core/src/airflow/api_fastapi/core_api/routes/public/log.py

Lines 136 to 153 in 96c6daa

	ifaccept==Mimetype.JSONoraccept==Mimetype.ANY: # default
	logs, metadata=task_log_reader.read_log_chunks(ti, try_number, metadata)
	encoded_token=None
	ifnotmetadata.get("end_of_log", False):
	encoded_token=URLSafeSerializer(request.app.state.secret_key).dumps(metadata)
	returnTaskInstancesLogResponse.model_construct(continuation_token=encoded_token, content=logs)
	# text/plain, or something else we don't understand. Return raw log content
	# We need to exhaust the iterator before we can generate the continuation token.
	# We could improve this by making it a streaming/async response, and by then setting the header using
	# HTTP Trailers
	logs="".join(task_log_reader.read_log_stream(ti, try_number, metadata))
	headers=None
	ifnotmetadata.get("end_of_log", False):
	headers= {
	"Airflow-Continuation-Token": URLSafeSerializer(request.app.state.secret_key).dumps(metadata)
	}
	returnResponse(media_type="application/x-ndjson", content=logs, headers=headers)

[bbovenzi]

@aarochuk mind rebasing and addressing PR comments?

[aarochuk]

@bbovenzi I will make sure to make another push by the end of the week sorry it has taken so long

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

[bbovenzi]

@aarochuk are you still able to finish this PR?

[phanikumv]

@aarochuk just checking in to see if you need help with this PR?

[uranusjr]

Removing from 3.1.0 due to lack of response and the PR not in a working state. Please comment if anyone is interested in picking this up, we can add it back if possible.

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

[aarochuk]

Added ability to download task instance logs in airflow 3 ui as described in issue #47689
closes: #47689

[tirkarthi]

This looks the filename for the downloaded log file. This should include dagId and taskInstance details in the name like Airflow 2 else this might keep overwriting the file on disk or create something like taskInstanceLogs(1).txt

[bbovenzi]

+1 on ${dagId}-${taskId}-${runId}-${mapIndex}-${tryNumber}.txt

[tirkarthi]

I guess you can pass text/plain in the accept here and get the raw value of the log file in the storage which is the value users usually want to reduce the code for formatting.

accept?: "application/json" | "text/plain" | "*/*";

airflow/airflow-core/src/airflow/api_fastapi/core_api/routes/public/log.py

Lines 136 to 153 in 96c6daa

	ifaccept==Mimetype.JSONoraccept==Mimetype.ANY: # default
	logs, metadata=task_log_reader.read_log_chunks(ti, try_number, metadata)
	encoded_token=None
	ifnotmetadata.get("end_of_log", False):
	encoded_token=URLSafeSerializer(request.app.state.secret_key).dumps(metadata)
	returnTaskInstancesLogResponse.model_construct(continuation_token=encoded_token, content=logs)
	# text/plain, or something else we don't understand. Return raw log content
	# We need to exhaust the iterator before we can generate the continuation token.
	# We could improve this by making it a streaming/async response, and by then setting the header using
	# HTTP Trailers
	logs="".join(task_log_reader.read_log_stream(ti, try_number, metadata))
	headers=None
	ifnotmetadata.get("end_of_log", False):
	headers= {
	"Airflow-Continuation-Token": URLSafeSerializer(request.app.state.secret_key).dumps(metadata)
	}
	returnResponse(media_type="application/x-ndjson", content=logs, headers=headers)

[bbovenzi]

@aarochuk mind rebasing and addressing PR comments?

[aarochuk]

@bbovenzi I will make sure to make another push by the end of the week sorry it has taken so long

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

[bbovenzi]

@aarochuk are you still able to finish this PR?

[phanikumv]

@aarochuk just checking in to see if you need help with this PR?

[uranusjr]

Removing from 3.1.0 due to lack of response and the PR not in a working state. Please comment if anyone is interested in picking this up, we can add it back if possible.

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

[aarochuk]

Added ability to download task instance logs in airflow 3 ui as described in issue #47689
closes: #47689

[tirkarthi]

This looks the filename for the downloaded log file. This should include dagId and taskInstance details in the name like Airflow 2 else this might keep overwriting the file on disk or create something like taskInstanceLogs(1).txt

[bbovenzi]

+1 on ${dagId}-${taskId}-${runId}-${mapIndex}-${tryNumber}.txt

[tirkarthi]

I guess you can pass text/plain in the accept here and get the raw value of the log file in the storage which is the value users usually want to reduce the code for formatting.

accept?: "application/json" | "text/plain" | "*/*";

airflow/airflow-core/src/airflow/api_fastapi/core_api/routes/public/log.py

Lines 136 to 153 in 96c6daa

	ifaccept==Mimetype.JSONoraccept==Mimetype.ANY: # default
	logs, metadata=task_log_reader.read_log_chunks(ti, try_number, metadata)
	encoded_token=None
	ifnotmetadata.get("end_of_log", False):
	encoded_token=URLSafeSerializer(request.app.state.secret_key).dumps(metadata)
	returnTaskInstancesLogResponse.model_construct(continuation_token=encoded_token, content=logs)
	# text/plain, or something else we don't understand. Return raw log content
	# We need to exhaust the iterator before we can generate the continuation token.
	# We could improve this by making it a streaming/async response, and by then setting the header using
	# HTTP Trailers
	logs="".join(task_log_reader.read_log_stream(ti, try_number, metadata))
	headers=None
	ifnotmetadata.get("end_of_log", False):
	headers= {
	"Airflow-Continuation-Token": URLSafeSerializer(request.app.state.secret_key).dumps(metadata)
	}
	returnResponse(media_type="application/x-ndjson", content=logs, headers=headers)

[bbovenzi]

@aarochuk mind rebasing and addressing PR comments?

[aarochuk]

@bbovenzi I will make sure to make another push by the end of the week sorry it has taken so long

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

[bbovenzi]

@aarochuk are you still able to finish this PR?

[phanikumv]

@aarochuk just checking in to see if you need help with this PR?

[uranusjr]

Removing from 3.1.0 due to lack of response and the PR not in a working state. Please comment if anyone is interested in picking this up, we can add it back if possible.

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

[aarochuk]

Added ability to download task instance logs in airflow 3 ui as described in issue #47689
closes: #47689

[tirkarthi]

This looks the filename for the downloaded log file. This should include dagId and taskInstance details in the name like Airflow 2 else this might keep overwriting the file on disk or create something like taskInstanceLogs(1).txt

[bbovenzi]

+1 on ${dagId}-${taskId}-${runId}-${mapIndex}-${tryNumber}.txt

[tirkarthi]

I guess you can pass text/plain in the accept here and get the raw value of the log file in the storage which is the value users usually want to reduce the code for formatting.

accept?: "application/json" | "text/plain" | "*/*";

airflow/airflow-core/src/airflow/api_fastapi/core_api/routes/public/log.py

Lines 136 to 153 in 96c6daa

	ifaccept==Mimetype.JSONoraccept==Mimetype.ANY: # default
	logs, metadata=task_log_reader.read_log_chunks(ti, try_number, metadata)
	encoded_token=None
	ifnotmetadata.get("end_of_log", False):
	encoded_token=URLSafeSerializer(request.app.state.secret_key).dumps(metadata)
	returnTaskInstancesLogResponse.model_construct(continuation_token=encoded_token, content=logs)
	# text/plain, or something else we don't understand. Return raw log content
	# We need to exhaust the iterator before we can generate the continuation token.
	# We could improve this by making it a streaming/async response, and by then setting the header using
	# HTTP Trailers
	logs="".join(task_log_reader.read_log_stream(ti, try_number, metadata))
	headers=None
	ifnotmetadata.get("end_of_log", False):
	headers= {
	"Airflow-Continuation-Token": URLSafeSerializer(request.app.state.secret_key).dumps(metadata)
	}
	returnResponse(media_type="application/x-ndjson", content=logs, headers=headers)

[bbovenzi]

@aarochuk mind rebasing and addressing PR comments?

[aarochuk]

@bbovenzi I will make sure to make another push by the end of the week sorry it has taken so long

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

[bbovenzi]

@aarochuk are you still able to finish this PR?

[phanikumv]

@aarochuk just checking in to see if you need help with this PR?

[uranusjr]

Removing from 3.1.0 due to lack of response and the PR not in a working state. Please comment if anyone is interested in picking this up, we can add it back if possible.

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

[aarochuk]

Added ability to download task instance logs in airflow 3 ui as described in issue #47689
closes: #47689

[tirkarthi]

This looks the filename for the downloaded log file. This should include dagId and taskInstance details in the name like Airflow 2 else this might keep overwriting the file on disk or create something like taskInstanceLogs(1).txt

[bbovenzi]

+1 on ${dagId}-${taskId}-${runId}-${mapIndex}-${tryNumber}.txt

[tirkarthi]

I guess you can pass text/plain in the accept here and get the raw value of the log file in the storage which is the value users usually want to reduce the code for formatting.

accept?: "application/json" | "text/plain" | "*/*";

airflow/airflow-core/src/airflow/api_fastapi/core_api/routes/public/log.py

Lines 136 to 153 in 96c6daa

	ifaccept==Mimetype.JSONoraccept==Mimetype.ANY: # default
	logs, metadata=task_log_reader.read_log_chunks(ti, try_number, metadata)
	encoded_token=None
	ifnotmetadata.get("end_of_log", False):
	encoded_token=URLSafeSerializer(request.app.state.secret_key).dumps(metadata)
	returnTaskInstancesLogResponse.model_construct(continuation_token=encoded_token, content=logs)
	# text/plain, or something else we don't understand. Return raw log content
	# We need to exhaust the iterator before we can generate the continuation token.
	# We could improve this by making it a streaming/async response, and by then setting the header using
	# HTTP Trailers
	logs="".join(task_log_reader.read_log_stream(ti, try_number, metadata))
	headers=None
	ifnotmetadata.get("end_of_log", False):
	headers= {
	"Airflow-Continuation-Token": URLSafeSerializer(request.app.state.secret_key).dumps(metadata)
	}
	returnResponse(media_type="application/x-ndjson", content=logs, headers=headers)

[bbovenzi]

@aarochuk mind rebasing and addressing PR comments?

[aarochuk]

@bbovenzi I will make sure to make another push by the end of the week sorry it has taken so long

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

[bbovenzi]

@aarochuk are you still able to finish this PR?

[phanikumv]

@aarochuk just checking in to see if you need help with this PR?

[uranusjr]

Removing from 3.1.0 due to lack of response and the PR not in a working state. Please comment if anyone is interested in picking this up, we can add it back if possible.

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

[aarochuk]

Added ability to download task instance logs in airflow 3 ui as described in issue #47689
closes: #47689

[tirkarthi]

This looks the filename for the downloaded log file. This should include dagId and taskInstance details in the name like Airflow 2 else this might keep overwriting the file on disk or create something like taskInstanceLogs(1).txt

[bbovenzi]

+1 on ${dagId}-${taskId}-${runId}-${mapIndex}-${tryNumber}.txt

[tirkarthi]

I guess you can pass text/plain in the accept here and get the raw value of the log file in the storage which is the value users usually want to reduce the code for formatting.

accept?: "application/json" | "text/plain" | "*/*";

airflow/airflow-core/src/airflow/api_fastapi/core_api/routes/public/log.py

Lines 136 to 153 in 96c6daa

	ifaccept==Mimetype.JSONoraccept==Mimetype.ANY: # default
	logs, metadata=task_log_reader.read_log_chunks(ti, try_number, metadata)
	encoded_token=None
	ifnotmetadata.get("end_of_log", False):
	encoded_token=URLSafeSerializer(request.app.state.secret_key).dumps(metadata)
	returnTaskInstancesLogResponse.model_construct(continuation_token=encoded_token, content=logs)
	# text/plain, or something else we don't understand. Return raw log content
	# We need to exhaust the iterator before we can generate the continuation token.
	# We could improve this by making it a streaming/async response, and by then setting the header using
	# HTTP Trailers
	logs="".join(task_log_reader.read_log_stream(ti, try_number, metadata))
	headers=None
	ifnotmetadata.get("end_of_log", False):
	headers= {
	"Airflow-Continuation-Token": URLSafeSerializer(request.app.state.secret_key).dumps(metadata)
	}
	returnResponse(media_type="application/x-ndjson", content=logs, headers=headers)

[bbovenzi]

@aarochuk mind rebasing and addressing PR comments?

[aarochuk]

@bbovenzi I will make sure to make another push by the end of the week sorry it has taken so long

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

[bbovenzi]

@aarochuk are you still able to finish this PR?

[phanikumv]

@aarochuk just checking in to see if you need help with this PR?

[uranusjr]

Removing from 3.1.0 due to lack of response and the PR not in a working state. Please comment if anyone is interested in picking this up, we can add it back if possible.

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

[aarochuk]

Added ability to download task instance logs in airflow 3 ui as described in issue #47689
closes: #47689

[tirkarthi]

This looks the filename for the downloaded log file. This should include dagId and taskInstance details in the name like Airflow 2 else this might keep overwriting the file on disk or create something like taskInstanceLogs(1).txt

[bbovenzi]

+1 on ${dagId}-${taskId}-${runId}-${mapIndex}-${tryNumber}.txt

[tirkarthi]

I guess you can pass text/plain in the accept here and get the raw value of the log file in the storage which is the value users usually want to reduce the code for formatting.

accept?: "application/json" | "text/plain" | "*/*";

airflow/airflow-core/src/airflow/api_fastapi/core_api/routes/public/log.py

Lines 136 to 153 in 96c6daa

	ifaccept==Mimetype.JSONoraccept==Mimetype.ANY: # default
	logs, metadata=task_log_reader.read_log_chunks(ti, try_number, metadata)
	encoded_token=None
	ifnotmetadata.get("end_of_log", False):
	encoded_token=URLSafeSerializer(request.app.state.secret_key).dumps(metadata)
	returnTaskInstancesLogResponse.model_construct(continuation_token=encoded_token, content=logs)
	# text/plain, or something else we don't understand. Return raw log content
	# We need to exhaust the iterator before we can generate the continuation token.
	# We could improve this by making it a streaming/async response, and by then setting the header using
	# HTTP Trailers
	logs="".join(task_log_reader.read_log_stream(ti, try_number, metadata))
	headers=None
	ifnotmetadata.get("end_of_log", False):
	headers= {
	"Airflow-Continuation-Token": URLSafeSerializer(request.app.state.secret_key).dumps(metadata)
	}
	returnResponse(media_type="application/x-ndjson", content=logs, headers=headers)

[bbovenzi]

@aarochuk mind rebasing and addressing PR comments?

[aarochuk]

@bbovenzi I will make sure to make another push by the end of the week sorry it has taken so long

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

[bbovenzi]

@aarochuk are you still able to finish this PR?

[phanikumv]

@aarochuk just checking in to see if you need help with this PR?

[uranusjr]

Removing from 3.1.0 due to lack of response and the PR not in a working state. Please comment if anyone is interested in picking this up, we can add it back if possible.

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.