Uh oh!
There was an error while loading. Please reload this page.
Allow variables to be printed to STDOUT with airflow variables export. - #31854
Allow variables to be printed to STDOUT with airflow variables export.#31854vedantlodha wants to merge 7 commits into
Conversation
vedantlodha
commented
Jun 12, 2023
If by console, you mean the UI, that makes sense, however this allows the cli to do so as well, similar to connections. In the past i have had to switch between the cli and the UI for connections and variables. |
In stdout only. I was thinking if we could print in different formats like json/yaml etc But I see we do print in different formats for connection export like |
There was a problem hiding this comment.
Can we please use more meaningful variable names instead of qry, var, val?
There was a problem hiding this comment.
Fair point. I just refactored the helper function into this. Will update the patch.
There was a problem hiding this comment.
| exceptException: | |
| exceptjson.decoder.JSONDecodeError: |
There was a problem hiding this comment.
It's always a good idea to handle specific exceptions as close to the source as possible.
There was a problem hiding this comment.
This method doesn't really add much value, I would directly use fileio.name == "<stdout>" in line 96. Saves one lookup to a different method. :)
There was a problem hiding this comment.
Sure, this was just to maintain some consistency with connection_command.py's export. Will update the patch
vedantlodha
commented
Jun 13, 2023
Thanks for the reviews @utkarsharma2. Updated the patch. Please take a look when you have some time. |
There was a problem hiding this comment.
| query=session.query(Variable).all() | |
| data=json.JSONDecoder() | |
| forvariableinquery: | |
| try: | |
| value=data.decode(variable.val) | |
| exceptjson.decoder.JSONDecodeError: | |
| value=variable.val | |
| var_dict[variable.key] =value | |
| rows=session.query(Variable).all() | |
| data=json.JSONDecoder() | |
| forrowinrows: | |
| try: | |
| value=data.decode(row.val) | |
| exceptjson.decoder.JSONDecodeError: | |
| value=row.val | |
| var_dict[row.key] =value |
Maybe something like this?
There was a problem hiding this comment.
I think we can drop the all() as well since it’s only iterated once. Also why is the decoder called the data?
There was a problem hiding this comment.
Apologies for the delay, had been unwell lately.
Thanks for the inputs. Ive updated the patch.
Similar to connections, this change allows the cli to export the variables to the stdout using 'airflow variables export -' Fixesapache#31851
- handle JSONDecodeError instead of Exception
Uh oh!
There was an error while loading. Please reload this page.
| var_dict[row.key] = value | ||
| with args.file as f: | ||
| f.write(json.dumps(var_dict, sort_keys=True, indent=4)) |
There was a problem hiding this comment.
| f.write(json.dumps(var_dict, sort_keys=True, indent=4)) | |
| json.dump(f, var_dict, sort_keys=True, indent=4) |
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions. |
This change allows airflow cli to export variables to stdout similar to connections using
airflow variables export -.Fixes#31851
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.