[Enhancement] Enhancement of query profile: make enable_profile always true

[RoanHeNaN]

Search before asking

• I had searched in the issues and found no similar issues.

Description

We need to make global variable enable_profile always true.

Benefit:
profile of query that has bad performance can be seen directly without retry

To realize the goal, we need to refactor the current report profile mechanism, in which profile data is sent from BE to FE
during almost all the time the query is running, and profile resides in FE until there are too many profile, and this makes memory usage on FE explode.

Solution

The refactor process will be step by step:

• when enable_profile is set true, query will only report profile after it is finished(current mechanism is still kept until we finished all the refactor process)
• support real-time profile fetching
• support profile persistent
• remove old profile report code until we finished above

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[RoanHeNaN]

step 1: #33331

[RoanHeNaN]

Step 2: #33383

Actually, this pr just finished real-time profile fetching on FE, we need more work on BE. For example, profile should be updated during each time pipeline task is executed instead of only update profile on close stage.

[RoanHeNaN]

Step 3: #33690

[github-actions]

We're closing this issue because it hasn't been updated in a while.
If you'd like to revive this issue, please reopen it and feel free a maintainer to remove the Stale tag!