[FLINK-40555][runtime] Isolate unreadable JobResultStore entries per job

[XComp]

A corrupted JobResultStore file would have lead to a JobManager failover. The new implementation utilizes the new configuration parameter introduced with FLINK-40554 to skips parsing the entry and quarantines the file under a different name for manual investigation and cleanup.

What is the purpose of the change

• Extends functionality from FLINK-40554 to cover also corrupted JRS entry files.

Brief change log

• Adds error log handling for corrupted files which will be renamed

Verifying this change

• Unit tests added

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): no
• The serializers: no
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: yes
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? yes
• If yes, how is the feature documented? docs

[XComp]

See FLINK-40554 PR comment for closing remark.