Skip to content

Refine explore strategy, add prioritized sampling support; add DDQN example; add DQN test - #590

Merged
Jinyu-W merged 9 commits into
v0.3from
huoran/ddqn_and_prio_memory
May 23, 2023
Merged

Refine explore strategy, add prioritized sampling support; add DDQN example; add DQN test#590
Jinyu-W merged 9 commits into
v0.3from
huoran/ddqn_and_prio_memory

Conversation

@lihuoran

@lihuoranHuoran Li (lihuoran) commented May 6, 2023

Copy link
Copy Markdown
Contributor

Description

  • Refine the RL exploration strategy, prioritized sampling is supported now
  • Add DDQN example
  • Add DQN algorithm test, performance evaluation result appended to test part

Linked issue(s)/Pull request(s)

Type of Change

  • Non-breaking bug fix
  • Breaking bug fix
  • New feature
  • Test
  • Doc update
  • Docker update

Related Component

  • Simulation toolkit
  • RL toolkit
  • Distributed toolkit

Has Been Tested

  • OS:
    • Windows
    • Mac OS
    • Linux
  • Python version:
    • 3.7
    • 3.8
    • 3.9
  • Key information snapshot(s):

Needs Follow Up Actions

  • New release package
  • New docker image

Checklist

  • Add/update the related comments
  • Add/update the related tests
  • Add/update the related documentations
  • Update the dependent downstream modules usage

@lihuoran
Huoran Li (lihuoran) marked this pull request as ready for review May 12, 2023 03:40
Comment threadmaro/rl/exploration/strategies.py Outdated
**kwargs: Any,
) -> np.ndarray:
return np.array(
[act if np.random.random() > self._eps else np.random.randint(self._num_actions) for act in action],

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for batch operation?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add input and output description for newly added class/functions

Comment threadmaro/rl/exploration/strategies.py
Comment threadtests/rl/tasks/dqn/__init__.py Outdated
Comment threadtests/rl/tasks/dqn/__init__.py Outdated
Comment threadtests/rl/tasks/dqn/__init__.py
@Jinyu-W

Copy link
Copy Markdown
Contributor

Update the title and description of this PR

@Jinyu-W
Jinyu-Wforce-pushed the huoran/ddqn_and_prio_memory branch from 2811239 to a09750eCompareMay 22, 2023 06:36
Comment threadtests/rl/performance.md
@Jinyu-WJinyu-W changed the title Huoran/ddqn and prio memoryRefine explore strategy, add prioritized sampling support; add DDQN example; add DQN testMay 23, 2023
@Jinyu-W
Jinyu-W merged commit 607d3b6 into v0.3May 23, 2023
@Jinyu-W
Jinyu-W deleted the huoran/ddqn_and_prio_memory branch May 23, 2023 07:52
Jinyu-W added a commit that referenced this pull request Oct 27, 2023
* Refine explore strategy, add prioritized sampling support; add DDQN example; add DQN test (#590)
* Runnable. Should setup a benchmark and test performance.
* Refine logic
* Test DQN on GYM passed
* Refine explore strategy
* Minor
* Minor
* Add Dueling DQN in CIM scenario
* Resolve PR comments
* Add one more explanation
* fix env_sampler eval info list issue
* update version to 0.3.2a4
---------
Co-authored-by: Huoran Li <huoranli@microsoft.com>
Co-authored-by: Jinyu Wang <Wang.Jinyu@microsoft.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@lihuoran@Jinyu-W