Skip to content

V0.2 rl refinement task queue - #406

Merged
GQ.Chen (buptchan) merged 14 commits into
v0.2_rl_refinementfrom
v0.2_rl_refinement_taskq
Oct 29, 2021
Merged

V0.2 rl refinement task queue#406
GQ.Chen (buptchan) merged 14 commits into
v0.2_rl_refinementfrom
v0.2_rl_refinement_taskq

Conversation

@buptchan

@buptchanGQ.Chen (buptchan) commented Oct 9, 2021

Copy link
Copy Markdown
Contributor

Description

Add task_queue for data parallelism.
Through task queue, policies request gradient workers for each training iteration, instead of a fixed allocation.
WorkerAllocator is removed because it can be replaced by task queue.

Linked issue(s)/Pull request(s)

Type of Change

  • Non-breaking bug fix
  • Breaking bug fix
  • New feature
  • Test
  • Doc update
  • Docker update

Related Component

  • Simulation toolkit
  • RL toolkit
  • Distributed toolkit

Has Been Tested

  • OS:
    • Windows
    • Mac OS
    • Linux
  • Python version:
    • 3.6
    • 3.7
  • Key information snapshot(s):

Needs Follow Up Actions

  • New release package
  • New docker image

Checklist

  • Add/update the related comments
  • Add/update the related tests
  • Add/update the related documentations
  • Update the dependent downstream modules usage

ysqyang
ysqyang (ysqyang) reviewed Oct 13, 2021
View reviewed changes
Comment threadmaro/rl/learning/policy_manager.py Outdated
Comment threadmaro/rl/data_parallelism/task_queue.py
Comment threadmaro/rl/policy/ac.py
Comment threadmaro/rl/data_parallelism/task_queue.py
Comment threadmaro/rl/learning/policy_manager.py Outdated
Comment threadmaro/rl/learning/policy_manager.py Outdated
ysqyang
ysqyang (ysqyang) reviewed Oct 19, 2021
View reviewed changes
Comment threadexamples/rl/scripts/docker/docker_compose_yml.py Outdated
ysqyang
ysqyang (ysqyang) reviewed Oct 19, 2021
View reviewed changes
Comment threadexamples/rl/scripts/docker/docker_compose_yml.py Outdated
ysqyang
ysqyang (ysqyang) reviewed Oct 19, 2021
View reviewed changes
Comment threadmaro/rl/data_parallelism/__init__.py Outdated
ysqyang
ysqyang (ysqyang) reviewed Oct 19, 2021
View reviewed changes
Comment threadmaro/rl/workflows/policy_manager.py Outdated
ysqyang
ysqyang (ysqyang) reviewed Oct 19, 2021
View reviewed changes
Comment threadmaro/rl/learning/policy_manager.py Outdated
ysqyang
ysqyang (ysqyang) reviewed Oct 19, 2021
View reviewed changes
Comment threadmaro/rl/workflows/policy_manager.py Outdated
ysqyang
ysqyang (ysqyang) reviewed Oct 19, 2021
View reviewed changes
Comment threadmaro/rl/workflows/policy_host.py Outdated
ysqyang
ysqyang (ysqyang) reviewed Oct 19, 2021
View reviewed changes
Comment threadmaro/rl/workflows/policy_host.py Outdated
ysqyang
ysqyang (ysqyang) reviewed Oct 19, 2021
View reviewed changes
Comment threadmaro/rl/workflows/task_queue.py Outdated
ysqyang
ysqyang (ysqyang) reviewed Oct 19, 2021
View reviewed changes
Comment threadmaro/rl/workflows/task_queue.py Outdated
ysqyang
ysqyang (ysqyang) reviewed Oct 19, 2021
View reviewed changes
Comment threadmaro/rl/workflows/task_queue.py Outdated
ysqyang
ysqyang (ysqyang) reviewed Oct 19, 2021
View reviewed changes
Comment threadmaro/rl/workflows/task_queue.py Outdated
ysqyang
ysqyang (ysqyang) reviewed Oct 19, 2021
View reviewed changes
Comment threadmaro/rl/workflows/policy_manager.py Outdated
ysqyang
ysqyang (ysqyang) reviewed Oct 19, 2021
View reviewed changes
Comment threadmaro/rl/workflows/policy_manager.py Outdated
ysqyang
ysqyang (ysqyang) reviewed Oct 19, 2021
View reviewed changes
Comment threadmaro/rl/data_parallelism/task_queue.py Outdated
ysqyang
ysqyang (ysqyang) reviewed Oct 19, 2021
View reviewed changes
Comment threadmaro/rl/data_parallelism/task_queue.py Outdated
ysqyang
ysqyang (ysqyang) reviewed Oct 19, 2021
View reviewed changes
Comment threadmaro/rl/data_parallelism/task_queue.py Outdated
ysqyang
ysqyang (ysqyang) reviewed Oct 19, 2021
View reviewed changes
Comment threadmaro/rl/data_parallelism/task_queue.py Outdated
ysqyang
ysqyang (ysqyang) reviewed Oct 19, 2021
View reviewed changes
Comment threadmaro/rl/data_parallelism/task_queue.py Outdated
ysqyang
ysqyang (ysqyang) reviewed Oct 19, 2021
View reviewed changes
Comment threadmaro/rl/data_parallelism/task_queue.py Outdated
ysqyang
ysqyang (ysqyang) reviewed Oct 19, 2021
View reviewed changes
Comment threadmaro/rl/data_parallelism/task_queue.py Outdated
ysqyang
ysqyang (ysqyang) reviewed Oct 19, 2021
View reviewed changes

@ysqyangysqyang (ysqyang) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More comments

@buptchan
1.Merge data_parallel and num_workers into data_parallelism in config…
6677572
…; 2.Assign recently used workers as possible in task_queue.
buptchan
GQ.Chen (buptchan) commented Oct 20, 2021
View reviewed changes
Comment threadmaro/rl/data_parallelism/task_queue.py Outdated
Jinyu-W
Jinyu-W reviewed Oct 24, 2021
View reviewed changes
Comment threadmaro/rl/learning/policy_manager.py
Comment threadmaro/rl/workflows/task_queue.py
Comment threadmaro/rl/data_parallelism/task_queue.py Outdated
Comment threadmaro/rl/data_parallelism/task_queue.py
Comment threadmaro/rl/data_parallelism/task_queue.py Outdated
Comment threadmaro/rl/data_parallelism/task_queue.py
Jinyu-W
Jinyu-W reviewed Oct 25, 2021
View reviewed changes
Comment threadmaro/rl/data_parallelism/task_queue.py Outdated
Jinyu-W
Jinyu-W reviewed Oct 25, 2021
View reviewed changes
Comment threadmaro/rl/data_parallelism/task_queue.py
Jinyu-W
Jinyu-W reviewed Oct 25, 2021
View reviewed changes
Comment threadmaro/rl/data_parallelism/task_queue.py Outdated
Jinyu-W
Jinyu-W reviewed Oct 25, 2021
View reviewed changes
Comment threadmaro/rl/learning/policy_manager.py Outdated
buptchan
GQ.Chen (buptchan) commented Oct 26, 2021
View reviewed changes
Comment threadmaro/rl/data_parallelism/task_queue.py
@buptchan
GQ.Chen (buptchan) merged commit 1c2d51c into v0.2_rl_refinementOct 29, 2021
@lihuoranHuoran Li (lihuoran) mentioned this pull request Oct 29, 2021
Merged
@buptchanGQ.Chen (buptchan) mentioned this pull request Oct 29, 2021
Merged
20 tasks
@lihuoran
Huoran Li (lihuoran) deleted the v0.2_rl_refinement_taskq branch October 29, 2021 06:55
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

2 more reviewers
@Jinyu-WJinyu-WJinyu-W left review comments
@ysqyangysqyang (ysqyang)ysqyang approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@buptchan@ysqyang@Jinyu-W