StatRecorder Class by xinchen-yang · Pull Request #31 · RyanNavillus/Syllabus

xinchen-yang · 2024-04-02T19:12:11Z

No description provided.

RyanNavillus

Great work! Left some suggestions

RyanNavillus · 2024-04-03T04:42:46Z

+ idx = info.index(item)
+ multiprocessing_sync_wrapper_envs = envs.venv.venv.envs # extract envs of class MultiProcessingSyncWrapper from envs of class VecNormalize, which has access to task id
+ episode_task = multiprocessing_sync_wrapper_envs[idx]._latest_task
+ curriculum.update_on_episode(item["episode"]["r"], item["episode"]["l"], episode_task, args.env_id)


This isn't necessary, the curriculum sync wrapper will call this automatically with the correct data.

RyanNavillus · 2024-04-03T04:45:54Z

+ def __init__(self, task_space: TaskSpace):
+ """Initialize the StatRecorder"""
+
+ self.write_path = '/Users/allisonyang/Downloads'


Make this an initializer argument and let people configure it when they create their curriculum

RyanNavillus · 2024-04-03T04:50:21Z

+ self.num_tasks = self.task_space.num_tasks
+
+ self.records = {task: [] for task in self.tasks}
+ self.stats = {task: {} for task in self.tasks}


Instead of tracking the full list, track these efficiently. If we train for 10M episodes then it would be impossible to take the average of these lists. You can look up the running mean formulas, I think it's average_mean = ((average_mean * N) + new_mean) / (N+1) or something like that. There might be one for variance too

Alternatively, it would be good to provide an option for only saving the past N episodes, so rather than taking the average over all of training, it's an average over the past N episodes. I think some normalization schemes prefer that method because returns change during training. It would be good to provide both options (let the user choose and configure each)

RyanNavillus

I think you can simplify this code quite a bit, but it looks more efficient now!

RyanNavillus · 2024-04-19T03:00:44Z

+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_return_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_return_var", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_length_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_length_var", 0, step)


Please simplify this code, you shouldn't need all these if statements and repeated code. Also I think you can ignore the task_names feature for now, it's sort of half implemented, I'll need to fix it at some point

RyanNavillus · 2024-04-19T03:03:14Z

+ else:
+ N_past = len(self.records[episode_task])
+
+ self.stats[episode_task]['mean_r'] =round((self.stats[episode_task]['mean_r'] * N_past + episode_return) / (N_past + 1), 4)


We probably shouldn't round the saved values, we should only round them when logging or printing

RyanNavillus · 2024-04-19T03:04:55Z

+ "l": episode_length,
+ "env_id": env_id
+ })
+ self.records[episode_task] = self.records[episode_task][-self.calc_past_N:]


It would be more efficient to implement this with a separate queue for return, length, and id's https://docs.python.org/3/library/collections.html#collections.deque

RyanNavillus · 2024-04-19T03:06:02Z

+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_return_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_return_var", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_length_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_length_var", 0, step)


simplify this if else, theres a lot of repeated code. Also why are we logging 0 if there are no stats? We can probably just skip logging in that case

RyanNavillus · 2024-04-19T03:07:15Z

@@ -34,45 +31,96 @@ def record(self, episode_return: float, episode_length: int, episode_task, env_i
 """


If you use defaultdicts for the stats you can cut out a lot of code. Change:
self.stats = {task: {} for task in self.tasks}
to:
self.stats = {task: defaultdict(float) for task in self.tasks}

from collections import defaultdict

RyanNavillus

Thanks for simplifying the code, looks much better!

RyanNavillus · 2024-04-19T20:54:22Z

+ self.episode_lengths[episode_task].append(episode_length)
+ self.env_ids[episode_task].append(env_id)
+
+ self.stats[episode_task]['mean_r'] = np.mean(list(self.episode_returns[episode_task])[-self.calc_past_N:]) # I am not sure whether there is a more efficient way to slice to deque. I temperorily convert it to a list then slice it, which should cost O(n)


You shouldn't need to slice a deque at all, it should automatically drop elements when it goes past keep_last_n. Check the documentation for it

Async plr

Co-authored-by: Ryan Sullivan <ryanpnavillus@gmail.com>

…episodes); implemented the log_metrics function of the StatRecorder class for visualization on weights & biases

… for the StatRecorder class

xinchen-yang · 2024-04-02T19:12:11Z

No description provided.

RyanNavillus

Great work! Left some suggestions

RyanNavillus · 2024-04-03T04:42:46Z

+ idx = info.index(item)
+ multiprocessing_sync_wrapper_envs = envs.venv.venv.envs # extract envs of class MultiProcessingSyncWrapper from envs of class VecNormalize, which has access to task id
+ episode_task = multiprocessing_sync_wrapper_envs[idx]._latest_task
+ curriculum.update_on_episode(item["episode"]["r"], item["episode"]["l"], episode_task, args.env_id)


This isn't necessary, the curriculum sync wrapper will call this automatically with the correct data.

RyanNavillus · 2024-04-03T04:45:54Z

+ def __init__(self, task_space: TaskSpace):
+ """Initialize the StatRecorder"""
+
+ self.write_path = '/Users/allisonyang/Downloads'


Make this an initializer argument and let people configure it when they create their curriculum

RyanNavillus · 2024-04-03T04:50:21Z

+ self.num_tasks = self.task_space.num_tasks
+
+ self.records = {task: [] for task in self.tasks}
+ self.stats = {task: {} for task in self.tasks}


Instead of tracking the full list, track these efficiently. If we train for 10M episodes then it would be impossible to take the average of these lists. You can look up the running mean formulas, I think it's average_mean = ((average_mean * N) + new_mean) / (N+1) or something like that. There might be one for variance too

Alternatively, it would be good to provide an option for only saving the past N episodes, so rather than taking the average over all of training, it's an average over the past N episodes. I think some normalization schemes prefer that method because returns change during training. It would be good to provide both options (let the user choose and configure each)

RyanNavillus

I think you can simplify this code quite a bit, but it looks more efficient now!

RyanNavillus · 2024-04-19T03:00:44Z

+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_return_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_return_var", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_length_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_length_var", 0, step)


Please simplify this code, you shouldn't need all these if statements and repeated code. Also I think you can ignore the task_names feature for now, it's sort of half implemented, I'll need to fix it at some point

RyanNavillus · 2024-04-19T03:03:14Z

+ else:
+ N_past = len(self.records[episode_task])
+
+ self.stats[episode_task]['mean_r'] =round((self.stats[episode_task]['mean_r'] * N_past + episode_return) / (N_past + 1), 4)


We probably shouldn't round the saved values, we should only round them when logging or printing

RyanNavillus · 2024-04-19T03:04:55Z

+ "l": episode_length,
+ "env_id": env_id
+ })
+ self.records[episode_task] = self.records[episode_task][-self.calc_past_N:]


It would be more efficient to implement this with a separate queue for return, length, and id's https://docs.python.org/3/library/collections.html#collections.deque

RyanNavillus · 2024-04-19T03:06:02Z

+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_return_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_return_var", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_length_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_length_var", 0, step)


simplify this if else, theres a lot of repeated code. Also why are we logging 0 if there are no stats? We can probably just skip logging in that case

RyanNavillus · 2024-04-19T03:07:15Z

@@ -34,45 +31,96 @@ def record(self, episode_return: float, episode_length: int, episode_task, env_i
 """


If you use defaultdicts for the stats you can cut out a lot of code. Change:
self.stats = {task: {} for task in self.tasks}
to:
self.stats = {task: defaultdict(float) for task in self.tasks}

from collections import defaultdict

RyanNavillus

Thanks for simplifying the code, looks much better!

RyanNavillus · 2024-04-19T20:54:22Z

+ self.episode_lengths[episode_task].append(episode_length)
+ self.env_ids[episode_task].append(env_id)
+
+ self.stats[episode_task]['mean_r'] = np.mean(list(self.episode_returns[episode_task])[-self.calc_past_N:]) # I am not sure whether there is a more efficient way to slice to deque. I temperorily convert it to a list then slice it, which should cost O(n)


You shouldn't need to slice a deque at all, it should automatically drop elements when it goes past keep_last_n. Check the documentation for it

Async plr

Co-authored-by: Ryan Sullivan <ryanpnavillus@gmail.com>

…episodes); implemented the log_metrics function of the StatRecorder class for visualization on weights & biases

… for the StatRecorder class

xinchen-yang · 2024-04-02T19:12:11Z

No description provided.

RyanNavillus

Great work! Left some suggestions

RyanNavillus · 2024-04-03T04:42:46Z

+ idx = info.index(item)
+ multiprocessing_sync_wrapper_envs = envs.venv.venv.envs # extract envs of class MultiProcessingSyncWrapper from envs of class VecNormalize, which has access to task id
+ episode_task = multiprocessing_sync_wrapper_envs[idx]._latest_task
+ curriculum.update_on_episode(item["episode"]["r"], item["episode"]["l"], episode_task, args.env_id)


This isn't necessary, the curriculum sync wrapper will call this automatically with the correct data.

RyanNavillus · 2024-04-03T04:45:54Z

+ def __init__(self, task_space: TaskSpace):
+ """Initialize the StatRecorder"""
+
+ self.write_path = '/Users/allisonyang/Downloads'


Make this an initializer argument and let people configure it when they create their curriculum

RyanNavillus · 2024-04-03T04:50:21Z

+ self.num_tasks = self.task_space.num_tasks
+
+ self.records = {task: [] for task in self.tasks}
+ self.stats = {task: {} for task in self.tasks}


Instead of tracking the full list, track these efficiently. If we train for 10M episodes then it would be impossible to take the average of these lists. You can look up the running mean formulas, I think it's average_mean = ((average_mean * N) + new_mean) / (N+1) or something like that. There might be one for variance too

Alternatively, it would be good to provide an option for only saving the past N episodes, so rather than taking the average over all of training, it's an average over the past N episodes. I think some normalization schemes prefer that method because returns change during training. It would be good to provide both options (let the user choose and configure each)

RyanNavillus

I think you can simplify this code quite a bit, but it looks more efficient now!

RyanNavillus · 2024-04-19T03:00:44Z

+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_return_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_return_var", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_length_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_length_var", 0, step)


Please simplify this code, you shouldn't need all these if statements and repeated code. Also I think you can ignore the task_names feature for now, it's sort of half implemented, I'll need to fix it at some point

RyanNavillus · 2024-04-19T03:03:14Z

+ else:
+ N_past = len(self.records[episode_task])
+
+ self.stats[episode_task]['mean_r'] =round((self.stats[episode_task]['mean_r'] * N_past + episode_return) / (N_past + 1), 4)


We probably shouldn't round the saved values, we should only round them when logging or printing

RyanNavillus · 2024-04-19T03:04:55Z

+ "l": episode_length,
+ "env_id": env_id
+ })
+ self.records[episode_task] = self.records[episode_task][-self.calc_past_N:]


It would be more efficient to implement this with a separate queue for return, length, and id's https://docs.python.org/3/library/collections.html#collections.deque

RyanNavillus · 2024-04-19T03:06:02Z

+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_return_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_return_var", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_length_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_length_var", 0, step)


simplify this if else, theres a lot of repeated code. Also why are we logging 0 if there are no stats? We can probably just skip logging in that case

RyanNavillus · 2024-04-19T03:07:15Z

@@ -34,45 +31,96 @@ def record(self, episode_return: float, episode_length: int, episode_task, env_i
 """


If you use defaultdicts for the stats you can cut out a lot of code. Change:
self.stats = {task: {} for task in self.tasks}
to:
self.stats = {task: defaultdict(float) for task in self.tasks}

from collections import defaultdict

RyanNavillus

Thanks for simplifying the code, looks much better!

RyanNavillus · 2024-04-19T20:54:22Z

+ self.episode_lengths[episode_task].append(episode_length)
+ self.env_ids[episode_task].append(env_id)
+
+ self.stats[episode_task]['mean_r'] = np.mean(list(self.episode_returns[episode_task])[-self.calc_past_N:]) # I am not sure whether there is a more efficient way to slice to deque. I temperorily convert it to a list then slice it, which should cost O(n)


You shouldn't need to slice a deque at all, it should automatically drop elements when it goes past keep_last_n. Check the documentation for it

Async plr

Co-authored-by: Ryan Sullivan <ryanpnavillus@gmail.com>

…episodes); implemented the log_metrics function of the StatRecorder class for visualization on weights & biases

… for the StatRecorder class

xinchen-yang · 2024-04-02T19:12:11Z

No description provided.

RyanNavillus

Great work! Left some suggestions

RyanNavillus · 2024-04-03T04:42:46Z

+ idx = info.index(item)
+ multiprocessing_sync_wrapper_envs = envs.venv.venv.envs # extract envs of class MultiProcessingSyncWrapper from envs of class VecNormalize, which has access to task id
+ episode_task = multiprocessing_sync_wrapper_envs[idx]._latest_task
+ curriculum.update_on_episode(item["episode"]["r"], item["episode"]["l"], episode_task, args.env_id)


This isn't necessary, the curriculum sync wrapper will call this automatically with the correct data.

RyanNavillus · 2024-04-03T04:45:54Z

+ def __init__(self, task_space: TaskSpace):
+ """Initialize the StatRecorder"""
+
+ self.write_path = '/Users/allisonyang/Downloads'


Make this an initializer argument and let people configure it when they create their curriculum

RyanNavillus · 2024-04-03T04:50:21Z

+ self.num_tasks = self.task_space.num_tasks
+
+ self.records = {task: [] for task in self.tasks}
+ self.stats = {task: {} for task in self.tasks}


Instead of tracking the full list, track these efficiently. If we train for 10M episodes then it would be impossible to take the average of these lists. You can look up the running mean formulas, I think it's average_mean = ((average_mean * N) + new_mean) / (N+1) or something like that. There might be one for variance too

Alternatively, it would be good to provide an option for only saving the past N episodes, so rather than taking the average over all of training, it's an average over the past N episodes. I think some normalization schemes prefer that method because returns change during training. It would be good to provide both options (let the user choose and configure each)

RyanNavillus

I think you can simplify this code quite a bit, but it looks more efficient now!

RyanNavillus · 2024-04-19T03:00:44Z

+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_return_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_return_var", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_length_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_length_var", 0, step)


Please simplify this code, you shouldn't need all these if statements and repeated code. Also I think you can ignore the task_names feature for now, it's sort of half implemented, I'll need to fix it at some point

RyanNavillus · 2024-04-19T03:03:14Z

+ else:
+ N_past = len(self.records[episode_task])
+
+ self.stats[episode_task]['mean_r'] =round((self.stats[episode_task]['mean_r'] * N_past + episode_return) / (N_past + 1), 4)


We probably shouldn't round the saved values, we should only round them when logging or printing

RyanNavillus · 2024-04-19T03:04:55Z

+ "l": episode_length,
+ "env_id": env_id
+ })
+ self.records[episode_task] = self.records[episode_task][-self.calc_past_N:]


It would be more efficient to implement this with a separate queue for return, length, and id's https://docs.python.org/3/library/collections.html#collections.deque

RyanNavillus · 2024-04-19T03:06:02Z

+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_return_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_return_var", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_length_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_length_var", 0, step)


simplify this if else, theres a lot of repeated code. Also why are we logging 0 if there are no stats? We can probably just skip logging in that case

RyanNavillus · 2024-04-19T03:07:15Z

@@ -34,45 +31,96 @@ def record(self, episode_return: float, episode_length: int, episode_task, env_i
 """


If you use defaultdicts for the stats you can cut out a lot of code. Change:
self.stats = {task: {} for task in self.tasks}
to:
self.stats = {task: defaultdict(float) for task in self.tasks}

from collections import defaultdict

RyanNavillus

Thanks for simplifying the code, looks much better!

RyanNavillus · 2024-04-19T20:54:22Z

+ self.episode_lengths[episode_task].append(episode_length)
+ self.env_ids[episode_task].append(env_id)
+
+ self.stats[episode_task]['mean_r'] = np.mean(list(self.episode_returns[episode_task])[-self.calc_past_N:]) # I am not sure whether there is a more efficient way to slice to deque. I temperorily convert it to a list then slice it, which should cost O(n)


You shouldn't need to slice a deque at all, it should automatically drop elements when it goes past keep_last_n. Check the documentation for it

Async plr

Co-authored-by: Ryan Sullivan <ryanpnavillus@gmail.com>

…episodes); implemented the log_metrics function of the StatRecorder class for visualization on weights & biases

… for the StatRecorder class

xinchen-yang · 2024-04-02T19:12:11Z

No description provided.

RyanNavillus

Great work! Left some suggestions

RyanNavillus · 2024-04-03T04:42:46Z

+ idx = info.index(item)
+ multiprocessing_sync_wrapper_envs = envs.venv.venv.envs # extract envs of class MultiProcessingSyncWrapper from envs of class VecNormalize, which has access to task id
+ episode_task = multiprocessing_sync_wrapper_envs[idx]._latest_task
+ curriculum.update_on_episode(item["episode"]["r"], item["episode"]["l"], episode_task, args.env_id)


This isn't necessary, the curriculum sync wrapper will call this automatically with the correct data.

RyanNavillus · 2024-04-03T04:45:54Z

+ def __init__(self, task_space: TaskSpace):
+ """Initialize the StatRecorder"""
+
+ self.write_path = '/Users/allisonyang/Downloads'


Make this an initializer argument and let people configure it when they create their curriculum

RyanNavillus · 2024-04-03T04:50:21Z

+ self.num_tasks = self.task_space.num_tasks
+
+ self.records = {task: [] for task in self.tasks}
+ self.stats = {task: {} for task in self.tasks}


Instead of tracking the full list, track these efficiently. If we train for 10M episodes then it would be impossible to take the average of these lists. You can look up the running mean formulas, I think it's average_mean = ((average_mean * N) + new_mean) / (N+1) or something like that. There might be one for variance too

Alternatively, it would be good to provide an option for only saving the past N episodes, so rather than taking the average over all of training, it's an average over the past N episodes. I think some normalization schemes prefer that method because returns change during training. It would be good to provide both options (let the user choose and configure each)

RyanNavillus

I think you can simplify this code quite a bit, but it looks more efficient now!

RyanNavillus · 2024-04-19T03:00:44Z

+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_return_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_return_var", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_length_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_length_var", 0, step)


Please simplify this code, you shouldn't need all these if statements and repeated code. Also I think you can ignore the task_names feature for now, it's sort of half implemented, I'll need to fix it at some point

RyanNavillus · 2024-04-19T03:03:14Z

+ else:
+ N_past = len(self.records[episode_task])
+
+ self.stats[episode_task]['mean_r'] =round((self.stats[episode_task]['mean_r'] * N_past + episode_return) / (N_past + 1), 4)


We probably shouldn't round the saved values, we should only round them when logging or printing

RyanNavillus · 2024-04-19T03:04:55Z

+ "l": episode_length,
+ "env_id": env_id
+ })
+ self.records[episode_task] = self.records[episode_task][-self.calc_past_N:]


It would be more efficient to implement this with a separate queue for return, length, and id's https://docs.python.org/3/library/collections.html#collections.deque

RyanNavillus · 2024-04-19T03:06:02Z

+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_return_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_return_var", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_length_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_length_var", 0, step)


simplify this if else, theres a lot of repeated code. Also why are we logging 0 if there are no stats? We can probably just skip logging in that case

RyanNavillus · 2024-04-19T03:07:15Z

@@ -34,45 +31,96 @@ def record(self, episode_return: float, episode_length: int, episode_task, env_i
 """


If you use defaultdicts for the stats you can cut out a lot of code. Change:
self.stats = {task: {} for task in self.tasks}
to:
self.stats = {task: defaultdict(float) for task in self.tasks}

from collections import defaultdict

RyanNavillus

Thanks for simplifying the code, looks much better!

RyanNavillus · 2024-04-19T20:54:22Z

+ self.episode_lengths[episode_task].append(episode_length)
+ self.env_ids[episode_task].append(env_id)
+
+ self.stats[episode_task]['mean_r'] = np.mean(list(self.episode_returns[episode_task])[-self.calc_past_N:]) # I am not sure whether there is a more efficient way to slice to deque. I temperorily convert it to a list then slice it, which should cost O(n)


You shouldn't need to slice a deque at all, it should automatically drop elements when it goes past keep_last_n. Check the documentation for it

Async plr

Co-authored-by: Ryan Sullivan <ryanpnavillus@gmail.com>

…episodes); implemented the log_metrics function of the StatRecorder class for visualization on weights & biases

… for the StatRecorder class

xinchen-yang · 2024-04-02T19:12:11Z

No description provided.

RyanNavillus

Great work! Left some suggestions

RyanNavillus · 2024-04-03T04:42:46Z

+ idx = info.index(item)
+ multiprocessing_sync_wrapper_envs = envs.venv.venv.envs # extract envs of class MultiProcessingSyncWrapper from envs of class VecNormalize, which has access to task id
+ episode_task = multiprocessing_sync_wrapper_envs[idx]._latest_task
+ curriculum.update_on_episode(item["episode"]["r"], item["episode"]["l"], episode_task, args.env_id)


This isn't necessary, the curriculum sync wrapper will call this automatically with the correct data.

RyanNavillus · 2024-04-03T04:45:54Z

+ def __init__(self, task_space: TaskSpace):
+ """Initialize the StatRecorder"""
+
+ self.write_path = '/Users/allisonyang/Downloads'


Make this an initializer argument and let people configure it when they create their curriculum

RyanNavillus · 2024-04-03T04:50:21Z

+ self.num_tasks = self.task_space.num_tasks
+
+ self.records = {task: [] for task in self.tasks}
+ self.stats = {task: {} for task in self.tasks}


Instead of tracking the full list, track these efficiently. If we train for 10M episodes then it would be impossible to take the average of these lists. You can look up the running mean formulas, I think it's average_mean = ((average_mean * N) + new_mean) / (N+1) or something like that. There might be one for variance too

Alternatively, it would be good to provide an option for only saving the past N episodes, so rather than taking the average over all of training, it's an average over the past N episodes. I think some normalization schemes prefer that method because returns change during training. It would be good to provide both options (let the user choose and configure each)

RyanNavillus

I think you can simplify this code quite a bit, but it looks more efficient now!

RyanNavillus · 2024-04-19T03:00:44Z

+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_return_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_return_var", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_length_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_length_var", 0, step)


Please simplify this code, you shouldn't need all these if statements and repeated code. Also I think you can ignore the task_names feature for now, it's sort of half implemented, I'll need to fix it at some point

RyanNavillus · 2024-04-19T03:03:14Z

+ else:
+ N_past = len(self.records[episode_task])
+
+ self.stats[episode_task]['mean_r'] =round((self.stats[episode_task]['mean_r'] * N_past + episode_return) / (N_past + 1), 4)


We probably shouldn't round the saved values, we should only round them when logging or printing

RyanNavillus · 2024-04-19T03:04:55Z

+ "l": episode_length,
+ "env_id": env_id
+ })
+ self.records[episode_task] = self.records[episode_task][-self.calc_past_N:]


It would be more efficient to implement this with a separate queue for return, length, and id's https://docs.python.org/3/library/collections.html#collections.deque

RyanNavillus · 2024-04-19T03:06:02Z

+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_return_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_return_var", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_length_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_length_var", 0, step)


simplify this if else, theres a lot of repeated code. Also why are we logging 0 if there are no stats? We can probably just skip logging in that case

RyanNavillus · 2024-04-19T03:07:15Z

@@ -34,45 +31,96 @@ def record(self, episode_return: float, episode_length: int, episode_task, env_i
 """


If you use defaultdicts for the stats you can cut out a lot of code. Change:
self.stats = {task: {} for task in self.tasks}
to:
self.stats = {task: defaultdict(float) for task in self.tasks}

from collections import defaultdict

RyanNavillus

Thanks for simplifying the code, looks much better!

RyanNavillus · 2024-04-19T20:54:22Z

+ self.episode_lengths[episode_task].append(episode_length)
+ self.env_ids[episode_task].append(env_id)
+
+ self.stats[episode_task]['mean_r'] = np.mean(list(self.episode_returns[episode_task])[-self.calc_past_N:]) # I am not sure whether there is a more efficient way to slice to deque. I temperorily convert it to a list then slice it, which should cost O(n)


You shouldn't need to slice a deque at all, it should automatically drop elements when it goes past keep_last_n. Check the documentation for it

Async plr

Co-authored-by: Ryan Sullivan <ryanpnavillus@gmail.com>

…episodes); implemented the log_metrics function of the StatRecorder class for visualization on weights & biases

… for the StatRecorder class

xinchen-yang · 2024-04-02T19:12:11Z

No description provided.

RyanNavillus

Great work! Left some suggestions

RyanNavillus · 2024-04-03T04:42:46Z

+ idx = info.index(item)
+ multiprocessing_sync_wrapper_envs = envs.venv.venv.envs # extract envs of class MultiProcessingSyncWrapper from envs of class VecNormalize, which has access to task id
+ episode_task = multiprocessing_sync_wrapper_envs[idx]._latest_task
+ curriculum.update_on_episode(item["episode"]["r"], item["episode"]["l"], episode_task, args.env_id)


This isn't necessary, the curriculum sync wrapper will call this automatically with the correct data.

RyanNavillus · 2024-04-03T04:45:54Z

+ def __init__(self, task_space: TaskSpace):
+ """Initialize the StatRecorder"""
+
+ self.write_path = '/Users/allisonyang/Downloads'


Make this an initializer argument and let people configure it when they create their curriculum

RyanNavillus · 2024-04-03T04:50:21Z

+ self.num_tasks = self.task_space.num_tasks
+
+ self.records = {task: [] for task in self.tasks}
+ self.stats = {task: {} for task in self.tasks}


Instead of tracking the full list, track these efficiently. If we train for 10M episodes then it would be impossible to take the average of these lists. You can look up the running mean formulas, I think it's average_mean = ((average_mean * N) + new_mean) / (N+1) or something like that. There might be one for variance too

Alternatively, it would be good to provide an option for only saving the past N episodes, so rather than taking the average over all of training, it's an average over the past N episodes. I think some normalization schemes prefer that method because returns change during training. It would be good to provide both options (let the user choose and configure each)

RyanNavillus

I think you can simplify this code quite a bit, but it looks more efficient now!

RyanNavillus · 2024-04-19T03:00:44Z

+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_return_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_return_var", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_length_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_length_var", 0, step)


Please simplify this code, you shouldn't need all these if statements and repeated code. Also I think you can ignore the task_names feature for now, it's sort of half implemented, I'll need to fix it at some point

RyanNavillus · 2024-04-19T03:03:14Z

+ else:
+ N_past = len(self.records[episode_task])
+
+ self.stats[episode_task]['mean_r'] =round((self.stats[episode_task]['mean_r'] * N_past + episode_return) / (N_past + 1), 4)


We probably shouldn't round the saved values, we should only round them when logging or printing

RyanNavillus · 2024-04-19T03:04:55Z

+ "l": episode_length,
+ "env_id": env_id
+ })
+ self.records[episode_task] = self.records[episode_task][-self.calc_past_N:]


It would be more efficient to implement this with a separate queue for return, length, and id's https://docs.python.org/3/library/collections.html#collections.deque

RyanNavillus · 2024-04-19T03:06:02Z

+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_return_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_return_var", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_length_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_length_var", 0, step)


simplify this if else, theres a lot of repeated code. Also why are we logging 0 if there are no stats? We can probably just skip logging in that case

RyanNavillus · 2024-04-19T03:07:15Z

@@ -34,45 +31,96 @@ def record(self, episode_return: float, episode_length: int, episode_task, env_i
 """


If you use defaultdicts for the stats you can cut out a lot of code. Change:
self.stats = {task: {} for task in self.tasks}
to:
self.stats = {task: defaultdict(float) for task in self.tasks}

from collections import defaultdict

RyanNavillus

Thanks for simplifying the code, looks much better!

RyanNavillus · 2024-04-19T20:54:22Z

+ self.episode_lengths[episode_task].append(episode_length)
+ self.env_ids[episode_task].append(env_id)
+
+ self.stats[episode_task]['mean_r'] = np.mean(list(self.episode_returns[episode_task])[-self.calc_past_N:]) # I am not sure whether there is a more efficient way to slice to deque. I temperorily convert it to a list then slice it, which should cost O(n)


You shouldn't need to slice a deque at all, it should automatically drop elements when it goes past keep_last_n. Check the documentation for it

Async plr

Co-authored-by: Ryan Sullivan <ryanpnavillus@gmail.com>

…episodes); implemented the log_metrics function of the StatRecorder class for visualization on weights & biases

… for the StatRecorder class

xinchen-yang · 2024-04-02T19:12:11Z

No description provided.

RyanNavillus

Great work! Left some suggestions

RyanNavillus · 2024-04-03T04:42:46Z

+ idx = info.index(item)
+ multiprocessing_sync_wrapper_envs = envs.venv.venv.envs # extract envs of class MultiProcessingSyncWrapper from envs of class VecNormalize, which has access to task id
+ episode_task = multiprocessing_sync_wrapper_envs[idx]._latest_task
+ curriculum.update_on_episode(item["episode"]["r"], item["episode"]["l"], episode_task, args.env_id)


This isn't necessary, the curriculum sync wrapper will call this automatically with the correct data.

RyanNavillus · 2024-04-03T04:45:54Z

+ def __init__(self, task_space: TaskSpace):
+ """Initialize the StatRecorder"""
+
+ self.write_path = '/Users/allisonyang/Downloads'


Make this an initializer argument and let people configure it when they create their curriculum

RyanNavillus · 2024-04-03T04:50:21Z

+ self.num_tasks = self.task_space.num_tasks
+
+ self.records = {task: [] for task in self.tasks}
+ self.stats = {task: {} for task in self.tasks}


Instead of tracking the full list, track these efficiently. If we train for 10M episodes then it would be impossible to take the average of these lists. You can look up the running mean formulas, I think it's average_mean = ((average_mean * N) + new_mean) / (N+1) or something like that. There might be one for variance too

Alternatively, it would be good to provide an option for only saving the past N episodes, so rather than taking the average over all of training, it's an average over the past N episodes. I think some normalization schemes prefer that method because returns change during training. It would be good to provide both options (let the user choose and configure each)

RyanNavillus

I think you can simplify this code quite a bit, but it looks more efficient now!

RyanNavillus · 2024-04-19T03:00:44Z

+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_return_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_return_var", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_length_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{idx}_episode_length_var", 0, step)


Please simplify this code, you shouldn't need all these if statements and repeated code. Also I think you can ignore the task_names feature for now, it's sort of half implemented, I'll need to fix it at some point

RyanNavillus · 2024-04-19T03:03:14Z

+ else:
+ N_past = len(self.records[episode_task])
+
+ self.stats[episode_task]['mean_r'] =round((self.stats[episode_task]['mean_r'] * N_past + episode_return) / (N_past + 1), 4)


We probably shouldn't round the saved values, we should only round them when logging or printing

RyanNavillus · 2024-04-19T03:04:55Z

+ "l": episode_length,
+ "env_id": env_id
+ })
+ self.records[episode_task] = self.records[episode_task][-self.calc_past_N:]


It would be more efficient to implement this with a separate queue for return, length, and id's https://docs.python.org/3/library/collections.html#collections.deque

RyanNavillus · 2024-04-19T03:06:02Z

+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_return_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_return_var", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_length_mean", 0, step)
+ writer.add_scalar(f"stats_per_task/task_{self.task_space.task_name(idx)}_episode_length_var", 0, step)


simplify this if else, theres a lot of repeated code. Also why are we logging 0 if there are no stats? We can probably just skip logging in that case

RyanNavillus · 2024-04-19T03:07:15Z

@@ -34,45 +31,96 @@ def record(self, episode_return: float, episode_length: int, episode_task, env_i
 """


If you use defaultdicts for the stats you can cut out a lot of code. Change:
self.stats = {task: {} for task in self.tasks}
to:
self.stats = {task: defaultdict(float) for task in self.tasks}

from collections import defaultdict

RyanNavillus

Thanks for simplifying the code, looks much better!

RyanNavillus · 2024-04-19T20:54:22Z

+ self.episode_lengths[episode_task].append(episode_length)
+ self.env_ids[episode_task].append(env_id)
+
+ self.stats[episode_task]['mean_r'] = np.mean(list(self.episode_returns[episode_task])[-self.calc_past_N:]) # I am not sure whether there is a more efficient way to slice to deque. I temperorily convert it to a list then slice it, which should cost O(n)


You shouldn't need to slice a deque at all, it should automatically drop elements when it goes past keep_last_n. Check the documentation for it

Async plr

Co-authored-by: Ryan Sullivan <ryanpnavillus@gmail.com>

…episodes); implemented the log_metrics function of the StatRecorder class for visualization on weights & biases

… for the StatRecorder class

RyanNavillus reviewed Apr 3, 2024
View reviewed changes

RyanNavillus reviewed Apr 19, 2024
View reviewed changes

xinchen-yang force-pushed the main branch from 31d2cfe to d721bf0Compare April 19, 2024 21:31

RyanNavillusand others added 13 commits April 22, 2024 03:08

Merge pull requestRyanNavillus#34from RyanNavillus/async-plr
63dc8f6
Async plr

Remove unused eval
78f450a

merge in upstram
9370a1c

minor changes
c470517

merge in upstream
4158368

Update syllabus/core/stat_recorder.py
d04b6a7
Co-authored-by: Ryan Sullivan <ryanpnavillus@gmail.com>

implemented efficient reward normalization (running average & last N …
d43deca
…episodes); implemented the log_metrics function of the StatRecorder class for visualization on weights & biases

Simply StatRecorder code, use deque and defaultdict structure
cd9494b

remove keep_past_n
a6de4f9

implemented the normalize function of stat_recorder, write test cases…
1e6938d
… for the StatRecorder class

syllabus/tests/__init__
c29887e

test cases for the StatRecorder class
a00f90b

merge in upstream
4441ba3

xinchen-yang force-pushed the main branch from 63da604 to 4441ba3Compare April 24, 2024 03:33

xinchen-yang added 2 commits April 24, 2024 13:13

access task_id info
d17a690

add info['task'] to reset, api for the normalize func
9f68e1b

RyanNavillus changed the base branch from main to stat-recorderApril 26, 2024 07:27

RyanNavillus reviewed Apr 3, 2024
View reviewed changes

RyanNavillus reviewed Apr 19, 2024
View reviewed changes

xinchen-yang force-pushed the main branch from 31d2cfe to d721bf0Compare April 19, 2024 21:31

RyanNavillusand others added 13 commits April 22, 2024 03:08

Merge pull requestRyanNavillus#34from RyanNavillus/async-plr
63dc8f6
Async plr

Remove unused eval
78f450a

merge in upstram
9370a1c

minor changes
c470517

merge in upstream
4158368

Update syllabus/core/stat_recorder.py
d04b6a7
Co-authored-by: Ryan Sullivan <ryanpnavillus@gmail.com>

implemented efficient reward normalization (running average & last N …
d43deca
…episodes); implemented the log_metrics function of the StatRecorder class for visualization on weights & biases

Simply StatRecorder code, use deque and defaultdict structure
cd9494b

remove keep_past_n
a6de4f9

implemented the normalize function of stat_recorder, write test cases…
1e6938d
… for the StatRecorder class

syllabus/tests/__init__
c29887e

test cases for the StatRecorder class
a00f90b

merge in upstream
4441ba3

xinchen-yang force-pushed the main branch from 63da604 to 4441ba3Compare April 24, 2024 03:33

xinchen-yang added 2 commits April 24, 2024 13:13

access task_id info
d17a690

add info['task'] to reset, api for the normalize func
9f68e1b

RyanNavillus changed the base branch from main to stat-recorderApril 26, 2024 07:27

RyanNavillus reviewed Apr 3, 2024
View reviewed changes

RyanNavillus reviewed Apr 19, 2024
View reviewed changes

xinchen-yang force-pushed the main branch from 31d2cfe to d721bf0Compare April 19, 2024 21:31

RyanNavillusand others added 13 commits April 22, 2024 03:08

Merge pull requestRyanNavillus#34from RyanNavillus/async-plr
63dc8f6
Async plr

Remove unused eval
78f450a

merge in upstram
9370a1c

minor changes
c470517

merge in upstream
4158368

Update syllabus/core/stat_recorder.py
d04b6a7
Co-authored-by: Ryan Sullivan <ryanpnavillus@gmail.com>

implemented efficient reward normalization (running average & last N …
d43deca
…episodes); implemented the log_metrics function of the StatRecorder class for visualization on weights & biases

Simply StatRecorder code, use deque and defaultdict structure
cd9494b

remove keep_past_n
a6de4f9

implemented the normalize function of stat_recorder, write test cases…
1e6938d
… for the StatRecorder class

syllabus/tests/__init__
c29887e

test cases for the StatRecorder class
a00f90b

merge in upstream
4441ba3

xinchen-yang force-pushed the main branch from 63da604 to 4441ba3Compare April 24, 2024 03:33

xinchen-yang added 2 commits April 24, 2024 13:13

access task_id info
d17a690

add info['task'] to reset, api for the normalize func
9f68e1b

RyanNavillus changed the base branch from main to stat-recorderApril 26, 2024 07:27

RyanNavillus reviewed Apr 3, 2024
View reviewed changes

RyanNavillus reviewed Apr 19, 2024
View reviewed changes

xinchen-yang force-pushed the main branch from 31d2cfe to d721bf0Compare April 19, 2024 21:31

RyanNavillusand others added 13 commits April 22, 2024 03:08

Merge pull requestRyanNavillus#34from RyanNavillus/async-plr
63dc8f6
Async plr

Remove unused eval
78f450a

merge in upstram
9370a1c

minor changes
c470517

merge in upstream
4158368

Update syllabus/core/stat_recorder.py
d04b6a7
Co-authored-by: Ryan Sullivan <ryanpnavillus@gmail.com>

implemented efficient reward normalization (running average & last N …
d43deca
…episodes); implemented the log_metrics function of the StatRecorder class for visualization on weights & biases

Simply StatRecorder code, use deque and defaultdict structure
cd9494b

remove keep_past_n
a6de4f9

implemented the normalize function of stat_recorder, write test cases…
1e6938d
… for the StatRecorder class

syllabus/tests/__init__
c29887e

test cases for the StatRecorder class
a00f90b

merge in upstream
4441ba3

xinchen-yang force-pushed the main branch from 63da604 to 4441ba3Compare April 24, 2024 03:33

xinchen-yang added 2 commits April 24, 2024 13:13

access task_id info
d17a690

add info['task'] to reset, api for the normalize func
9f68e1b

RyanNavillus changed the base branch from main to stat-recorderApril 26, 2024 07:27

RyanNavillus reviewed Apr 3, 2024
View reviewed changes

RyanNavillus reviewed Apr 19, 2024
View reviewed changes

xinchen-yang force-pushed the main branch from 31d2cfe to d721bf0Compare April 19, 2024 21:31

RyanNavillusand others added 13 commits April 22, 2024 03:08

Merge pull requestRyanNavillus#34from RyanNavillus/async-plr
63dc8f6
Async plr

Remove unused eval
78f450a

merge in upstram
9370a1c

minor changes
c470517

merge in upstream
4158368

Update syllabus/core/stat_recorder.py
d04b6a7
Co-authored-by: Ryan Sullivan <ryanpnavillus@gmail.com>

implemented efficient reward normalization (running average & last N …
d43deca
…episodes); implemented the log_metrics function of the StatRecorder class for visualization on weights & biases

Simply StatRecorder code, use deque and defaultdict structure
cd9494b

remove keep_past_n
a6de4f9

implemented the normalize function of stat_recorder, write test cases…
1e6938d
… for the StatRecorder class

syllabus/tests/__init__
c29887e

test cases for the StatRecorder class
a00f90b

merge in upstream
4441ba3

xinchen-yang force-pushed the main branch from 63da604 to 4441ba3Compare April 24, 2024 03:33

xinchen-yang added 2 commits April 24, 2024 13:13

access task_id info
d17a690

add info['task'] to reset, api for the normalize func
9f68e1b

RyanNavillus changed the base branch from main to stat-recorderApril 26, 2024 07:27

RyanNavillus reviewed Apr 3, 2024
View reviewed changes

RyanNavillus reviewed Apr 19, 2024
View reviewed changes

xinchen-yang force-pushed the main branch from 31d2cfe to d721bf0Compare April 19, 2024 21:31

RyanNavillusand others added 13 commits April 22, 2024 03:08

Merge pull requestRyanNavillus#34from RyanNavillus/async-plr
63dc8f6
Async plr

Remove unused eval
78f450a

merge in upstram
9370a1c

minor changes
c470517

merge in upstream
4158368

Update syllabus/core/stat_recorder.py
d04b6a7
Co-authored-by: Ryan Sullivan <ryanpnavillus@gmail.com>

implemented efficient reward normalization (running average & last N …
d43deca
…episodes); implemented the log_metrics function of the StatRecorder class for visualization on weights & biases

Simply StatRecorder code, use deque and defaultdict structure
cd9494b

remove keep_past_n
a6de4f9

implemented the normalize function of stat_recorder, write test cases…
1e6938d
… for the StatRecorder class

syllabus/tests/__init__
c29887e

test cases for the StatRecorder class
a00f90b

merge in upstream
4441ba3

xinchen-yang force-pushed the main branch from 63da604 to 4441ba3Compare April 24, 2024 03:33

xinchen-yang added 2 commits April 24, 2024 13:13

access task_id info
d17a690

add info['task'] to reset, api for the normalize func
9f68e1b

RyanNavillus changed the base branch from main to stat-recorderApril 26, 2024 07:27

RyanNavillus reviewed Apr 3, 2024
View reviewed changes

RyanNavillus reviewed Apr 19, 2024
View reviewed changes

xinchen-yang force-pushed the main branch from 31d2cfe to d721bf0Compare April 19, 2024 21:31

RyanNavillusand others added 13 commits April 22, 2024 03:08

Merge pull requestRyanNavillus#34from RyanNavillus/async-plr
63dc8f6
Async plr

Remove unused eval
78f450a

merge in upstram
9370a1c

minor changes
c470517

merge in upstream
4158368

Update syllabus/core/stat_recorder.py
d04b6a7
Co-authored-by: Ryan Sullivan <ryanpnavillus@gmail.com>

implemented efficient reward normalization (running average & last N …
d43deca
…episodes); implemented the log_metrics function of the StatRecorder class for visualization on weights & biases

Simply StatRecorder code, use deque and defaultdict structure
cd9494b

remove keep_past_n
a6de4f9

implemented the normalize function of stat_recorder, write test cases…
1e6938d
… for the StatRecorder class

syllabus/tests/__init__
c29887e

test cases for the StatRecorder class
a00f90b

merge in upstream
4441ba3

xinchen-yang force-pushed the main branch from 63da604 to 4441ba3Compare April 24, 2024 03:33

xinchen-yang added 2 commits April 24, 2024 13:13

access task_id info
d17a690

add info['task'] to reset, api for the normalize func
9f68e1b

RyanNavillus changed the base branch from main to stat-recorderApril 26, 2024 07:27

RyanNavillus reviewed Apr 3, 2024
View reviewed changes

RyanNavillus reviewed Apr 19, 2024
View reviewed changes

xinchen-yang force-pushed the main branch from 31d2cfe to d721bf0Compare April 19, 2024 21:31

RyanNavillusand others added 13 commits April 22, 2024 03:08

Merge pull requestRyanNavillus#34from RyanNavillus/async-plr
63dc8f6
Async plr

Remove unused eval
78f450a

merge in upstram
9370a1c

minor changes
c470517

merge in upstream
4158368

Update syllabus/core/stat_recorder.py
d04b6a7
Co-authored-by: Ryan Sullivan <ryanpnavillus@gmail.com>

implemented efficient reward normalization (running average & last N …
d43deca
…episodes); implemented the log_metrics function of the StatRecorder class for visualization on weights & biases

Simply StatRecorder code, use deque and defaultdict structure
cd9494b

remove keep_past_n
a6de4f9

implemented the normalize function of stat_recorder, write test cases…
1e6938d
… for the StatRecorder class

syllabus/tests/__init__
c29887e

test cases for the StatRecorder class
a00f90b

merge in upstream
4441ba3

xinchen-yang force-pushed the main branch from 63da604 to 4441ba3Compare April 24, 2024 03:33

xinchen-yang added 2 commits April 24, 2024 13:13

access task_id info
d17a690

add info['task'] to reset, api for the normalize func
9f68e1b

RyanNavillus changed the base branch from main to stat-recorderApril 26, 2024 07:27

		@@ -34,45 +31,96 @@ def record(self, episode_return: float, episode_length: int, episode_task, env_i
		"""

Conversation

xinchen-yang commented Apr 2, 2024

Uh oh!

RyanNavillus left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

RyanNavillus left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RyanNavillus left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

xinchen-yang commented Apr 2, 2024

Uh oh!

RyanNavillus left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

RyanNavillus left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RyanNavillus left a comment