Skip to content

Parameter usage

Jump to bottom
qinrj-mbp edited this page May 30, 2022 · 2 revisions

Usage of neorl.make()

NeoRL uses OpenAI Gym API, allowing users to create an env via neorl.make()

For neorl.make() func, the parameters are shown below:

paramtypedescription
taskstrThe task name you want to create. A full list of tasks is available here
reward_funcfuncA customized reward function, which should be provided if you want to calculate reward instead of using built-in reward of dataset.

The following code segment shows the usage of neorl with a customized reward function.

Example

import neorl
def customized_reward_func(data):
obs = data["obs"]
action = data["action"]
obs_next = data["next_obs"]
single_reward = False
if len(obs.shape) == 1:
single_reward = True
obs = obs.reshape(1, -1)
if len(action.shape) == 1:
action = action.reshape(1, -1)
if len(obs_next.shape) == 1:
obs_next = obs_next.reshape(1, -1)
CRF = 3.0
CRC = 1.0
fatigue = obs_next[:, -2]
consumption = obs_next[:, -1]
cost = CRF * fatigue + CRC * consumption
reward = -cost
if single_reward:
reward = reward[0].item()
else:
reward = reward.reshape(-1, 1)
return reward
env = neorl.make("ib", reward_func=customized_reward_func) # create the industrial benchmark env

Usage of get_dataset()

For get_dataset() func, the parameters are shown below:

paramtypedescription
task_name_versionstrThe name and version (if applicable) of the task, default is the same as task while making env
data_typestrWhich type of policy is used to collect data. It should be one of ["high", "medium", "low"], default to high
train_numintThe num of trajectory of training data. Note that the num should be less than 10,000, 100 by default
need_valboolWhether needs to download validation data, default to True
val_ratiofloatThe ratio of validation data to training data, default to 0.1
pathstrThe directory of data to load from or download to ./data/
use_data_rewardboolWhether uses default data reward. If false, a customized reward function should be provided by users while making env

Note that task_name_version is the same as task while making env by default. For instance, env = neorl.make("citylearn") will bind citylearn with env and dataset, which indicates env.get_dataset() will obtain citylearn data by default. For flexibility, task_name_version can be other task considering some people only intend to obtain data using an existing env instead of creating a neo one.

When calling get_dataset(), it will first look at local path for appropriate dataset ("appropriate" means the data type should match with the target data and the num of trajectories should not be less than the target data's). Meanwhile, MD5 is utilized to ensure dataset is complete and correct. If local dataset is not applicable, it will download the least appropriate dataset from remote server to path according to local data_map.json.

Example

import neorl
env = neorl.make("finance")
train_data, val_data = env.get_dataset(data_type="medium", train_num=100, need_val=True, val_ratio=0.2, use_data_reward=True)

It will load 100 trajectories for train_data and 10 trajectories for val_data, both using "medium" policy and built-in data reward. 

import neorl
env = neorl.make("citylearn")
train_data, _ = env.get_dataset(task_name_version="HalfCheetah-v3", data_type="low", train_num=50, need_val=False, use_data_reward=True)

It will load 50 trajectories for train_data without val_data, using "low" policy and built-in data reward.

Clone this wiki locally

, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Parameter usage · polixir/NeoRL Wiki · GitHub
Skip to content
Parameter usage
Jump to bottom

qinrj-mbp edited this page May 30, 2022
·
2 revisions
Usage of neorl.make()
NeoRL uses OpenAI Gym API, allowing users to create an env via neorl.make()
For neorl.make() func, the parameters are shown below:
paramtypedescription
taskstrThe task name you want to create. A full list of tasks is available here
reward_funcfuncA customized reward function, which should be provided if you want to calculate reward instead of using built-in reward of dataset.
The following code segment shows the usage of neorl with a customized reward function.
Example
import neorl
def customized_reward_func(data):
obs = data["obs"]
action = data["action"]
obs_next = data["next_obs"]
single_reward = False
if len(obs.shape) == 1:
single_reward = True
obs = obs.reshape(1, -1)
if len(action.shape) == 1:
action = action.reshape(1, -1)
if len(obs_next.shape) == 1:
obs_next = obs_next.reshape(1, -1)
CRF = 3.0
CRC = 1.0
fatigue = obs_next[:, -2]
consumption = obs_next[:, -1]
cost = CRF * fatigue + CRC * consumption
reward = -cost
if single_reward:
reward = reward[0].item()
else:
reward = reward.reshape(-1, 1)
return reward
env = neorl.make("ib", reward_func=customized_reward_func) # create the industrial benchmark env

Usage of get_dataset()
For get_dataset() func, the parameters are shown below:
paramtypedescription
task_name_versionstrThe name and version (if applicable) of the task, default is the same as task while making env
data_typestrWhich type of policy is used to collect data. It should be one of ["high", "medium", "low"], default to high
train_numintThe num of trajectory of training data. Note that the num should be less than 10,000, 100 by default
need_valboolWhether needs to download validation data, default to True
val_ratiofloatThe ratio of validation data to training data, default to 0.1
pathstrThe directory of data to load from or download to ./data/
use_data_rewardboolWhether uses default data reward. If false, a customized reward function should be provided by users while making env
Note that task_name_version is the same as task while making env by default. For instance, env = neorl.make("citylearn") will bind citylearn with env and dataset, which indicates env.get_dataset() will obtain citylearn data by default. For flexibility, task_name_version can be other task considering some people only intend to obtain data using an existing env instead of creating a neo one.
When calling get_dataset(), it will first look at local path for appropriate dataset ("appropriate" means the data type should match with the target data and the num of trajectories should not be less than the target data's). Meanwhile, MD5 is utilized to ensure dataset is complete and correct. If local dataset is not applicable, it will download the least appropriate dataset from remote server to path according to local data_map.json.
Example
import neorl
env = neorl.make("finance")
train_data, val_data = env.get_dataset(data_type="medium", train_num=100, need_val=True, val_ratio=0.2, use_data_reward=True)

It will load 100 trajectories for train_data and 10 trajectories for val_data, both using "medium" policy and built-in data reward.


import neorl
env = neorl.make("citylearn")
train_data, _ = env.get_dataset(task_name_version="HalfCheetah-v3", data_type="low", train_num=50, need_val=False, use_data_reward=True)

It will load 50 trajectories for train_data without val_data, using "low" policy and built-in data reward.


Clone this wiki locally

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Parameter usage · polixir/NeoRL Wiki · GitHub
Skip to content
Parameter usage
Jump to bottom

qinrj-mbp edited this page May 30, 2022
·
2 revisions
Usage of neorl.make()
NeoRL uses OpenAI Gym API, allowing users to create an env via neorl.make()
For neorl.make() func, the parameters are shown below:
paramtypedescription
taskstrThe task name you want to create. A full list of tasks is available here
reward_funcfuncA customized reward function, which should be provided if you want to calculate reward instead of using built-in reward of dataset.
The following code segment shows the usage of neorl with a customized reward function.
Example
import neorl
def customized_reward_func(data):
obs = data["obs"]
action = data["action"]
obs_next = data["next_obs"]
single_reward = False
if len(obs.shape) == 1:
single_reward = True
obs = obs.reshape(1, -1)
if len(action.shape) == 1:
action = action.reshape(1, -1)
if len(obs_next.shape) == 1:
obs_next = obs_next.reshape(1, -1)
CRF = 3.0
CRC = 1.0
fatigue = obs_next[:, -2]
consumption = obs_next[:, -1]
cost = CRF * fatigue + CRC * consumption
reward = -cost
if single_reward:
reward = reward[0].item()
else:
reward = reward.reshape(-1, 1)
return reward
env = neorl.make("ib", reward_func=customized_reward_func) # create the industrial benchmark env

Usage of get_dataset()
For get_dataset() func, the parameters are shown below:
paramtypedescription
task_name_versionstrThe name and version (if applicable) of the task, default is the same as task while making env
data_typestrWhich type of policy is used to collect data. It should be one of ["high", "medium", "low"], default to high
train_numintThe num of trajectory of training data. Note that the num should be less than 10,000, 100 by default
need_valboolWhether needs to download validation data, default to True
val_ratiofloatThe ratio of validation data to training data, default to 0.1
pathstrThe directory of data to load from or download to ./data/
use_data_rewardboolWhether uses default data reward. If false, a customized reward function should be provided by users while making env
Note that task_name_version is the same as task while making env by default. For instance, env = neorl.make("citylearn") will bind citylearn with env and dataset, which indicates env.get_dataset() will obtain citylearn data by default. For flexibility, task_name_version can be other task considering some people only intend to obtain data using an existing env instead of creating a neo one.
When calling get_dataset(), it will first look at local path for appropriate dataset ("appropriate" means the data type should match with the target data and the num of trajectories should not be less than the target data's). Meanwhile, MD5 is utilized to ensure dataset is complete and correct. If local dataset is not applicable, it will download the least appropriate dataset from remote server to path according to local data_map.json.
Example
import neorl
env = neorl.make("finance")
train_data, val_data = env.get_dataset(data_type="medium", train_num=100, need_val=True, val_ratio=0.2, use_data_reward=True)

It will load 100 trajectories for train_data and 10 trajectories for val_data, both using "medium" policy and built-in data reward.


import neorl
env = neorl.make("citylearn")
train_data, _ = env.get_dataset(task_name_version="HalfCheetah-v3", data_type="low", train_num=50, need_val=False, use_data_reward=True)

It will load 50 trajectories for train_data without val_data, using "low" policy and built-in data reward.


Clone this wiki locally

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Parameter usage · polixir/NeoRL Wiki · GitHub
Skip to content
Parameter usage
Jump to bottom

qinrj-mbp edited this page May 30, 2022
·
2 revisions
Usage of neorl.make()
NeoRL uses OpenAI Gym API, allowing users to create an env via neorl.make()
For neorl.make() func, the parameters are shown below:
paramtypedescription
taskstrThe task name you want to create. A full list of tasks is available here
reward_funcfuncA customized reward function, which should be provided if you want to calculate reward instead of using built-in reward of dataset.
The following code segment shows the usage of neorl with a customized reward function.
Example
import neorl
def customized_reward_func(data):
obs = data["obs"]
action = data["action"]
obs_next = data["next_obs"]
single_reward = False
if len(obs.shape) == 1:
single_reward = True
obs = obs.reshape(1, -1)
if len(action.shape) == 1:
action = action.reshape(1, -1)
if len(obs_next.shape) == 1:
obs_next = obs_next.reshape(1, -1)
CRF = 3.0
CRC = 1.0
fatigue = obs_next[:, -2]
consumption = obs_next[:, -1]
cost = CRF * fatigue + CRC * consumption
reward = -cost
if single_reward:
reward = reward[0].item()
else:
reward = reward.reshape(-1, 1)
return reward
env = neorl.make("ib", reward_func=customized_reward_func) # create the industrial benchmark env

Usage of get_dataset()
For get_dataset() func, the parameters are shown below:
paramtypedescription
task_name_versionstrThe name and version (if applicable) of the task, default is the same as task while making env
data_typestrWhich type of policy is used to collect data. It should be one of ["high", "medium", "low"], default to high
train_numintThe num of trajectory of training data. Note that the num should be less than 10,000, 100 by default
need_valboolWhether needs to download validation data, default to True
val_ratiofloatThe ratio of validation data to training data, default to 0.1
pathstrThe directory of data to load from or download to ./data/
use_data_rewardboolWhether uses default data reward. If false, a customized reward function should be provided by users while making env
Note that task_name_version is the same as task while making env by default. For instance, env = neorl.make("citylearn") will bind citylearn with env and dataset, which indicates env.get_dataset() will obtain citylearn data by default. For flexibility, task_name_version can be other task considering some people only intend to obtain data using an existing env instead of creating a neo one.
When calling get_dataset(), it will first look at local path for appropriate dataset ("appropriate" means the data type should match with the target data and the num of trajectories should not be less than the target data's). Meanwhile, MD5 is utilized to ensure dataset is complete and correct. If local dataset is not applicable, it will download the least appropriate dataset from remote server to path according to local data_map.json.
Example
import neorl
env = neorl.make("finance")
train_data, val_data = env.get_dataset(data_type="medium", train_num=100, need_val=True, val_ratio=0.2, use_data_reward=True)

It will load 100 trajectories for train_data and 10 trajectories for val_data, both using "medium" policy and built-in data reward.


import neorl
env = neorl.make("citylearn")
train_data, _ = env.get_dataset(task_name_version="HalfCheetah-v3", data_type="low", train_num=50, need_val=False, use_data_reward=True)

It will load 50 trajectories for train_data without val_data, using "low" policy and built-in data reward.


Clone this wiki locally

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
Parameter usage · polixir/NeoRL Wiki · GitHub
Skip to content
Parameter usage
Jump to bottom

qinrj-mbp edited this page May 30, 2022
·
2 revisions
Usage of neorl.make()
NeoRL uses OpenAI Gym API, allowing users to create an env via neorl.make()
For neorl.make() func, the parameters are shown below:
paramtypedescription
taskstrThe task name you want to create. A full list of tasks is available here
reward_funcfuncA customized reward function, which should be provided if you want to calculate reward instead of using built-in reward of dataset.
The following code segment shows the usage of neorl with a customized reward function.
Example
import neorl
def customized_reward_func(data):
obs = data["obs"]
action = data["action"]
obs_next = data["next_obs"]
single_reward = False
if len(obs.shape) == 1:
single_reward = True
obs = obs.reshape(1, -1)
if len(action.shape) == 1:
action = action.reshape(1, -1)
if len(obs_next.shape) == 1:
obs_next = obs_next.reshape(1, -1)
CRF = 3.0
CRC = 1.0
fatigue = obs_next[:, -2]
consumption = obs_next[:, -1]
cost = CRF * fatigue + CRC * consumption
reward = -cost
if single_reward:
reward = reward[0].item()
else:
reward = reward.reshape(-1, 1)
return reward
env = neorl.make("ib", reward_func=customized_reward_func) # create the industrial benchmark env

Usage of get_dataset()
For get_dataset() func, the parameters are shown below:
paramtypedescription
task_name_versionstrThe name and version (if applicable) of the task, default is the same as task while making env
data_typestrWhich type of policy is used to collect data. It should be one of ["high", "medium", "low"], default to high
train_numintThe num of trajectory of training data. Note that the num should be less than 10,000, 100 by default
need_valboolWhether needs to download validation data, default to True
val_ratiofloatThe ratio of validation data to training data, default to 0.1
pathstrThe directory of data to load from or download to ./data/
use_data_rewardboolWhether uses default data reward. If false, a customized reward function should be provided by users while making env
Note that task_name_version is the same as task while making env by default. For instance, env = neorl.make("citylearn") will bind citylearn with env and dataset, which indicates env.get_dataset() will obtain citylearn data by default. For flexibility, task_name_version can be other task considering some people only intend to obtain data using an existing env instead of creating a neo one.
When calling get_dataset(), it will first look at local path for appropriate dataset ("appropriate" means the data type should match with the target data and the num of trajectories should not be less than the target data's). Meanwhile, MD5 is utilized to ensure dataset is complete and correct. If local dataset is not applicable, it will download the least appropriate dataset from remote server to path according to local data_map.json.
Example
import neorl
env = neorl.make("finance")
train_data, val_data = env.get_dataset(data_type="medium", train_num=100, need_val=True, val_ratio=0.2, use_data_reward=True)

It will load 100 trajectories for train_data and 10 trajectories for val_data, both using "medium" policy and built-in data reward.


import neorl
env = neorl.make("citylearn")
train_data, _ = env.get_dataset(task_name_version="HalfCheetah-v3", data_type="low", train_num=50, need_val=False, use_data_reward=True)

It will load 50 trajectories for train_data without val_data, using "low" policy and built-in data reward.


Clone this wiki locally

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Parameter usage · polixir/NeoRL Wiki · GitHub
Skip to content
Parameter usage
Jump to bottom

qinrj-mbp edited this page May 30, 2022
·
2 revisions
Usage of neorl.make()
NeoRL uses OpenAI Gym API, allowing users to create an env via neorl.make()
For neorl.make() func, the parameters are shown below:
paramtypedescription
taskstrThe task name you want to create. A full list of tasks is available here
reward_funcfuncA customized reward function, which should be provided if you want to calculate reward instead of using built-in reward of dataset.
The following code segment shows the usage of neorl with a customized reward function.
Example
import neorl
def customized_reward_func(data):
obs = data["obs"]
action = data["action"]
obs_next = data["next_obs"]
single_reward = False
if len(obs.shape) == 1:
single_reward = True
obs = obs.reshape(1, -1)
if len(action.shape) == 1:
action = action.reshape(1, -1)
if len(obs_next.shape) == 1:
obs_next = obs_next.reshape(1, -1)
CRF = 3.0
CRC = 1.0
fatigue = obs_next[:, -2]
consumption = obs_next[:, -1]
cost = CRF * fatigue + CRC * consumption
reward = -cost
if single_reward:
reward = reward[0].item()
else:
reward = reward.reshape(-1, 1)
return reward
env = neorl.make("ib", reward_func=customized_reward_func) # create the industrial benchmark env

Usage of get_dataset()
For get_dataset() func, the parameters are shown below:
paramtypedescription
task_name_versionstrThe name and version (if applicable) of the task, default is the same as task while making env
data_typestrWhich type of policy is used to collect data. It should be one of ["high", "medium", "low"], default to high
train_numintThe num of trajectory of training data. Note that the num should be less than 10,000, 100 by default
need_valboolWhether needs to download validation data, default to True
val_ratiofloatThe ratio of validation data to training data, default to 0.1
pathstrThe directory of data to load from or download to ./data/
use_data_rewardboolWhether uses default data reward. If false, a customized reward function should be provided by users while making env
Note that task_name_version is the same as task while making env by default. For instance, env = neorl.make("citylearn") will bind citylearn with env and dataset, which indicates env.get_dataset() will obtain citylearn data by default. For flexibility, task_name_version can be other task considering some people only intend to obtain data using an existing env instead of creating a neo one.
When calling get_dataset(), it will first look at local path for appropriate dataset ("appropriate" means the data type should match with the target data and the num of trajectories should not be less than the target data's). Meanwhile, MD5 is utilized to ensure dataset is complete and correct. If local dataset is not applicable, it will download the least appropriate dataset from remote server to path according to local data_map.json.
Example
import neorl
env = neorl.make("finance")
train_data, val_data = env.get_dataset(data_type="medium", train_num=100, need_val=True, val_ratio=0.2, use_data_reward=True)

It will load 100 trajectories for train_data and 10 trajectories for val_data, both using "medium" policy and built-in data reward.


import neorl
env = neorl.make("citylearn")
train_data, _ = env.get_dataset(task_name_version="HalfCheetah-v3", data_type="low", train_num=50, need_val=False, use_data_reward=True)

It will load 50 trajectories for train_data without val_data, using "low" policy and built-in data reward.


Clone this wiki locally

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Parameter usage · polixir/NeoRL Wiki · GitHub
Skip to content
Parameter usage
Jump to bottom

qinrj-mbp edited this page May 30, 2022
·
2 revisions
Usage of neorl.make()
NeoRL uses OpenAI Gym API, allowing users to create an env via neorl.make()
For neorl.make() func, the parameters are shown below:
paramtypedescription
taskstrThe task name you want to create. A full list of tasks is available here
reward_funcfuncA customized reward function, which should be provided if you want to calculate reward instead of using built-in reward of dataset.
The following code segment shows the usage of neorl with a customized reward function.
Example
import neorl
def customized_reward_func(data):
obs = data["obs"]
action = data["action"]
obs_next = data["next_obs"]
single_reward = False
if len(obs.shape) == 1:
single_reward = True
obs = obs.reshape(1, -1)
if len(action.shape) == 1:
action = action.reshape(1, -1)
if len(obs_next.shape) == 1:
obs_next = obs_next.reshape(1, -1)
CRF = 3.0
CRC = 1.0
fatigue = obs_next[:, -2]
consumption = obs_next[:, -1]
cost = CRF * fatigue + CRC * consumption
reward = -cost
if single_reward:
reward = reward[0].item()
else:
reward = reward.reshape(-1, 1)
return reward
env = neorl.make("ib", reward_func=customized_reward_func) # create the industrial benchmark env

Usage of get_dataset()
For get_dataset() func, the parameters are shown below:
paramtypedescription
task_name_versionstrThe name and version (if applicable) of the task, default is the same as task while making env
data_typestrWhich type of policy is used to collect data. It should be one of ["high", "medium", "low"], default to high
train_numintThe num of trajectory of training data. Note that the num should be less than 10,000, 100 by default
need_valboolWhether needs to download validation data, default to True
val_ratiofloatThe ratio of validation data to training data, default to 0.1
pathstrThe directory of data to load from or download to ./data/
use_data_rewardboolWhether uses default data reward. If false, a customized reward function should be provided by users while making env
Note that task_name_version is the same as task while making env by default. For instance, env = neorl.make("citylearn") will bind citylearn with env and dataset, which indicates env.get_dataset() will obtain citylearn data by default. For flexibility, task_name_version can be other task considering some people only intend to obtain data using an existing env instead of creating a neo one.
When calling get_dataset(), it will first look at local path for appropriate dataset ("appropriate" means the data type should match with the target data and the num of trajectories should not be less than the target data's). Meanwhile, MD5 is utilized to ensure dataset is complete and correct. If local dataset is not applicable, it will download the least appropriate dataset from remote server to path according to local data_map.json.
Example
import neorl
env = neorl.make("finance")
train_data, val_data = env.get_dataset(data_type="medium", train_num=100, need_val=True, val_ratio=0.2, use_data_reward=True)

It will load 100 trajectories for train_data and 10 trajectories for val_data, both using "medium" policy and built-in data reward.


import neorl
env = neorl.make("citylearn")
train_data, _ = env.get_dataset(task_name_version="HalfCheetah-v3", data_type="low", train_num=50, need_val=False, use_data_reward=True)

It will load 50 trajectories for train_data without val_data, using "low" policy and built-in data reward.


Clone this wiki locally

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
Parameter usage · polixir/NeoRL Wiki · GitHub
Skip to content
Parameter usage
Jump to bottom

qinrj-mbp edited this page May 30, 2022
·
2 revisions
Usage of neorl.make()
NeoRL uses OpenAI Gym API, allowing users to create an env via neorl.make()
For neorl.make() func, the parameters are shown below:
paramtypedescription
taskstrThe task name you want to create. A full list of tasks is available here
reward_funcfuncA customized reward function, which should be provided if you want to calculate reward instead of using built-in reward of dataset.
The following code segment shows the usage of neorl with a customized reward function.
Example
import neorl
def customized_reward_func(data):
obs = data["obs"]
action = data["action"]
obs_next = data["next_obs"]
single_reward = False
if len(obs.shape) == 1:
single_reward = True
obs = obs.reshape(1, -1)
if len(action.shape) == 1:
action = action.reshape(1, -1)
if len(obs_next.shape) == 1:
obs_next = obs_next.reshape(1, -1)
CRF = 3.0
CRC = 1.0
fatigue = obs_next[:, -2]
consumption = obs_next[:, -1]
cost = CRF * fatigue + CRC * consumption
reward = -cost
if single_reward:
reward = reward[0].item()
else:
reward = reward.reshape(-1, 1)
return reward
env = neorl.make("ib", reward_func=customized_reward_func) # create the industrial benchmark env

Usage of get_dataset()
For get_dataset() func, the parameters are shown below:
paramtypedescription
task_name_versionstrThe name and version (if applicable) of the task, default is the same as task while making env
data_typestrWhich type of policy is used to collect data. It should be one of ["high", "medium", "low"], default to high
train_numintThe num of trajectory of training data. Note that the num should be less than 10,000, 100 by default
need_valboolWhether needs to download validation data, default to True
val_ratiofloatThe ratio of validation data to training data, default to 0.1
pathstrThe directory of data to load from or download to ./data/
use_data_rewardboolWhether uses default data reward. If false, a customized reward function should be provided by users while making env
Note that task_name_version is the same as task while making env by default. For instance, env = neorl.make("citylearn") will bind citylearn with env and dataset, which indicates env.get_dataset() will obtain citylearn data by default. For flexibility, task_name_version can be other task considering some people only intend to obtain data using an existing env instead of creating a neo one.
When calling get_dataset(), it will first look at local path for appropriate dataset ("appropriate" means the data type should match with the target data and the num of trajectories should not be less than the target data's). Meanwhile, MD5 is utilized to ensure dataset is complete and correct. If local dataset is not applicable, it will download the least appropriate dataset from remote server to path according to local data_map.json.
Example
import neorl
env = neorl.make("finance")
train_data, val_data = env.get_dataset(data_type="medium", train_num=100, need_val=True, val_ratio=0.2, use_data_reward=True)

It will load 100 trajectories for train_data and 10 trajectories for val_data, both using "medium" policy and built-in data reward.


import neorl
env = neorl.make("citylearn")
train_data, _ = env.get_dataset(task_name_version="HalfCheetah-v3", data_type="low", train_num=50, need_val=False, use_data_reward=True)

It will load 50 trajectories for train_data without val_data, using "low" policy and built-in data reward.


Clone this wiki locally