Skip to content
BranchesTags

Repository files navigation

The goal of this project is to provide tools for working with large network traffic datasets and to facilitate research in the traffic classification area. The core functions of the cesnet-datazoo package are:

  • A common API for downloading, configuring, and loading of four public datasets of encrypted network traffic.
  • Extensive configuration options for:
    • Selection of train, validation, and test periods.
    • Selection of application classes and splitting classes between known and unknown.
    • Data transformations, such as feature scaling.
  • Built on suitable data structures for experiments with large datasets. There are several caching mechanisms to make repeated runs faster, for example, when searching for the best model configuration.
  • Datasets are offered in multiple sizes to give users an option to start the experiments at a smaller scale (also faster dataset download, disk space, etc.). The default is the S size containing 25 million samples.

🧠 🧠 See a related project CESNET Models providing pretrained neural networks for traffic classification. 🧠 🧠

📓 📓 Example Jupyter notebooks are included in a separate Traffic Classification Examples repository. 📓 📓

🚀 🚀 Transfer Learning Codebase for reproducing experiments from our TNSM paper — covering ten downstream traffic classification tasks with three transfer approaches (k-NN, linear probing, and full model fine-tuning). 🚀 🚀

Datasets

The cesnet-datazoo package currently provides four datasets with details in the following table (you might need to scroll the table horizontally to see all datasets).

  1. CESNET-TLS22
  2. CESNET-QUIC22
  3. CESNET-TLS-Year22
  4. CESNET-QUICEXT-25
NameCESNET-TLS22CESNET-QUIC22CESNET-TLS-Year22CESNET-QUICEXT-25
ProtocolTLSQUICTLSQUIC
Published in2022202320242025
Collection duration2 weeks4 weeks1 year1 year
Collection period4.10.2021 - 17.10.202131.10.2022 - 27.11.20221.1.2022 - 31.12.20221.6.2024 - 31.5.2025
Application count19110218050
Available samples141392195153226273507739073194296462
Available dataset sizesXS, S, M, LXS, S, M, LXS, S, M, LXS, S, M, L
Cite10.1016/j.comnet.2022.10946710.1016/j.dib.2023.10888810.1038/s41597-024-03927-4In preparation
Zenodo URLzenodo.org/record/7965515zenodo.org/record/7963302zenodo.org/records/10608607zenodo.org/records/17249078
Related papers10.23919/TMA58422.2023.1019905210.1145/3768988

Installation

Install the package from pip with:

pip install cesnet-datazoo

or for editable install with:

pip install -e git+https://github.com/CESNET/cesnet-datazoo

Examples

Initialize dataset to create train, validation, and test dataframes

fromcesnet_datazoo.datasetsimportCESNET_QUIC22fromcesnet_datazoo.configimportDatasetConfig, AppSelectiondataset=CESNET_QUIC22("/datasets/CESNET-QUIC22/", size="XS")
dataset_config=DatasetConfig(
dataset=dataset,
apps_selection=AppSelection.ALL_KNOWN,
train_period_name="W-2022-44",
test_period_name="W-2022-45",
)
dataset.set_dataset_config_and_initialize(dataset_config)
train_dataframe=dataset.get_train_df()
val_dataframe=dataset.get_val_df()
test_dataframe=dataset.get_test_df()

The DatasetConfig class handles the configuration of datasets, and calling set_dataset_config_and_initialize initializes train, validation, and test sets with the desired configuration. Data can be read into Pandas DataFrames as shown here or via PyTorch DataLoaders. See CesnetDataset reference.

See more examples in the documentation.

Papers

Acknowledgments

This project was supported by the Ministry of the Interior of the Czech Republic, grant No. VJ02010024: Flow-Based Encrypted Traffic Analysis.

About

CESNET DataZoo: A toolset for large network traffic datasets

cesnet.github.io/cesnet-datazoo/

Topics

computer-networksdatasetsdeep-learningencrypted-trafficmachine-learningtraffic-classification

Resources

Readme
BSD-3-Clause license
Activity
Custom properties

Stars

49 stars

Watchers

16 watching

Forks

6 forks
Report repository

Releases

Packages

Used by

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
GitHub - CESNET/cesnet-datazoo: CESNET DataZoo: A toolset for large network traffic datasets · GitHub
Skip to content
BranchesTags
Repository files navigation
The goal of this project is to provide tools for working with large network traffic datasets and to facilitate research in the traffic classification area. The core functions of the cesnet-datazoo package are:
  • A common API for downloading, configuring, and loading of four public datasets of encrypted network traffic.
  • Extensive configuration options for:

    • Selection of train, validation, and test periods.
    • Selection of application classes and splitting classes between known and unknown.
    • Data transformations, such as feature scaling.
  • Built on suitable data structures for experiments with large datasets. There are several caching mechanisms to make repeated runs faster, for example, when searching for the best model configuration.
  • Datasets are offered in multiple sizes to give users an option to start the experiments at a smaller scale (also faster dataset download, disk space, etc.). The default is the S size containing 25 million samples.
🧠 🧠 See a related project CESNET Models providing pretrained neural networks for traffic classification. 🧠 🧠
📓 📓 Example Jupyter notebooks are included in a separate Traffic Classification Examples repository. 📓 📓
🚀 🚀 Transfer Learning Codebase for reproducing experiments from our TNSM paper — covering ten downstream traffic classification tasks with three transfer approaches (k-NN, linear probing, and full model fine-tuning). 🚀 🚀
Datasets
The cesnet-datazoo package currently provides four datasets with details in the following table (you might need to scroll the table horizontally to see all datasets).
  1. CESNET-TLS22
  2. CESNET-QUIC22
  3. CESNET-TLS-Year22
  4. CESNET-QUICEXT-25
NameCESNET-TLS22CESNET-QUIC22CESNET-TLS-Year22CESNET-QUICEXT-25
ProtocolTLSQUICTLSQUIC
Published in2022202320242025
Collection duration2 weeks4 weeks1 year1 year
Collection period4.10.2021 - 17.10.202131.10.2022 - 27.11.20221.1.2022 - 31.12.20221.6.2024 - 31.5.2025
Application count19110218050
Available samples141392195153226273507739073194296462
Available dataset sizesXS, S, M, LXS, S, M, LXS, S, M, LXS, S, M, L
Cite10.1016/j.comnet.2022.10946710.1016/j.dib.2023.10888810.1038/s41597-024-03927-4In preparation
Zenodo URLzenodo.org/record/7965515zenodo.org/record/7963302zenodo.org/records/10608607zenodo.org/records/17249078
Related papers10.23919/TMA58422.2023.1019905210.1145/3768988
Installation
Install the package from pip with:
pip install cesnet-datazoo
or for editable install with:
pip install -e git+https://github.com/CESNET/cesnet-datazoo
Examples
Initialize dataset to create train, validation, and test dataframes
fromcesnet_datazoo.datasetsimportCESNET_QUIC22fromcesnet_datazoo.configimportDatasetConfig, AppSelectiondataset=CESNET_QUIC22("/datasets/CESNET-QUIC22/", size="XS")
dataset_config=DatasetConfig(
dataset=dataset,
apps_selection=AppSelection.ALL_KNOWN,
train_period_name="W-2022-44",
test_period_name="W-2022-45",
)
dataset.set_dataset_config_and_initialize(dataset_config)
train_dataframe=dataset.get_train_df()
val_dataframe=dataset.get_val_df()
test_dataframe=dataset.get_test_df()
The DatasetConfig class handles the configuration of datasets, and calling set_dataset_config_and_initialize initializes train, validation, and test sets with the desired configuration.
Data can be read into Pandas DataFrames as shown here or via PyTorch DataLoaders. See CesnetDataset reference.
See more examples in the documentation.
Papers
Acknowledgments
This project was supported by the Ministry of the Interior of the Czech Republic, grant No. VJ02010024: Flow-Based Encrypted Traffic Analysis.
About
CESNET DataZoo: A toolset for large network traffic datasets
cesnet.github.io/cesnet-datazoo/
Topics
computer-networksdatasetsdeep-learningencrypted-trafficmachine-learningtraffic-classification
Resources
Readme
BSD-3-Clause license
Activity
Custom properties
Stars
49 stars
Watchers
16 watching
Forks
6 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
GitHub - CESNET/cesnet-datazoo: CESNET DataZoo: A toolset for large network traffic datasets · GitHub
Skip to content
BranchesTags
Repository files navigation
The goal of this project is to provide tools for working with large network traffic datasets and to facilitate research in the traffic classification area. The core functions of the cesnet-datazoo package are:
  • A common API for downloading, configuring, and loading of four public datasets of encrypted network traffic.
  • Extensive configuration options for:

    • Selection of train, validation, and test periods.
    • Selection of application classes and splitting classes between known and unknown.
    • Data transformations, such as feature scaling.
  • Built on suitable data structures for experiments with large datasets. There are several caching mechanisms to make repeated runs faster, for example, when searching for the best model configuration.
  • Datasets are offered in multiple sizes to give users an option to start the experiments at a smaller scale (also faster dataset download, disk space, etc.). The default is the S size containing 25 million samples.
🧠 🧠 See a related project CESNET Models providing pretrained neural networks for traffic classification. 🧠 🧠
📓 📓 Example Jupyter notebooks are included in a separate Traffic Classification Examples repository. 📓 📓
🚀 🚀 Transfer Learning Codebase for reproducing experiments from our TNSM paper — covering ten downstream traffic classification tasks with three transfer approaches (k-NN, linear probing, and full model fine-tuning). 🚀 🚀
Datasets
The cesnet-datazoo package currently provides four datasets with details in the following table (you might need to scroll the table horizontally to see all datasets).
  1. CESNET-TLS22
  2. CESNET-QUIC22
  3. CESNET-TLS-Year22
  4. CESNET-QUICEXT-25
NameCESNET-TLS22CESNET-QUIC22CESNET-TLS-Year22CESNET-QUICEXT-25
ProtocolTLSQUICTLSQUIC
Published in2022202320242025
Collection duration2 weeks4 weeks1 year1 year
Collection period4.10.2021 - 17.10.202131.10.2022 - 27.11.20221.1.2022 - 31.12.20221.6.2024 - 31.5.2025
Application count19110218050
Available samples141392195153226273507739073194296462
Available dataset sizesXS, S, M, LXS, S, M, LXS, S, M, LXS, S, M, L
Cite10.1016/j.comnet.2022.10946710.1016/j.dib.2023.10888810.1038/s41597-024-03927-4In preparation
Zenodo URLzenodo.org/record/7965515zenodo.org/record/7963302zenodo.org/records/10608607zenodo.org/records/17249078
Related papers10.23919/TMA58422.2023.1019905210.1145/3768988
Installation
Install the package from pip with:
pip install cesnet-datazoo
or for editable install with:
pip install -e git+https://github.com/CESNET/cesnet-datazoo
Examples
Initialize dataset to create train, validation, and test dataframes
fromcesnet_datazoo.datasetsimportCESNET_QUIC22fromcesnet_datazoo.configimportDatasetConfig, AppSelectiondataset=CESNET_QUIC22("/datasets/CESNET-QUIC22/", size="XS")
dataset_config=DatasetConfig(
dataset=dataset,
apps_selection=AppSelection.ALL_KNOWN,
train_period_name="W-2022-44",
test_period_name="W-2022-45",
)
dataset.set_dataset_config_and_initialize(dataset_config)
train_dataframe=dataset.get_train_df()
val_dataframe=dataset.get_val_df()
test_dataframe=dataset.get_test_df()
The DatasetConfig class handles the configuration of datasets, and calling set_dataset_config_and_initialize initializes train, validation, and test sets with the desired configuration.
Data can be read into Pandas DataFrames as shown here or via PyTorch DataLoaders. See CesnetDataset reference.
See more examples in the documentation.
Papers
Acknowledgments
This project was supported by the Ministry of the Interior of the Czech Republic, grant No. VJ02010024: Flow-Based Encrypted Traffic Analysis.
About
CESNET DataZoo: A toolset for large network traffic datasets
cesnet.github.io/cesnet-datazoo/
Topics
computer-networksdatasetsdeep-learningencrypted-trafficmachine-learningtraffic-classification
Resources
Readme
BSD-3-Clause license
Activity
Custom properties
Stars
49 stars
Watchers
16 watching
Forks
6 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
GitHub - CESNET/cesnet-datazoo: CESNET DataZoo: A toolset for large network traffic datasets · GitHub
Skip to content
BranchesTags
Repository files navigation
The goal of this project is to provide tools for working with large network traffic datasets and to facilitate research in the traffic classification area. The core functions of the cesnet-datazoo package are:
  • A common API for downloading, configuring, and loading of four public datasets of encrypted network traffic.
  • Extensive configuration options for:

    • Selection of train, validation, and test periods.
    • Selection of application classes and splitting classes between known and unknown.
    • Data transformations, such as feature scaling.
  • Built on suitable data structures for experiments with large datasets. There are several caching mechanisms to make repeated runs faster, for example, when searching for the best model configuration.
  • Datasets are offered in multiple sizes to give users an option to start the experiments at a smaller scale (also faster dataset download, disk space, etc.). The default is the S size containing 25 million samples.
🧠 🧠 See a related project CESNET Models providing pretrained neural networks for traffic classification. 🧠 🧠
📓 📓 Example Jupyter notebooks are included in a separate Traffic Classification Examples repository. 📓 📓
🚀 🚀 Transfer Learning Codebase for reproducing experiments from our TNSM paper — covering ten downstream traffic classification tasks with three transfer approaches (k-NN, linear probing, and full model fine-tuning). 🚀 🚀
Datasets
The cesnet-datazoo package currently provides four datasets with details in the following table (you might need to scroll the table horizontally to see all datasets).
  1. CESNET-TLS22
  2. CESNET-QUIC22
  3. CESNET-TLS-Year22
  4. CESNET-QUICEXT-25
NameCESNET-TLS22CESNET-QUIC22CESNET-TLS-Year22CESNET-QUICEXT-25
ProtocolTLSQUICTLSQUIC
Published in2022202320242025
Collection duration2 weeks4 weeks1 year1 year
Collection period4.10.2021 - 17.10.202131.10.2022 - 27.11.20221.1.2022 - 31.12.20221.6.2024 - 31.5.2025
Application count19110218050
Available samples141392195153226273507739073194296462
Available dataset sizesXS, S, M, LXS, S, M, LXS, S, M, LXS, S, M, L
Cite10.1016/j.comnet.2022.10946710.1016/j.dib.2023.10888810.1038/s41597-024-03927-4In preparation
Zenodo URLzenodo.org/record/7965515zenodo.org/record/7963302zenodo.org/records/10608607zenodo.org/records/17249078
Related papers10.23919/TMA58422.2023.1019905210.1145/3768988
Installation
Install the package from pip with:
pip install cesnet-datazoo
or for editable install with:
pip install -e git+https://github.com/CESNET/cesnet-datazoo
Examples
Initialize dataset to create train, validation, and test dataframes
fromcesnet_datazoo.datasetsimportCESNET_QUIC22fromcesnet_datazoo.configimportDatasetConfig, AppSelectiondataset=CESNET_QUIC22("/datasets/CESNET-QUIC22/", size="XS")
dataset_config=DatasetConfig(
dataset=dataset,
apps_selection=AppSelection.ALL_KNOWN,
train_period_name="W-2022-44",
test_period_name="W-2022-45",
)
dataset.set_dataset_config_and_initialize(dataset_config)
train_dataframe=dataset.get_train_df()
val_dataframe=dataset.get_val_df()
test_dataframe=dataset.get_test_df()
The DatasetConfig class handles the configuration of datasets, and calling set_dataset_config_and_initialize initializes train, validation, and test sets with the desired configuration.
Data can be read into Pandas DataFrames as shown here or via PyTorch DataLoaders. See CesnetDataset reference.
See more examples in the documentation.
Papers
Acknowledgments
This project was supported by the Ministry of the Interior of the Czech Republic, grant No. VJ02010024: Flow-Based Encrypted Traffic Analysis.
About
CESNET DataZoo: A toolset for large network traffic datasets
cesnet.github.io/cesnet-datazoo/
Topics
computer-networksdatasetsdeep-learningencrypted-trafficmachine-learningtraffic-classification
Resources
Readme
BSD-3-Clause license
Activity
Custom properties
Stars
49 stars
Watchers
16 watching
Forks
6 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
GitHub - CESNET/cesnet-datazoo: CESNET DataZoo: A toolset for large network traffic datasets · GitHub
Skip to content
BranchesTags
Repository files navigation
The goal of this project is to provide tools for working with large network traffic datasets and to facilitate research in the traffic classification area. The core functions of the cesnet-datazoo package are:
  • A common API for downloading, configuring, and loading of four public datasets of encrypted network traffic.
  • Extensive configuration options for:

    • Selection of train, validation, and test periods.
    • Selection of application classes and splitting classes between known and unknown.
    • Data transformations, such as feature scaling.
  • Built on suitable data structures for experiments with large datasets. There are several caching mechanisms to make repeated runs faster, for example, when searching for the best model configuration.
  • Datasets are offered in multiple sizes to give users an option to start the experiments at a smaller scale (also faster dataset download, disk space, etc.). The default is the S size containing 25 million samples.
🧠 🧠 See a related project CESNET Models providing pretrained neural networks for traffic classification. 🧠 🧠
📓 📓 Example Jupyter notebooks are included in a separate Traffic Classification Examples repository. 📓 📓
🚀 🚀 Transfer Learning Codebase for reproducing experiments from our TNSM paper — covering ten downstream traffic classification tasks with three transfer approaches (k-NN, linear probing, and full model fine-tuning). 🚀 🚀
Datasets
The cesnet-datazoo package currently provides four datasets with details in the following table (you might need to scroll the table horizontally to see all datasets).
  1. CESNET-TLS22
  2. CESNET-QUIC22
  3. CESNET-TLS-Year22
  4. CESNET-QUICEXT-25
NameCESNET-TLS22CESNET-QUIC22CESNET-TLS-Year22CESNET-QUICEXT-25
ProtocolTLSQUICTLSQUIC
Published in2022202320242025
Collection duration2 weeks4 weeks1 year1 year
Collection period4.10.2021 - 17.10.202131.10.2022 - 27.11.20221.1.2022 - 31.12.20221.6.2024 - 31.5.2025
Application count19110218050
Available samples141392195153226273507739073194296462
Available dataset sizesXS, S, M, LXS, S, M, LXS, S, M, LXS, S, M, L
Cite10.1016/j.comnet.2022.10946710.1016/j.dib.2023.10888810.1038/s41597-024-03927-4In preparation
Zenodo URLzenodo.org/record/7965515zenodo.org/record/7963302zenodo.org/records/10608607zenodo.org/records/17249078
Related papers10.23919/TMA58422.2023.1019905210.1145/3768988
Installation
Install the package from pip with:
pip install cesnet-datazoo
or for editable install with:
pip install -e git+https://github.com/CESNET/cesnet-datazoo
Examples
Initialize dataset to create train, validation, and test dataframes
fromcesnet_datazoo.datasetsimportCESNET_QUIC22fromcesnet_datazoo.configimportDatasetConfig, AppSelectiondataset=CESNET_QUIC22("/datasets/CESNET-QUIC22/", size="XS")
dataset_config=DatasetConfig(
dataset=dataset,
apps_selection=AppSelection.ALL_KNOWN,
train_period_name="W-2022-44",
test_period_name="W-2022-45",
)
dataset.set_dataset_config_and_initialize(dataset_config)
train_dataframe=dataset.get_train_df()
val_dataframe=dataset.get_val_df()
test_dataframe=dataset.get_test_df()
The DatasetConfig class handles the configuration of datasets, and calling set_dataset_config_and_initialize initializes train, validation, and test sets with the desired configuration.
Data can be read into Pandas DataFrames as shown here or via PyTorch DataLoaders. See CesnetDataset reference.
See more examples in the documentation.
Papers
Acknowledgments
This project was supported by the Ministry of the Interior of the Czech Republic, grant No. VJ02010024: Flow-Based Encrypted Traffic Analysis.
About
CESNET DataZoo: A toolset for large network traffic datasets
cesnet.github.io/cesnet-datazoo/
Topics
computer-networksdatasetsdeep-learningencrypted-trafficmachine-learningtraffic-classification
Resources
Readme
BSD-3-Clause license
Activity
Custom properties
Stars
49 stars
Watchers
16 watching
Forks
6 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
GitHub - CESNET/cesnet-datazoo: CESNET DataZoo: A toolset for large network traffic datasets · GitHub
Skip to content
BranchesTags
Repository files navigation
The goal of this project is to provide tools for working with large network traffic datasets and to facilitate research in the traffic classification area. The core functions of the cesnet-datazoo package are:
  • A common API for downloading, configuring, and loading of four public datasets of encrypted network traffic.
  • Extensive configuration options for:

    • Selection of train, validation, and test periods.
    • Selection of application classes and splitting classes between known and unknown.
    • Data transformations, such as feature scaling.
  • Built on suitable data structures for experiments with large datasets. There are several caching mechanisms to make repeated runs faster, for example, when searching for the best model configuration.
  • Datasets are offered in multiple sizes to give users an option to start the experiments at a smaller scale (also faster dataset download, disk space, etc.). The default is the S size containing 25 million samples.
🧠 🧠 See a related project CESNET Models providing pretrained neural networks for traffic classification. 🧠 🧠
📓 📓 Example Jupyter notebooks are included in a separate Traffic Classification Examples repository. 📓 📓
🚀 🚀 Transfer Learning Codebase for reproducing experiments from our TNSM paper — covering ten downstream traffic classification tasks with three transfer approaches (k-NN, linear probing, and full model fine-tuning). 🚀 🚀
Datasets
The cesnet-datazoo package currently provides four datasets with details in the following table (you might need to scroll the table horizontally to see all datasets).
  1. CESNET-TLS22
  2. CESNET-QUIC22
  3. CESNET-TLS-Year22
  4. CESNET-QUICEXT-25
NameCESNET-TLS22CESNET-QUIC22CESNET-TLS-Year22CESNET-QUICEXT-25
ProtocolTLSQUICTLSQUIC
Published in2022202320242025
Collection duration2 weeks4 weeks1 year1 year
Collection period4.10.2021 - 17.10.202131.10.2022 - 27.11.20221.1.2022 - 31.12.20221.6.2024 - 31.5.2025
Application count19110218050
Available samples141392195153226273507739073194296462
Available dataset sizesXS, S, M, LXS, S, M, LXS, S, M, LXS, S, M, L
Cite10.1016/j.comnet.2022.10946710.1016/j.dib.2023.10888810.1038/s41597-024-03927-4In preparation
Zenodo URLzenodo.org/record/7965515zenodo.org/record/7963302zenodo.org/records/10608607zenodo.org/records/17249078
Related papers10.23919/TMA58422.2023.1019905210.1145/3768988
Installation
Install the package from pip with:
pip install cesnet-datazoo
or for editable install with:
pip install -e git+https://github.com/CESNET/cesnet-datazoo
Examples
Initialize dataset to create train, validation, and test dataframes
fromcesnet_datazoo.datasetsimportCESNET_QUIC22fromcesnet_datazoo.configimportDatasetConfig, AppSelectiondataset=CESNET_QUIC22("/datasets/CESNET-QUIC22/", size="XS")
dataset_config=DatasetConfig(
dataset=dataset,
apps_selection=AppSelection.ALL_KNOWN,
train_period_name="W-2022-44",
test_period_name="W-2022-45",
)
dataset.set_dataset_config_and_initialize(dataset_config)
train_dataframe=dataset.get_train_df()
val_dataframe=dataset.get_val_df()
test_dataframe=dataset.get_test_df()
The DatasetConfig class handles the configuration of datasets, and calling set_dataset_config_and_initialize initializes train, validation, and test sets with the desired configuration.
Data can be read into Pandas DataFrames as shown here or via PyTorch DataLoaders. See CesnetDataset reference.
See more examples in the documentation.
Papers
Acknowledgments
This project was supported by the Ministry of the Interior of the Czech Republic, grant No. VJ02010024: Flow-Based Encrypted Traffic Analysis.
About
CESNET DataZoo: A toolset for large network traffic datasets
cesnet.github.io/cesnet-datazoo/
Topics
computer-networksdatasetsdeep-learningencrypted-trafficmachine-learningtraffic-classification
Resources
Readme
BSD-3-Clause license
Activity
Custom properties
Stars
49 stars
Watchers
16 watching
Forks
6 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
GitHub - CESNET/cesnet-datazoo: CESNET DataZoo: A toolset for large network traffic datasets · GitHub
Skip to content
BranchesTags
Repository files navigation
The goal of this project is to provide tools for working with large network traffic datasets and to facilitate research in the traffic classification area. The core functions of the cesnet-datazoo package are:
  • A common API for downloading, configuring, and loading of four public datasets of encrypted network traffic.
  • Extensive configuration options for:

    • Selection of train, validation, and test periods.
    • Selection of application classes and splitting classes between known and unknown.
    • Data transformations, such as feature scaling.
  • Built on suitable data structures for experiments with large datasets. There are several caching mechanisms to make repeated runs faster, for example, when searching for the best model configuration.
  • Datasets are offered in multiple sizes to give users an option to start the experiments at a smaller scale (also faster dataset download, disk space, etc.). The default is the S size containing 25 million samples.
🧠 🧠 See a related project CESNET Models providing pretrained neural networks for traffic classification. 🧠 🧠
📓 📓 Example Jupyter notebooks are included in a separate Traffic Classification Examples repository. 📓 📓
🚀 🚀 Transfer Learning Codebase for reproducing experiments from our TNSM paper — covering ten downstream traffic classification tasks with three transfer approaches (k-NN, linear probing, and full model fine-tuning). 🚀 🚀
Datasets
The cesnet-datazoo package currently provides four datasets with details in the following table (you might need to scroll the table horizontally to see all datasets).
  1. CESNET-TLS22
  2. CESNET-QUIC22
  3. CESNET-TLS-Year22
  4. CESNET-QUICEXT-25
NameCESNET-TLS22CESNET-QUIC22CESNET-TLS-Year22CESNET-QUICEXT-25
ProtocolTLSQUICTLSQUIC
Published in2022202320242025
Collection duration2 weeks4 weeks1 year1 year
Collection period4.10.2021 - 17.10.202131.10.2022 - 27.11.20221.1.2022 - 31.12.20221.6.2024 - 31.5.2025
Application count19110218050
Available samples141392195153226273507739073194296462
Available dataset sizesXS, S, M, LXS, S, M, LXS, S, M, LXS, S, M, L
Cite10.1016/j.comnet.2022.10946710.1016/j.dib.2023.10888810.1038/s41597-024-03927-4In preparation
Zenodo URLzenodo.org/record/7965515zenodo.org/record/7963302zenodo.org/records/10608607zenodo.org/records/17249078
Related papers10.23919/TMA58422.2023.1019905210.1145/3768988
Installation
Install the package from pip with:
pip install cesnet-datazoo
or for editable install with:
pip install -e git+https://github.com/CESNET/cesnet-datazoo
Examples
Initialize dataset to create train, validation, and test dataframes
fromcesnet_datazoo.datasetsimportCESNET_QUIC22fromcesnet_datazoo.configimportDatasetConfig, AppSelectiondataset=CESNET_QUIC22("/datasets/CESNET-QUIC22/", size="XS")
dataset_config=DatasetConfig(
dataset=dataset,
apps_selection=AppSelection.ALL_KNOWN,
train_period_name="W-2022-44",
test_period_name="W-2022-45",
)
dataset.set_dataset_config_and_initialize(dataset_config)
train_dataframe=dataset.get_train_df()
val_dataframe=dataset.get_val_df()
test_dataframe=dataset.get_test_df()
The DatasetConfig class handles the configuration of datasets, and calling set_dataset_config_and_initialize initializes train, validation, and test sets with the desired configuration.
Data can be read into Pandas DataFrames as shown here or via PyTorch DataLoaders. See CesnetDataset reference.
See more examples in the documentation.
Papers
Acknowledgments
This project was supported by the Ministry of the Interior of the Czech Republic, grant No. VJ02010024: Flow-Based Encrypted Traffic Analysis.
About
CESNET DataZoo: A toolset for large network traffic datasets
cesnet.github.io/cesnet-datazoo/
Topics
computer-networksdatasetsdeep-learningencrypted-trafficmachine-learningtraffic-classification
Resources
Readme
BSD-3-Clause license
Activity
Custom properties
Stars
49 stars
Watchers
16 watching
Forks
6 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
GitHub - CESNET/cesnet-datazoo: CESNET DataZoo: A toolset for large network traffic datasets · GitHub
Skip to content
BranchesTags
Repository files navigation
The goal of this project is to provide tools for working with large network traffic datasets and to facilitate research in the traffic classification area. The core functions of the cesnet-datazoo package are:
  • A common API for downloading, configuring, and loading of four public datasets of encrypted network traffic.
  • Extensive configuration options for:

    • Selection of train, validation, and test periods.
    • Selection of application classes and splitting classes between known and unknown.
    • Data transformations, such as feature scaling.
  • Built on suitable data structures for experiments with large datasets. There are several caching mechanisms to make repeated runs faster, for example, when searching for the best model configuration.
  • Datasets are offered in multiple sizes to give users an option to start the experiments at a smaller scale (also faster dataset download, disk space, etc.). The default is the S size containing 25 million samples.
🧠 🧠 See a related project CESNET Models providing pretrained neural networks for traffic classification. 🧠 🧠
📓 📓 Example Jupyter notebooks are included in a separate Traffic Classification Examples repository. 📓 📓
🚀 🚀 Transfer Learning Codebase for reproducing experiments from our TNSM paper — covering ten downstream traffic classification tasks with three transfer approaches (k-NN, linear probing, and full model fine-tuning). 🚀 🚀
Datasets
The cesnet-datazoo package currently provides four datasets with details in the following table (you might need to scroll the table horizontally to see all datasets).
  1. CESNET-TLS22
  2. CESNET-QUIC22
  3. CESNET-TLS-Year22
  4. CESNET-QUICEXT-25
NameCESNET-TLS22CESNET-QUIC22CESNET-TLS-Year22CESNET-QUICEXT-25
ProtocolTLSQUICTLSQUIC
Published in2022202320242025
Collection duration2 weeks4 weeks1 year1 year
Collection period4.10.2021 - 17.10.202131.10.2022 - 27.11.20221.1.2022 - 31.12.20221.6.2024 - 31.5.2025
Application count19110218050
Available samples141392195153226273507739073194296462
Available dataset sizesXS, S, M, LXS, S, M, LXS, S, M, LXS, S, M, L
Cite10.1016/j.comnet.2022.10946710.1016/j.dib.2023.10888810.1038/s41597-024-03927-4In preparation
Zenodo URLzenodo.org/record/7965515zenodo.org/record/7963302zenodo.org/records/10608607zenodo.org/records/17249078
Related papers10.23919/TMA58422.2023.1019905210.1145/3768988
Installation
Install the package from pip with:
pip install cesnet-datazoo
or for editable install with:
pip install -e git+https://github.com/CESNET/cesnet-datazoo
Examples
Initialize dataset to create train, validation, and test dataframes
fromcesnet_datazoo.datasetsimportCESNET_QUIC22fromcesnet_datazoo.configimportDatasetConfig, AppSelectiondataset=CESNET_QUIC22("/datasets/CESNET-QUIC22/", size="XS")
dataset_config=DatasetConfig(
dataset=dataset,
apps_selection=AppSelection.ALL_KNOWN,
train_period_name="W-2022-44",
test_period_name="W-2022-45",
)
dataset.set_dataset_config_and_initialize(dataset_config)
train_dataframe=dataset.get_train_df()
val_dataframe=dataset.get_val_df()
test_dataframe=dataset.get_test_df()
The DatasetConfig class handles the configuration of datasets, and calling set_dataset_config_and_initialize initializes train, validation, and test sets with the desired configuration.
Data can be read into Pandas DataFrames as shown here or via PyTorch DataLoaders. See CesnetDataset reference.
See more examples in the documentation.
Papers
Acknowledgments
This project was supported by the Ministry of the Interior of the Czech Republic, grant No. VJ02010024: Flow-Based Encrypted Traffic Analysis.
About
CESNET DataZoo: A toolset for large network traffic datasets
cesnet.github.io/cesnet-datazoo/
Topics
computer-networksdatasetsdeep-learningencrypted-trafficmachine-learningtraffic-classification
Resources
Readme
BSD-3-Clause license
Activity
Custom properties
Stars
49 stars
Watchers
16 watching
Forks
6 forks
Report repository
Releases
Packages
Used by
Contributors
Languages