BranchesTags

Repository files navigation

kgof

Build Statuslicense

11th July 2019: For an implementation of our test in Julia, see this repository by Tor Erlend Fjelde.

UPDATE: On 8th Mar 2018, we have updated the code to support Python 3 (with futurize). If you find any problem, please let us know. Thanks.

This repository contains a Python 2.7/3 implementation of the nonparametric linear-time goodness-of-fit test described in our paper

A Linear-Time Kernel Goodness-of-Fit Test
Wittawat Jitkrittum, Wenkai Xu, Zoltan Szabo, Kenji Fukumizu, Arthur Gretton
NIPS 2017 (Best paper)
https://arxiv.org/abs/1705.07673

How to install?

The package can be installed with the pip command.

pip install git+https://github.com/wittawatj/kernel-gof.git

Once installed, you should be able to do import kgof without any error. pip will also resolve the following dependency automatically.

Dependency

The following Python packages were used during development. Ideally, the following packages with the specified version numbers or newer should be used. However, older versions may work as well. We did not specifically rely on newest features in these specified versions.

autograd == 1.1.7
matplotlib == 2.0.0
numpy == 1.11.3
scipy == 0.19.0

Demo

To get started, check demo_kgof.ipynb. This is a Jupyter notebook which will guide you through from the beginning. It can also be viewed on the web. There are many Jupyter notebooks in ipynb folder demonstrating other implemented tests. Be sure to check them if you would like to explore.

Reproduce experimental results

Each experiment is defined in its own Python file with a name starting with exXX where XX is a number. All the experiment files are in kgof/ex folder. Each file is runnable with a command line argument. For example in ex1_vary_n.py, we aim to check the test power of each testing algorithm as a function of the sample size n. The script ex1_vary_n.py takes a dataset name as its argument. See run_ex1.sh which is a standalone Bash script on how to execute ex1_power_vs_n.py.

We used independent-jobs package to parallelize our experiments over a Slurm cluster (the package is not needed if you just need to use our developed tests). For example, for ex1_vary_n.py, a job is created for each combination of

(dataset, test algorithm, n, trial)

If you do not use Slurm, you can change the line

engine = SlurmComputationEngine(batch_parameters)

to

engine = SerialComputationEngine()

which will instruct the computation engine to just use a normal for-loop on a single machine (will take a lot of time). Other computation engines that you use might be supported. See independent-jobs's repository page. Running simulation will create a lot of result files (one for each tuple above) saved as Pickle. Also, the independent-jobs package requires a scratch folder to save temporary files for communication among computing nodes. Path to the folder containing the saved results can be specified in kgof/config.py by changing the value of expr_results_path:

# Full path to the directory to store experimental results.
'expr_results_path': '/full/path/to/where/you/want/to/save/results/',

The scratch folder needed by the independent-jobs package can be specified in the same file by changing the value of scratch_path

# Full path to the directory to store temporary files when running experiments
'scratch_path': '/full/path/to/a/temporary/folder/',

To plot the results, see the experiment's corresponding Jupyter notebook in the ipynb/ folder. For example, for ex1_vary_n.py see ipynb/ex1_results.ipynb to plot the results.

Some note

  • When adding a new Kernel or new UnnormalizedDensity, use np.dot(X, Y) instead of X.dot(Y). autograd cannot differentiate the latter. Also, do not use x += .... Use x = x + .. instead.

  • The sub-module kgof.intertst depends on the linear-time two-sample test of Jitkrittum et al., 2016 (NIPS 2016) implemented in the freqopttest Python package which can be found here.

If you have questions or comments about anything related to this work, please do not hesitate to contact Wittawat Jitkrittum.

About

NeurIPS 2017 best paper. An interpretable linear-time kernel goodness-of-fit test.

Topics

goodness-of-fitkernel-methodsmachine-learningpythonstatistical-tests

Resources

Readme
License
Activity

Stars

67 stars

Watchers

4 watching

Forks

16 forks
Report repository

Releases

Packages

Used by

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
BranchesTags
Repository files navigation
kgof
Build Statuslicense
11th July 2019: For an implementation of our test in Julia, see this
repository by Tor Erlend Fjelde.
UPDATE: On 8th Mar 2018, we have updated the code to support Python 3 (with
futurize). If you find any problem, please let us know. Thanks.
This repository contains a Python 2.7/3 implementation of the nonparametric
linear-time goodness-of-fit test described in our paper
A Linear-Time Kernel Goodness-of-Fit Test
Wittawat Jitkrittum, Wenkai Xu, Zoltan Szabo, Kenji Fukumizu, Arthur Gretton
NIPS 2017 (Best paper)
https://arxiv.org/abs/1705.07673

How to install?
The package can be installed with the pip command.
pip install git+https://github.com/wittawatj/kernel-gof.git

Once installed, you should be able to do import kgof without any error.
pip will also resolve the following dependency automatically.
Dependency
The following Python packages were used during development. Ideally, the
following packages with the specified version numbers or newer should be used.
However, older versions may work as well. We did not specifically rely on
newest features in these specified versions.
autograd == 1.1.7
matplotlib == 2.0.0
numpy == 1.11.3
scipy == 0.19.0

Demo
To get started, check
demo_kgof.ipynb.
This is a Jupyter notebook which will guide you through from the beginning. It
can also be viewed on the web. There are many Jupyter notebooks in ipynb
folder demonstrating other implemented tests. Be sure to check them if you
would like to explore.
Reproduce experimental results
Each experiment is defined in its own Python file with a name starting with
exXX where XX is a number. All the experiment files are in kgof/ex
folder. Each file is runnable with a command line argument. For example in
ex1_vary_n.py, we aim to check the test power of each testing algorithm
as a function of the sample size n. The script ex1_vary_n.py takes a
dataset name as its argument. See run_ex1.sh which is a standalone Bash
script on how to execute ex1_power_vs_n.py.
We used independent-jobs
package to parallelize our experiments over a
Slurm cluster (the package is not needed if you
just need to use our developed tests). For example, for
ex1_vary_n.py, a job is created for each combination of
(dataset, test algorithm, n, trial)

If you do not use Slurm, you can change the line
engine = SlurmComputationEngine(batch_parameters)

to
engine = SerialComputationEngine()

which will instruct the computation engine to just use a normal for-loop on a
single machine (will take a lot of time). Other computation engines that you
use might be supported. See independent-jobs's repository
page. Running simulation will
create a lot of result files (one for each tuple above) saved as Pickle. Also,
the independent-jobs package requires a scratch folder to save temporary
files for communication among computing nodes. Path to the folder containing
the saved results can be specified in kgof/config.py by changing the value of
expr_results_path:
# Full path to the directory to store experimental results.
'expr_results_path': '/full/path/to/where/you/want/to/save/results/',

The scratch folder needed by the independent-jobs package can be specified in
the same file by changing the value of scratch_path
# Full path to the directory to store temporary files when running experiments
'scratch_path': '/full/path/to/a/temporary/folder/',

To plot the results, see the experiment's corresponding Jupyter notebook in the
ipynb/ folder. For example, for ex1_vary_n.py see ipynb/ex1_results.ipynb
to plot the results.
Some note
  • When adding a new Kernel or new UnnormalizedDensity, use np.dot(X, Y)
instead of X.dot(Y). autograd cannot differentiate the latter. Also, do
not use x += .... Use x = x + .. instead.
  • The sub-module kgof.intertst depends on the linear-time two-sample test of
Jitkrittum et al., 2016 (NIPS
2016)
implemented in the freqopttest Python package which can be found
here.
If you have questions or comments about anything related to this work, please
do not hesitate to contact Wittawat Jitkrittum.
About
NeurIPS 2017 best paper. An interpretable linear-time kernel goodness-of-fit test. 
Topics
goodness-of-fitkernel-methodsmachine-learningpythonstatistical-tests
Resources
Readme
License
Activity
Stars
67 stars
Watchers
4 watching
Forks
16 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
kgof
Build Statuslicense
11th July 2019: For an implementation of our test in Julia, see this
repository by Tor Erlend Fjelde.
UPDATE: On 8th Mar 2018, we have updated the code to support Python 3 (with
futurize). If you find any problem, please let us know. Thanks.
This repository contains a Python 2.7/3 implementation of the nonparametric
linear-time goodness-of-fit test described in our paper
A Linear-Time Kernel Goodness-of-Fit Test
Wittawat Jitkrittum, Wenkai Xu, Zoltan Szabo, Kenji Fukumizu, Arthur Gretton
NIPS 2017 (Best paper)
https://arxiv.org/abs/1705.07673

How to install?
The package can be installed with the pip command.
pip install git+https://github.com/wittawatj/kernel-gof.git

Once installed, you should be able to do import kgof without any error.
pip will also resolve the following dependency automatically.
Dependency
The following Python packages were used during development. Ideally, the
following packages with the specified version numbers or newer should be used.
However, older versions may work as well. We did not specifically rely on
newest features in these specified versions.
autograd == 1.1.7
matplotlib == 2.0.0
numpy == 1.11.3
scipy == 0.19.0

Demo
To get started, check
demo_kgof.ipynb.
This is a Jupyter notebook which will guide you through from the beginning. It
can also be viewed on the web. There are many Jupyter notebooks in ipynb
folder demonstrating other implemented tests. Be sure to check them if you
would like to explore.
Reproduce experimental results
Each experiment is defined in its own Python file with a name starting with
exXX where XX is a number. All the experiment files are in kgof/ex
folder. Each file is runnable with a command line argument. For example in
ex1_vary_n.py, we aim to check the test power of each testing algorithm
as a function of the sample size n. The script ex1_vary_n.py takes a
dataset name as its argument. See run_ex1.sh which is a standalone Bash
script on how to execute ex1_power_vs_n.py.
We used independent-jobs
package to parallelize our experiments over a
Slurm cluster (the package is not needed if you
just need to use our developed tests). For example, for
ex1_vary_n.py, a job is created for each combination of
(dataset, test algorithm, n, trial)

If you do not use Slurm, you can change the line
engine = SlurmComputationEngine(batch_parameters)

to
engine = SerialComputationEngine()

which will instruct the computation engine to just use a normal for-loop on a
single machine (will take a lot of time). Other computation engines that you
use might be supported. See independent-jobs's repository
page. Running simulation will
create a lot of result files (one for each tuple above) saved as Pickle. Also,
the independent-jobs package requires a scratch folder to save temporary
files for communication among computing nodes. Path to the folder containing
the saved results can be specified in kgof/config.py by changing the value of
expr_results_path:
# Full path to the directory to store experimental results.
'expr_results_path': '/full/path/to/where/you/want/to/save/results/',

The scratch folder needed by the independent-jobs package can be specified in
the same file by changing the value of scratch_path
# Full path to the directory to store temporary files when running experiments
'scratch_path': '/full/path/to/a/temporary/folder/',

To plot the results, see the experiment's corresponding Jupyter notebook in the
ipynb/ folder. For example, for ex1_vary_n.py see ipynb/ex1_results.ipynb
to plot the results.
Some note
  • When adding a new Kernel or new UnnormalizedDensity, use np.dot(X, Y)
instead of X.dot(Y). autograd cannot differentiate the latter. Also, do
not use x += .... Use x = x + .. instead.
  • The sub-module kgof.intertst depends on the linear-time two-sample test of
Jitkrittum et al., 2016 (NIPS
2016)
implemented in the freqopttest Python package which can be found
here.
If you have questions or comments about anything related to this work, please
do not hesitate to contact Wittawat Jitkrittum.
About
NeurIPS 2017 best paper. An interpretable linear-time kernel goodness-of-fit test. 
Topics
goodness-of-fitkernel-methodsmachine-learningpythonstatistical-tests
Resources
Readme
License
Activity
Stars
67 stars
Watchers
4 watching
Forks
16 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
kgof
Build Statuslicense
11th July 2019: For an implementation of our test in Julia, see this
repository by Tor Erlend Fjelde.
UPDATE: On 8th Mar 2018, we have updated the code to support Python 3 (with
futurize). If you find any problem, please let us know. Thanks.
This repository contains a Python 2.7/3 implementation of the nonparametric
linear-time goodness-of-fit test described in our paper
A Linear-Time Kernel Goodness-of-Fit Test
Wittawat Jitkrittum, Wenkai Xu, Zoltan Szabo, Kenji Fukumizu, Arthur Gretton
NIPS 2017 (Best paper)
https://arxiv.org/abs/1705.07673

How to install?
The package can be installed with the pip command.
pip install git+https://github.com/wittawatj/kernel-gof.git

Once installed, you should be able to do import kgof without any error.
pip will also resolve the following dependency automatically.
Dependency
The following Python packages were used during development. Ideally, the
following packages with the specified version numbers or newer should be used.
However, older versions may work as well. We did not specifically rely on
newest features in these specified versions.
autograd == 1.1.7
matplotlib == 2.0.0
numpy == 1.11.3
scipy == 0.19.0

Demo
To get started, check
demo_kgof.ipynb.
This is a Jupyter notebook which will guide you through from the beginning. It
can also be viewed on the web. There are many Jupyter notebooks in ipynb
folder demonstrating other implemented tests. Be sure to check them if you
would like to explore.
Reproduce experimental results
Each experiment is defined in its own Python file with a name starting with
exXX where XX is a number. All the experiment files are in kgof/ex
folder. Each file is runnable with a command line argument. For example in
ex1_vary_n.py, we aim to check the test power of each testing algorithm
as a function of the sample size n. The script ex1_vary_n.py takes a
dataset name as its argument. See run_ex1.sh which is a standalone Bash
script on how to execute ex1_power_vs_n.py.
We used independent-jobs
package to parallelize our experiments over a
Slurm cluster (the package is not needed if you
just need to use our developed tests). For example, for
ex1_vary_n.py, a job is created for each combination of
(dataset, test algorithm, n, trial)

If you do not use Slurm, you can change the line
engine = SlurmComputationEngine(batch_parameters)

to
engine = SerialComputationEngine()

which will instruct the computation engine to just use a normal for-loop on a
single machine (will take a lot of time). Other computation engines that you
use might be supported. See independent-jobs's repository
page. Running simulation will
create a lot of result files (one for each tuple above) saved as Pickle. Also,
the independent-jobs package requires a scratch folder to save temporary
files for communication among computing nodes. Path to the folder containing
the saved results can be specified in kgof/config.py by changing the value of
expr_results_path:
# Full path to the directory to store experimental results.
'expr_results_path': '/full/path/to/where/you/want/to/save/results/',

The scratch folder needed by the independent-jobs package can be specified in
the same file by changing the value of scratch_path
# Full path to the directory to store temporary files when running experiments
'scratch_path': '/full/path/to/a/temporary/folder/',

To plot the results, see the experiment's corresponding Jupyter notebook in the
ipynb/ folder. For example, for ex1_vary_n.py see ipynb/ex1_results.ipynb
to plot the results.
Some note
  • When adding a new Kernel or new UnnormalizedDensity, use np.dot(X, Y)
instead of X.dot(Y). autograd cannot differentiate the latter. Also, do
not use x += .... Use x = x + .. instead.
  • The sub-module kgof.intertst depends on the linear-time two-sample test of
Jitkrittum et al., 2016 (NIPS
2016)
implemented in the freqopttest Python package which can be found
here.
If you have questions or comments about anything related to this work, please
do not hesitate to contact Wittawat Jitkrittum.
About
NeurIPS 2017 best paper. An interpretable linear-time kernel goodness-of-fit test. 
Topics
goodness-of-fitkernel-methodsmachine-learningpythonstatistical-tests
Resources
Readme
License
Activity
Stars
67 stars
Watchers
4 watching
Forks
16 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
BranchesTags
Repository files navigation
kgof
Build Statuslicense
11th July 2019: For an implementation of our test in Julia, see this
repository by Tor Erlend Fjelde.
UPDATE: On 8th Mar 2018, we have updated the code to support Python 3 (with
futurize). If you find any problem, please let us know. Thanks.
This repository contains a Python 2.7/3 implementation of the nonparametric
linear-time goodness-of-fit test described in our paper
A Linear-Time Kernel Goodness-of-Fit Test
Wittawat Jitkrittum, Wenkai Xu, Zoltan Szabo, Kenji Fukumizu, Arthur Gretton
NIPS 2017 (Best paper)
https://arxiv.org/abs/1705.07673

How to install?
The package can be installed with the pip command.
pip install git+https://github.com/wittawatj/kernel-gof.git

Once installed, you should be able to do import kgof without any error.
pip will also resolve the following dependency automatically.
Dependency
The following Python packages were used during development. Ideally, the
following packages with the specified version numbers or newer should be used.
However, older versions may work as well. We did not specifically rely on
newest features in these specified versions.
autograd == 1.1.7
matplotlib == 2.0.0
numpy == 1.11.3
scipy == 0.19.0

Demo
To get started, check
demo_kgof.ipynb.
This is a Jupyter notebook which will guide you through from the beginning. It
can also be viewed on the web. There are many Jupyter notebooks in ipynb
folder demonstrating other implemented tests. Be sure to check them if you
would like to explore.
Reproduce experimental results
Each experiment is defined in its own Python file with a name starting with
exXX where XX is a number. All the experiment files are in kgof/ex
folder. Each file is runnable with a command line argument. For example in
ex1_vary_n.py, we aim to check the test power of each testing algorithm
as a function of the sample size n. The script ex1_vary_n.py takes a
dataset name as its argument. See run_ex1.sh which is a standalone Bash
script on how to execute ex1_power_vs_n.py.
We used independent-jobs
package to parallelize our experiments over a
Slurm cluster (the package is not needed if you
just need to use our developed tests). For example, for
ex1_vary_n.py, a job is created for each combination of
(dataset, test algorithm, n, trial)

If you do not use Slurm, you can change the line
engine = SlurmComputationEngine(batch_parameters)

to
engine = SerialComputationEngine()

which will instruct the computation engine to just use a normal for-loop on a
single machine (will take a lot of time). Other computation engines that you
use might be supported. See independent-jobs's repository
page. Running simulation will
create a lot of result files (one for each tuple above) saved as Pickle. Also,
the independent-jobs package requires a scratch folder to save temporary
files for communication among computing nodes. Path to the folder containing
the saved results can be specified in kgof/config.py by changing the value of
expr_results_path:
# Full path to the directory to store experimental results.
'expr_results_path': '/full/path/to/where/you/want/to/save/results/',

The scratch folder needed by the independent-jobs package can be specified in
the same file by changing the value of scratch_path
# Full path to the directory to store temporary files when running experiments
'scratch_path': '/full/path/to/a/temporary/folder/',

To plot the results, see the experiment's corresponding Jupyter notebook in the
ipynb/ folder. For example, for ex1_vary_n.py see ipynb/ex1_results.ipynb
to plot the results.
Some note
  • When adding a new Kernel or new UnnormalizedDensity, use np.dot(X, Y)
instead of X.dot(Y). autograd cannot differentiate the latter. Also, do
not use x += .... Use x = x + .. instead.
  • The sub-module kgof.intertst depends on the linear-time two-sample test of
Jitkrittum et al., 2016 (NIPS
2016)
implemented in the freqopttest Python package which can be found
here.
If you have questions or comments about anything related to this work, please
do not hesitate to contact Wittawat Jitkrittum.
About
NeurIPS 2017 best paper. An interpretable linear-time kernel goodness-of-fit test. 
Topics
goodness-of-fitkernel-methodsmachine-learningpythonstatistical-tests
Resources
Readme
License
Activity
Stars
67 stars
Watchers
4 watching
Forks
16 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
kgof
Build Statuslicense
11th July 2019: For an implementation of our test in Julia, see this
repository by Tor Erlend Fjelde.
UPDATE: On 8th Mar 2018, we have updated the code to support Python 3 (with
futurize). If you find any problem, please let us know. Thanks.
This repository contains a Python 2.7/3 implementation of the nonparametric
linear-time goodness-of-fit test described in our paper
A Linear-Time Kernel Goodness-of-Fit Test
Wittawat Jitkrittum, Wenkai Xu, Zoltan Szabo, Kenji Fukumizu, Arthur Gretton
NIPS 2017 (Best paper)
https://arxiv.org/abs/1705.07673

How to install?
The package can be installed with the pip command.
pip install git+https://github.com/wittawatj/kernel-gof.git

Once installed, you should be able to do import kgof without any error.
pip will also resolve the following dependency automatically.
Dependency
The following Python packages were used during development. Ideally, the
following packages with the specified version numbers or newer should be used.
However, older versions may work as well. We did not specifically rely on
newest features in these specified versions.
autograd == 1.1.7
matplotlib == 2.0.0
numpy == 1.11.3
scipy == 0.19.0

Demo
To get started, check
demo_kgof.ipynb.
This is a Jupyter notebook which will guide you through from the beginning. It
can also be viewed on the web. There are many Jupyter notebooks in ipynb
folder demonstrating other implemented tests. Be sure to check them if you
would like to explore.
Reproduce experimental results
Each experiment is defined in its own Python file with a name starting with
exXX where XX is a number. All the experiment files are in kgof/ex
folder. Each file is runnable with a command line argument. For example in
ex1_vary_n.py, we aim to check the test power of each testing algorithm
as a function of the sample size n. The script ex1_vary_n.py takes a
dataset name as its argument. See run_ex1.sh which is a standalone Bash
script on how to execute ex1_power_vs_n.py.
We used independent-jobs
package to parallelize our experiments over a
Slurm cluster (the package is not needed if you
just need to use our developed tests). For example, for
ex1_vary_n.py, a job is created for each combination of
(dataset, test algorithm, n, trial)

If you do not use Slurm, you can change the line
engine = SlurmComputationEngine(batch_parameters)

to
engine = SerialComputationEngine()

which will instruct the computation engine to just use a normal for-loop on a
single machine (will take a lot of time). Other computation engines that you
use might be supported. See independent-jobs's repository
page. Running simulation will
create a lot of result files (one for each tuple above) saved as Pickle. Also,
the independent-jobs package requires a scratch folder to save temporary
files for communication among computing nodes. Path to the folder containing
the saved results can be specified in kgof/config.py by changing the value of
expr_results_path:
# Full path to the directory to store experimental results.
'expr_results_path': '/full/path/to/where/you/want/to/save/results/',

The scratch folder needed by the independent-jobs package can be specified in
the same file by changing the value of scratch_path
# Full path to the directory to store temporary files when running experiments
'scratch_path': '/full/path/to/a/temporary/folder/',

To plot the results, see the experiment's corresponding Jupyter notebook in the
ipynb/ folder. For example, for ex1_vary_n.py see ipynb/ex1_results.ipynb
to plot the results.
Some note
  • When adding a new Kernel or new UnnormalizedDensity, use np.dot(X, Y)
instead of X.dot(Y). autograd cannot differentiate the latter. Also, do
not use x += .... Use x = x + .. instead.
  • The sub-module kgof.intertst depends on the linear-time two-sample test of
Jitkrittum et al., 2016 (NIPS
2016)
implemented in the freqopttest Python package which can be found
here.
If you have questions or comments about anything related to this work, please
do not hesitate to contact Wittawat Jitkrittum.
About
NeurIPS 2017 best paper. An interpretable linear-time kernel goodness-of-fit test. 
Topics
goodness-of-fitkernel-methodsmachine-learningpythonstatistical-tests
Resources
Readme
License
Activity
Stars
67 stars
Watchers
4 watching
Forks
16 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
kgof
Build Statuslicense
11th July 2019: For an implementation of our test in Julia, see this
repository by Tor Erlend Fjelde.
UPDATE: On 8th Mar 2018, we have updated the code to support Python 3 (with
futurize). If you find any problem, please let us know. Thanks.
This repository contains a Python 2.7/3 implementation of the nonparametric
linear-time goodness-of-fit test described in our paper
A Linear-Time Kernel Goodness-of-Fit Test
Wittawat Jitkrittum, Wenkai Xu, Zoltan Szabo, Kenji Fukumizu, Arthur Gretton
NIPS 2017 (Best paper)
https://arxiv.org/abs/1705.07673

How to install?
The package can be installed with the pip command.
pip install git+https://github.com/wittawatj/kernel-gof.git

Once installed, you should be able to do import kgof without any error.
pip will also resolve the following dependency automatically.
Dependency
The following Python packages were used during development. Ideally, the
following packages with the specified version numbers or newer should be used.
However, older versions may work as well. We did not specifically rely on
newest features in these specified versions.
autograd == 1.1.7
matplotlib == 2.0.0
numpy == 1.11.3
scipy == 0.19.0

Demo
To get started, check
demo_kgof.ipynb.
This is a Jupyter notebook which will guide you through from the beginning. It
can also be viewed on the web. There are many Jupyter notebooks in ipynb
folder demonstrating other implemented tests. Be sure to check them if you
would like to explore.
Reproduce experimental results
Each experiment is defined in its own Python file with a name starting with
exXX where XX is a number. All the experiment files are in kgof/ex
folder. Each file is runnable with a command line argument. For example in
ex1_vary_n.py, we aim to check the test power of each testing algorithm
as a function of the sample size n. The script ex1_vary_n.py takes a
dataset name as its argument. See run_ex1.sh which is a standalone Bash
script on how to execute ex1_power_vs_n.py.
We used independent-jobs
package to parallelize our experiments over a
Slurm cluster (the package is not needed if you
just need to use our developed tests). For example, for
ex1_vary_n.py, a job is created for each combination of
(dataset, test algorithm, n, trial)

If you do not use Slurm, you can change the line
engine = SlurmComputationEngine(batch_parameters)

to
engine = SerialComputationEngine()

which will instruct the computation engine to just use a normal for-loop on a
single machine (will take a lot of time). Other computation engines that you
use might be supported. See independent-jobs's repository
page. Running simulation will
create a lot of result files (one for each tuple above) saved as Pickle. Also,
the independent-jobs package requires a scratch folder to save temporary
files for communication among computing nodes. Path to the folder containing
the saved results can be specified in kgof/config.py by changing the value of
expr_results_path:
# Full path to the directory to store experimental results.
'expr_results_path': '/full/path/to/where/you/want/to/save/results/',

The scratch folder needed by the independent-jobs package can be specified in
the same file by changing the value of scratch_path
# Full path to the directory to store temporary files when running experiments
'scratch_path': '/full/path/to/a/temporary/folder/',

To plot the results, see the experiment's corresponding Jupyter notebook in the
ipynb/ folder. For example, for ex1_vary_n.py see ipynb/ex1_results.ipynb
to plot the results.
Some note
  • When adding a new Kernel or new UnnormalizedDensity, use np.dot(X, Y)
instead of X.dot(Y). autograd cannot differentiate the latter. Also, do
not use x += .... Use x = x + .. instead.
  • The sub-module kgof.intertst depends on the linear-time two-sample test of
Jitkrittum et al., 2016 (NIPS
2016)
implemented in the freqopttest Python package which can be found
here.
If you have questions or comments about anything related to this work, please
do not hesitate to contact Wittawat Jitkrittum.
About
NeurIPS 2017 best paper. An interpretable linear-time kernel goodness-of-fit test. 
Topics
goodness-of-fitkernel-methodsmachine-learningpythonstatistical-tests
Resources
Readme
License
Activity
Stars
67 stars
Watchers
4 watching
Forks
16 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
BranchesTags
Repository files navigation
kgof
Build Statuslicense
11th July 2019: For an implementation of our test in Julia, see this
repository by Tor Erlend Fjelde.
UPDATE: On 8th Mar 2018, we have updated the code to support Python 3 (with
futurize). If you find any problem, please let us know. Thanks.
This repository contains a Python 2.7/3 implementation of the nonparametric
linear-time goodness-of-fit test described in our paper
A Linear-Time Kernel Goodness-of-Fit Test
Wittawat Jitkrittum, Wenkai Xu, Zoltan Szabo, Kenji Fukumizu, Arthur Gretton
NIPS 2017 (Best paper)
https://arxiv.org/abs/1705.07673

How to install?
The package can be installed with the pip command.
pip install git+https://github.com/wittawatj/kernel-gof.git

Once installed, you should be able to do import kgof without any error.
pip will also resolve the following dependency automatically.
Dependency
The following Python packages were used during development. Ideally, the
following packages with the specified version numbers or newer should be used.
However, older versions may work as well. We did not specifically rely on
newest features in these specified versions.
autograd == 1.1.7
matplotlib == 2.0.0
numpy == 1.11.3
scipy == 0.19.0

Demo
To get started, check
demo_kgof.ipynb.
This is a Jupyter notebook which will guide you through from the beginning. It
can also be viewed on the web. There are many Jupyter notebooks in ipynb
folder demonstrating other implemented tests. Be sure to check them if you
would like to explore.
Reproduce experimental results
Each experiment is defined in its own Python file with a name starting with
exXX where XX is a number. All the experiment files are in kgof/ex
folder. Each file is runnable with a command line argument. For example in
ex1_vary_n.py, we aim to check the test power of each testing algorithm
as a function of the sample size n. The script ex1_vary_n.py takes a
dataset name as its argument. See run_ex1.sh which is a standalone Bash
script on how to execute ex1_power_vs_n.py.
We used independent-jobs
package to parallelize our experiments over a
Slurm cluster (the package is not needed if you
just need to use our developed tests). For example, for
ex1_vary_n.py, a job is created for each combination of
(dataset, test algorithm, n, trial)

If you do not use Slurm, you can change the line
engine = SlurmComputationEngine(batch_parameters)

to
engine = SerialComputationEngine()

which will instruct the computation engine to just use a normal for-loop on a
single machine (will take a lot of time). Other computation engines that you
use might be supported. See independent-jobs's repository
page. Running simulation will
create a lot of result files (one for each tuple above) saved as Pickle. Also,
the independent-jobs package requires a scratch folder to save temporary
files for communication among computing nodes. Path to the folder containing
the saved results can be specified in kgof/config.py by changing the value of
expr_results_path:
# Full path to the directory to store experimental results.
'expr_results_path': '/full/path/to/where/you/want/to/save/results/',

The scratch folder needed by the independent-jobs package can be specified in
the same file by changing the value of scratch_path
# Full path to the directory to store temporary files when running experiments
'scratch_path': '/full/path/to/a/temporary/folder/',

To plot the results, see the experiment's corresponding Jupyter notebook in the
ipynb/ folder. For example, for ex1_vary_n.py see ipynb/ex1_results.ipynb
to plot the results.
Some note
  • When adding a new Kernel or new UnnormalizedDensity, use np.dot(X, Y)
instead of X.dot(Y). autograd cannot differentiate the latter. Also, do
not use x += .... Use x = x + .. instead.
  • The sub-module kgof.intertst depends on the linear-time two-sample test of
Jitkrittum et al., 2016 (NIPS
2016)
implemented in the freqopttest Python package which can be found
here.
If you have questions or comments about anything related to this work, please
do not hesitate to contact Wittawat Jitkrittum.
About
NeurIPS 2017 best paper. An interpretable linear-time kernel goodness-of-fit test. 
Topics
goodness-of-fitkernel-methodsmachine-learningpythonstatistical-tests
Resources
Readme
License
Activity
Stars
67 stars
Watchers
4 watching
Forks
16 forks
Report repository
Releases
Packages
Used by
Contributors
Languages