BranchesTags

Repository files navigation

Funcom

Funcom Source Code Summarization Tool - Public Release

This repository contains the public release code for Funcom, a tool for source code summarization. Code summarization is the task of automatically generating natural language descriptions of source code.

Publications related to this work include:

LeClair, A., McMillan, C., "Recommendations for Datasets for Source Code Summarization", in Proc. of the 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL'19), Short Research Paper Track, Minneapolis, USA, June 2-7, 2019.

LeClair, A., Jiang, S., McMillan, C., "A Neural Model for Generating Natural Language Summaries of Program Subroutines", in Proc. of the 41st ACE/IEEE International Conference on Software Engineering (ICSE'19), Montreal, QC, Canada, May 25-31, 2019.
https://arxiv.org/abs/1902.01954

Example Output

Randomly sampled example output from the ast-attendgru model compared to reference good human-written summaries:

PROTOTYPE OUTPUT - HUMAN REFERENCE
returns the duration of the movie - get the full length of this movie in seconds
write a string to the client - write a string to all the connected clients
this method is called to indicate the next page in the page - call to explicitly go to the next page from within a single draw
returns a list of all the ids that match the given gene - get a list of superfamily ids for a gene name
compares two nodes by their UNK - compare nodes n1 and n2 by their dx entry
this method updates the tree panel - updates the tree panel with a new tree
returns the number of residues in the sequence - get number of interacting residues in domain b
returns true if the network is found - return true if passed inet address match a network which was used
log status message - log the status of the current message as info

Update Notice

This repository is archival for the ICSE'19 paper mentioned above. It is a good place to get started, but you may also want to look at our newer projects:

https://github.com/aakashba/callcon-public

https://github.com/Attn-to-FC/Attn-to-FC

https://github.com/acleclair/ICPC2020_GNN

USAGE

Step 0: Dependencies

We assume Ubuntu 18.04, Python 3.6, Keras 2.2.4, TensorFlow 1.12. Your milage may vary on different systems.

Step 1: Obtain Dataset

We provide a dataset of 2.1m Java methods and method comments, already cleaned and separated into training/val/test sets:

https://s3.us-east-2.amazonaws.com/icse2018/index.html

(Note: this paper is now several years old. Please see an update of data here: https://github.com/aakashba/callcon-public)

Extract the dataset to a directory (/scratch/ is the assumed default) so that you have a directory structure:
/scratch/funcom/data/standard/dataset.pkl
etc. in accordance with the files described on the site above.

To be consistent with defaults, create the following directories:
/scratch/funcom/data/outdir/models/
/scratch/funcom/data/outdir/histories/
/scratch/funcom/data/outdir/predictions/

Step 2: Train a Model

you@server:~/dev/funcom$ time python3 train.py --model-type=attendgru --gpu=0

Model types are defined in model.py. The ICSE'19 version is ast-attendgru, if you are seeking to reproduce it for comparision to your own models. Note that history information for each epoch is stored in a pkl file e.g. /scratch/funcom/data/outdir/histories/attendgru_hist_1551297717.pkl. The integer at the end of the file is the Epoch time at which training started, and is used to connect history, configuration, model, and prediction data. For example, training attendgru to epoch 5 would produce:

/scratch/funcom/data/outdir/histories/attendgru_conf_1551297717.pkl
/scratch/funcom/data/outdir/histories/attendgru_hist_1551297717.pkl
/scratch/funcom/data/outdir/models/attendgru_E01_1551297717.h5
/scratch/funcom/data/outdir/models/attendgru_E02_1551297717.h5
/scratch/funcom/data/outdir/models/attendgru_E03_1551297717.h5
/scratch/funcom/data/outdir/models/attendgru_E04_1551297717.h5
/scratch/funcom/data/outdir/models/attendgru_E05_1551297717.h5

A good baseline for initial work is the attendgru model. Comments in the file (models/attendgru.py) explain its behavior in detail, and it trains relatively quickly: about 45 minutes per epoch using batch size 200 on a single Quadro P5000, with maximum performance on the validation set at epoch 5.

Step 3: Inference / Prediction

you@server:~/dev/funcom$ time python3 predict.py /scratch/funcom/data/outdir/models/attendgru_E05_1551297717.h5 --gpu=0

The only necessary input to predict.py on the command line is the model file, but configuration information is read from the pkl files mentioned above. Output predictions will be written to a file e.g.:

/scratch/funcom/data/outdir/predictions/predict-attendgru_E05_1551297717.txt

Note that CPU prediction is possible in principle, but by default the attendgru and ast-attendgru models use CuDNNGRU instead of standard GRU, which necessitates using a GPU during prediction.

Step 4: Calculate Metrics

you@server:~/dev/funcom$ time python3 bleu.py /scratch/funcom/data/outdir/predictions/predict-attendgru_E05_1551297717.txt

This will output a BLEU score for the prediction file.

About

Funcom Source Code Summarization Tool - Public Release

www.leclair.tech/data/funcom

Resources

Readme
GPL-3.0 license
Activity

Stars

34 stars

Watchers

3 watching

Forks

20 forks
Report repository

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
BranchesTags
Repository files navigation
Funcom
Funcom Source Code Summarization Tool - Public Release
This repository contains the public release code for Funcom, a tool for source code summarization. Code summarization is the task of automatically generating natural language descriptions of source code.
Publications related to this work include:
LeClair, A., McMillan, C., "Recommendations for Datasets for Source Code Summarization", in Proc. of the 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL'19), Short Research Paper Track, Minneapolis, USA, June 2-7, 2019.
LeClair, A., Jiang, S., McMillan, C., "A Neural Model for Generating Natural Language Summaries of Program Subroutines", in Proc. of the 41st ACE/IEEE International Conference on Software Engineering (ICSE'19), Montreal, QC, Canada, May 25-31, 2019.
https://arxiv.org/abs/1902.01954
Example Output
Randomly sampled example output from the ast-attendgru model compared to reference good human-written summaries:
PROTOTYPE OUTPUT - HUMAN REFERENCE

returns the duration of the movie - get the full length of this movie in seconds

write a string to the client - write a string to all the connected clients

this method is called to indicate the next page in the page - call to explicitly go to the next page from within a single draw

returns a list of all the ids that match the given gene - get a list of superfamily ids for a gene name

compares two nodes by their UNK - compare nodes n1 and n2 by their dx entry

this method updates the tree panel - updates the tree panel with a new tree

returns the number of residues in the sequence - get number of interacting residues in domain b

returns true if the network is found - return true if passed inet address match a network which was used

log status message - log the status of the current message as info
Update Notice
This repository is archival for the ICSE'19 paper mentioned above. It is a good place to get started, but you may also want to look at our newer projects:
https://github.com/aakashba/callcon-public
https://github.com/Attn-to-FC/Attn-to-FC
https://github.com/acleclair/ICPC2020_GNN
USAGE
Step 0: Dependencies
We assume Ubuntu 18.04, Python 3.6, Keras 2.2.4, TensorFlow 1.12. Your milage may vary on different systems.
Step 1: Obtain Dataset
We provide a dataset of 2.1m Java methods and method comments, already cleaned and separated into training/val/test sets:
https://s3.us-east-2.amazonaws.com/icse2018/index.html
(Note: this paper is now several years old. Please see an update of data here: https://github.com/aakashba/callcon-public)
Extract the dataset to a directory (/scratch/ is the assumed default) so that you have a directory structure:

/scratch/funcom/data/standard/dataset.pkl

etc. in accordance with the files described on the site above.
To be consistent with defaults, create the following directories:

/scratch/funcom/data/outdir/models/

/scratch/funcom/data/outdir/histories/

/scratch/funcom/data/outdir/predictions/
Step 2: Train a Model
you@server:~/dev/funcom$ time python3 train.py --model-type=attendgru --gpu=0
Model types are defined in model.py. The ICSE'19 version is ast-attendgru, if you are seeking to reproduce it for comparision to your own models. Note that history information for each epoch is stored in a pkl file e.g. /scratch/funcom/data/outdir/histories/attendgru_hist_1551297717.pkl. The integer at the end of the file is the Epoch time at which training started, and is used to connect history, configuration, model, and prediction data. For example, training attendgru to epoch 5 would produce:
/scratch/funcom/data/outdir/histories/attendgru_conf_1551297717.pkl

/scratch/funcom/data/outdir/histories/attendgru_hist_1551297717.pkl

/scratch/funcom/data/outdir/models/attendgru_E01_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E02_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E03_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E04_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E05_1551297717.h5
A good baseline for initial work is the attendgru model. Comments in the file (models/attendgru.py) explain its behavior in detail, and it trains relatively quickly: about 45 minutes per epoch using batch size 200 on a single Quadro P5000, with maximum performance on the validation set at epoch 5.
Step 3: Inference / Prediction
you@server:~/dev/funcom$ time python3 predict.py /scratch/funcom/data/outdir/models/attendgru_E05_1551297717.h5 --gpu=0
The only necessary input to predict.py on the command line is the model file, but configuration information is read from the pkl files mentioned above. Output predictions will be written to a file e.g.:
/scratch/funcom/data/outdir/predictions/predict-attendgru_E05_1551297717.txt
Note that CPU prediction is possible in principle, but by default the attendgru and ast-attendgru models use CuDNNGRU instead of standard GRU, which necessitates using a GPU during prediction.
Step 4: Calculate Metrics
you@server:~/dev/funcom$ time python3 bleu.py /scratch/funcom/data/outdir/predictions/predict-attendgru_E05_1551297717.txt
This will output a BLEU score for the prediction file.
About
Funcom Source Code Summarization Tool - Public Release
www.leclair.tech/data/funcom
Resources
Readme
GPL-3.0 license
Activity
Stars
34 stars
Watchers
3 watching
Forks
20 forks
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
Funcom
Funcom Source Code Summarization Tool - Public Release
This repository contains the public release code for Funcom, a tool for source code summarization. Code summarization is the task of automatically generating natural language descriptions of source code.
Publications related to this work include:
LeClair, A., McMillan, C., "Recommendations for Datasets for Source Code Summarization", in Proc. of the 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL'19), Short Research Paper Track, Minneapolis, USA, June 2-7, 2019.
LeClair, A., Jiang, S., McMillan, C., "A Neural Model for Generating Natural Language Summaries of Program Subroutines", in Proc. of the 41st ACE/IEEE International Conference on Software Engineering (ICSE'19), Montreal, QC, Canada, May 25-31, 2019.
https://arxiv.org/abs/1902.01954
Example Output
Randomly sampled example output from the ast-attendgru model compared to reference good human-written summaries:
PROTOTYPE OUTPUT - HUMAN REFERENCE

returns the duration of the movie - get the full length of this movie in seconds

write a string to the client - write a string to all the connected clients

this method is called to indicate the next page in the page - call to explicitly go to the next page from within a single draw

returns a list of all the ids that match the given gene - get a list of superfamily ids for a gene name

compares two nodes by their UNK - compare nodes n1 and n2 by their dx entry

this method updates the tree panel - updates the tree panel with a new tree

returns the number of residues in the sequence - get number of interacting residues in domain b

returns true if the network is found - return true if passed inet address match a network which was used

log status message - log the status of the current message as info
Update Notice
This repository is archival for the ICSE'19 paper mentioned above. It is a good place to get started, but you may also want to look at our newer projects:
https://github.com/aakashba/callcon-public
https://github.com/Attn-to-FC/Attn-to-FC
https://github.com/acleclair/ICPC2020_GNN
USAGE
Step 0: Dependencies
We assume Ubuntu 18.04, Python 3.6, Keras 2.2.4, TensorFlow 1.12. Your milage may vary on different systems.
Step 1: Obtain Dataset
We provide a dataset of 2.1m Java methods and method comments, already cleaned and separated into training/val/test sets:
https://s3.us-east-2.amazonaws.com/icse2018/index.html
(Note: this paper is now several years old. Please see an update of data here: https://github.com/aakashba/callcon-public)
Extract the dataset to a directory (/scratch/ is the assumed default) so that you have a directory structure:

/scratch/funcom/data/standard/dataset.pkl

etc. in accordance with the files described on the site above.
To be consistent with defaults, create the following directories:

/scratch/funcom/data/outdir/models/

/scratch/funcom/data/outdir/histories/

/scratch/funcom/data/outdir/predictions/
Step 2: Train a Model
you@server:~/dev/funcom$ time python3 train.py --model-type=attendgru --gpu=0
Model types are defined in model.py. The ICSE'19 version is ast-attendgru, if you are seeking to reproduce it for comparision to your own models. Note that history information for each epoch is stored in a pkl file e.g. /scratch/funcom/data/outdir/histories/attendgru_hist_1551297717.pkl. The integer at the end of the file is the Epoch time at which training started, and is used to connect history, configuration, model, and prediction data. For example, training attendgru to epoch 5 would produce:
/scratch/funcom/data/outdir/histories/attendgru_conf_1551297717.pkl

/scratch/funcom/data/outdir/histories/attendgru_hist_1551297717.pkl

/scratch/funcom/data/outdir/models/attendgru_E01_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E02_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E03_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E04_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E05_1551297717.h5
A good baseline for initial work is the attendgru model. Comments in the file (models/attendgru.py) explain its behavior in detail, and it trains relatively quickly: about 45 minutes per epoch using batch size 200 on a single Quadro P5000, with maximum performance on the validation set at epoch 5.
Step 3: Inference / Prediction
you@server:~/dev/funcom$ time python3 predict.py /scratch/funcom/data/outdir/models/attendgru_E05_1551297717.h5 --gpu=0
The only necessary input to predict.py on the command line is the model file, but configuration information is read from the pkl files mentioned above. Output predictions will be written to a file e.g.:
/scratch/funcom/data/outdir/predictions/predict-attendgru_E05_1551297717.txt
Note that CPU prediction is possible in principle, but by default the attendgru and ast-attendgru models use CuDNNGRU instead of standard GRU, which necessitates using a GPU during prediction.
Step 4: Calculate Metrics
you@server:~/dev/funcom$ time python3 bleu.py /scratch/funcom/data/outdir/predictions/predict-attendgru_E05_1551297717.txt
This will output a BLEU score for the prediction file.
About
Funcom Source Code Summarization Tool - Public Release
www.leclair.tech/data/funcom
Resources
Readme
GPL-3.0 license
Activity
Stars
34 stars
Watchers
3 watching
Forks
20 forks
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
Funcom
Funcom Source Code Summarization Tool - Public Release
This repository contains the public release code for Funcom, a tool for source code summarization. Code summarization is the task of automatically generating natural language descriptions of source code.
Publications related to this work include:
LeClair, A., McMillan, C., "Recommendations for Datasets for Source Code Summarization", in Proc. of the 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL'19), Short Research Paper Track, Minneapolis, USA, June 2-7, 2019.
LeClair, A., Jiang, S., McMillan, C., "A Neural Model for Generating Natural Language Summaries of Program Subroutines", in Proc. of the 41st ACE/IEEE International Conference on Software Engineering (ICSE'19), Montreal, QC, Canada, May 25-31, 2019.
https://arxiv.org/abs/1902.01954
Example Output
Randomly sampled example output from the ast-attendgru model compared to reference good human-written summaries:
PROTOTYPE OUTPUT - HUMAN REFERENCE

returns the duration of the movie - get the full length of this movie in seconds

write a string to the client - write a string to all the connected clients

this method is called to indicate the next page in the page - call to explicitly go to the next page from within a single draw

returns a list of all the ids that match the given gene - get a list of superfamily ids for a gene name

compares two nodes by their UNK - compare nodes n1 and n2 by their dx entry

this method updates the tree panel - updates the tree panel with a new tree

returns the number of residues in the sequence - get number of interacting residues in domain b

returns true if the network is found - return true if passed inet address match a network which was used

log status message - log the status of the current message as info
Update Notice
This repository is archival for the ICSE'19 paper mentioned above. It is a good place to get started, but you may also want to look at our newer projects:
https://github.com/aakashba/callcon-public
https://github.com/Attn-to-FC/Attn-to-FC
https://github.com/acleclair/ICPC2020_GNN
USAGE
Step 0: Dependencies
We assume Ubuntu 18.04, Python 3.6, Keras 2.2.4, TensorFlow 1.12. Your milage may vary on different systems.
Step 1: Obtain Dataset
We provide a dataset of 2.1m Java methods and method comments, already cleaned and separated into training/val/test sets:
https://s3.us-east-2.amazonaws.com/icse2018/index.html
(Note: this paper is now several years old. Please see an update of data here: https://github.com/aakashba/callcon-public)
Extract the dataset to a directory (/scratch/ is the assumed default) so that you have a directory structure:

/scratch/funcom/data/standard/dataset.pkl

etc. in accordance with the files described on the site above.
To be consistent with defaults, create the following directories:

/scratch/funcom/data/outdir/models/

/scratch/funcom/data/outdir/histories/

/scratch/funcom/data/outdir/predictions/
Step 2: Train a Model
you@server:~/dev/funcom$ time python3 train.py --model-type=attendgru --gpu=0
Model types are defined in model.py. The ICSE'19 version is ast-attendgru, if you are seeking to reproduce it for comparision to your own models. Note that history information for each epoch is stored in a pkl file e.g. /scratch/funcom/data/outdir/histories/attendgru_hist_1551297717.pkl. The integer at the end of the file is the Epoch time at which training started, and is used to connect history, configuration, model, and prediction data. For example, training attendgru to epoch 5 would produce:
/scratch/funcom/data/outdir/histories/attendgru_conf_1551297717.pkl

/scratch/funcom/data/outdir/histories/attendgru_hist_1551297717.pkl

/scratch/funcom/data/outdir/models/attendgru_E01_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E02_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E03_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E04_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E05_1551297717.h5
A good baseline for initial work is the attendgru model. Comments in the file (models/attendgru.py) explain its behavior in detail, and it trains relatively quickly: about 45 minutes per epoch using batch size 200 on a single Quadro P5000, with maximum performance on the validation set at epoch 5.
Step 3: Inference / Prediction
you@server:~/dev/funcom$ time python3 predict.py /scratch/funcom/data/outdir/models/attendgru_E05_1551297717.h5 --gpu=0
The only necessary input to predict.py on the command line is the model file, but configuration information is read from the pkl files mentioned above. Output predictions will be written to a file e.g.:
/scratch/funcom/data/outdir/predictions/predict-attendgru_E05_1551297717.txt
Note that CPU prediction is possible in principle, but by default the attendgru and ast-attendgru models use CuDNNGRU instead of standard GRU, which necessitates using a GPU during prediction.
Step 4: Calculate Metrics
you@server:~/dev/funcom$ time python3 bleu.py /scratch/funcom/data/outdir/predictions/predict-attendgru_E05_1551297717.txt
This will output a BLEU score for the prediction file.
About
Funcom Source Code Summarization Tool - Public Release
www.leclair.tech/data/funcom
Resources
Readme
GPL-3.0 license
Activity
Stars
34 stars
Watchers
3 watching
Forks
20 forks
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
BranchesTags
Repository files navigation
Funcom
Funcom Source Code Summarization Tool - Public Release
This repository contains the public release code for Funcom, a tool for source code summarization. Code summarization is the task of automatically generating natural language descriptions of source code.
Publications related to this work include:
LeClair, A., McMillan, C., "Recommendations for Datasets for Source Code Summarization", in Proc. of the 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL'19), Short Research Paper Track, Minneapolis, USA, June 2-7, 2019.
LeClair, A., Jiang, S., McMillan, C., "A Neural Model for Generating Natural Language Summaries of Program Subroutines", in Proc. of the 41st ACE/IEEE International Conference on Software Engineering (ICSE'19), Montreal, QC, Canada, May 25-31, 2019.
https://arxiv.org/abs/1902.01954
Example Output
Randomly sampled example output from the ast-attendgru model compared to reference good human-written summaries:
PROTOTYPE OUTPUT - HUMAN REFERENCE

returns the duration of the movie - get the full length of this movie in seconds

write a string to the client - write a string to all the connected clients

this method is called to indicate the next page in the page - call to explicitly go to the next page from within a single draw

returns a list of all the ids that match the given gene - get a list of superfamily ids for a gene name

compares two nodes by their UNK - compare nodes n1 and n2 by their dx entry

this method updates the tree panel - updates the tree panel with a new tree

returns the number of residues in the sequence - get number of interacting residues in domain b

returns true if the network is found - return true if passed inet address match a network which was used

log status message - log the status of the current message as info
Update Notice
This repository is archival for the ICSE'19 paper mentioned above. It is a good place to get started, but you may also want to look at our newer projects:
https://github.com/aakashba/callcon-public
https://github.com/Attn-to-FC/Attn-to-FC
https://github.com/acleclair/ICPC2020_GNN
USAGE
Step 0: Dependencies
We assume Ubuntu 18.04, Python 3.6, Keras 2.2.4, TensorFlow 1.12. Your milage may vary on different systems.
Step 1: Obtain Dataset
We provide a dataset of 2.1m Java methods and method comments, already cleaned and separated into training/val/test sets:
https://s3.us-east-2.amazonaws.com/icse2018/index.html
(Note: this paper is now several years old. Please see an update of data here: https://github.com/aakashba/callcon-public)
Extract the dataset to a directory (/scratch/ is the assumed default) so that you have a directory structure:

/scratch/funcom/data/standard/dataset.pkl

etc. in accordance with the files described on the site above.
To be consistent with defaults, create the following directories:

/scratch/funcom/data/outdir/models/

/scratch/funcom/data/outdir/histories/

/scratch/funcom/data/outdir/predictions/
Step 2: Train a Model
you@server:~/dev/funcom$ time python3 train.py --model-type=attendgru --gpu=0
Model types are defined in model.py. The ICSE'19 version is ast-attendgru, if you are seeking to reproduce it for comparision to your own models. Note that history information for each epoch is stored in a pkl file e.g. /scratch/funcom/data/outdir/histories/attendgru_hist_1551297717.pkl. The integer at the end of the file is the Epoch time at which training started, and is used to connect history, configuration, model, and prediction data. For example, training attendgru to epoch 5 would produce:
/scratch/funcom/data/outdir/histories/attendgru_conf_1551297717.pkl

/scratch/funcom/data/outdir/histories/attendgru_hist_1551297717.pkl

/scratch/funcom/data/outdir/models/attendgru_E01_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E02_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E03_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E04_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E05_1551297717.h5
A good baseline for initial work is the attendgru model. Comments in the file (models/attendgru.py) explain its behavior in detail, and it trains relatively quickly: about 45 minutes per epoch using batch size 200 on a single Quadro P5000, with maximum performance on the validation set at epoch 5.
Step 3: Inference / Prediction
you@server:~/dev/funcom$ time python3 predict.py /scratch/funcom/data/outdir/models/attendgru_E05_1551297717.h5 --gpu=0
The only necessary input to predict.py on the command line is the model file, but configuration information is read from the pkl files mentioned above. Output predictions will be written to a file e.g.:
/scratch/funcom/data/outdir/predictions/predict-attendgru_E05_1551297717.txt
Note that CPU prediction is possible in principle, but by default the attendgru and ast-attendgru models use CuDNNGRU instead of standard GRU, which necessitates using a GPU during prediction.
Step 4: Calculate Metrics
you@server:~/dev/funcom$ time python3 bleu.py /scratch/funcom/data/outdir/predictions/predict-attendgru_E05_1551297717.txt
This will output a BLEU score for the prediction file.
About
Funcom Source Code Summarization Tool - Public Release
www.leclair.tech/data/funcom
Resources
Readme
GPL-3.0 license
Activity
Stars
34 stars
Watchers
3 watching
Forks
20 forks
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
Funcom
Funcom Source Code Summarization Tool - Public Release
This repository contains the public release code for Funcom, a tool for source code summarization. Code summarization is the task of automatically generating natural language descriptions of source code.
Publications related to this work include:
LeClair, A., McMillan, C., "Recommendations for Datasets for Source Code Summarization", in Proc. of the 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL'19), Short Research Paper Track, Minneapolis, USA, June 2-7, 2019.
LeClair, A., Jiang, S., McMillan, C., "A Neural Model for Generating Natural Language Summaries of Program Subroutines", in Proc. of the 41st ACE/IEEE International Conference on Software Engineering (ICSE'19), Montreal, QC, Canada, May 25-31, 2019.
https://arxiv.org/abs/1902.01954
Example Output
Randomly sampled example output from the ast-attendgru model compared to reference good human-written summaries:
PROTOTYPE OUTPUT - HUMAN REFERENCE

returns the duration of the movie - get the full length of this movie in seconds

write a string to the client - write a string to all the connected clients

this method is called to indicate the next page in the page - call to explicitly go to the next page from within a single draw

returns a list of all the ids that match the given gene - get a list of superfamily ids for a gene name

compares two nodes by their UNK - compare nodes n1 and n2 by their dx entry

this method updates the tree panel - updates the tree panel with a new tree

returns the number of residues in the sequence - get number of interacting residues in domain b

returns true if the network is found - return true if passed inet address match a network which was used

log status message - log the status of the current message as info
Update Notice
This repository is archival for the ICSE'19 paper mentioned above. It is a good place to get started, but you may also want to look at our newer projects:
https://github.com/aakashba/callcon-public
https://github.com/Attn-to-FC/Attn-to-FC
https://github.com/acleclair/ICPC2020_GNN
USAGE
Step 0: Dependencies
We assume Ubuntu 18.04, Python 3.6, Keras 2.2.4, TensorFlow 1.12. Your milage may vary on different systems.
Step 1: Obtain Dataset
We provide a dataset of 2.1m Java methods and method comments, already cleaned and separated into training/val/test sets:
https://s3.us-east-2.amazonaws.com/icse2018/index.html
(Note: this paper is now several years old. Please see an update of data here: https://github.com/aakashba/callcon-public)
Extract the dataset to a directory (/scratch/ is the assumed default) so that you have a directory structure:

/scratch/funcom/data/standard/dataset.pkl

etc. in accordance with the files described on the site above.
To be consistent with defaults, create the following directories:

/scratch/funcom/data/outdir/models/

/scratch/funcom/data/outdir/histories/

/scratch/funcom/data/outdir/predictions/
Step 2: Train a Model
you@server:~/dev/funcom$ time python3 train.py --model-type=attendgru --gpu=0
Model types are defined in model.py. The ICSE'19 version is ast-attendgru, if you are seeking to reproduce it for comparision to your own models. Note that history information for each epoch is stored in a pkl file e.g. /scratch/funcom/data/outdir/histories/attendgru_hist_1551297717.pkl. The integer at the end of the file is the Epoch time at which training started, and is used to connect history, configuration, model, and prediction data. For example, training attendgru to epoch 5 would produce:
/scratch/funcom/data/outdir/histories/attendgru_conf_1551297717.pkl

/scratch/funcom/data/outdir/histories/attendgru_hist_1551297717.pkl

/scratch/funcom/data/outdir/models/attendgru_E01_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E02_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E03_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E04_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E05_1551297717.h5
A good baseline for initial work is the attendgru model. Comments in the file (models/attendgru.py) explain its behavior in detail, and it trains relatively quickly: about 45 minutes per epoch using batch size 200 on a single Quadro P5000, with maximum performance on the validation set at epoch 5.
Step 3: Inference / Prediction
you@server:~/dev/funcom$ time python3 predict.py /scratch/funcom/data/outdir/models/attendgru_E05_1551297717.h5 --gpu=0
The only necessary input to predict.py on the command line is the model file, but configuration information is read from the pkl files mentioned above. Output predictions will be written to a file e.g.:
/scratch/funcom/data/outdir/predictions/predict-attendgru_E05_1551297717.txt
Note that CPU prediction is possible in principle, but by default the attendgru and ast-attendgru models use CuDNNGRU instead of standard GRU, which necessitates using a GPU during prediction.
Step 4: Calculate Metrics
you@server:~/dev/funcom$ time python3 bleu.py /scratch/funcom/data/outdir/predictions/predict-attendgru_E05_1551297717.txt
This will output a BLEU score for the prediction file.
About
Funcom Source Code Summarization Tool - Public Release
www.leclair.tech/data/funcom
Resources
Readme
GPL-3.0 license
Activity
Stars
34 stars
Watchers
3 watching
Forks
20 forks
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
Funcom
Funcom Source Code Summarization Tool - Public Release
This repository contains the public release code for Funcom, a tool for source code summarization. Code summarization is the task of automatically generating natural language descriptions of source code.
Publications related to this work include:
LeClair, A., McMillan, C., "Recommendations for Datasets for Source Code Summarization", in Proc. of the 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL'19), Short Research Paper Track, Minneapolis, USA, June 2-7, 2019.
LeClair, A., Jiang, S., McMillan, C., "A Neural Model for Generating Natural Language Summaries of Program Subroutines", in Proc. of the 41st ACE/IEEE International Conference on Software Engineering (ICSE'19), Montreal, QC, Canada, May 25-31, 2019.
https://arxiv.org/abs/1902.01954
Example Output
Randomly sampled example output from the ast-attendgru model compared to reference good human-written summaries:
PROTOTYPE OUTPUT - HUMAN REFERENCE

returns the duration of the movie - get the full length of this movie in seconds

write a string to the client - write a string to all the connected clients

this method is called to indicate the next page in the page - call to explicitly go to the next page from within a single draw

returns a list of all the ids that match the given gene - get a list of superfamily ids for a gene name

compares two nodes by their UNK - compare nodes n1 and n2 by their dx entry

this method updates the tree panel - updates the tree panel with a new tree

returns the number of residues in the sequence - get number of interacting residues in domain b

returns true if the network is found - return true if passed inet address match a network which was used

log status message - log the status of the current message as info
Update Notice
This repository is archival for the ICSE'19 paper mentioned above. It is a good place to get started, but you may also want to look at our newer projects:
https://github.com/aakashba/callcon-public
https://github.com/Attn-to-FC/Attn-to-FC
https://github.com/acleclair/ICPC2020_GNN
USAGE
Step 0: Dependencies
We assume Ubuntu 18.04, Python 3.6, Keras 2.2.4, TensorFlow 1.12. Your milage may vary on different systems.
Step 1: Obtain Dataset
We provide a dataset of 2.1m Java methods and method comments, already cleaned and separated into training/val/test sets:
https://s3.us-east-2.amazonaws.com/icse2018/index.html
(Note: this paper is now several years old. Please see an update of data here: https://github.com/aakashba/callcon-public)
Extract the dataset to a directory (/scratch/ is the assumed default) so that you have a directory structure:

/scratch/funcom/data/standard/dataset.pkl

etc. in accordance with the files described on the site above.
To be consistent with defaults, create the following directories:

/scratch/funcom/data/outdir/models/

/scratch/funcom/data/outdir/histories/

/scratch/funcom/data/outdir/predictions/
Step 2: Train a Model
you@server:~/dev/funcom$ time python3 train.py --model-type=attendgru --gpu=0
Model types are defined in model.py. The ICSE'19 version is ast-attendgru, if you are seeking to reproduce it for comparision to your own models. Note that history information for each epoch is stored in a pkl file e.g. /scratch/funcom/data/outdir/histories/attendgru_hist_1551297717.pkl. The integer at the end of the file is the Epoch time at which training started, and is used to connect history, configuration, model, and prediction data. For example, training attendgru to epoch 5 would produce:
/scratch/funcom/data/outdir/histories/attendgru_conf_1551297717.pkl

/scratch/funcom/data/outdir/histories/attendgru_hist_1551297717.pkl

/scratch/funcom/data/outdir/models/attendgru_E01_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E02_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E03_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E04_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E05_1551297717.h5
A good baseline for initial work is the attendgru model. Comments in the file (models/attendgru.py) explain its behavior in detail, and it trains relatively quickly: about 45 minutes per epoch using batch size 200 on a single Quadro P5000, with maximum performance on the validation set at epoch 5.
Step 3: Inference / Prediction
you@server:~/dev/funcom$ time python3 predict.py /scratch/funcom/data/outdir/models/attendgru_E05_1551297717.h5 --gpu=0
The only necessary input to predict.py on the command line is the model file, but configuration information is read from the pkl files mentioned above. Output predictions will be written to a file e.g.:
/scratch/funcom/data/outdir/predictions/predict-attendgru_E05_1551297717.txt
Note that CPU prediction is possible in principle, but by default the attendgru and ast-attendgru models use CuDNNGRU instead of standard GRU, which necessitates using a GPU during prediction.
Step 4: Calculate Metrics
you@server:~/dev/funcom$ time python3 bleu.py /scratch/funcom/data/outdir/predictions/predict-attendgru_E05_1551297717.txt
This will output a BLEU score for the prediction file.
About
Funcom Source Code Summarization Tool - Public Release
www.leclair.tech/data/funcom
Resources
Readme
GPL-3.0 license
Activity
Stars
34 stars
Watchers
3 watching
Forks
20 forks
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
BranchesTags
Repository files navigation
Funcom
Funcom Source Code Summarization Tool - Public Release
This repository contains the public release code for Funcom, a tool for source code summarization. Code summarization is the task of automatically generating natural language descriptions of source code.
Publications related to this work include:
LeClair, A., McMillan, C., "Recommendations for Datasets for Source Code Summarization", in Proc. of the 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL'19), Short Research Paper Track, Minneapolis, USA, June 2-7, 2019.
LeClair, A., Jiang, S., McMillan, C., "A Neural Model for Generating Natural Language Summaries of Program Subroutines", in Proc. of the 41st ACE/IEEE International Conference on Software Engineering (ICSE'19), Montreal, QC, Canada, May 25-31, 2019.
https://arxiv.org/abs/1902.01954
Example Output
Randomly sampled example output from the ast-attendgru model compared to reference good human-written summaries:
PROTOTYPE OUTPUT - HUMAN REFERENCE

returns the duration of the movie - get the full length of this movie in seconds

write a string to the client - write a string to all the connected clients

this method is called to indicate the next page in the page - call to explicitly go to the next page from within a single draw

returns a list of all the ids that match the given gene - get a list of superfamily ids for a gene name

compares two nodes by their UNK - compare nodes n1 and n2 by their dx entry

this method updates the tree panel - updates the tree panel with a new tree

returns the number of residues in the sequence - get number of interacting residues in domain b

returns true if the network is found - return true if passed inet address match a network which was used

log status message - log the status of the current message as info
Update Notice
This repository is archival for the ICSE'19 paper mentioned above. It is a good place to get started, but you may also want to look at our newer projects:
https://github.com/aakashba/callcon-public
https://github.com/Attn-to-FC/Attn-to-FC
https://github.com/acleclair/ICPC2020_GNN
USAGE
Step 0: Dependencies
We assume Ubuntu 18.04, Python 3.6, Keras 2.2.4, TensorFlow 1.12. Your milage may vary on different systems.
Step 1: Obtain Dataset
We provide a dataset of 2.1m Java methods and method comments, already cleaned and separated into training/val/test sets:
https://s3.us-east-2.amazonaws.com/icse2018/index.html
(Note: this paper is now several years old. Please see an update of data here: https://github.com/aakashba/callcon-public)
Extract the dataset to a directory (/scratch/ is the assumed default) so that you have a directory structure:

/scratch/funcom/data/standard/dataset.pkl

etc. in accordance with the files described on the site above.
To be consistent with defaults, create the following directories:

/scratch/funcom/data/outdir/models/

/scratch/funcom/data/outdir/histories/

/scratch/funcom/data/outdir/predictions/
Step 2: Train a Model
you@server:~/dev/funcom$ time python3 train.py --model-type=attendgru --gpu=0
Model types are defined in model.py. The ICSE'19 version is ast-attendgru, if you are seeking to reproduce it for comparision to your own models. Note that history information for each epoch is stored in a pkl file e.g. /scratch/funcom/data/outdir/histories/attendgru_hist_1551297717.pkl. The integer at the end of the file is the Epoch time at which training started, and is used to connect history, configuration, model, and prediction data. For example, training attendgru to epoch 5 would produce:
/scratch/funcom/data/outdir/histories/attendgru_conf_1551297717.pkl

/scratch/funcom/data/outdir/histories/attendgru_hist_1551297717.pkl

/scratch/funcom/data/outdir/models/attendgru_E01_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E02_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E03_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E04_1551297717.h5

/scratch/funcom/data/outdir/models/attendgru_E05_1551297717.h5
A good baseline for initial work is the attendgru model. Comments in the file (models/attendgru.py) explain its behavior in detail, and it trains relatively quickly: about 45 minutes per epoch using batch size 200 on a single Quadro P5000, with maximum performance on the validation set at epoch 5.
Step 3: Inference / Prediction
you@server:~/dev/funcom$ time python3 predict.py /scratch/funcom/data/outdir/models/attendgru_E05_1551297717.h5 --gpu=0
The only necessary input to predict.py on the command line is the model file, but configuration information is read from the pkl files mentioned above. Output predictions will be written to a file e.g.:
/scratch/funcom/data/outdir/predictions/predict-attendgru_E05_1551297717.txt
Note that CPU prediction is possible in principle, but by default the attendgru and ast-attendgru models use CuDNNGRU instead of standard GRU, which necessitates using a GPU during prediction.
Step 4: Calculate Metrics
you@server:~/dev/funcom$ time python3 bleu.py /scratch/funcom/data/outdir/predictions/predict-attendgru_E05_1551297717.txt
This will output a BLEU score for the prediction file.
About
Funcom Source Code Summarization Tool - Public Release
www.leclair.tech/data/funcom
Resources
Readme
GPL-3.0 license
Activity
Stars
34 stars
Watchers
3 watching
Forks
20 forks
Report repository
Releases
Packages
Contributors
Languages