Uh oh!
There was an error while loading. Please reload this page.
Created samples for 'ProduceWordBags' and 'ProduceHashedWordBags' API. - #3183
Merged
Conversation
Codecov Report
@@ Coverage Diff @@## master #3183 +/- ##
==========================================
+ Coverage 72.54% 72.58% +0.04%
==========================================
Files 807 807 Lines 144774 144956 +182 Branches 16208 16212 +4 ==========================================
+ Hits 105021 105223 +202 + Misses 35339 35318 -21 - Partials 4414 4415 +1
|
| new TextData(){ Text = "This is an example to compute bag-of-word features using hashing." }, | ||
| new TextData(){ Text = "ML.NET's ProduceHashedWordBags API produces count of Ngrams and hashes it as an index into a vector of given bit length." }, | ||
| new TextData(){ Text = "It does so by first tokenizing text/string into words/tokens then " }, | ||
| new TextData(){ Text = "computing Ngram and hash them to the index given by hash value." }, |
Contributor
There was a problem hiding this comment.
Ngram [](start = 50, length = 5)
Ngrams ? #Resolved
| new TextData(){ Text = "ML.NET's ProduceHashedWordBags API produces count of Ngrams and hashes it as an index into a vector of given bit length." }, | ||
| new TextData(){ Text = "It does so by first tokenizing text/string into words/tokens then " }, | ||
| new TextData(){ Text = "computing Ngram and hash them to the index given by hash value." }, | ||
| new TextData(){ Text = "The hashing schem reduces the size of the output feature vector" }, |
Contributor
There was a problem hiding this comment.
schem [](start = 52, length = 5)
schema #Resolved
| // Expected output: | ||
| // Number of Features: 256 | ||
| // Features: 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 |
Contributor
There was a problem hiding this comment.
0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 [](start = 30, length = 78)
This is kinda anticlimactic.
Maybe change numberOfBits to smaller value to get some results different from zero here? #Resolved
| new TextData(){ Text = "ML.NET's ProduceWordBags API produces bag-of-word features from input text." }, | ||
| new TextData(){ Text = "It does so by first tokenizing text/string into words/tokens then " }, | ||
| new TextData(){ Text = "computing Ngram and their neumeric values." }, | ||
| new TextData(){ Text = "Each position in the output vector corresponds to a particular Ngram." }, |
Contributor
There was a problem hiding this comment.
Ngram [](start = 103, length = 5)
https://en.wikipedia.org/wiki/N-gram
We call it NGram in code because you can't use - in variable, but for explanation maybe we should use n-gram? #Resolved
| new TextData(){ Text = "Each position in the output vector corresponds to a particular Ngram." }, | ||
| new TextData(){ Text = "The value at each position corresponds to," }, | ||
| new TextData(){ Text = "the number of times Ngram occured in the data (Tf), or" }, | ||
| new TextData(){ Text = "the inverse of the number of documents contain the Ngram (Idf), or." }, |
Contributor
There was a problem hiding this comment.
or. [](start = 104, length = 3)
double or, plus dot in the end. I would omit this or. #Resolved
| var transformedDataView = textTransformer.Transform(dataview); | ||
| // Create the prediction engine to get the bag-of-word features extracted from the text. | ||
| var predictionEngine = mlContext.Model.CreatePredictionEngine<TextData, TransformedTextData>(textTransformer); |
Contributor
There was a problem hiding this comment.
Why predictionEngine rather than TakeRows and ConvertToEnumerable? I would use the one we would recommend people to use to inspect data in practice.
| // Preview of the produced n-grams. | ||
| // Get the slot names from the column's metadata. | ||
| // If the column is a vector column the slot names corresponds to the names associated with each position in the vector. |
Contributor
There was a problem hiding this comment.
If the column is a vector column [](start = 15, length = 32)
The If clause is confusing here. I would just drop it. #Resolved
rogancarr
approved these changes
Apr 4, 2019
zeahmed added a commit
to zeahmed/machinelearning
that referenced
this pull request
Apr 8, 2019
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related to #1209.