Uh oh!
There was an error while loading. Please reload this page.
add document for GenAI - #7170
Conversation
@NiklasGustafsson@stephentoub@SteveSandersonMS can you please take a look and provide feedback. This is the design document for the GenAI library which consolidates the models ported to TorchSharp (Phi, Llama, StableDiffusion) into a single library for easier consumption. |
Uh oh!
There was an error while loading. Please reload this page.
| { | ||
| public virtual ( | ||
| Tensor, // output token ids [batch_size, sequence_length] | ||
| Tensor // output logits [batch_size, sequence_length, vocab_size] |
There was a problem hiding this comment.
Is this the TorchSharp tensor type? Can / should it become the System.Numerics.Tensors one?
There was a problem hiding this comment.
Is this the TorchSharp tensor type
Yes
should it become the System.Numerics.Tensors one?
If System.Numerics.Tensors can work with libtorch, then maybe yes.
There was a problem hiding this comment.
We're some ways from having Tensor integrated with TorchSharp, it is still experimental and subject to change.
There was a problem hiding this comment.
And it would be the TorchSharp shim implementing ITensor, not Tensor from SNT.
There was a problem hiding this comment.
Presumably it would be ITensor<T> such that a Tensor<T> could be passed in? The implementation could type test if it were a TorchSharp implementation and act accordingly, presumably.
| where TTokenizer : ITokenizer | ||
| where TCausalLM : nn.Module<CausalLanguageModelInput, CausalLanguageModelOutput> | ||
| { | ||
| public CausalLMPipeline<LLama2Tokenizer, Phi3ForCasualLM> Create(LLama2Tokenizer tokenizer, Phi3ForCasualLM model); |
There was a problem hiding this comment.
Presumably this is intended to instead be a static method on the base class?
There was a problem hiding this comment.
Hmmm on the second thought, maybe not. Because CasualLMPipeline and CasualLMPipeline<TTokenizer, TCasualLM> will live in GenAI.Core while Phi3ForCasualLM lives in GenAI.Phi3.
In that case, this Create method is no longer necessary exist, using a constructor to take TTokenzier and TCausalLM would be easier to understand
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
We are excited to review your PR.
So we can do the best job, please check:
Fixes #nnnnin your description to cause GitHub to automatically close the issue(s) when your PR is merged.Add desgin document for Add GenAI packages #7169