feat(template): read jinja templates from gguf files

[mudler]

Description

This PR adds automatic detection and parsing of jinja templates in gguf files. If we fail to identify a variant and we do not have already a specific template, it injects the jinja templates which is part of the model metadata if one is found.

Alternatively, it is possible to enable jinja templates manually in the model config file, in the template config section with jinja_template: true.

Notes for Reviewers

• This is extracted from feat: Realtime API support #3722 as it refactors message templating in a way that is more re-usable by other endpoints.
• Refactors mainly how the template is processed, removing the business logic from the handler and separate it so can be re-used from other parts (e.g. realtime needs same templating)
• Refactors some of the dependency passed down to the routes as it adds another component (Template Evaluator) aside the model loader
• It might cross with feat: Centralized Request Processing middleware #3847 ( didn't checked closely ) but with other PRs too as it refactors some core parts, however, if we don't move this forward there will be too many changes that requires this adaptation or introduces dups

Signed commits

• Yes, I signed my commits.

[netlify]

✅ Deploy Preview for localai ready!

Name	Link
🔨 Latest commit	56f6ab4
🔍 Latest deploy log	https://app.netlify.com/sites/localai/deploys/67557dade0632500089601d4
😎 Deploy Preview	https://deploy-preview-4332--localai.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[mudler]

WIP as need to still add mapping between the transformer tokenizer and the templates (see TODO note in code comments)

[mudler]

basic support should work (tested with llama3 prompt), probably is not going to cover all cases as gonja has limitations, but, since this kicks-in when no other template was defined it is safe to merge without drawbacks.

[mudler]

Description

This PR adds automatic detection and parsing of jinja templates in gguf files. If we fail to identify a variant and we do not have already a specific template, it injects the jinja templates which is part of the model metadata if one is found.

Alternatively, it is possible to enable jinja templates manually in the model config file, in the template config section with jinja_template: true.

Notes for Reviewers

• This is extracted from feat: Realtime API support #3722 as it refactors message templating in a way that is more re-usable by other endpoints.
• Refactors mainly how the template is processed, removing the business logic from the handler and separate it so can be re-used from other parts (e.g. realtime needs same templating)
• Refactors some of the dependency passed down to the routes as it adds another component (Template Evaluator) aside the model loader
• It might cross with feat: Centralized Request Processing middleware #3847 ( didn't checked closely ) but with other PRs too as it refactors some core parts, however, if we don't move this forward there will be too many changes that requires this adaptation or introduces dups

Signed commits

• Yes, I signed my commits.

[netlify]

✅ Deploy Preview for localai ready!

Name	Link
🔨 Latest commit	56f6ab4
🔍 Latest deploy log	https://app.netlify.com/sites/localai/deploys/67557dade0632500089601d4
😎 Deploy Preview	https://deploy-preview-4332--localai.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[mudler]

WIP as need to still add mapping between the transformer tokenizer and the templates (see TODO note in code comments)

[mudler]

basic support should work (tested with llama3 prompt), probably is not going to cover all cases as gonja has limitations, but, since this kicks-in when no other template was defined it is safe to merge without drawbacks.

[mudler]

Description

This PR adds automatic detection and parsing of jinja templates in gguf files. If we fail to identify a variant and we do not have already a specific template, it injects the jinja templates which is part of the model metadata if one is found.

Alternatively, it is possible to enable jinja templates manually in the model config file, in the template config section with jinja_template: true.

Notes for Reviewers

• This is extracted from feat: Realtime API support #3722 as it refactors message templating in a way that is more re-usable by other endpoints.
• Refactors mainly how the template is processed, removing the business logic from the handler and separate it so can be re-used from other parts (e.g. realtime needs same templating)
• Refactors some of the dependency passed down to the routes as it adds another component (Template Evaluator) aside the model loader
• It might cross with feat: Centralized Request Processing middleware #3847 ( didn't checked closely ) but with other PRs too as it refactors some core parts, however, if we don't move this forward there will be too many changes that requires this adaptation or introduces dups

Signed commits

• Yes, I signed my commits.

[netlify]

✅ Deploy Preview for localai ready!

Name	Link
🔨 Latest commit	56f6ab4
🔍 Latest deploy log	https://app.netlify.com/sites/localai/deploys/67557dade0632500089601d4
😎 Deploy Preview	https://deploy-preview-4332--localai.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[mudler]

WIP as need to still add mapping between the transformer tokenizer and the templates (see TODO note in code comments)

[mudler]

basic support should work (tested with llama3 prompt), probably is not going to cover all cases as gonja has limitations, but, since this kicks-in when no other template was defined it is safe to merge without drawbacks.

[mudler]

Description

This PR adds automatic detection and parsing of jinja templates in gguf files. If we fail to identify a variant and we do not have already a specific template, it injects the jinja templates which is part of the model metadata if one is found.

Alternatively, it is possible to enable jinja templates manually in the model config file, in the template config section with jinja_template: true.

Notes for Reviewers

• This is extracted from feat: Realtime API support #3722 as it refactors message templating in a way that is more re-usable by other endpoints.
• Refactors mainly how the template is processed, removing the business logic from the handler and separate it so can be re-used from other parts (e.g. realtime needs same templating)
• Refactors some of the dependency passed down to the routes as it adds another component (Template Evaluator) aside the model loader
• It might cross with feat: Centralized Request Processing middleware #3847 ( didn't checked closely ) but with other PRs too as it refactors some core parts, however, if we don't move this forward there will be too many changes that requires this adaptation or introduces dups

Signed commits

• Yes, I signed my commits.

[netlify]

✅ Deploy Preview for localai ready!

Name	Link
🔨 Latest commit	56f6ab4
🔍 Latest deploy log	https://app.netlify.com/sites/localai/deploys/67557dade0632500089601d4
😎 Deploy Preview	https://deploy-preview-4332--localai.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[mudler]

WIP as need to still add mapping between the transformer tokenizer and the templates (see TODO note in code comments)

[mudler]

basic support should work (tested with llama3 prompt), probably is not going to cover all cases as gonja has limitations, but, since this kicks-in when no other template was defined it is safe to merge without drawbacks.

[mudler]

Description

This PR adds automatic detection and parsing of jinja templates in gguf files. If we fail to identify a variant and we do not have already a specific template, it injects the jinja templates which is part of the model metadata if one is found.

Alternatively, it is possible to enable jinja templates manually in the model config file, in the template config section with jinja_template: true.

Notes for Reviewers

• This is extracted from feat: Realtime API support #3722 as it refactors message templating in a way that is more re-usable by other endpoints.
• Refactors mainly how the template is processed, removing the business logic from the handler and separate it so can be re-used from other parts (e.g. realtime needs same templating)
• Refactors some of the dependency passed down to the routes as it adds another component (Template Evaluator) aside the model loader
• It might cross with feat: Centralized Request Processing middleware #3847 ( didn't checked closely ) but with other PRs too as it refactors some core parts, however, if we don't move this forward there will be too many changes that requires this adaptation or introduces dups

Signed commits

• Yes, I signed my commits.

[netlify]

✅ Deploy Preview for localai ready!

Name	Link
🔨 Latest commit	56f6ab4
🔍 Latest deploy log	https://app.netlify.com/sites/localai/deploys/67557dade0632500089601d4
😎 Deploy Preview	https://deploy-preview-4332--localai.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[mudler]

WIP as need to still add mapping between the transformer tokenizer and the templates (see TODO note in code comments)

[mudler]

basic support should work (tested with llama3 prompt), probably is not going to cover all cases as gonja has limitations, but, since this kicks-in when no other template was defined it is safe to merge without drawbacks.

[mudler]

Description

This PR adds automatic detection and parsing of jinja templates in gguf files. If we fail to identify a variant and we do not have already a specific template, it injects the jinja templates which is part of the model metadata if one is found.

Alternatively, it is possible to enable jinja templates manually in the model config file, in the template config section with jinja_template: true.

Notes for Reviewers

• This is extracted from feat: Realtime API support #3722 as it refactors message templating in a way that is more re-usable by other endpoints.
• Refactors mainly how the template is processed, removing the business logic from the handler and separate it so can be re-used from other parts (e.g. realtime needs same templating)
• Refactors some of the dependency passed down to the routes as it adds another component (Template Evaluator) aside the model loader
• It might cross with feat: Centralized Request Processing middleware #3847 ( didn't checked closely ) but with other PRs too as it refactors some core parts, however, if we don't move this forward there will be too many changes that requires this adaptation or introduces dups

Signed commits

• Yes, I signed my commits.

[netlify]

✅ Deploy Preview for localai ready!

Name	Link
🔨 Latest commit	56f6ab4
🔍 Latest deploy log	https://app.netlify.com/sites/localai/deploys/67557dade0632500089601d4
😎 Deploy Preview	https://deploy-preview-4332--localai.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[mudler]

WIP as need to still add mapping between the transformer tokenizer and the templates (see TODO note in code comments)

[mudler]

basic support should work (tested with llama3 prompt), probably is not going to cover all cases as gonja has limitations, but, since this kicks-in when no other template was defined it is safe to merge without drawbacks.

[mudler]

Description

This PR adds automatic detection and parsing of jinja templates in gguf files. If we fail to identify a variant and we do not have already a specific template, it injects the jinja templates which is part of the model metadata if one is found.

Alternatively, it is possible to enable jinja templates manually in the model config file, in the template config section with jinja_template: true.

Notes for Reviewers

• This is extracted from feat: Realtime API support #3722 as it refactors message templating in a way that is more re-usable by other endpoints.
• Refactors mainly how the template is processed, removing the business logic from the handler and separate it so can be re-used from other parts (e.g. realtime needs same templating)
• Refactors some of the dependency passed down to the routes as it adds another component (Template Evaluator) aside the model loader
• It might cross with feat: Centralized Request Processing middleware #3847 ( didn't checked closely ) but with other PRs too as it refactors some core parts, however, if we don't move this forward there will be too many changes that requires this adaptation or introduces dups

Signed commits

• Yes, I signed my commits.

[netlify]

✅ Deploy Preview for localai ready!

Name	Link
🔨 Latest commit	56f6ab4
🔍 Latest deploy log	https://app.netlify.com/sites/localai/deploys/67557dade0632500089601d4
😎 Deploy Preview	https://deploy-preview-4332--localai.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[mudler]

WIP as need to still add mapping between the transformer tokenizer and the templates (see TODO note in code comments)

[mudler]

basic support should work (tested with llama3 prompt), probably is not going to cover all cases as gonja has limitations, but, since this kicks-in when no other template was defined it is safe to merge without drawbacks.

[mudler]

Description

This PR adds automatic detection and parsing of jinja templates in gguf files. If we fail to identify a variant and we do not have already a specific template, it injects the jinja templates which is part of the model metadata if one is found.

Alternatively, it is possible to enable jinja templates manually in the model config file, in the template config section with jinja_template: true.

Notes for Reviewers

• This is extracted from feat: Realtime API support #3722 as it refactors message templating in a way that is more re-usable by other endpoints.
• Refactors mainly how the template is processed, removing the business logic from the handler and separate it so can be re-used from other parts (e.g. realtime needs same templating)
• Refactors some of the dependency passed down to the routes as it adds another component (Template Evaluator) aside the model loader
• It might cross with feat: Centralized Request Processing middleware #3847 ( didn't checked closely ) but with other PRs too as it refactors some core parts, however, if we don't move this forward there will be too many changes that requires this adaptation or introduces dups

Signed commits

• Yes, I signed my commits.

[netlify]

✅ Deploy Preview for localai ready!

Name	Link
🔨 Latest commit	56f6ab4
🔍 Latest deploy log	https://app.netlify.com/sites/localai/deploys/67557dade0632500089601d4
😎 Deploy Preview	https://deploy-preview-4332--localai.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[mudler]

WIP as need to still add mapping between the transformer tokenizer and the templates (see TODO note in code comments)

[mudler]

basic support should work (tested with llama3 prompt), probably is not going to cover all cases as gonja has limitations, but, since this kicks-in when no other template was defined it is safe to merge without drawbacks.