All issues

Search results

Replace custom HF tokenizer with C++ bindings for HF tokenizer

Status: Open.

#153In meta-pytorch/tokenizers;

· jackzhxng opened on Oct 28, 2025

Use prompt template from HF tokenizer

enhancementNew feature or requestNew feature or request

Status: Open.

#131In meta-pytorch/tokenizers;

· metascroy opened on Oct 2, 2025

Package tokenizers into swift PM

Status: Open.

#83In meta-pytorch/tokenizers;

· larryliu0820 opened on Jun 11, 2025

Distinguish between public and internal headers

Status: Open.

#52In meta-pytorch/tokenizers;

· kirklandsign opened on Apr 17, 2025

[Feature request] Java binding for tokenizers

Status: Open.

#40In meta-pytorch/tokenizers;

· kirklandsign opened on Mar 31, 2025