Tag
#tokenizer
17 repositories
Repos
A faster than tiktoken tokenizer with first-class support for Vercel's AI SDK.
C++ Lexer Toolkit Library (LexerTk) https://www.partow.net/programming/lexertk/index.html
A C++ library to interact with TI-z80 (82/83/84 series) calculators files (programs, lists, matrices, etc.)
Converts a string into an array of tokens, with useful methods for looking ahead and behind, capturing, matching, et cetera.
3DPass mobile light wallet for Android and iOS. 3D object recognition and wallet functions.
A (nicer) tokenizer you want to use for model inference and training: with all known peventable gotchas normalized or auto-fixed.
Powder is my attempt to program a scripting language that compiles down to byte codes that are interpreted and executed by a virtual machine. The language itself is a cross between C and Python, with a few unique syntax features.
MOAT is a Solana-based token powered by Token-2022 extensions and Anchor framework, designed to provide a secure, transparent, and scalable ecosystem for long-term tokenomics.
J'échange 🍕 contre ce token, jure il va prendre de la valeur !
Ce projet permet d'apprendre les bases du web3 en créant un token sur une blockchain.