j_r_r_token (1.0.4-x86_64-linux)

Logan Bresnahan
2025-08-02T04:01:04Z

Ruby wrapper for the tiktoken Rust library, providing fast tokenization for OpenAI models.

JRRToken is a Ruby gem that wraps the tiktoken Rust library, enabling fast and efficient tokenization for OpenAI models. It supports multiple models including o200k_base, cl100k_base, p50k_base, p50k_edit, and r50k_base.