Hi HN! I'm just sharing a project I've been working on during the LLM Efficiency Challenge - you can now finetune Llama with QLoRA 5x faster than Huggingface's original implementation on your own local GPU. Some highlights:
1. Manual autograd engine - hand derived backprop steps.
2. QLoRA / LoRA 80% faster, 50% less memory.
3. All kernels written in OpenAI's Triton language.
4. 0% loss in accuracy - no approximation methods - all exact.
5. No change of hardware necessary. Supports NVIDIA GPUs since 2018+. CUDA 7.5+.
6. Flash Attention support via Xformers.
7. Supports 4bit and 16bit LoRA finetuning.
8. Train Slim Orca fully locally in 260 hours from 1301 hours (5x faster).
9. Open source version trains 5x faster or you can check out Unsloth Pro and Max codepaths for 30x faster training!
Hopefully you can try it out! Wrote a blog post at https://unsloth.ai/introducing if you want to learn more about our manual hand derived backprop or Triton kernels and stuff! Thanks once again!
Hey HN! We’re building FinetuneDB (https://finetunedb.com/), an LLM fine-tuning platform. It enables teams to easily create and manage high-quality datasets, and streamlines the entire workflow from fine-tuning to serving and evaluating models with domain experts. You can check out our docs here: (https://docs.finetunedb.com/)
FinetuneDB exists because creating and managing high-quality datasets is a real bottleneck when fine-tuning LLMs. The quality of your data directly impacts the performance of your fine-tuned models, and existing tools didn’t offer an easy way for teams to build, organize, and iterate on their datasets. We’ve been working closely with our pilot customers, both AI startups and more traditional businesses like a large newspaper, which is fine-tuning models on their articles to automate content generation in their tone of voice.
The platform is built with an end-to-end workflow in mind, from dataset building, fine-tuning, serving, and evaluating outputs. The centerpiece is a version-controlled, no-code dataset manager where you can upload existing datasets in JSONL, use production data, or collaborate with domain experts to create high-quality datasets for custom use cases. We also offer evaluation workflows that allow non-technical contributors to annotate data, review model outputs, and refine responses (LLM-as-judge also available).
We offer:
- A free tier for developers and hobbyists who want to streamline dataset management.
- Business-tier with full feature access for teams, using per-seat pricing.
- A custom tier for model hosting, custom integrations, and self-hosting.
Most users still use OpenAI models, but if you're working with open-source LLMs, we offer pay-as-you-go pricing for serverless inference for Llama and Mistral models with up to €100 in free credits to get started.
We're in public beta right now, so any feedback—whether it’s about features, usability, or anything else—would be incredibly valuable.
If you've worked on fine-tuning models before or are curious about custom LLMs, we’d love to hear from you. Our goal is to make the fine-tuning process more accessible and help more companies leverage their data and domain experts to create custom LLMs.
1. Manual autograd engine - hand derived backprop steps.
2. QLoRA / LoRA 80% faster, 50% less memory.
3. All kernels written in OpenAI's Triton language.
4. 0% loss in accuracy - no approximation methods - all exact.
5. No change of hardware necessary. Supports NVIDIA GPUs since 2018+. CUDA 7.5+.
6. Flash Attention support via Xformers.
7. Supports 4bit and 16bit LoRA finetuning.
8. Train Slim Orca fully locally in 260 hours from 1301 hours (5x faster).
9. Open source version trains 5x faster or you can check out Unsloth Pro and Max codepaths for 30x faster training!
https://www.reddit.com/r/LocalLLaMA/comments/188197j/80_fast... has more info about Unsloth!
Hopefully you can try it out! Wrote a blog post at https://unsloth.ai/introducing if you want to learn more about our manual hand derived backprop or Triton kernels and stuff! Thanks once again!