Fine-tuning performance between Apple and Nvidia

David Pickett

22 May 2025 — 1 min read

People:

David

Idea:

Comparing fine-tuning performance on MacBook M3 Max, Mac Studio M1 Ultra, and Nvidia 4090 using MLX and Unsloth

Details:

Tested fine-tuning Phi-3-mini-4k-instruct model
Followed this Jan 2025 MLX guide for Apple hardware
Used Unsloth library for Nvidia GPU
Dataset had 627 examples and used 500 training steps
M1 Ultra achieved ~260-325 tokens/sec
M3 Max MacBook faster at ~350-420 tokens/sec
Nvidia 4090 (Unsloth) completed training in about 6.42 minutes (probably can optimize a lot more?)
Peak memory usage was consistently ~8.1 GB
Had to convert MLX dataset into Parquet format for Nvidia GPU
Unsloth v1 currently supports only single GPU configurations
Experimented with Llama-Factory using Deepspeed for multi-GPU training but seems like the example dataset from the above would need to be reformatted

Read more

Benchmarking Runpod cloud GPUs

People David Idea Exploring cloud GPU performance, cost, and usability for running open-source AI models (in comparison to local hardware and in context of recent learning on software to handle concurrent users) Details * Compared RunPod serverless and pods using Nvidia (vLLM) and AMD (sglang) * Benchmarked Nvidia 3090, 4090, RTX 6000

Comparing Onyx vs Morphik (whole PMF Deep Research)

People: * David Idea: * Comparing two open-source projects, Onyx and Morphik, for self-hosted semantic search (and deep research) across Google Drive files (PDFs, Docs, Sheets, and Slides) Details: * Onyx: Great text-based search, flexible embedding models * Morphik: Powerful multi-modal search (text, images, graphs) * Onyx easily connects and syncs with Google Drive * Morphik

Designing Po‘owai with Magicpatterns.com

A Real-World Test of Prompt-to-UI AI Creating a usable interface for a brand-new app often eats up early project time. To see how far AI can shorten that stage, we pointed Magicpatterns.com at a brand idea called Po‘owai, a cash-flow tool for non-profits that have to juggle restricted

From Bolt.ai to Your Laptop in Four Easy Steps

Bringing the Ho‘omo‘o App Home — No Tech Jargon Required You played with Ho‘omo‘o in Bolt.ai’s browser editor and now you’d like to run it on your own computer and save it on GitHub for safekeeping. Good news: it’s basically a three-minute job.