Learn MLX by reading a port from the inside out.
Start with arrays, lazy execution, and unified memory. Then read a source model, translate PyTorch and CUDA assumptions, prove the first matching MLX path, and only then use the executable tools and optimization runbooks.
What MLX is—and what changes
MLX is an array framework for machine learning on Apple silicon. Its Python surface is deliberately familiar, but its execution model matters more than the syntax: computations are lazy, CPU and GPU operate over unified memory, and function transforms compose with ordinary Python.
NumPy-like foundations
mx.array and mlx.nn make the readable port feel familiar before optimization begins.
Unified, not magical
Operations can use CPU or GPU without explicit tensor transfers. Interop boundaries may still copy or force evaluation.
Lazy by default
Expressions build a graph. Place mx.eval where a dependency, comparison, or measurement needs realized values.
Compile stable work
mx.compile belongs around pure, stable, reused regions after the eager path is correct.
Keep parameters inspectable
mlx.nn.Module organizes layers, parameters, and nested state so the readable graph and weight map stay reviewable.
Compose behavior around functions
Gradient, value-and-gradient, vectorization, and related function transforms wrap ordinary functions instead of changing the model contract.
Place operations deliberately
Devices and streams belong to operation placement. Extra streams are an explicit scheduling tool, not an automatic overlap or speed guarantee.
MLX is an array framework for machine learning on Apple silicon with a NumPy-like API.
Lazy execution and unified memory change evaluation, state, placement, and measurement boundaries.
Source projects commonly assume eager tensors plus explicit CPU-to-CUDA movement.
Build with arrays and modules; choose streams at operations and evaluate only at deliberate boundaries.
Create an array expression, then call mx.eval only where its result becomes observable.
Treating unified memory as a promise that every framework bridge is zero-copy.
Explain when an expression is built and when its result is actually needed.
Learn what a port must preserve beyond weights.
What is actually being ported?
It reproduces the source computation, preprocessing, state transitions, weight meaning, numerical policy, and user-visible outputs in a new runtime.
Moves representation
Renames, reshapes, transposes, splits, merges, or changes dtype. Necessary, never sufficient.
Reproduces behavior
Rebuilds operators, control flow, masks, positions, cache or recurrent state, and task semantics in MLX.
Packages a proven path
Adds serving, lifecycle, observability, and distribution after correctness and workload gates exist.
A checkpoint can load while the model is wrong. A tensor with the right shape can still be transposed incorrectly. Matching final text can hide a broken cache that fails on the next request. The proof must follow the computation.
A port reproduces a source model's behavior in a new runtime.
Conversion alone cannot preserve control flow, layouts, masks, cache semantics, preprocessing, or generation.
The pinned source implementation and its outputs are the contract, not only its state_dict.
Rebuild the computation, transform every weight explicitly, and compare staged outputs to a source oracle.
Match inputs, one primitive, one block, full output, stateful decode, and task behavior in order.
Calling successfully loaded tensors a completed port before output parity exists.
Name the semantics, transforms, state, and outputs that need evidence.
Read the model before choosing an implementation route.
Read the model before choosing a method
Model selection starts with architecture recognition. A task label such as “speech” or “vision” describes the input and output; it does not describe the repeated graph. Keep modality, architecture family, source format, state, and workload as separate decisions.
What crosses the boundary?
Text IDs, pixels, waveforms, mel features, graph nodes, latents, timestamps, or structured values.
What repeats?
Decoder blocks, encoder blocks, expert routing, recurrent scans, denoising steps, convolution stacks, or message passing.
What state survives?
KV cache, recurrent state, streaming buffers, scheduler state, prefix features, or no mutable inference state.
What surrounds the network?
Tokenizer, image processor, audio frontend, scheduler, codec, VAE, sampling loop, or task post-processing.
What is the artifact?
Safetensors, a local module, ONNX, GGUF, Keras, Flax/Orbax, TensorFlow, Core ML, or an executable-risk format.
What must the workload do?
Single request, long-context prefill, cached decode, batch serving, streaming, fixed-seed generation, or training support.
Whisper combines speech preprocessing with an encoder-decoder Transformer. A LLaVA-style model combines an image processor, encoder Transformer, projector, and dense language decoder. Route every component through its native runbook.
Inventory artifacts, inputs, outputs, repeated blocks, state, and the surrounding pipeline.
Modality, architecture, and workload are separate decisions with different runbooks and proof gates.
A class name may hide a vision tower, audio frontend, decoder, scheduler, or custom cache.
Route every component through its native family and record uncertainty instead of forcing one label.
Whisper is speech plus encoder-decoder; LLaVA composes vision, projection, and language components.
Using modality as the architecture family or treating a composition as one homogeneous block.
The plan lists source format, modality, components, state, preprocessing, and output contract.
Translate runtime assumptions, then capture a source oracle.
Translate PyTorch and CUDA thinking into MLX
These are translation lenses, not one-to-one replacements. For every row, preserve the source behavior first and choose the MLX mechanism that makes the contract inspectable.
| Source habit | MLX translation | Proof question |
|---|---|---|
| torch.Tensor and .to("cuda") | mx.array in unified memory; select device or stream at the operation. | Do shape, dtype, value, placement, and interop-copy behavior match? |
| Eager execution | Lazy graph construction with deliberate mx.eval boundaries. | Did timing realize the intended work exactly once? |
| state_dict loading | Deterministic implementation plus an explicit weight map for rename, transpose, reshape, split, merge, and dtype. | Is source-key coverage complete and every transform justified? |
| Global NCHW assumptions | Operation-specific layout with every transition visible at its consumer. | Are named axes correct at convolution, attention, and image boundaries? |
| AMP or autocast | Explicit dtype and accumulation by numerical role. | Do norms, softmax, recurrence, FFT, and distances meet the declared policy? |
| Hidden module mutation | Thread cache and recurrent values as explicit state inputs and outputs. | Do update order, reset, positions, and reuse match? |
| torch.compile or CUDA graphs | mx.compile over pure, stable, reused regions after eager parity. | Do compiled and eager outputs match across representative shapes? |
| CUDA extensions | Native MLX operations first, then mx.fast or compilation, then custom Metal with a readable fallback. | Did profiling justify the maintenance and portability cost? |
| CUDA timing and synchronization | Synchronize through evaluation only at dependency or measurement boundaries. | Are construction, execution, warm state, and transfers charged consistently? |
| NumPy or DLPack bridges | Use documented framework conversion for oracle comparison, outside the measured native path. | Are copy, ownership, dtype, and forced-evaluation effects recorded? |
Read the official framework conversion and DLPack guidance ↗
Fast scaled-dot-product attention changes a compatible attention operator. Speculative decoding changes an autoregressive generation algorithm. Compilation changes graph execution. Caches change reusable state. None is a child of the other.
A translation lens turns each source-runtime assumption into an explicit MLX choice.
Familiar syntax can hide different execution, layout, dtype, state, and synchronization behavior.
Annotate device moves, eager boundaries, layouts, autocast, mutation, compilation, extensions, and timing.
Choose the inspectable MLX mechanism that preserves the behavior; do not force one-to-one API replacements.
Replace implicit state_dict loading with a reviewed map of names, axes, shapes, and dtypes.
Porting a CUDA optimization before reproducing the semantics it accelerated.
Compare at each translated boundary before moving to the next one.
Use the ordered correctness rail to localize the first mismatch.
The correctness rail
Every step earns the right to take the next. The stable rail is the same for a decoder, speech system, diffusion pipeline, or multimodal composition; the architecture-specific checkpoints change inside it.
- 01
Inspect
Pin the source and inventory artifacts, modality, architecture, state, preprocessing, and output contract.
- 02
Capture oracle
Freeze deterministic source inputs, intermediate tensors, state, and task output.
- 03
Implement
Build the smallest readable eager MLX graph with explicit axes, numerical roles, and state.
- 04
Map weights
Declare every source-to-target name, shape, layout, split, merge, and dtype transform.
- 05
Prove parity
Compare shapes → primitive → block → output → state → task. Stop at the first divergence.
- 06
Profile
Measure the representative workload with intentional evaluation and synchronization boundaries.
- 07
Optimize
Unlock one bottleneck-specific branch only after parity and quality gates exist.
- 08
Publish
Package provenance, transforms, validation, receipts, limitations, and rollback together.
Debug the first divergence
Start at the earliest failed rung and move outward only when it passes: shapes and dtypes, primitive math, one block, full output, mutable state, then the task metric. Later failures contain every earlier error.
Move from the smallest comparable value to the complete task and stop at the first divergence.
Later errors compound, so an end-to-end failure gives little help locating the original semantic mismatch.
Capture deterministic source tensors, layouts, masks, positions, dtypes, and state at named checkpoints.
Compare the readable eager path at primitive, block, output, state, and task rungs before compilation.
A wrong attention mask should fail at attention output before later residual blocks hide it.
Loosening tolerance, profiling, or compiling before explaining the earliest mismatch.
The fixed rung passes with declared inputs and policy while every earlier rung remains green.
Trace the same rail through the pinned Qwen proof, then profile the target workload.
Trace one proof rail through four different models
The eight checkpoints stay fixed while modality, architecture, state, proof, and workload decisions change. Choose a journey, inspect each node, compare the same checkpoint across routes, and export a port plan. Selection means navigation—not completion or validation.
Proven means this repository has a pinned, reproducible proof packet for the named checkpoint. Simulation means the path is assembled from canonical runbooks and evidence; it is not a completed checkpoint port.
Linear study guide
JavaScript is optional. Every journey, checkpoint, proof boundary, and runbook remains available below.
Proven — pinned checkpoint proofQwen2.5-0.5B-Instruct
Text · safetensors · dense decoder Transformer. Tokenizer contract → Embedding → Decoder block loop → KV cache state → Norm, head, and generation.
- InspectPin the exact Qwen2.5-0.5B-Instruct revision and identify the dense decoder route.
- Capture oracleCapture tokenizer IDs, primitive tensors, repeated blocks, logits, KV state, and greedy outputs.
- RebuildBuild a readable standalone MLX decoder with explicit RoPE, GQA, RMSNorm, masks, and cache.
- Map weightsApply the checked-in schema-2 rename and transform contract with full source-key coverage.
- Prove parityRun the ordered 29-rung ladder under its declared floating-point policy.
- ProfileMeasure load, prefill, and cached decode separately; keep observations tied to the local receipt.
- OptimizeOnly after parity, test attention, compile, dtype, quantization, cache, or decode-algorithm branches independently.
- PublishPublish the exact checkpoint boundary, parity packet, receipts, and limitations without generalizing to all Qwen models.
Proof boundary: Claim one pinned local real-model port, exact eight-token greedy agreement for the F32 artifact, and one held F32-versus-BF16 wall-time observation. Do not generalize the tolerance, BF16 quality window, performance ratio, longer-context behavior, batching, or publication readiness.
Simulation — not a completed checkpoint portWhisper-style ASR
Audio-to-text · safetensors · ASR plus encoder-decoder Transformer. Waveform contract → Mel frontend → Audio encoder → Cross-attention decoder → Transcript and timestamps.
- InspectClassify speech modality separately from the encoder-decoder architecture and frontend.
- Capture oracleCapture waveform, mel features, encoder outputs, cross-attention decoder checkpoints, and timestamps.
- RebuildRebuild the frontend, audio encoder, text decoder, masks, positions, and distinct cache lifetimes.
- Map weightsMap convolution, encoder, decoder, cross-attention, embedding, and head tensors explicitly.
- Prove parityProve features, encoder, decoder, state, logits, transcript, and timestamp behavior in order.
- ProfileSeparate frontend, encoder, autoregressive decoder, and task post-processing costs.
- OptimizeFor offline Whisper, start with attention, compilation, and evaluation boundaries only when their prerequisites hold.
- PublishKeep this route labeled simulation until a pinned checkpoint and task-quality packet are reproduced.
Proof boundary: This is a runbook simulation, not a completed Whisper checkpoint port. The official MLX example proves an implementation path exists; it does not provide this repository's pinned parity and task-quality packet.
Simulation — not a completed checkpoint portFLUX-style diffusion/flow
Text-to-image · safetensors · diffusion and flow. Prompt encoder → Latent and scheduler → Repeated flow or denoiser step → Latent trajectory → VAE decode and image.
- InspectInventory every pipeline component, scheduler, source artifact, and conditioning path.
- Capture oracleCapture conditioning, initial latent, schedule, selected denoiser blocks, latent steps, and final decode.
- RebuildRebuild components independently, then compose a readable single-step pipeline before iteration.
- Map weightsKeep encoder, denoiser, and VAE weight namespaces and transforms separately auditable.
- Prove parityCompare scheduler identity, one-step updates, fixed latent checkpoints, VAE output, and quality.
- ProfileSeparate prompt encoding, repeated denoiser work, scheduler overhead, transfers, and VAE decode.
- OptimizeCurrent registry candidates are stable compilation, evaluation boundaries, block streaming, and the spatial grid-sample kernel.
- PublishLabel the route FLUX-style simulation until an exact model source and reproducible MLX proof are checked in.
Proof boundary: This is a generic FLUX-style teaching route, not an exact FLUX checkpoint claim. Fixed-seed output still requires declared image-quality checks; pixel identity is not promised by default.
Simulation — not a completed checkpoint portLLaVA-style vision-language model
Image-to-text · safetensors · vision encoder, projector, and dense decoder. Image processor → Vision encoder → Multimodal projector → Multimodal token assembly → Language decoder and output.
- InspectRoute image processing, the encoder tower, projector, dense decoder, and KV state as separate components.
- Capture oracleCapture pixels, vision features, projected embeddings, token assembly, logits, cache, and final output.
- RebuildRebuild each component through its native runbook before composing the multimodal path.
- Map weightsKeep vision, projector, and language tensor transforms distinct with complete source-key coverage.
- Prove parityProve every modality boundary before evaluating the final image-conditioned task.
- ProfileSeparate image processing, vision encoding, projection, language prefill, cached decode, and serving reuse.
- OptimizeSelect attention, compile, memory, cache, batching, spatial, or research branches only after their own gates.
- PublishKeep Apple MLX, third-party MLX-VLM, paper-only research, and local reproduction states visibly distinct.
Proof boundary: This is a composed-route simulation with no pinned model outcome or local proof ladder. MLX-VLM evidence is third-party and must not be presented as Apple-official or locally reproduced.
Multimodal runbook ↗ · encoder runbook ↗ · decoder runbook ↗
Qwen2.5-0.5B-Instruct
Tokenizer → embedding → decoder blocks → KV state → logits
Clear the filter to compare this checkpoint across all four routes. Alternatives stay visible as teaching context; none becomes validated.
Export this journey as a text port plan
The export preserves all eight gates, the model status, and the exact evidence boundary.
The proven lab: Qwen2.5-0.5B-Instruct
Proven guided lab
The repository's one complete guided port is bound to the pinned Qwen/Qwen2.5-0.5B-Instruct checkpoint. It teaches tokenizer identity, embeddings, RoPE, RMSNorm, grouped-query attention, repeated decoder blocks, KV state, the final head, and greedy generation.
The standalone MLX path passed all 29 ordered parity rungs under its declared floating-point policy. Input IDs and the eight greedy output token IDs matched exactly. Separate timing receipts remain scoped observations, not a portable speed claim.
This result does not establish every Qwen variant, every dense decoder, or any Whisper, FLUX, or LLaVA checkpoint. Those routes remain simulations until their own artifacts and quality gates exist.
This is one pinned dense-decoder checkpoint with a checked-in MLX proof packet.
A concrete port shows how component, weight, state, and task evidence connect.
The source oracle supplies exact inputs plus ordered tensors, state, logits, and greedy IDs.
Readable embeddings, RoPE, RMSNorm, grouped-query attention, decoder blocks, KV state, and head are mapped explicitly.
Tokens → embedding → decoder blocks → KV state → logits → eight greedy token IDs.
Generalizing one checkpoint result to every Qwen variant, dense decoder, or Apple target.
Locate each of the 29 source and target rungs and the exact-token policy in the report.
Profile a declared target workload without turning its timing into a portable claim.
Profile the real bottleneck before choosing an optimization
Parity earns permission to measure; measurement earns permission to optimize. Split the target workload into the phases a user actually experiences, place evaluation so lazy work is charged exactly once, and select a technique only from the measured branch.
Preprocessing
Tokenizer, image processor, audio frontend, scheduler setup, or feature extraction.
Prefill or encode
Prompt prefill, vision or audio encoding, conditioning, or the first full graph realization.
Decode or denoise
Cached token steps, recurrent scans, diffusion iterations, or another repeated state transition.
State and residency
KV cache, recurrent buffers, activations, weights, temporary arrays, and peak pressure.
Serving and overlap
Batching, queues, streams, transfers, post-processing, and end-to-end latency.
Quality and rollback
Every speed experiment keeps the parity and task-quality checks that can reject it.
Fast attention is a native operator choice. Compilation changes graph execution. Quantization changes representation. Caching changes reusable state. Speculative decoding changes the inference algorithm. Batching changes serving. Keep them as separate branches with separate prerequisites, proof gates, and rollback conditions.
Measure the phase that limits the target workload before choosing an optimization.
Prefill, decode, preprocessing, denoising, streaming, memory, and serving demand different techniques.
CUDA intuition about a hot kernel does not prove the same bottleneck exists on MLX and Apple silicon.
Place evaluation around measured regions so lazy work is included exactly once and synchronization is intentional.
Separate prompt prefill from cached token decode instead of reporting one blended generation time.
Timing graph construction while computation is realized outside the measured region.
The receipt names workload, warm state, evaluation boundaries, hardware, software, and quality gate.
Select one technique from the measured bottleneck branch and keep a rollback.
Choose an optimization from the bottleneck—not from a buzzword
Parity first. Profile second. One experiment at a time. The map below separates operator choices, graph execution, layout, state, compression, inference algorithms, serving, and custom backend work so a fast-attention kernel never gets confused with speculative decoding or a cache policy.
Declaring parity here only unlocks an experiment plan. The page does not inspect your artifacts, prove the selected model supports a method, or promote a numeric claim.
Eight measured-bottleneck branches
JavaScript is optional for the taxonomy. Open the canonical guidance registry for applicability, validation gates, evidence, trade-offs, and rollback details.
Evaluation and scheduling 1 method
Lazy graph lifetime, synchronization, or misplaced evaluation.
- Deliberate lazy evaluation boundaries native MLX
Native operators and compilation 2 methods
Compatible repeated operators or stable graph regions.
- MLX fast scaled-dot-product attention native MLX operator
- Compile stable repeated regions native MLX execution
Layout and numerical representation 1 method
Dispatch layout, gathers, dtype, or accumulation.
- Gather matmul for indexed expert projections native MLX
State and memory 3 methods
Weight residency or KV state exceeds the useful memory budget.
- Repeated-block weight streaming
- Uniform KV cache quantization
- Adaptive mixed-precision KV research
Compression 4 methods
Weight, state, or token representation dominates memory or bandwidth.
- BF16 weight casting
- Native low-bit weight quantization
- Visual-token pruning or merging research
- MoE expert dispatch and quantization
Inference algorithms 3 methods
Autoregressive target-model calls dominate after decoder parity.
- Independent draft-model speculative decoding
- Prompt lookup / n-gram speculation research
- EAGLE / trained drafters research boundary
Serving and pipeline 11 methods
Batching, preprocessing, cache reuse, streaming, or scheduling.
- Prompt cache
- Continuous batching
- Block-level automatic prefix caching rejected / superseded
- Audio streaming and cache
- Cache privacy and tenant isolation research
- Vision feature cache
- Multimodal prefix cache
- Video input budgeting
- Qwen3-TTS batch generation
- Audio reference conditioning cache
- Generic audio prefix cache research
Custom backend work 3 methods
A proven hot operation has no adequate native path.
- MoE gate/up projection fusion research
- Grid-sample Metal kernel
- CUDA Graphs capture rejected—NVIDIA-specific
Select what profiling found
Journey context: Qwen2.5-0.5B-Instruct
Choose one measured branch before viewing or adding experiment hypotheses.
Change the model journey in the atlas ↑- Family proof gate
- First select a measured bottleneck.
- Family rollback
- Every selected experiment must retain the last parity-passing baseline.
Selected hypotheses
Each entry keeps its prerequisite, proof and quality gates, evidence state, canonical sources, and rollback. Nothing is described as enabled or recommended.
Benchmark without fooling yourself
A benchmark is a scoped observation. Keep model revision, inputs, shapes, cache state, precision, warmup, evaluation boundaries, hardware, software, quality, and raw output together so another engineer can reproduce what the number actually means.
Bind the workload
Use the same prompt or input, output length, batch, cache state, dtype policy, and task-quality gate.
Charge lazy work once
Separate construction from execution and declare cold start, warmup, synchronization, repetitions, and aggregation.
Publish the boundary
Preserve raw evidence, environment, baseline, limitations, rollback, and why a result is withheld or effective.
A benchmark is a scoped observation, not a portable promise.
Shape, cache, dtype, model revision, quality, and measurement boundaries can change the result.
Framework habits still need explicit warmup, synchronization, inputs, and quality checks.
Force the intended lazy work inside the timed region and preserve raw evidence plus rollback conditions.
Compare the same prompt, decode length, cache state, dtype policy, and output-quality gate.
Publishing a source-reported or local range as an effective recommendation.
Another engineer can reproduce the workload and see why the number is or is not promotion-ready.
Package the port and its proof boundary together.
Publish the proof boundary, not just the artifact
Ship the readable implementation with its source pin, component route, weight map, parity report, benchmark metadata, supported workload, known gaps, and rollback. A user should be able to tell exactly what was reproduced and what remains a simulation.
Name the source
Checkpoint identity, immutable revision, artifact tree, license evidence, dependencies, and preprocessing.
Attach the proof
Weight-map coverage, staged parity, state behavior, task-quality gate, receipts, and tested environment.
Keep claims narrow
Supported model and workload, limitations, unresolved risks, experimental paths, and rollback conditions.
Ship implementation, provenance, validation, limitations, and a rollback story together.
Users need to know which checkpoint, workload, target, and behavior were reproduced.
A converted artifact without source revision, transforms, and validation cannot be audited.
Record source pin, weight map, parity report, benchmark metadata, supported route, and known gaps.
Label one pinned Qwen checkpoint proven while keeping neighboring Qwen architectures unclaimed.
Promoting indexed research or a passing smoke test into supported guidance.
Every claim points to evidence and every risky change names a rollback condition.
Use the executable field manual to run the same rail on a real model.
Glossary: the proof vocabulary
- MLX
- Apple's array framework for machine learning on Apple silicon, with lazy evaluation, transforms, and unified-memory operation.
- Eager graph
- The simplest readable MLX implementation used to prove semantics before compilation or specialized optimization.
- Source oracle
- Deterministic tensors, state, and outputs captured from the pinned source implementation.
- Parity
- Evidence that source and MLX behavior agree under a declared exact or numerical comparison policy.
- Route
- The selected architecture-family and capability path that determines runbooks, risks, and proof gates.
- Runbook
- Architecture-specific implementation and validation guidance with supported, experimental, and blocked boundaries.
- Checkpoint
- A named intermediate tensor, state, or output used to localize correctness.
- Weight map
- The explicit rename, transpose, reshape, split, merge, dtype, and destination contract for source tensors.
- KV cache
- Stored attention keys and values reused across autoregressive decoding steps.
- Schema-2
- The structured weight-map and conversion contract with explicit tensor transforms and coverage.
- Golden scenario
- A small representative input and expected behavior used as a stable end-to-end validation target.
- Receipt
- Structured raw evidence for one bounded benchmark or validation run, including its exact context.
- Promotion-ready
- An evidence state that has satisfied provenance, workload, quality, attestation, and scope gates.
- Execution attestation
- Evidence binding a measurement to its command, dependencies, outputs, and trusted verification boundary.
- TargetProfile
- The controlled hardware, software, model, workload, and capability context used for recommendation eligibility.
- Source-key coverage
- The disposition of every source checkpoint key under the weight-map contract.
- Proof boundary
- The exact model, revision, target, workload, behavior, and evidence scope a claim may cover.
The glossary is the shared vocabulary for routes, checkpoints, evidence, and claims.
Precise terms keep source support, local proof, task quality, and promotion state from collapsing into “works.”
Familiar framework labels do not replace this repository's explicit evidence and publication states.
Routes select runbooks; checkpoints establish parity; receipts bind measurements; proof boundaries scope claims.
A promotion-ready receipt is narrower than “validated” and says exactly which experiment survived which gates.
Using “validated” without saying source, local, task, or promotion state.
Every public claim uses a defined evidence state and names its boundary.
Keep the glossary nearby while using the executable field manual.
Port for real: the field manual
The concepts above explain why each command exists. The reference below is the inspectable implementation path.
Quick start
Run static inspection first. It reads local metadata and safe tensor headers, classifies the architecture, emits risk blockers, and recommends one or more runbooks. It does not execute project code.
python3 mlx-model-porting/scripts/inspect_model.py /path/to/model \
--output inspection.json \
--markdown inspection.md
python3 mlx-model-porting/scripts/make_port_plan.py inspection.json \
--artifact-root /path/to/model \
--output PORT_PLAN.mdinspection.json contains controlled family ids, architecture traits, routing status, risks, blockers, tensor metadata, source-format observations, and recommended runbooks. Ambiguity stays explicit.
For an existing MLX codebase, inspect that surface separately:
python3 mlx-model-porting/scripts/inspect_mlx_project.py /path/to/project \
--model /path/to/local/model \
--output mlx-inspection.json \
--markdown MLX_INSPECTION.mdInstallation
Client presets select a documented destination and copy or symlink mode. Always dry-run first when the target matters; --force atomically replaces an existing target.
python3 mlx-model-porting/scripts/install_skill.py \
--client codex \
--dry-run
# Inspect the printed source, destination, target, and mode.
# Then repeat without --dry-run.Preset clients
Claude Code, Codex, Cursor, Gemini CLI, Windsurf, and GitHub Copilot.
Explicit destination
Use --dest ~/.agents/skills --mode symlink when a product version or local convention differs.
The installer rejects recursive destinations, ancestor targets that contain the source, path escapes, and symlink cycles. It compares content and modes for idempotence.
The workflow contract
The order is a correctness device. A profiler cannot rescue a mismapped checkpoint, and a throughput number cannot replace task quality.
For an unblocked dense decoder, capture_oracle.py, scaffold_port.py, convert_checkpoint.py, capture_mlx.py, and run_parity.py implement the rail above. Other routed families still require the runbook-defined MLX module and model-specific capture wiring. Exact output is the only controlled built-in task metric.
Architecture routes
The registry declares 17 routes. Each has a golden scenario for routing, source-key coverage, seeded parity-bug detection, and optimization policy. That is not evidence of 17 completed real-checkpoint ports.
See the generated architecture registry. Interactive route cards require the local generated data file.
Qwen3-Next, Falcon-H1, Nemotron-H, Whisper, Qwen2-Audio, Moshi, RNN-T, Voxtral Realtime, and Gemma 3 vision-style compositions are routed through capability-aware profiles. Conflicting or weak signals remain ambiguous.
Static inspection
The inspector recognizes local directories, individual files, and Hugging Face ids. Network access is off by default. Local intake hashes every inventoried file into a portable SHA-256 tree identity and binds accepted license evidence to that same tree. Incomplete identity or missing evidence blocks recommendations. When network access is enabled, pin a revision; weight downloads are a separate opt-in.
python3 mlx-model-porting/scripts/inspect_model.py org/model \
--allow-network \
--revision 0123456789abcdef0123456789abcdef01234567 \
--output inspection.jsonSafe metadata paths
Config JSON, tokenizer metadata, safetensors headers, bounded GGUF/ONNX/protobuf metadata, and preflighted archive tables.
Blocked confidence
Pickle weights, dynamic imports, custom model code, escaping symlinks, truncated inventories, missing license evidence, oversized structures, and conflicting identities.
ONNX, GGUF, Flax/Orbax, TensorFlow/Keras, and Core ML are intake and mapping surfaces here. Static recognition does not lower arbitrary graphs into executable MLX.
Build a source oracle
Freeze a small, representative fixture before porting. Record source revision, preprocessing, tokenizer or processor, seed, train/eval state, dtype, device, generation settings, and cache/state behavior.
python3 mlx-model-porting/scripts/capture_oracle.py /path/to/model \
--token-ids 1 42 17 9 \
--generate-steps 4 \
--output source-oracle.npz- Capture embeddings or earliest feature transforms.
- Capture one block at the first non-trivial mask, routing, recurrence, or multimodal boundary.
- Capture final logits, waveform, image, embedding, forecast, or graph output.
- Capture mutable state: KV cache, convolution buffer, recurrent state, diffusion schedule, or streaming flush state.
- Define a task-quality metric before measuring speed.
A final-output mismatch tells you that something is wrong. A staged oracle tells you where semantics first diverged.
Validate the weight map
For a dense decoder, generate the eager package and draft the schema-2 map. Review the draft, resolve every entry, and set draft to false before validation and conversion.
python3 mlx-model-porting/scripts/scaffold_port.py inspection.json \
--artifact-root /path/to/model \
--output mlx_port
python3 mlx-model-porting/scripts/convert_checkpoint.py \
--source inspection.json \
--scaffold-manifest mlx_port/scaffold-manifest.json \
--emit-draft-map WEIGHT_MAP.draft.jsonThe mapping records each source key, target key, transform, expected shape, dtype decision, and reason for any intentionally unmapped value.
python3 mlx-model-porting/scripts/validate_weight_map.py \
--source source-manifest.json \
--target target-manifest.json \
--mapping WEIGHT_MAP.json \
--output weight-map-report.jsonThis validates key and shape-transform coverage. It does not prove operator semantics or numerical parity. --allow-unmapped weakens the default release gate.
Climb the parity ladder
After conversion, one command captures both runtimes and stops at the first dense-decoder divergence:
python3 mlx-model-porting/scripts/run_parity.py \
--source-model /path/to/model \
--package mlx_port \
--weights converted \
--token-ids 1 42 17 9 \
--generate-steps 4 \
--output parity-report.jsonUse the family-neutral comparator for additional captures and runbook-defined checkpoints:
python3 mlx-model-porting/scripts/compare_tensors.py \
source.npz target.npz \
--mapping mapping.json \
--atol 1e-5 --rtol 1e-4 --cosine-min 0.99 \
--output parity.json| Level | Compare | Typical failure |
|---|---|---|
| Tensor | Shapes, dtypes, exact transforms | Transpose, split/merge, padding, layout |
| Block | Norm, attention/SSM, MLP/expert, residual | Axis, mask, router, recurrence, evaluation timing |
| Output | Logits, image latent, waveform, embedding | Head, denormalization, processor, decoder |
| State | Cache, buffer, recurrent state, streaming flush | Offset, window, mutation, boundary continuity |
| Task | Accuracy, WER/CER, perceptual or domain metric | Numerically close but behaviorally wrong |
--no-fail is report-only. --allow-empty must not be used as a release gate.
Optimization is a controlled experiment
The recommender matches controlled family, capability, workload, software, and hardware identifiers. Raw guidance numbers are scrubbed; the generated effective-claim catalog is the only numeric authority. It currently contains 10 numeric records, all withheld. Any future local promotion requires the full canonical TargetProfile.experiment_fingerprint, exact receipt-derived model, target, and workload descriptors, exact hardware and software, a non-empty controlled workload set, and an external signature verified against an out-of-repository trust anchor. A copied digest or generic profile cannot unlock a number.
python3 mlx-model-porting/scripts/recommend_optimizations.py inspection.json \
--target-profile target-profile.json \
--objective peak-memory \
--output recommendations.json \
--markdown OPTIMIZATIONS.md| Advisor bucket | Meaning | Execution policy |
|---|---|---|
| Validated locally | A local parity fixture, benchmark, or skill gate reproduced the behavior in its captured scope. | Keep the exact model, inputs, target, and validation gate attached; do not generalize. |
| Validated by source or theory | Native MLX or official project path exists. | Confirm on the target; no portable speed promise. |
| Benchmark required | Prior MLX port evidence exists. | Profile and reproduce against the named baseline. |
| Experimental approach | Research candidate or incomplete MLX path. | Explicit user opt-in before implementation. |
| Rejected / do not use | Superseded, conflicting, regressing, or unsafe. | Cannot execute as a recommendation. |
Blocked intake holds candidates by default. --allow-blocked is a deliberate override, not a way to erase the blocker.
Benchmark receipts
A generic command benchmark can record bounded wall-time output and optional RSS. With --receipt-spec, the family-neutral external-command-wall-time adapter produces controlled observation evidence. Its exact argv template executes one digest-pinned Python runner at argv position 1 and binds models.target.id, models.target.revision, checked-in workload evidence, and semantic variant_config. The harness hashes the resolved interpreter and package metadata, sanitizes and binds the ambient environment, starts Python with -I -B, statically rejects symlink components, snapshots the quality contract before execution, and requires every measured run to recreate the declared output under quality/outputs/<label>/. Only parent-measured wall time counts; child stdout cannot attest performance.
A digest-pinned generic script can ignore its model/workload arguments, and the legacy MLX-LM lane does not bind imported package bytes or per-run output; both remain observations. The narrow repository-owned attested-mlx-port-wall-time Qwen adapter binds a fresh parent challenge, reviewed runner and loaded dependency bytes, model/workload identity, and every output before the validator sets execution_attested=true. It does not attest another model or runner.
python3 mlx-model-porting/scripts/benchmark_command.py \
--warmup 1 --runs 5 --timeout 600 \
--output benchmark.json \
-- COMMAND ...python3 mlx-model-porting/scripts/benchmark_command.py \
--receipt-spec bench/candidate-spec.json \
--quality-contract bench/quality-contract.json \
--baseline-receipt baseline.json \
--warmup 1 --runs 5 --timeout 600 \
--output bench/candidate.jsonFamily-neutral receipt spec
{
"schema_version": 1,
"label": "candidate",
"argv_template": [
{"literal": "python3"},
{"source": ["workload", "artifacts", 0, "path"]},
{"literal": "--model"},
{"source": ["models", "target", "id"]},
{"literal": "--revision"},
{"source": ["models", "target", "revision"]},
{"literal": "--input"},
{"source": ["workload", "artifacts", 1, "path"]},
{"literal": "--mode"},
{"source": ["variant_config", "mode"]},
{"literal": "--output"},
{"source": ["variant_config", "quality_output_path"]}
],
"models": {"target": {
"id": "owner/model", "revision": "<pinned 40-64 hex>",
"lineage_id": "controlled-lineage", "source_id": "owner/source-model",
"source_revision": "<pinned 40-64 hex>"
}},
"workload": {
"id": "controlled-workload",
"artifacts": [
{"role": "runner", "path": "runner.py", "sha256": "...", "size_bytes": 1234},
{"role": "input", "path": "input.bin", "sha256": "...", "size_bytes": 5678}
],
"parameters": {"shape": "fixed"}
},
"variant_config": {
"mode": "candidate",
"quality_output_path": "quality/outputs/candidate/result.txt"
},
"enabled_methods": ["method-id"],
"comparison_role": "candidate",
"rollback_condition": "Rollback on quality failure or a gain within noise."
}Legacy MLX-LM schema-2 candidate (observation-only)
python3 mlx-model-porting/scripts/benchmark_generation.py \
--label candidate \
--runs 5 --warmup 1 \
--target-model "$TARGET_MODEL" \
--target-revision "$TARGET_REVISION" \
--lineage-id "$LINEAGE_ID" \
--source-id "$SOURCE_MODEL" \
--source-revision "$SOURCE_REVISION" \
--workload-id deterministic-smoke-v1 \
--workload-artifact prompt=bench/prompt.txt \
--workload-params-json '{"max_tokens":128,"temperature":0.0,"seed":7}' \
--quality-contract bench/quality-contract.json \
--rollback-condition "Rollback on quality or throughput regression." \
--baseline-receipt bench/baseline.json \
--enabled-method native-low-bit-weight-quantization \
--output bench/candidate.json \
-- python3 -m mlx_lm generate \
--model "$TARGET_MODEL" --prompt "$PROMPT" \
--max-tokens 128 --temp 0.0 --seed 7Current controlled quality contract
{
"schema_version": 2,
"validator": {"id": "mlx-benchmark-exact-output-parity", "version": 1},
"metric": "exact-output-parity",
"reference_artifact": {"path": "quality/reference.txt", "size_bytes": 123, "sha256": "..."},
"candidate_artifact": {"path": "quality/outputs/candidate/result.txt", "size_bytes": 123, "sha256": "..."}
}The validator independently rereads two distinct, bounded, size- and digest-bound artifacts and compares their bytes. Schema-1 arbitrary JSON scores, legacy --quality-artifact, and Python --quality-evaluator inputs remain observations; they cannot authorize promotion. Lossy or task-specific candidates stay held until a controlled built-in metric exists.
Receipts can pool only when their exact baseline file and digest, warmup/run/timeout protocol, semantic experiment, and exact-output quality contract match. The lowest compatible ratio supplies any future range and its receipt-specific full fingerprint; missing, rejected, or heterogeneous repetitions withhold the number.
python3 mlx-model-porting/scripts/validate_benchmarks.py generate \
--root mlx-model-porting/assets/benchmarks
python3 mlx-model-porting/scripts/validate_benchmarks.py check \
--root mlx-model-porting/assets/benchmarksEvidence states are separate axes
Review depth says how deeply a source has been read. Support scope says who maintains the implementation or whether it is paper-only. Benchmark classification says whether a local measurement can authorize a number. Do not collapse these into one “validated” badge.
| Axis | States | Safe public wording |
|---|---|---|
| Review depth | indexed, screened, synthesized | Catalogued; limitations reviewed; directly informed a rule/runbook. |
| Support scope | official_mlx, official_mlx_project, third_party_pinned, paper_only, context_only, local_reproduced | Apple framework; Apple project; pinned third-party prior art; research candidate; discovery context; captured local result. |
| Receipt class | Observation, promotion-ready, rejected | Historical measurement; all gates pass; negative or invalid evidence. |
13 total: 12 observations, 0 promotion-ready, and 1 rejected. The retained Qwen challenge and digest bundle supports reproducibility-on-request, not forgery-proof attestation; SHA-256 is not a signature.
Package validation
Run deterministic offline gates before any release. Live URL health and upstream pin drift are separate networked checks so a transient outage cannot corrupt prior evidence.
python3 mlx-model-porting/scripts/audit_skill.py --strict mlx-model-porting
python3 mlx-model-porting/scripts/validate_sources.py mlx-model-porting
python3 mlx-model-porting/scripts/knowledge_curator.py --check-backlog
python3 mlx-model-porting/scripts/validate_benchmarks.py check
python3 mlx-model-porting/scripts/generate_claim_catalog.py --check
python3 mlx-model-porting/scripts/generate_evidence_index.py --check
python3 mlx-model-porting/scripts/generate_site_data.py --check
python3 mlx-model-porting/scripts/manifest.py check
python3 -m unittest discover -s tests -vRun validate_sources.py --check-urls only when network validation is intended. Missing or unhealthy URLs fail loudly without upgrading or rewriting evidence.
Registry-current primary sources
“Current” means the pinned registry snapshot reviewed on 2026-07-12, not an unverified claim about live upstream latest versions.
| Surface | Recorded immutable reference | Scope |
|---|---|---|
| Apple MLX | mlx-repo · 96296e9 ↗ | official_mlx |
| Apple MLX-LM | mlx-lm-repo · 2c008fd ↗ | Apple-maintained project; model/workload confirmation still required. |
| MLX source snapshot | Registered framework tree ↗ | Canonical mlx-repo snapshot; not a speed guarantee. |
| MLX-Audio | mlx-audio-repo · 412cf7c ↗ | third_party_pinned; not Apple-official. |
| MLX-VLM / vLLM-MLX | MLX-VLM 6a8cdff ↗ · vLLM-MLX batching source ↗ | Pinned third-party implementation evidence. |
The generated evidence index preserves every source id, URL, review depth, affected surface, and limitation.
Contribution flow
- Collect a candidate as review-only evidence. Daily automation may update candidates and open a PR; it cannot change supported guidance or merge itself.
- Pin the source and record review depth, classification, support scope, affected rule/runbook, limitations, and access date.
- Provide a reproducible MLX path, correctness or quality gate, benchmark metadata when performance is claimed, and rollback condition.
- Update the smallest concrete asset or runbook, regenerate derived artifacts, and add a test that fails if the requirement drifts.
- Keep rejected and superseded evidence visible. Never average contradictory sources into a vague compromise.
Limitations
The Qwen2.5 packet proves one dense-decoder checkpoint, 29 parity rungs, and exact greedy-token output. Golden scenarios prove routing and guards for all 17 families; they do not prove end-to-end conversion across every family, and exact output does not replace domain evaluation.
- Forecasting coverage does not imply tabular, ranking, or recommender support.
- Current CV route coverage centers on backbone/classifier patterns; dense vision, promptable masks, OCR, depth, and pose remain candidate tracks.
- Graph coverage proves GCN-style message-passing routing, not point-cloud, equivariant, molecular, or scientific ML.
- Training is a supporting workflow for adapters, distillation, QAT, or architecture artifacts—not a generic port target.
- All current local benchmark measurements are observations; none has the externally signed attestation required for promotion.
- Live Apple Silicon execution, external researcher agents, network collectors, URL health, and upstream revision drift are not proven by deterministic offline tests.
For the complete boundary, read VALIDATION.md and the generated benchmark assessment.
After the native path is exhausted
Advanced work starts after a readable parity path and a measured bottleneck. It does not bypass them.
Custom Metal
Keep a readable native MLX fallback, compare representative shapes, and roll back on semantic mismatch, maintenance risk, or no end-to-end gain.
Experimental algorithms
Speculative drafters, adaptive compression, token pruning, or new cache policies need explicit opt-in and task-quality gates.
Training and fine-tuning
Use training only as a supporting path for adapters, distillation, quantization-aware artifacts, or architecture-specific work—not as proof of an inference port.
Unsupported families
Contribute a reproducible source oracle, MLX implementation, validation ladder, benchmark metadata, limitations, and rollback before changing support language.
Native operations → stable compilation → measured custom backend. An unfamiliar CUDA kernel is evidence to inspect, not an instruction to rewrite it first.