analyze_files
Read multiple files and answer a question about them.
resolve_model
Resolve a model name to engine-specific artifacts including HuggingFace repo, filename, engine, and architecture.
list_models
List all available models in the Octomil catalog with publisher, parameter count, engines, and variants.
detect_engines
Detect which inference engines are available on this machine.
run_inference
Run raw inference through the local on-device model without system prompt wrapping.
get_metrics
Get current model, engine, hardware, and device status.
deploy_model
Deploy a model to edge devices via the Octomil platform. Requires OCTOMIL_API_KEY.
convert_model
Convert a local PyTorch model to edge-optimized formats (ONNX, CoreML, TFLite).
optimize_model
Optimize a model for on-device deployment via server-side pruning and quantization. Requires OCTOMIL_API_KEY.
detect_hardware
Detect full hardware capabilities including CPU, GPU, RAM, and recommended inference backend.
benchmark_model
Benchmark inference engines for a specific model, measuring tokens/second and latency.
recommend_model
Recommend the best model configuration for this hardware based on available RAM, GPU, and CPU.
scan_codebase
Scan a codebase to find all ML inference points across iOS, Android, and Python code.
compress_prompt
Compress a prompt to reduce token count before inference, saving cost without losing meaning.
plan_deployment
Dry-run a deployment to see the plan without executing it. Requires OCTOMIL_API_KEY.
embed_text
Generate text embeddings using the Octomil platform. Requires OCTOMIL_API_KEY.