Migrating from closed to open source models, Together

Together AI has outlined a five-stage playbook for companies moving from closed source to open source models: discover, evaluate, adapt, decide, and production. In a company blog post, it said this type of migration can take weeks to months rather than months to years, particularly when a managed service is used.

The discover stage centers on defining the use case and workload in plain terms, then using existing benchmarks as a filtering tool. Together AI pointed to sources including The Open Frontier, Artificial Analysis, Epoch AI, Vals AI, Scale's leaderboards, Intelligence and Arena, with suggested benchmarks such as FrontierCode, DeepSWE and Terminal Bench 4 for coding agents, LMArena and AA-Intelligence Index for chat, τ-bench and Agent Arena for agents, and Vision Arena for vision. It described three tiers of models: Frontier Open, Value Tier, and Small and edge-deployable.

For evaluation, Together AI said teams should assess both accuracy and performance, and that replaying existing traffic is the best way to benchmark a workload. The adapt stage lists levers in rough order of effort: system prompt engineering, model inference settings such as temperature and top_k, context engineering and the harness, and fine-tuning or distillation of model weights.

The decide stage covers packaging effort, risks, and return on investment for stakeholders. Together AI said it has seen up to 70% cost reduction in some cases when customers transition to open source, and that ROI can be calculated from token generation per dollar and quality comparisons against closed source.

For production, Together AI said the step is often the lightest, citing canary deployments that start with 10% of traffic to open source. It added that technical implementation can be as simple as switching out an endpoint, and directed questions to sales@together.ai.