TogetherAI by TogetherAI | AI Agents

Agent framework

•

By TogetherAI

TogetherAI's AI agent is a cutting-edge solution designed to leverage the collective strengths of multiple large language models (LLMs) to enhance state-of-the-art quality and performance. This innovative approach, known as the Mixture of Agents (MoA), employs a layered architecture where each layer comprises several LLM agents. These agents utilize outputs from the previous layer as auxiliary information to generate refined responses, thereby integrating diverse capabilities and insights from various models.

Features

The TogetherAI agent boasts a range of features that enhance its functionality and adaptability for different applications. The combination of a layered architecture, fine-tuning capabilities, high-performance hardware, and a comprehensive API allows users to customize and scale their AI solutions effectively. Below is an overview of the exact features:

Feature	Description
Layered Architecture	Utilizes a multi-layer approach where each layer contributes to the output, integrating the strengths of various LLMs.
Fine-Tuning Capabilities	Allows users to customize open-source models using their private data for improved task accuracy.
High-Performance GPU Clusters	Offers scalable GPU clusters ranging from 16 to 2048 GPUs, powered by NVIDIA A100 and H100 hardware for large-scale training.
AI Inference Technology	Provides a fast and efficient inference stack, supporting large-scale deployments with cost savings.
Comprehensive API	Includes SDKs for multiple programming languages and detailed documentation for easy integration.
Multi-Agent Workflows	Supports frameworks like Axiomic for creating portable and steerable chat agents, facilitating structured decision-making.

Use Cases

The TogetherAI agent can be utilized across a variety of applications, demonstrating its versatility and effectiveness in different scenarios:

Custom AI Models: Users can personalize AI models by fine-tuning them with specific datasets, achieving greater accuracy in applications requiring domain-specific knowledge.
Chat Applications: The multi-agent workflow enables the creation of sophisticated chatbots that can gather information, decide on actions, generate responses, and ensure safety through guardrails.
AI Workloads: TogetherAI captures the end-to-end needs for AI workloads, providing both compute and software solutions that cater to customers needing dedicated clusters for training or inference.

How to get started

To begin using TogetherAI's AI agent, interested users can access resources for trial and integration. For more information, users are encouraged to visit the official TogetherAI website, where they can find documentation, API details, and options to contact the support team for personalized assistance.

Together AI Pricing Overview

The pricing for Together AI is structured based on token usage, model type, hosting, and dedicated endpoints. Below are the details:

Inference Pricing

Per 1K Tokens:
- Up to 3B: $0.0001
- 3.1B - 7B: $0.0002
- 7.1B - 20B: $0.0004
- 20.1B - 40B: $0.001
- 40.1B - 70B: $0.003

Chat, Language, and Code Models

Per 1K Tokens:
- Up to 3B: $0.0001
- 3.1B - 7B: $0.0002
- 7.1B - 20B: $0.0004 (Coming soon)
- 20.1B - 40B: $0.001 (Coming soon)
- 40.1B - 70B: $0.003 (Coming soon)
Per Hour Hosting:
- Up to 3B: $0.52
- 3.1B - 7B: $0.52
- 7.1B - 20B: Coming soon
- 20.1B - 40B: Coming soon
- 40.1B - 70B: Coming soon

Image Models

Pricing remains the same:
- 25 steps: $0.001
- 50 steps: $0.002
- 75 steps: $0.0035
- 100 steps: $0.005

Dedicated Endpoints

GPU Pricing per Minute:
- 1x RTX-6000 48GB: $0.034
- 1x L40 48GB: $0.034
- 1x L40S 48GB: $0.048
- 1x A100 PCIe 80GB: $0.050
- 1x A100 SXM 40GB: $0.050
- 1x A100 SXM 80GB: $0.054
- 1x H100 80GB: $0.098

Together AI Consumption Units Packages

Monthly Contracts:
- $5,000: $5,000.00
- $10,000: $10,000.00
- $20,000: $20,000.00
- Custom Together AI Units: $1.00

Additional Usage Costs

Additional Usage: $1.00 per unit

Inference API Pricing

General Models: $0.10 per 1M tokens

Note: The prices listed are subject to change and might not reflect the most up-to-date information. For the latest pricing, it is recommended to visit the Together AI pricing page directly.