Baidu: ERNIE 4.5 21B A3B

Baidu: ERNIE 4.5 21B A3B is a cutting-edge text-based Mixture-of-Experts (MoE) model, boasting 21 billion total parameters with 3 billion activated per token. This architecture delivers exceptional multimodal understanding and generation capabilities, powered by heterogeneous MoE structures and modality-isolated routing. It supports an extensive 131K token context length, allowing for deep and comprehensive processing of information. The model achieves efficient inference through multi-expert parallel collaboration and quantization techniques. Advanced post-training methods, including SFT (Supervised Fine-Tuning), DPO (Direct Preference Optimization), and UPO (Unsupervised Preference Optimization), ensure optimized performance across diverse applications. Specialized routing and balancing losses further enhance its ability to handle various tasks with superior precision. It offers 'functions' and 'streaming' capabilities, making it versatile for interactive applications. Pricing is competitive at $0.07 per 1M input tokens and $0.28 per 1M output tokens, with a generous 120K token context window and 4K token max output.