Google Unveils Gemini 1.5: Improving AI

Google recently introduced its latest AI model, Gemini 1.5, which boasts an “experimental” one million token context window. This revolutionary feature enables Gemini 1.5 to process long text passages of up to one million characters to better understand context and meaning. The capability of Gemini 1.5 surpasses previous AI systems like Claude 2.1 and GPT-4 Turbo, which are limited to 200,000 and 128,000 tokens, respectively.

Mixture-of-Experts Architecture

The efficiency of Google’s latest model is attributed to its innovative Mixture-of-Experts (MoE) architecture. This architecture divides the model into smaller “expert” neural networks and enables selective activation of the most relevant expert pathways, significantly enhancing the model’s efficiency. According to Google researchers, this architecture has improved the performance of Gemini 1.5, achieving near-perfect recall on long-context retrieval tasks across modalities and setting new benchmarks across various domains.

Power of the 1M Token Context Window

To demonstrate the power of the 1M token context window, Google showcased how Gemini 1.5 could accurately process and summarize the entire 326,914-token Apollo 11 flight transcript and a 684,000-token silent film when prompted. Such capabilities highlight the revolutionary potential of Gemini 1.5 in understanding and processing complex, real-world text.

Availability and Future Release

Initially, Google is providing developers and enterprises with free access to a limited Gemini 1.5 preview with the one million token context window. A general release with 128,000 tokens for the public is planned to follow, along with pricing details. While the one million token capability remains experimental for now, Gemini 1.5 has the potential to become a new standard for AI’s understanding of complex text.

Testing and Access

Interested developers can sign up to test Gemini 1.5 Pro in AI Studio, while enterprise customers can reach out to their Vertex AI account team to explore access to the model.