Mirai Boosts On-Device AI: Faster Models for Your Phone

Why You Care

Ever wonder why your phone struggles with AI features, or why some apps require an internet connection for seemingly simple tasks? What if your device could handle AI locally, without sending your data to the cloud? This is precisely what Mirai, a new London-based startup, aims to achieve. They are working to bring AI directly to your smartphone, making it faster and more private.

What Actually Happened

Mirai, a 14-person technical team, recently emerged to address a essential gap in the AI landscape. While much of the AI conversation centers on massive cloud data centers, Mirai focuses on improving on-device model inference – that is, running AI models directly on your phone or tablet. The company, backed by a $10 million seed round led by Uncork Capital, was founded last year by Dima Shevts and Alexey Moiseenkov, as mentioned in the release. Both co-founders have a strong background in building consumer applications. Shevts co-founded the popular face-swapping app Reface, and Moiseenkov was the CEO and co-founder of Prisma, a viral AI filter app, the company reports. Their combined experience in consumer apps drives Mirai’s mission to enable complex AI tasks on mobile devices.

Why This Matters to You

Mirai is developing a structure to help AI models perform better on your devices. They have built an inference engine specifically for Apple Silicon, which significantly optimizes on-device throughput (the amount of data processed over time). Imagine using an AI writing assistant that summarizes documents instantly on your iPad, without any lag or privacy concerns. The company reports that its upcoming SDK (Software creation Kit) will allow developers to integrate this runtime with just a few lines of code. “One of the visions why we started the company was that we wanted to give developers, like this Stripe-like, eight lines of code [integration] experience,” Shevts stated, emphasizing ease of use for developers.

Mirai’s Impact on Your Apps

This means your future apps could be faster, more private, and work seamlessly offline. Think of it as having a AI supercomputer in your pocket. What kind of AI features would you love to see run perfectly on your device, even without an internet connection?

The Surprising Finding

Here’s an interesting twist: Mirai’s engine, built in Rust, can bump up a model’s generation speed by up to 37%, the company claims. This is quite surprising given the common assumption that on-device AI is inherently slower and less capable than cloud-based solutions. The team revealed that while tuning models for different platforms, they intentionally avoid altering model weights (the parameters that define how an AI model works). This ensures there is no loss in the quality of the AI’s output, a crucial detail often overlooked when optimizing for speed. This approach challenges the notion that speed improvements must come at the expense of accuracy or quality.

What Happens Next

Mirai’s current focus is on improving text and voice modalities on its system, as mentioned in the release. However, the team plans to support vision capabilities in the future. They are actively working with frontier model providers (companies creating AI models) to improve these models for edge use (running on devices). The company is also in talks with various chipmakers, indicating future hardware integrations. Later, Mirai intends to bring its engine to Android devices, expanding its reach significantly. The team also aims to release on-device benchmarks. This will allow model makers to test and compare on-device performance, fostering creation across the industry. For you, this means a future where your devices are smarter, more responsive, and more capable than ever before.