Hybrid Edge-AI Framework for Intelligent Mobile Applications: Leveraging Large Language Models for On-device Contextual Assistance and Code-Aware Automation

Abstract

The integration of large language models (LLMs) into mobile development workflows has been fundamentally constrained by three competing requirements: computational efficiency, contextual awareness, and real-time responsiveness. While cloud-based LLMs offer unparalleled reasoning capabilities, their reliance on remote infrastructure introduces prohibitive latency, privacy risks, and energy inefficiencies for mobile environments. Conversely, on-device models, though responsive and privacy-preserving, often lack the contextual depth required for complex code understanding and automation tasks. To address these challenges, we present SolidGPT, a hybrid edge-cloud framework that achieves an optimal balance between these competing demands through three key architectural innovations.

First, we introduce a Markov Decision Process (MDP)-based dynamic routing system that intelligently allocates tasks between on-device lightweight models (DistilGPT, TinyLLaMA) and cloud-based LLMs (GPT-4). This system evaluates real-time parameters—including contextual complexity, hardware constraints, and network conditions—to minimize energy consumption (28.6% reduction) while maintaining high accuracy (91% diagnostic accuracy). Second, our deep integration with Android’s Model-View-ViewModel (MVVM) architecture enables semantic-aware analysis across UI layouts, business logic, and runtime logs, bridging the gap between static code analysis and dynamic mobile runtime environments. Third, a novel prompt engineering pipeline preserves codebase-specific context across execution boundaries, ensuring continuity between local and cloud processing.

To validate our framework, we conducted a 12-week deployment with United Airlines’ Android application (128,500 LOC), involving 43 developers across six feature teams. The results demonstrate significant improvements: bug resolution time decreased by 64.1% (*p*<0.001), cloud API calls were reduced by 56.3%, and 87% of developer queries were resolved with sub-second latency. Notably, the system maintained a median energy consumption of 0.81mJ/token for on-device operations, outperforming cloud-only alternatives. These advancements highlight the framework’s ability to harmonize the strengths of edge and cloud computing while addressing critical challenges in energy efficiency, privacy preservation, and toolchain integration.

Beyond mobile development, SolidGPT establishes a template for deploying LLM-powered assistants in resource-constrained edge environments, such as IoT devices and embedded systems. By combining adaptive task allocation, platform-aware semantic analysis, and context-preserving prompt design, our work paves the way for next-generation AI tools that are both powerful and pragmatic—capable of scaling across domains without compromising responsiveness or user trust.