Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 16 additions & 10 deletions ai-agent-local/BUILDING.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -45,8 +45,14 @@ ndk.dir=/Users/your-username/Library/Android/sdk/ndk/27.0.12077973

That's the only setup a normal build needs. The native llama.cpp library is
already committed as `ai-agent-local/libs/v8/llama-v8-release.aar` (the JNI
wrapper plus `.so` files for arm64-v8a, armeabi-v7a, x86 and x86_64), with its
tiny interface jar at `ai-agent-local/libs/llama-api.jar`.
wrapper plus the `.so` files), with its tiny interface jar at
`ai-agent-local/libs/llama-api.jar`.

The plugin packages **arm64-v8a only**. The IDE extracts a single ABI
(`PluginLoader.extractNativeLibs` reads `lib/${Build.SUPPORTED_ABIS[0]}/`) and
CoGo itself ships no x86 build, so any other ABI would be download weight no
device can execute. `assemblePlugin` checks the packaged `.cgp` as its last step
and fails the build if it carries anything else.

---

Expand DownExpand Up@@ -141,8 +147,8 @@ normal build.

### Native library not found (`UnsatisfiedLinkError`)

The prebuilt AAR bundles arm64-v8a, armeabi-v7a, x86 and x86_64. If you rebuilt
it with a restricted ABI set, confirm your device's ABI is included:
The plugin packages arm64-v8a only, so it needs a 64-bit ARM device. Confirm
what the AAR carries:

```bash
unzip -l ai-agent-local/libs/v8/llama-v8-release.aar | grep 'jni/'
Expand DownExpand Up@@ -366,21 +372,21 @@ jarsigner -verbose -sigalg SHA256withRSA -digestalg SHA-256 \
### Size Optimization

Current release sizes:
- **ai-agent-local**: ~5 MB (includes llama.cpp native libraries)
- **ai-agent-local**: ~15 MB (includes the arm64-v8a llama.cpp native libraries)
- **ai-assistant-plugin**: ~2 MB (UI and business logic)

To reduce size:
1. Build only arm64-v8a ABI (skip v7/x86)
2. Enable R8 full mode in `gradle.properties`
3. Strip debug symbols from native libraries
To reduce size further:
1. Enable R8 full mode in `gradle.properties`
2. Strip debug symbols from native libraries

---

## Contributing

When contributing changes to the AI plugins:

1. Test on **physical ARM64 device** (emulators may not support native code)
1. Test on a **physical ARM64 device** — the plugin packages arm64-v8a only, and
an x86_64 emulator cannot run CoGo at all (there is no x86 build of the IDE)
2. Verify **ai-core**, **ai-agent-local**, **ai-agent-gemini** and **ai-assistant** work together
3. Ensure **llama.cpp submodule** updates are documented
4. Run ProGuard/R8 release build to catch reflection issues
Expand Down
3 changes: 2 additions & 1 deletion ai-agent-local/README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -6,7 +6,8 @@ what `ai-core`'s Agent chat, `code-suggestions-plugin`, `speech-to-text-plugin`
`vector-search-plugin` actually talk to.

Runs `.gguf` models through a bundled, prebuilt **llama.cpp** AAR. Declares no
INTERNET permission — prompts and code never leave the device.
INTERNET permission — prompts and code never leave the device. Requires a 64-bit
ARM device (`arm64-v8a`).

## Building

Expand Down
7 changes: 4 additions & 3 deletions ai-agent-local/ai-agent-local.html
Original file line numberDiff line numberDiff line change
Expand Up@@ -96,9 +96,10 @@ <h2>Technical architecture</h2>
Core mounts: model browser, load-from-saved, SHA-256 verification and the
prompt-format toggle, plus the low-memory warning dialog.</td></tr>
</table>
<p>The native library is committed prebuilt (four ABIs: arm64-v8a,
armeabi-v7a, x86, x86_64), so a normal build needs no NDK, CMake, or the
llama.cpp submodule.</p>
<p>The native library is committed prebuilt, so a normal build needs no NDK,
CMake, or the llama.cpp submodule. Packaging is limited to <code>arm64-v8a</code>:
the IDE extracts a single ABI and ships no x86 build, so this plugin requires a
64-bit ARM device.</p>

<h2>Usage</h2>
<ol>
Expand Down
85 changes: 83 additions & 2 deletions ai-agent-local/build.gradle.kts
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,8 @@
import java.io.File
import java.util.Enumeration
import java.util.zip.ZipEntry
import java.util.zip.ZipFile

plugins {
id("com.android.application")
id("org.jetbrains.kotlin.android")
Expand All@@ -16,8 +21,12 @@ android {
applicationId = "com.itsaky.androidide.plugins.aiagentlocal"
minSdk = 33
targetSdk = 36
versionCode = 1
versionName = "1.0.0"
versionCode = 2
versionName = "1.0.1"

ndk {
abiFilters += listOf("arm64-v8a")
}
}

buildTypes {
Expand DownExpand Up@@ -74,6 +83,78 @@ dependencies {
testImplementation("io.mockk:mockk:1.13.8")
}

// The one ABI this plugin ships. Shared by the packaging check and the unit tests.
val expectedAbi = "arm64-v8a"

// Matched on the variant suffix rather than an exact name: plugin-builder owns the file name.
fun File.isPluginArtifactFor(isDebug: Boolean) =
name.endsWith(".cgp") && name.endsWith("-debug.cgp") == isDebug

// Unit tests read the committed AAR; pass its path so they do not depend on the working directory.
tasks.withType<Test>().configureEach {
systemProperty("prebuiltAarPath", file("libs/v8/llama-v8-release.aar").absolutePath)
systemProperty("expectedAbi", expectedAbi)
}

// Guards the abiFilters above: a regenerated AAR or a dropped filter would silently re-inflate the .cgp.
// The check runs as the assemble task's last action rather than as a finalizer: a finalizer executes even
// after a failed assemble, so it would bury the real compile error under a "no .cgp found" failure in CI.
// It stays inline rather than moving to an applied script: apply(from = "*.gradle.kts") crashes
// lintVitalAnalyzeRelease ("Cannot find a KaModule for the VirtualFile") with this AGP/lint version.
// plugin-builder creates these tasks untyped inside its own afterEvaluate, so the task name is the only
// handle; tasks.named fails loudly if that name changes, where a name filter would silently stop wiring.
afterEvaluate {
mapOf("assemblePlugin" to false, "assemblePluginDebug" to true).forEach { (assembleTaskName, isDebug) ->
// Captured as a provider and read inside the action, so no path resolves at configuration time.
val pluginDir = layout.buildDirectory.dir("plugin")
val abi = expectedAbi

tasks.named(assembleTaskName) {
// Runs before plugin-builder copies the new artifact in, so the check below can only ever see
// this run's output. A leftover .cgp (an earlier build under a different pluginName, say) is
// released by CI verbatim, so dropping it here keeps the shipped set and the checked set equal.
doFirst {
pluginDir.get().asFile.listFiles()
?.filter { it.isPluginArtifactFor(isDebug) }
?.forEach { it.delete() }
}

doLast {
val dir = pluginDir.get().asFile
val artifacts = dir.listFiles()
?.filter { it.isPluginArtifactFor(isDebug) }
?.sortedBy { it.name }
.orEmpty()
if (artifacts.isEmpty()) {
throw GradleException("$assembleTaskName produced no .cgp under $dir.")
}

artifacts.forEach { cgp ->
Comment thread
jatezzz marked this conversation as resolved.
val abis = sortedSetOf<String>()
ZipFile(cgp).use { zip ->
val entries: Enumeration<out ZipEntry> = zip.entries()
while (entries.hasMoreElements()) {
val entry: ZipEntry = entries.nextElement()
val name: String = entry.name
if (!entry.isDirectory && name.startsWith("lib/") && name.endsWith(".so")) {
abis.add(name.removePrefix("lib/").substringBefore('/'))
}
}
}

if (abis != sortedSetOf(abi)) {
throw GradleException(
"${cgp.name} packages native libraries for $abis but must package exactly [$abi]. " +
"Check the ndk.abiFilters block in build.gradle.kts.",
)
}
logger.lifecycle("${cgp.name}: native libraries limited to $abi")
}
}
}
}
}

// AAR metadata checks are disabled by convention for these application-as-library
// plugins. The prebuilt llama .aar carries a "core library desugaring required"
// flag, but this module's minSdk (33) makes desugaring unnecessary at runtime,
Expand Down
Binary file modifiedai-agent-local/libs/llama-api.jar
Binary file not shown.
Binary file modifiedai-agent-local/libs/v8/llama-v8-release.aar
Binary file not shown.
3 changes: 1 addition & 2 deletions ai-agent-local/llama-impl/build.gradle.kts
Original file line numberDiff line numberDiff line change
Expand Up@@ -17,8 +17,7 @@ android {
minSdk = 33
consumerProguardFiles("proguard-rules.pro")
ndk {
// Add NDK properties if wanted, e.g.
// abiFilters += listOf("arm64-v8a")
abiFilters += listOf("arm64-v8a")
}
externalNativeBuild {
cmake {
Expand Down
2 changes: 1 addition & 1 deletion ai-agent-local/src/main/assets/docs/index.html
Original file line numberDiff line numberDiff line change
Expand Up@@ -44,7 +44,7 @@ <h1>AI Agent Local — On-Device Inference</h1>
<h2>What it does</h2>
<ul>
<li>Runs a <code>.gguf</code> model through a bundled, prebuilt llama.cpp
library (arm64-v8a, armeabi-v7a, x86 and x86_64).</li>
library. Requires a 64-bit ARM device (<code>arm64-v8a</code>).</li>
<li>Streams tokens as they are generated, and stops early when you press
<b>Stop</b>.</li>
<li>Needs <b>no network access</b> — this plugin declares no INTERNET
Expand Down
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
package com.itsaky.androidide.plugins.aiagentlocal.packaging

import java.io.File
import java.util.zip.ZipFile
import org.junit.Assert.assertEquals
import org.junit.Assert.assertTrue
import org.junit.Test

/**
* Guards the committed llama.cpp AAR against a bad regeneration.
* The plugin filters packaging to one ABI, so the AAR losing that ABI - or a
* partial native build - would only surface as an UnsatisfiedLinkError on device.
*/
class PrebuiltAarAbiTest {

// Gradle passes the AAR path; the relative fallback keeps an IDE test runner - which sets no
// system properties but does run from the module directory - reporting a real assertion.
private val aar: File = File(System.getProperty("prebuiltAarPath") ?: AAR_RELATIVE_PATH)

private val expectedAbi: String = System.getProperty("expectedAbi") ?: DEFAULT_ABI

// One pass over the archive; every test reads this map instead of reopening it.
private val jniLibs: Map<String, Long> by lazy {
ZipFile(aar).use { zip ->
zip.entries()
.asSequence()
.filter { !it.isDirectory && it.name.startsWith("jni/") && it.name.endsWith(".so") }
.associate { it.name to it.size }
}
}

private fun abisPresent(): Set<String> =
jniLibs.keys.map { it.removePrefix("jni/").substringBefore('/') }.toSet()

@Test
fun givenTheCommittedAar_whenLocated_thenItExists() {
assertTrue("Missing prebuilt AAR at ${aar.absolutePath}", aar.isFile)
}

@Test
fun givenTheCommittedAar_whenInspected_thenItShipsTheExpectedAbi() {
assertTrue(
"AAR carries $expectedAbi? Found ${abisPresent()}",
abisPresent().contains(expectedAbi),
)
}

@Test
fun givenTheCommittedAar_whenInspected_thenTheExpectedAbiCarriesEveryNativeLibrary() {
// A partial native build drops one .so and fails at load time, not at build time.
val actual = jniLibs.keys
.filter { it.startsWith("jni/$expectedAbi/") }
.map { it.substringAfterLast('/') }
.toSet()
// Subset, not equality: an upstream llama.cpp bump may legitimately add a library.
assertEquals(
"Prebuilt AAR is missing native libraries (found $actual)",
emptySet<String>(),
REQUIRED_LIBS - actual,
)
}

@Test
fun givenTheCommittedAar_whenInspected_thenTheJniWrapperIsNotEmpty() {
val size = jniLibs["jni/$expectedAbi/libllama-android.so"] ?: -1
assertTrue("libllama-android.so is missing or empty (size=$size)", size > 0)
}

private companion object {
const val DEFAULT_ABI = "arm64-v8a"

const val AAR_RELATIVE_PATH = "libs/v8/llama-v8-release.aar"

/** The libraries llama.cpp has to produce for the wrapper to load at all. */
val REQUIRED_LIBS = setOf(
"libggml-base.so",
"libggml-cpu.so",
"libggml.so",
"libllama-android.so",
"libllama.so",
"libomp.so",
)
}
}
Loading