Skip to content

Commit 31c01ba

Browse files
authored
ADFA-4589 | Add suite of AI-powered developer plugins for Code on the Go (#41)
* feat: add code-suggestions plugin Inline ghost-text AI code completions; restored onto main's split layout. * feat: add speech-to-text plugin Voice-to-code speech input; restored onto main's split layout. * feat: add vector-search plugin Semantic code search via embeddings; restored onto main's split layout. * fix(ai-core): load the user-selected model instead of newest .gguf in Downloads Stream the selected content:// URI into a private cache file; drop the Downloads fallback that silently loaded the wrong model (ADFA-4388). * fix(ai-core): reject embedding models on the chat path instead of crashing Add GgufModelInspector + UserFeedback; classify the .gguf and throw IncompatibleModelException before native decode SIGABRTs the IDE (ADFA-4388). * fix(ai-assistant): fetch live Gemini models and drop retired gemini-1.5-flash listModels now queries the live v1beta ListModels catalog (generateContent-only); default bumped to gemini-2.5-flash and retired models migrated on load. * fix(ai-assistant): settings dialog UX and backend selection bugs Readable settings title; real model name shown and persisted across reopen; and the chat backend indicator + routing now follow the selected backend, refreshing when the settings dialog closes. * fix(code-suggestions-plugin): LLM service lookup * feat(vector-search): wire semantic project search, also fixed code-suggestion-plugin and vector-search-plugin names * refactor: Address AI plugin review feedback - Add version ranges, README/docs, and tiered help coverage for AI plugins - Cancel plugin coroutine scopes during disposal - Move Gemini listModels API key from URL query to x-goog-api-key header - Document Gemini API key storage tradeoff and remove misleading encryption note - Update AI plugin review status with verification results * fix(ai-core): shut down Llm-RunLoop executor on plugin dispose LLamaAndroid's single-thread run-loop executor was never shut down, so the non-daemon Llm-RunLoop thread outlived the plugin and pinned its classloader. Expose LLamaAndroid.shutdown() (closes the ExecutorCoroutineDispatcher) and call it from LocalLlmBackend.close() after the model unload completes. Regenerate the prebuilt llama AAR so the new API is visible to the plugin build. * refactor: add icons day and night for AI plugins * fix(ai-assistant): surface LLM-not-configured feedback and follow IDE light/dark theme - Agent tab now shows an error bubble + Snackbar ("Open Settings...") when no local model or Gemini key is configured, instead of silently doing nothing. - Theme UI to the IDE day/night setting: route inflation through PluginFragmentHelper.getPluginInflater, make PluginTheme a Material3 DayNight theme, add values-night/ colors, and replace hardcoded hex with @color refs. - Fix AI Settings toolbar contrast (colorOnPrimary title/icon on the primary bar). * refactor: extract hardcoded strings to strings.xml and improve user feedback * fix(ai-plugins): resolve PR review findings across the AI plugin suite llama run-loop now recreates after per-plugin dispose, embedding-dim guard, synced suggestion cache, capped file walk, fallback-safe Gemini model migration and vector search indexes in the background (bounded wait) * fix(ai-plugins): honor selected backend and resolve Gemini OkHttp crash - Resolve backend via "auto" in ai-core - Migrate Gemini to REST instead of the google-genai SDK to fix OkHttp 3.x conflict - Implement context-aware suggestion cache key - Add STT raw-transcript fallback and vector-search roots - Refactor code: remove unused code and permissions - Fix version numbers and update plugins documentation * chore(ai-plugins): align IDE version range, docs, and icons across AI suite Refresh manifests, README/HTML docs, and day/night icons for ai-core, ai-assistant, code-suggestions, speech-to-text, and vector-search; update PathGuard and search handler. * fix(ai-core): format local-LLM prompts as ChatML so generation isn't empty * refactor: update docs for the tab key action and improve prompt for suggestions generation
1 parent a61431e commit 31c01ba

99 files changed

Lines changed: 6397 additions & 579 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

README.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,9 @@ See the official [plugin documentation](https://www.appdevforall.org/codeonthego
2525
| [`ai-core/`](ai-core/) | Shared on-device LLM inference backend (bundled llama.cpp AAR) plus a Gemini API backend, exposed to other plugins as a runtime service. |
2626
| [`ai-assistant/`](ai-assistant/) | In-IDE AI chat assistant with tool calling; talks to `ai-core` for inference over local or Gemini models. |
2727
| [`flutter-template/`](flutter-template/) | Adds Flutter starter project templates (Basic, BLoC, Provider, GetX, Riverpod) to the New Project screen. |
28+
| [`code-suggestions-plugin/`](code-suggestions-plugin/) | Inline ghost-text code completions powered by AI. |
29+
| [`speech-to-text-plugin/`](speech-to-text-plugin/) | Voice-to-code: converts speech to code with AI generation. |
30+
| [`vector-search-plugin/`](vector-search-plugin/) | Semantic code search using embeddings and vector similarity. |
2831

2932
## Building a plugin
3033

ai-assistant/README.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,22 @@ The build resolves `plugin-api.jar` from the repo-root `../libs/`.
5656
- `fragments/AiSettingsFragment.kt`, `viewmodel/AiSettingsViewModel.kt` — model/backend config
5757
- `tool/` — the agent tool-loop (executor, router, per-tool handlers, approval)
5858

59+
## Security
60+
61+
- **Gemini API key at rest.** When you use the Gemini backend, the API key is
62+
stored in this plugin's private `SharedPreferences` (`AgentSettings`). That
63+
store is **app-sandboxed** — other apps cannot read it — but it is **not
64+
encrypted at rest**, so it is recoverable on a rooted or otherwise compromised
65+
device. Remove it any time from **AI Settings** (or `clearGeminiApiKey()`).
66+
Encryption is deliberately not layered on here: the key is shared at runtime
67+
with the sibling `ai-core` plugin (which performs the Gemini calls), and
68+
`EncryptedSharedPreferences` would force both independent plugins to agree on a
69+
master-key alias and crypto library version — a fragile cross-plugin coupling.
70+
A host-provided secure-storage service is the correct long-term fix.
71+
- **Gemini API key in transit.** The key is sent to Google as an `x-goog-api-key`
72+
request header (and via the SDK on the chat path), never in a URL query string.
73+
- **File tools are confined to the project root** — see the in-IDE help page.
74+
5975
## License
6076

6177
GPL-3.0 — same as AndroidIDE / CodeOnTheGo.

ai-assistant/ai-assistant.html

Lines changed: 2 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -54,9 +54,8 @@ <h2>1. Executive Overview</h2>
5454
<li><b>AI Core</b> — the inference engine. Provides an
5555
<code>LlmInferenceService</code> with two backends: a local, on-device
5656
model (llama.cpp) and Google Gemini. Install this first.</li>
57-
<li><b>AI Assistant</b> — the user interface. Contributes the Agent tab, the
58-
editor context-menu actions, and the agent tool-loop that drives the
59-
model.</li>
57+
<li><b>AI Assistant</b> — the user interface. Contributes the Agent tab and
58+
the agent tool-loop that drives the model.</li>
6059
</ul>
6160
<p>The two plugins are bridged by a shared service registry
6261
(<code>SharedServices</code>), so the UI plugin discovers the inference
@@ -71,8 +70,6 @@ <h2>2. Core Functionality</h2>
7170
and generate code from templates.</li>
7271
<li><b>Dual inference backends</b> — fully offline on-device inference, or
7372
Gemini in the cloud, selectable in Settings.</li>
74-
<li><b>Editor context actions</b><b>Explain Code</b> and
75-
<b>Generate Code</b> from the editor selection.</li>
7673
<li><b>Safety controls</b> — filesystem tools are confined to the project
7774
root, and mutating tools require explicit user approval.</li>
7875
</ul>

ai-assistant/src/main/AndroidManifest.xml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -25,15 +25,15 @@
2525

2626
<meta-data
2727
android:name="plugin.author"
28-
android:value="AndroidIDE" />
28+
android:value="App Dev for All" />
2929

3030
<meta-data
3131
android:name="plugin.min_ide_version"
3232
android:value="26.17" />
3333

3434
<meta-data
3535
android:name="plugin.max_ide_version"
36-
android:value="26.28" />
36+
android:value="26.30" />
3737

3838
<!-- Number of editor tabs contributed via getEditorTabs() (the "Agent" tab) -->
3939
<meta-data

ai-assistant/src/main/assets/docs/index.html

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -64,10 +64,6 @@ <h2>What the agent can do</h2>
6464
device.
6565
</div>
6666

67-
<h2>Context-menu actions</h2>
68-
<p>Select code in the editor and open the context menu for
69-
<b>Explain Code</b> and <b>Generate Code</b> shortcuts.</p>
70-
7167
<h2>Troubleshooting</h2>
7268
<ul>
7369
<li><b>"No model configured"</b> — select a <code>.gguf</code> file (Local)
9.13 KB
Loading
9.43 KB
Loading

ai-assistant/src/main/kotlin/com/itsaky/androidide/plugins/aiassistant/AiAssistantPlugin.kt

Lines changed: 57 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,6 @@ package com.itsaky.androidide.plugins.aiassistant
33
import com.itsaky.androidide.plugins.IPlugin
44
import com.itsaky.androidide.plugins.PluginContext
55
import com.itsaky.androidide.plugins.extensions.UIExtension
6-
import com.itsaky.androidide.plugins.extensions.ContextMenuContext
76
import com.itsaky.androidide.plugins.extensions.DocumentationExtension
87
import com.itsaky.androidide.plugins.extensions.MenuItem
98
import com.itsaky.androidide.plugins.extensions.PluginTooltipButton
@@ -20,8 +19,20 @@ class AiAssistantPlugin : IPlugin, UIExtension, DocumentationExtension {
2019
private var llmService: LlmInferenceService? = null
2120

2221
companion object {
22+
/** Must match `plugin.id` in AndroidManifest.xml — keys the host's plugin Context lookup
23+
* used by [com.itsaky.androidide.plugins.base.PluginFragmentHelper.getPluginInflater]. */
24+
const val PLUGIN_ID = "com.itsaky.androidide.plugins.aiassistant"
25+
26+
/** Tooltip category for this plugin (strict `plugin_<pluginId>` convention); shared by the tab and the AI Settings screen. */
27+
const val TOOLTIP_CATEGORY = "plugin_$PLUGIN_ID"
28+
2329
const val TOOLTIP_TAG_TAB = "agent_chat_tab"
2430

31+
// Tags for the interactive controls on the AI Settings dialog (see AiSettingsFragment).
32+
const val TOOLTIP_TAG_SETTINGS_BACKEND = "ai_settings_backend"
33+
const val TOOLTIP_TAG_SETTINGS_LOCAL_MODEL = "ai_settings_local_model"
34+
const val TOOLTIP_TAG_SETTINGS_GEMINI_KEY = "ai_settings_gemini_key"
35+
2536
@Volatile
2637
private var pluginContext: PluginContext? = null
2738

@@ -87,30 +98,6 @@ class AiAssistantPlugin : IPlugin, UIExtension, DocumentationExtension {
8798
)
8899
}
89100

90-
override fun getContextMenuItems(menuContext: ContextMenuContext): List<MenuItem> {
91-
val selectedText = menuContext.selectedText
92-
if (selectedText.isNullOrBlank()) {
93-
return emptyList()
94-
}
95-
96-
return listOf(
97-
MenuItem(
98-
id = "ai_explain_code",
99-
title = "Explain Code",
100-
isEnabled = true,
101-
isVisible = true,
102-
action = { context.logger.info("Explain Code clicked") }
103-
),
104-
MenuItem(
105-
id = "ai_generate_code",
106-
title = "Generate Code",
107-
isEnabled = true,
108-
isVisible = true,
109-
action = { context.logger.info("Generate Code clicked") }
110-
)
111-
)
112-
}
113-
114101
override fun getMainMenuItems(): List<MenuItem> = emptyList()
115102

116103
// --- DocumentationExtension: three-tier tooltip help for the Agent tab ---
@@ -120,7 +107,7 @@ class AiAssistantPlugin : IPlugin, UIExtension, DocumentationExtension {
120107
// Tier 3 = `buttons[].uri` (offline HTML page served from
121108
// src/main/assets/docs/ at localhost)
122109

123-
override fun getTooltipCategory(): String = "plugin_ai_assistant"
110+
override fun getTooltipCategory(): String = TOOLTIP_CATEGORY
124111

125112
override fun getTooltipEntries(): List<PluginTooltipEntry> = listOf(
126113
PluginTooltipEntry(
@@ -147,6 +134,50 @@ class AiAssistantPlugin : IPlugin, UIExtension, DocumentationExtension {
147134
order = 0
148135
)
149136
)
137+
),
138+
PluginTooltipEntry(
139+
tag = TOOLTIP_TAG_SETTINGS_BACKEND,
140+
summary = "Choose which model powers the Agent: on-device Local (llama.cpp) or cloud Gemini.",
141+
detail = """
142+
<p>Selects the active inference backend:</p>
143+
<ul>
144+
<li><b>Local</b> — runs a <code>.gguf</code> model entirely on
145+
the device; nothing leaves the phone.</li>
146+
<li><b>Gemini</b> — calls Google's cloud API over HTTPS; needs
147+
an API key.</li>
148+
</ul>
149+
<p>The choice below changes which settings appear.</p>
150+
""".trimIndent(),
151+
buttons = listOf(
152+
PluginTooltipButton(description = "AI Assistant guide", uri = "index.html", order = 0)
153+
)
154+
),
155+
PluginTooltipEntry(
156+
tag = TOOLTIP_TAG_SETTINGS_LOCAL_MODEL,
157+
summary = "Pick a local .gguf chat model to run on-device.",
158+
detail = """
159+
<p>Browse for a <code>.gguf</code> model file to load with
160+
llama.cpp. Use a <b>chat/instruct</b> model — embedding-only
161+
models can't generate replies. Larger models are slower and use
162+
more memory; the file is copied into the app's private storage on
163+
first use.</p>
164+
""".trimIndent(),
165+
buttons = listOf(
166+
PluginTooltipButton(description = "AI Assistant guide", uri = "index.html", order = 0)
167+
)
168+
),
169+
PluginTooltipEntry(
170+
tag = TOOLTIP_TAG_SETTINGS_GEMINI_KEY,
171+
summary = "Enter your Google Gemini API key. It is stored only on this device.",
172+
detail = """
173+
<p>Paste a Gemini API key to enable the cloud backend. The key is
174+
kept in this plugin's private preferences on-device and is sent
175+
only to Google's API over HTTPS. Requests (your prompts and
176+
project context) leave the device when Gemini is selected.</p>
177+
""".trimIndent(),
178+
buttons = listOf(
179+
PluginTooltipButton(description = "AI Assistant guide", uri = "index.html", order = 0)
180+
)
150181
)
151182
)
152183

@@ -229,8 +260,6 @@ class AiAssistantPlugin : IPlugin, UIExtension, DocumentationExtension {
229260
}
230261
}
231262

232-
// Note: Encrypted Gemini API key migration handled by EncryptedPrefs
233-
234263
if (migratedCount > 0) {
235264
context.logger.info("Migrated $migratedCount settings from app to plugin")
236265
} else {

ai-assistant/src/main/kotlin/com/itsaky/androidide/plugins/aiassistant/adapters/ChatAdapter.kt

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,6 @@ import java.util.Date
2727
import java.util.Locale
2828

2929
class ChatAdapter(
30-
private val pluginContext: Context,
3130
private val markwon: Markwon,
3231
private val onMessageAction: (action: String, message: ChatMessage) -> Unit
3332
) : ListAdapter<ChatMessage, RecyclerView.ViewHolder>(DiffCallback) {
@@ -84,8 +83,8 @@ class ChatAdapter(
8483

8584
override fun onCreateViewHolder(parent: ViewGroup, viewType: Int): RecyclerView.ViewHolder {
8685
android.util.Log.d("ChatAdapter", "onCreateViewHolder called, viewType=$viewType")
87-
// Use plugin context for inflating layouts to access plugin resources
88-
val inflater = LayoutInflater.from(pluginContext)
86+
// Inflate from the RecyclerView's Context so item views follow the IDE day/night theme.
87+
val inflater = LayoutInflater.from(parent.context)
8988
return when (viewType) {
9089
VIEW_TYPE_SYSTEM -> {
9190
val view = inflater.inflate(R.layout.list_item_chat_system_message, parent, false)

0 commit comments

Comments
 (0)