Skip to content

feat(memory-islands): rewrite plugin with modular mixin architecture and OWUI compatibility fixes #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
skorphil wants to merge 3 commits into
base: trunk
Choose a base branch
from
Open

feat(memory-islands): rewrite plugin with modular mixin architecture and OWUI compatibility fixes #13

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3ca0d4f
feat(iterative-research): implement pre-flight validation and empty m…
skorphil Aug 16, 2026
3cee9e1
merge(iterative-research): resolve conflicts with origin/trunk
skorphil Aug 16, 2026
6f069fd
feat(memory-islands): rewrite plugin with modular mixin architecture …
skorphil Aug 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions plugins/iterative-research/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]

### Added
## [Unreleased]

### Added
- Pre-flight configuration validation to ensure at least one search engine (SearXNG or Tavily) is configured and a backend model is specified in the valves before execution.
- Prominent inline, user-facing critical markdown warning banners at the very top of responses when configurations or backend models are missing or invalid.
- Robust unit test coverage for validation rules, including only-Tavily, only-SearXNG, and unexpected falsy valve types.
- Explicit `requirements` metadata field for automated dependency provisioning in isolated environments (v10+).

### Changed
- Removed model auto-detection logic completely to strictly enforce manual MODEL configuration in valves and avoid unexpected database model lookups.

### Fixed
- Misleading exit state ("All Information Gaps Resolved") when the planning LLM call failed or returned an empty response.
- Runtime LLM failure handling in both planning and synthesis phases to raise clear inline errors and halt execution immediately, preventing token and API waste.
- **Open WebUI v11 Compatibility**: Refactored internal imports (`Models`, `generate_chat_completion`) to support both legacy and modern `open_webui` namespaces.
- **Resilience**: Implemented dynamic sync/async resolution for `Users` model calls to ensure stability across database driver upgrades.

Expand Down
2 changes: 1 addition & 1 deletion plugins/iterative-research/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ The plugin can be fully configured by administrators through the Open WebUI Admi
| **CHECK_PREVIEW_RELEASES** | `bool` | `False` | Also checks for pre-release, dev, and preview tags on GitHub. |
| **SEARXNG_URL** | `str` | `""` | The base URL of your SearXNG instance (e.g., `http://searxng:8080`) for private, self-hosted web search. |
| **TAVILY_API_KEY** | `str` | `""` | API key for Tavily Search (alternative or primary search engine API). |
| **MODEL** | `str` | `""` | The internal LLM model ID used for planning, reasoning, and synthesis. (Left blank to auto-detect from active models). |
| **MODEL** | `str` | `""` | The internal LLM model ID used for planning, reasoning, and synthesis. (A valid model ID must be configured). |
| **MAX_STEPS** | `int` | `3` | Maximum number of search-reasoning loops to perform (range: `1` to `10`). |
| **MAX_PAGES_TO_SCRAPE** | `int` | `3` | Maximum number of new web pages to crawl and digest concurrently per iteration (range: `1` to `10`). |
| **CO_STORM_STEERING** | `bool` | `False` | Pauses the research loop after Step 1 to allow interactive user feedback (Co-STORM style). |
Expand Down
9 changes: 6 additions & 3 deletions plugins/iterative-research/install.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,14 +114,17 @@ async def install():
except RuntimeError:
logger.warning(f"Could not activate function: {pid}")

# Configure Default Valves (Enable Pre-release checking by default for this install)
# Configure Default Valves (Enable Pre-release checking and set stepfun model for testing)
try:
valves_to_update = {"CHECK_PREVIEW_RELEASES": True}
valves_to_update = {
"CHECK_PREVIEW_RELEASES": True,
"MODEL": "stepfun/step-3.7-flash:free",
}
if ptype == PTYPE_TOOL:
await client.update_tool_valves(pid, valves_to_update)
else:
await client.update_function_valves(pid, valves_to_update)
logger.info(f"Enabled pre-release checking by default for {pid}")
logger.info(f"Configured default valves for {pid} (stepfun model set for testing)")
except RuntimeError as e:
logger.warning(f"Could not set default valves for {pid}: {e}")

Expand Down
118 changes: 76 additions & 42 deletions plugins/iterative-research/plugin.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,42 +82,13 @@ def _get_datetime_context(self) -> str:
return f"Current Date: {now.strftime('%Y-%m-%d')} ({now.strftime('%A')}) UTC\n"

async def _get_backend_model(self, body: dict[str, Any]) -> str:
"""Resolve the best backend model to use for completion."""
# 1. Check if MODEL is configured in valves
"""Resolve the best backend model to use for completion.

Returns "" if no model is configured.
"""
if hasattr(self, "valves") and self.valves.MODEL:
return self.valves.MODEL

# 2. Check if we can find a non-pipe model in the workspace
# We use noqa: N806 for variable naming since Models is a class name being imported dynamically.
Models = None # noqa: N806
try:
from open_webui.models.models import Models
except ImportError:
try:
from open_webui.apps.webui.models.models import Models
except ImportError:
logger.warning("Could not import Models from open_webui")
Models = None # noqa: N806

if Models is not None:
try:
all_models = await Models.get_all_models()
for m in all_models:
m_id = getattr(m, "id", None)
if not m_id and isinstance(m, dict):
m_id = m.get("id")
if (
m_id
and m_id != "iterative_research"
and m_id != "iterative_research_pipe"
and "pipe" not in m_id
):
return m_id
except Exception as e:
logger.warning("Could not list models from Models: %s", e)

# 3. Fallback to some common default model ID
return "gpt-4o-mini"
return ""

async def _call_llm(
self,
Expand Down Expand Up @@ -289,23 +260,46 @@ async def _stream_research(
except Exception as e:
logger.warning("Could not load user object: %s", e)

# Warn if no search engine configured
# Pre-flight Search Engine Validation
if not tavily_key and not searxng_url:
error_msg = (
"⚠️ **Configuration Error**: No search engine is configured. "
"Please configure either `TAVILY_API_KEY` or `SEARXNG_URL` in the admin valves."
)
logger.error("Pre-flight Validation Failed: No search engine configured.")
if __event_emitter__:
await __event_emitter__(
{
"type": "status",
"data": {
"description": (
"Warning: No search engine (Tavily/SearXNG) configured. "
"Relying on internal knowledge."
),
"done": False,
"description": "Configuration Error: No search engine configured.",
"done": True,
},
}
)
yield error_msg
return

# Pre-flight Model Validation
backend_model = await self._get_backend_model(body)
if not backend_model:
error_msg = (
"⚠️ **Configuration Error**: No backend model is specified. "
"Please specify a valid `MODEL` in the admin valves."
)
logger.error("Pre-flight Validation Failed: No backend model specified.")
if __event_emitter__:
await __event_emitter__(
{
"type": "status",
"data": {
"description": "Configuration Error: No backend model specified.",
"done": True,
},
}
)
yield error_msg
return
scraped_sources: dict[str, dict[str, str]] = {}

# Check for updates and show notification at the very top of the response
Expand Down Expand Up @@ -391,6 +385,27 @@ async def _stream_research(
planning_response = await self._call_llm(
__request__, user_obj, system_prompt, user_prompt, backend_model
)
if not planning_response or not planning_response.strip():
yield "</thinking>\n\n"
yield (
f"> ⚠️ **Error**: Failed to generate planning strategy using backend "
f"model '{backend_model}'. The model may be invalid, offline, or misconfigured.\n"
)
if __event_emitter__:
await __event_emitter__(
{
"type": "status",
"data": {
"description": (
f"Critical Error: Failed to generate planning strategy "
f"using backend model '{backend_model}'."
),
"done": True,
},
}
)
return

plan = self._parse_json_completions(planning_response)
gaps = plan.get("gaps", [])
queries = plan.get("queries", [])
Expand Down Expand Up @@ -822,6 +837,26 @@ async def _synthesize_report(
report = await self._call_llm(
__request__, user_obj, system_prompt, user_prompt, backend_model
)
if not report or not report.strip():
yield (
f"> ⚠️ **Error**: Final synthesis failed due to an empty response "
f"from model '{backend_model}'.\n"
)
if __event_emitter__:
await __event_emitter__(
{
"type": "status",
"data": {
"description": (
f"Critical Error: Final synthesis failed due to an empty "
f"response from model '{backend_model}'."
),
"done": True,
},
}
)
return

yield report

if __event_emitter__:
Expand Down Expand Up @@ -1111,8 +1146,7 @@ class Valves(BaseModel):
MODEL: str = Field(
default="",
description="The internal LLM model ID to use for research planning, gap analysis, "
"and final synthesis. If left empty, the plugin will attempt to auto-detect "
"an available model from the workspace.",
"and final synthesis. A valid model ID must be configured.",
)
MAX_STEPS: int = Field(
default=3,
Expand Down
39 changes: 5 additions & 34 deletions plugins/iterative-research/src/_llm.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,42 +31,13 @@ def _get_datetime_context(self) -> str:
return f"Current Date: {now.strftime('%Y-%m-%d')} ({now.strftime('%A')}) UTC\n"

async def _get_backend_model(self, body: dict[str, Any]) -> str:
"""Resolve the best backend model to use for completion."""
# 1. Check if MODEL is configured in valves
"""Resolve the best backend model to use for completion.

Returns "" if no model is configured.
"""
if hasattr(self, "valves") and self.valves.MODEL:
return self.valves.MODEL

# 2. Check if we can find a non-pipe model in the workspace
# We use noqa: N806 for variable naming since Models is a class name being imported dynamically.
Models = None # noqa: N806
try:
from open_webui.models.models import Models
except ImportError:
try:
from open_webui.apps.webui.models.models import Models
except ImportError:
logger.warning("Could not import Models from open_webui")
Models = None # noqa: N806

if Models is not None:
try:
all_models = await Models.get_all_models()
for m in all_models:
m_id = getattr(m, "id", None)
if not m_id and isinstance(m, dict):
m_id = m.get("id")
if (
m_id
and m_id != "iterative_research"
and m_id != "iterative_research_pipe"
and "pipe" not in m_id
):
return m_id
except Exception as e:
logger.warning("Could not list models from Models: %s", e)

# 3. Fallback to some common default model ID
return "gpt-4o-mini"
return ""

async def _call_llm(
self,
Expand Down
56 changes: 50 additions & 6 deletions plugins/iterative-research/src/_research.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,23 +89,46 @@ async def _stream_research(
except Exception as e:
logger.warning("Could not load user object: %s", e)

# Warn if no search engine configured
# Pre-flight Search Engine Validation
if not tavily_key and not searxng_url:
error_msg = (
"⚠️ **Configuration Error**: No search engine is configured. "
"Please configure either `TAVILY_API_KEY` or `SEARXNG_URL` in the admin valves."
)
logger.error("Pre-flight Validation Failed: No search engine configured.")
if __event_emitter__:
await __event_emitter__(
{
"type": "status",
"data": {
"description": (
"Warning: No search engine (Tavily/SearXNG) configured. "
"Relying on internal knowledge."
),
"done": False,
"description": "Configuration Error: No search engine configured.",
"done": True,
},
}
)
yield error_msg
return

# Pre-flight Model Validation
backend_model = await self._get_backend_model(body)
if not backend_model:
error_msg = (
"⚠️ **Configuration Error**: No backend model is specified. "
"Please specify a valid `MODEL` in the admin valves."
)
logger.error("Pre-flight Validation Failed: No backend model specified.")
if __event_emitter__:
await __event_emitter__(
{
"type": "status",
"data": {
"description": "Configuration Error: No backend model specified.",
"done": True,
},
}
)
yield error_msg
return
scraped_sources: dict[str, dict[str, str]] = {}

# Check for updates and show notification at the very top of the response
Expand Down Expand Up @@ -191,6 +214,27 @@ async def _stream_research(
planning_response = await self._call_llm(
__request__, user_obj, system_prompt, user_prompt, backend_model
)
if not planning_response or not planning_response.strip():
yield "</thinking>\n\n"
yield (
f"> ⚠️ **Error**: Failed to generate planning strategy using backend "
f"model '{backend_model}'. The model may be invalid, offline, or misconfigured.\n"
)
if __event_emitter__:
await __event_emitter__(
{
"type": "status",
"data": {
"description": (
f"Critical Error: Failed to generate planning strategy "
f"using backend model '{backend_model}'."
),
"done": True,
},
}
)
return

plan = self._parse_json_completions(planning_response)
gaps = plan.get("gaps", [])
queries = plan.get("queries", [])
Expand Down
20 changes: 20 additions & 0 deletions plugins/iterative-research/src/_synthesis.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,26 @@ async def _synthesize_report(
report = await self._call_llm(
__request__, user_obj, system_prompt, user_prompt, backend_model
)
if not report or not report.strip():
yield (
f"> ⚠️ **Error**: Final synthesis failed due to an empty response "
f"from model '{backend_model}'.\n"
)
if __event_emitter__:
await __event_emitter__(
{
"type": "status",
"data": {
"description": (
f"Critical Error: Final synthesis failed due to an empty "
f"response from model '{backend_model}'."
),
"done": True,
},
}
)
return

yield report

if __event_emitter__:
Expand Down
3 changes: 1 addition & 2 deletions plugins/iterative-research/src/_valves.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,7 @@ class Valves(BaseModel):
MODEL: str = Field(
default="",
description="The internal LLM model ID to use for research planning, gap analysis, "
"and final synthesis. If left empty, the plugin will attempt to auto-detect "
"an available model from the workspace.",
"and final synthesis. A valid model ID must be configured.",
)
MAX_STEPS: int = Field(
default=3,
Expand Down
Loading
Loading