Agent Configuration — How Everything Connects¶
This document maps out the Zweistein Agent Configuration Panel — how the different options interact, which settings unlock others, and how the data flows from the UI to the AI engine.
Source file: zweistein-dev/admin/src/common/agents/AgentDetailsForm.tsx (1,311 lines)
The Big Picture¶
The agent configuration has 6 sections. The key insight is that many options are conditional — selecting one option unlocks or hides others. The diagram below shows every dependency.
flowchart TB
subgraph CONFIG["🤖 AGENT CONFIGURATION PANEL"]
direction TB
subgraph S1["1️⃣ BASIC INFO"]
NAME["Name"]
DESC["Short Description"]
WELCOME["Welcome Message"]
THUMB["Thumbnail"]
LONGDESC["Long Description"]
WHEN["When to use"]
end
subgraph S2["2️⃣ TOOLS — The Heart of the Agent"]
direction TB
subgraph DOCTOOLS["📄 Document Analysis"]
SMART["Talk to Your Files\n(Smart Data Agent)"]
FILEREADER["File Reader"]
PDFFILLER["PDF Filler"]
end
subgraph KNOWLEDGE["🔍 Knowledge & Search"]
KB["Zweistein Knowledge Base"]
SEARCH["Internet Search"]
YT["YouTube"]
URL["URL Loader"]
DEEP["Deep Research"]
end
subgraph IMGTOOLS["🖼️ Image Tools"]
VLM["Image Understanding\n(BlinkIn VLM)"]
OVHM["Image Understanding\n(OVH Mistral)"]
OVHQ["Image Understanding\n(OVH Qwen)"]
FLUX["Image Generation\n(Flux.1)"]
GPTIMG["Image Generation\n(GPT-image-1)"]
NANO["Image Generation\n(Nano 🍌)"]
end
subgraph MEDIATOOLS["🎤 Media & Communication"]
VIDEO["Video Analyzer"]
TTS["Text to Speech\n(ElevenLabs)"]
EMAIL["Email Sender"]
end
subgraph ADVTOOLS["⚙️ Integrations"]
MCP["Model Context Protocol"]
ENFORCE["Enforce Tool Usage"]
end
end
subgraph S3["3️⃣ REASONING"]
RMODE["Reasoning Mode"]
REFLECT["Self-Reflection"]
end
subgraph S4["4️⃣ DATA CONTROLS"]
DCTOGGLE["Enable Data Controls"]
end
subgraph S5["5️⃣ MODEL & PARAMETERS"]
LLM["LLM Model Selection"]
PARAMS["Model-Specific Parameters"]
end
subgraph S6["6️⃣ PUBLISHING"]
PUBLISH["Visibility Mode"]
ELEVEN["ElevenLabs API"]
end
end
style CONFIG fill:#f8fafc,stroke:#e2e8f0,stroke-width:2px
style S1 fill:#eff6ff,stroke:#bfdbfe
style S2 fill:#fef3c7,stroke:#fde68a
style S3 fill:#f0fdf4,stroke:#bbf7d0
style S4 fill:#fdf4ff,stroke:#f0abfc
style S5 fill:#fff1f2,stroke:#fecdd3
style S6 fill:#f0f9ff,stroke:#bae6fd
style DOCTOOLS fill:#fff,stroke:#e2e8f0
style KNOWLEDGE fill:#fff,stroke:#e2e8f0
style IMGTOOLS fill:#fff,stroke:#e2e8f0
style MEDIATOOLS fill:#fff,stroke:#e2e8f0
style ADVTOOLS fill:#fff,stroke:#e2e8f0The Dependency Map — What Unlocks What¶
This is the most important diagram. It shows how selecting one option enables, disables, or changes other options.
flowchart LR
subgraph MASTER_SWITCH["🔀 MASTER SWITCH"]
SMART["✅ Talk to Your Files\n(Smart Data Agent)"]
end
subgraph WHEN_ON["When ENABLED ✅"]
direction TB
ON1["Requires Gemini model"]
ON2["Unlocks Smart Data sub-tools:\n• OCR Injection\n• Code Generation\n• Internet Search\n• URL Loader\n• Additional OCR Grounding"]
ON3["File upload area appears\nfor agent documents"]
end
subgraph WHEN_OFF["When DISABLED ❌"]
direction TB
OFF1["File Reader tool available"]
OFF2["PDF Filler tool available"]
OFF3["Zweistein Knowledge Base available"]
OFF4["Internet Search available"]
OFF5["YouTube / URL Loader available"]
OFF6["Deep Research available"]
OFF7["Data Controls section visible"]
end
SMART -->|ON| WHEN_ON
SMART -->|OFF| WHEN_OFF
style MASTER_SWITCH fill:#fef3c7,stroke:#f59e0b,stroke-width:2px
style WHEN_ON fill:#dcfce7,stroke:#86efac
style WHEN_OFF fill:#fee2e2,stroke:#fca5a5Model Selection → Parameter Cascade¶
Choosing a model unlocks specific parameter controls. No two models have the same set of parameters.
flowchart TB
LLM["🧠 LLM Model Selection"]
subgraph GPT5_FAMILY["GPT-5 / 5.1 / 5.2 / Mini / Nano"]
GPT_REASON["Reasoning Depth\nNone | Low | Medium | High"]
GPT_VERBOSE["Output Verbosity\nLow | Medium | High"]
end
subgraph CLAUDE_OPUS["Claude Opus 4.5 / 4.6 / Sonnet 4.5 / Haiku 4.5"]
CLAUDE_THINK["Extended Thinking toggle"]
CLAUDE_MODE["Thinking Mode\nSummarized | Interleaved"]
CLAUDE_EFFORT["Effort Level\nNone | Low | Medium | High"]
CLAUDE_CONTEXT["Context Editing toggle"]
CLAUDE_THINK -->|ON| CLAUDE_MODE
end
subgraph CLAUDE37["Claude 3.7 Sonnet (Thinking)"]
C37_BUDGET["Max Thinking Tokens\n(number)"]
end
subgraph GEMINI25["Gemini 2.5 Flash / Pro"]
G25_BUDGET["Max Thinking Tokens\n(number)"]
end
subgraph GEMINI3["Gemini 3 Pro"]
G3_THINK["Thinking Level\nHigh (default) | Low"]
G3_MEDIA["Media Resolution\nAuto | Low | Medium | High | Ultra High"]
end
subgraph GEMINI31["Gemini 3.1 Pro"]
G31_THINK["Thinking Level\nHigh (default) | Medium | Low"]
G31_MEDIA["Media Resolution\nAuto | Low | Medium | High | Ultra High"]
end
subgraph GEMINI3F["Gemini 3 Flash"]
G3F_THINK["Thinking Level\nHigh | Medium (default) | Low | Minimal"]
G3F_MEDIA["Media Resolution\nAuto | Low | Medium | High | Ultra High"]
end
subgraph ALL_MODELS["All Models"]
MAXOUT["Max Output Tokens\n(number)"]
end
LLM --> GPT5_FAMILY
LLM --> CLAUDE_OPUS
LLM --> CLAUDE37
LLM --> GEMINI25
LLM --> GEMINI3
LLM --> GEMINI31
LLM --> GEMINI3F
LLM --> ALL_MODELS
style LLM fill:#4f46e5,color:#fff,stroke:#4f46e5
style GPT5_FAMILY fill:#eff6ff,stroke:#93c5fd
style CLAUDE_OPUS fill:#faf5ff,stroke:#d8b4fe
style CLAUDE37 fill:#faf5ff,stroke:#d8b4fe
style GEMINI25 fill:#ecfdf5,stroke:#6ee7b7
style GEMINI3 fill:#ecfdf5,stroke:#6ee7b7
style GEMINI31 fill:#ecfdf5,stroke:#6ee7b7
style GEMINI3F fill:#ecfdf5,stroke:#6ee7b7
style ALL_MODELS fill:#f8fafc,stroke:#e2e8f0Deep Research Provider → Settings Cascade¶
When Deep Research is enabled, the provider selection unlocks provider-specific settings.
flowchart LR
DEEP["🔬 Deep Research\ntoggle ON"]
DEEP --> PROVIDER["Research Provider"]
PROVIDER --> GEMINI_DR["Gemini Deep Research\n(no extra settings)"]
PROVIDER --> CLAUDE_DR["Claude Opus 4.6\n(Extended Thinking)"]
PROVIDER --> GPT5_DR["GPT-5.2 Deep Research"]
PROVIDER --> SONAR_DR["Sonar Deep Research\n(Perplexity)\n(no extra settings)"]
PROVIDER --> KIMI_DR["Kimi K2.5 Agent Swarm"]
CLAUDE_DR --> C_BUDGET["Thinking Budget\n4K | 8K | 16K | 32K tokens"]
CLAUDE_DR --> C_VERBOSE["Report Verbosity\nConcise | Detailed | Comprehensive"]
GPT5_DR --> G_EFFORT["Reasoning Effort\nLow | Medium | High"]
GPT5_DR --> G_SOURCE["Sourcing Depth\nLow | Medium | High"]
KIMI_DR --> K_EFFORT["Reasoning Effort\nLow | Medium | High"]
KIMI_DR --> K_SOURCE["Sourcing Depth\nLow | Medium | High"]
style DEEP fill:#f59e0b,color:#fff,stroke:#f59e0b
style PROVIDER fill:#fef3c7,stroke:#fbbf24
style CLAUDE_DR fill:#faf5ff,stroke:#d8b4fe
style GPT5_DR fill:#eff6ff,stroke:#93c5fd
style KIMI_DR fill:#ecfdf5,stroke:#6ee7b7Reasoning Mode → Options Cascade¶
flowchart LR
RMODE["🧭 Reasoning Mode"]
RMODE --> AUTO["Auto\n'Let the agent decide'"]
RMODE --> CUSTOM["Custom\n'I want to provide guidelines'"]
RMODE --> PLAN["Plan\n'Prepare a plan first'"]
CUSTOM --> GUIDELINES["Reasoning Guidelines\n(bullet editor)"]
PLAN --> GUIDELINES
PLAN --> PHINTS["Planning Hints\n(bullet editor)"]
PLAN --> PMODE["Planning Mode\n• Plan and Execute\n• LLM Compiler"]
subgraph SELFREF["Self-Reflection (independent)"]
TOGGLE_REF["Self-Reflection toggle"]
TOGGLE_REF -->|ON| REF_GUIDE["Reflection Guidelines\n(bullet editor)"]
end
style RMODE fill:#16a34a,color:#fff,stroke:#16a34a
style AUTO fill:#f0fdf4,stroke:#86efac
style CUSTOM fill:#f0fdf4,stroke:#86efac
style PLAN fill:#f0fdf4,stroke:#86efac
style SELFREF fill:#f0fdf4,stroke:#86efacText-to-Speech Settings¶
flowchart LR
TTS["🔊 Text to Speech\ntoggle ON"]
TTS --> VOICE_ID["Voice ID\n(e.g. TX3LPaxmHKxFdv7VOQHJ)"]
TTS --> STABILITY["Stability\n0.0 — 1.0"]
TTS --> SIMILARITY["Similarity Boost\n0.0 — 1.0"]
TTS --> STYLE["Style\n0.0 — 1.0"]
TTS --> SPEED["Speed\n0.25 — 4.0"]
TTS --> BOOST["Use Speaker Boost\ntoggle"]
style TTS fill:#7c3aed,color:#fff,stroke:#7c3aedImage Tool Mutual Exclusions¶
flowchart TB
subgraph IMAGE_UNDERSTAND["Image Understanding (pick one or more)"]
VLM["BlinkIn VLM"]
OVHM["OVH Mistral"]
OVHQ["OVH Qwen"]
end
subgraph IMAGE_GENERATE["Image Generation (avoid combining Flux + GPT)"]
FLUX["Flux.1"]
GPTIMG["GPT-image-1"]
NANO["Nano 🍌"]
end
OVHM <-.->|"⚠️ Can't use both\n(auto-disables other)"| OVHQ
FLUX <-.->|"⚠️ Warning if both\nenabled simultaneously"| GPTIMG
style IMAGE_UNDERSTAND fill:#eff6ff,stroke:#93c5fd
style IMAGE_GENERATE fill:#fef3c7,stroke:#fbbf24Complete Data Flow: Config → Query Engine¶
This shows how the configuration you set in the admin panel ultimately affects how the agent behaves at runtime.
sequenceDiagram
participant Admin as 👤 Admin Panel
participant Server as 🖥️ NestJS Server
participant DB as 🗄️ PostgreSQL
participant QE as 🐍 Query Engine
Admin->>Server: POST /api/agents (save config)
Server->>DB: Save AgentEntity (all fields as JSON columns)
Note over Admin,DB: — Later, when a user chats with this agent —
QE->>DB: Load agent config by ID
DB-->>QE: AgentEntity (all settings)
Note over QE: Build agent based on config:
alt Talk to Your Files = ON
QE->>QE: Initialize Smart Data Agent (Gemini)
QE->>QE: Load uploaded files
QE->>QE: Enable OCR / Code Gen / Search sub-tools
else Talk to Your Files = OFF
alt Knowledge Base = ON
QE->>QE: Connect to Qdrant vector search
end
alt Internet Search = ON
QE->>QE: Enable Tavily/Exa search tool
end
alt Deep Research = ON
QE->>QE: Initialize research provider
QE->>QE: Set thinking budget & verbosity
end
end
QE->>QE: Configure LLM with selected model
QE->>QE: Apply model parameters (thinking, effort, tokens)
QE->>QE: Set reasoning mode (auto/custom/plan)
QE->>QE: Enable self-reflection if toggled
alt Reasoning Mode = Plan
QE->>QE: Create execution plan first
QE->>QE: Then execute step-by-step
end
QE-->>Admin: Agent ready for conversationsAll Available LLM Models¶
| Provider | Model | Key Feature |
|---|---|---|
| OpenAI | GPT-4o | General purpose |
| GPT-4o mini | Fast & cheap | |
| GPT o3-mini | Reasoning | |
| GPT o1 | Advanced reasoning | |
| GPT 4.1 | Upgraded GPT-4 | |
| GPT-5 | Latest generation | |
| GPT-5.1 | Improved GPT-5 | |
| GPT-5.2 | Latest OpenAI | |
| GPT-5 Mini | Smaller GPT-5 | |
| GPT-5 Nano | Smallest GPT-5 | |
| Anthropic | Claude 3.5 Sonnet | Balanced |
| Claude 3.7 Sonnet | Extended thinking | |
| Claude 3.7 (Thinking) | Built-in thinking budget | |
| Claude Opus 4.5 | Extended thinking + effort levels | |
| Claude Sonnet 4.5 | Extended thinking + effort levels | |
| Claude Haiku 4.5 | Fast + extended thinking | |
| Claude Opus 4.6 | Latest Claude | |
| Gemini 2.0 Flash | Fast | |
| Gemini 2.5 Flash | Thinking budget | |
| Gemini 2.5 Pro | Thinking budget | |
| Gemini 3 Pro | Thinking levels + media resolution | |
| Gemini 3.1 Pro | 3 thinking levels + media resolution | |
| Gemini 3 Flash | 4 thinking levels + media resolution | |
| Bosch | Gemini 2.5 Flash (Bosch) | Org-restricted |
| Gemini 2.5 Pro (Bosch) | Org-restricted | |
| OVH | OVH/GPT-OSS-120B | Open-source |
Quick Reference: Form Section Visibility Rules¶
| Condition | What Changes |
|---|---|
| Smart Data Agent = ON | Hides: File Reader, PDF Filler, Knowledge Base, Search, YouTube, URL Loader, Deep Research, Data Controls. Shows: Smart Data sub-tools (OCR, Code Gen, etc.) |
| Smart Data Agent = ON + non-Gemini model | Shows warning: "requires Gemini-based model" |
| Deep Research = ON | Shows: Provider selector + provider-specific settings |
| Knowledge Base = ON | Shows: Knowledge Space dropdown |
| MCP = ON | Shows: MCP server integration multi-select |
| Text to Speech = ON | Shows: Voice ID, Stability, Similarity, Style, Speed, Speaker Boost |
| PDF Filler = ON | Shows: Single file upload |
| Reasoning = Custom | Shows: Reasoning guidelines editor |
| Reasoning = Plan | Shows: Guidelines + Planning hints + Planning mode |
| Self-Reflection = ON | Shows: Reflection guidelines editor |
| Data Controls = ON | Shows: Control selection multi-select |
| Claude Extended Thinking = ON | Shows: Thinking mode (Summarized/Interleaved) |
| isInternalAgent = ON | Shows: Default execution action dropdown |
| Nano Image Gen = ON | Shows: Aspect ratio, image size, reference images |
| OVH Mistral = ON | Auto-disables OVH Qwen (and vice versa) |