Skip to content

Data Flows — End-to-End System Flows

This document traces how data moves through the entire Blinkin platform, across all four repositories (picasso-fe, studio-api, zweistein server, and zweistein python services). Each flow includes a Mermaid sequence diagram and a written explanation so developers can understand the complete journey of data through the system.

How to read the diagrams: Each Mermaid sequence diagram below shows actors (systems/services) across the top and time flowing downward. Arrows represent HTTP requests, WebSocket messages, or queue operations. Solid arrows are requests; dashed arrows are responses.

Table of Contents

This flow describes what happens when a creator (e.g., a product manager or content author) uses the Picasso Editor to build and publish an interactive flow (called a "Blink"). The Picasso Editor is a React-based single-page app that communicates with the Studio API backend. All creator-facing API routes live under the picasso/ prefix.

Key source files
File Purpose
picasso-editor/ Editor frontend (React)
flow.controller.ts Flow CRUD — @Controller('picasso/flows')
upload.controller.ts Media upload — @Controller('picasso/upload')
zweistein.controller.ts AI features proxy — @Controller('picasso/zweistein')
collaboration.gateway.ts Real-time collaboration WebSocket
flow.entity.ts Flow entity with JSON columns 
sequenceDiagram
    actor Creator
    participant Picasso as Picasso Editor<br/>(picasso-fe)
    participant Auth0 as Auth0
    participant StudioAPI as Studio API<br/>(studio-api)
    participant PostgreSQL as PostgreSQL
    participant Azure as Azure Blob<br/>Storage
    participant Zweistein as Zweistein<br/>(AI Services)
    participant WS as WebSocket<br/>(Socket.io)

    Creator->>Picasso: Opens Picasso Editor
    Picasso->>Auth0: Redirect to Auth0 Universal Login
    Auth0-->>Picasso: Returns JWT token (user claims)
    Note over Picasso: Token stored in browser,<br/>sent as Bearer token on all requests

    Creator->>Picasso: Creates new flow
    Picasso->>StudioAPI: POST /picasso/flows<br/>(with JWT Bearer token)
    Note over StudioAPI: JwtAuthGuard validates JWT<br/>via JwtStudioStrategy (JWKS)
    StudioAPI->>PostgreSQL: INSERT Flow entity<br/>(uuid, name, status=DRAFT)
    PostgreSQL-->>StudioAPI: Flow created (with ID + UUID)
    StudioAPI-->>Picasso: Response: { data: Flow }

    Creator->>Picasso: Adds steps, forms, AI components
    Picasso->>StudioAPI: POST /picasso/flows/:uuid/save
    Note over StudioAPI: saveFlowData() persists<br/>nodes, components, connections,<br/>forms as JSONB columns
    StudioAPI->>PostgreSQL: UPDATE Flow SET nodes, components,<br/>connections, forms
    PostgreSQL-->>StudioAPI: Updated
    StudioAPI-->>Picasso: { status: "ok", updatedAt }

    Creator->>Picasso: Uploads media (images, videos)
    Picasso->>StudioAPI: POST /picasso/upload<br/>(multipart file upload)
    StudioAPI->>Azure: Upload file to Azure Blob Storage
    Azure-->>StudioAPI: File URL returned
    StudioAPI->>PostgreSQL: INSERT Media entity (URL, metadata)
    StudioAPI-->>Picasso: { data: { url, mediaId } }

    Creator->>Picasso: Uses AI features (generate steps, images, media search)
    Picasso->>StudioAPI: POST /picasso/zweistein/search-media<br/>or POST /picasso/flows/generate-steps-with-ai
    StudioAPI->>Zweistein: Forward request to Zweistein API<br/>(search-media, generate steps, talk-to-image)
    Zweistein-->>StudioAPI: AI-generated content
    StudioAPI-->>Picasso: AI results returned to editor

    Creator->>Picasso: Joins collaborative editing session
    Picasso->>WS: WebSocket connect (Socket.io, transport: websocket)
    Note over WS: WsRBACGuard authenticates<br/>via JWT in handshake
    Picasso->>WS: emit('join-room', { room: flowUuid })
    WS-->>Picasso: Joined room confirmation
    Note over WS: Other editors in same room<br/>receive real-time updates<br/>via 'room-message' events

    Creator->>Picasso: Publishes flow
    Picasso->>StudioAPI: POST /picasso/flows/:uuid/publish
    Note over StudioAPI: createSnapshot() creates a<br/>FlowHistory record — an immutable<br/>snapshot of the current flow state
    StudioAPI->>PostgreSQL: INSERT FlowHistory<br/>(snapshot of nodes, components,<br/>connections, forms)
    PostgreSQL-->>StudioAPI: Snapshot created
    StudioAPI-->>Picasso: { data: FlowHistory }
    Note over Picasso: Flow is now live and<br/>accessible via Houston

What is saved in PostgreSQL: The Flow entity stores the flow's structure using JSONB columns: nodes (the visual steps/screens), components (UI elements within nodes), connections (links between nodes), and forms (form field definitions). When a flow is published, a FlowHistory record captures an immutable snapshot of all of this data.

Flow 2: End-User Interacts with a Published Flow

This flow describes what happens when an end-user visits a published Blink — either directly on the Houston web app or through an embedded widget. The Houston app fetches the published snapshot from the Studio API, renders the interactive flow, and submits collected data back. The Studio API then processes that submission through a Bull queue, triggering integrations, notifications, and data forwarding.

Key source files
File Purpose
houston/ End-user facing app (React)
flow.controller.ts Houston flow endpoints — @Controller('houston/flows')
form-submission.controller.ts Form submission upload
flow-submission.service.ts Queue service — enqueues submission jobs
flow-submission.processor.ts Main processor — @Processor('flow-submission-queue')
mail-sending.processor.ts Email queue — @Processor('mail-sending-queue')
send-results-to-zs.processor.ts Zweistein forwarding queue
integration.service.ts Google Sheets integration
common-flow-webhook.service.ts Webhook delivery 
sequenceDiagram
    actor EndUser
    participant Houston as Houston App<br/>(picasso-fe/houston)
    participant StudioAPI as Studio API<br/>(studio-api)
    participant PostgreSQL as PostgreSQL
    participant SubmitQ as flow-submission-queue<br/>(Bull/Redis)
    participant MailQ as mail-sending-queue<br/>(Bull/Redis)
    participant ZsQ as send-results-to-zs-queue<br/>(Bull/Redis)
    participant Integrations as External Integrations<br/>(Google Sheets, Webhooks)
    participant Zweistein as Zweistein<br/>(AI Services)
    participant Azure as Azure Blob<br/>Storage

    EndUser->>Houston: Visits Houston app or embedded widget<br/>(/b/:slug or direct link)
    Houston->>StudioAPI: GET /houston/flows/:id-:slug
    Note over StudioAPI: No auth required for public flows.<br/>Returns the latest published<br/>FlowHistory snapshot.
    StudioAPI->>PostgreSQL: SELECT FlowHistory<br/>WHERE flow.id = :id<br/>ORDER BY createdAt DESC LIMIT 1
    PostgreSQL-->>StudioAPI: FlowHistory (snapshot data)
    StudioAPI-->>Houston: { data: FlowHistory }<br/>(nodes, components, connections, forms)

    EndUser->>Houston: Interacts with flow<br/>(navigates steps, fills forms,<br/>records audio/video, takes photos)

    EndUser->>Houston: Uploads file attachment during flow
    Houston->>StudioAPI: POST /houston/form-submissions/upload<br/>(multipart, up to 500MB)
    StudioAPI->>Azure: Upload to Azure Blob Storage<br/>(tenant/default/forms/ container)
    Azure-->>StudioAPI: File URL + path
    StudioAPI->>PostgreSQL: INSERT FormAttachment
    StudioAPI-->>Houston: { data: FormAttachment }

    EndUser->>Houston: Completes flow and submits
    Houston->>StudioAPI: POST /houston/flows/v2/submit<br/>{ flowId, sessionId, trackingId,<br/>formSubmissions[], aiSubmissions[] }
    Note over StudioAPI: submitFlowResult() checks<br/>quota limits, generates<br/>checkProcessingStatusToken,<br/>then enqueues to Bull queue

    StudioAPI->>SubmitQ: flowSubmissionQueue.add(payload)
    StudioAPI-->>Houston: { data: { checkProcessingStatusToken } }

    Note over SubmitQ: FlowSubmissionQueueProcessor<br/>picks up the job asynchronously

    SubmitQ->>PostgreSQL: Fetch Flow, FlowHistory snapshot,<br/>enabled integrations
    SubmitQ->>PostgreSQL: INSERT FormSubmission records<br/>(one per form in the flow)
    SubmitQ->>PostgreSQL: INSERT/UPDATE FlowResult<br/>(formResults + aiResults as JSONB)

    alt Has Google Sheets Integration
        SubmitQ->>Integrations: Google Sheets API:<br/>Append row to spreadsheet<br/>(OAuth2 refresh token)
    end

    alt Has Webhooks Configured
        SubmitQ->>Integrations: HTTP POST to webhook URLs<br/>(Make.com, Zapier, custom)
    end

    alt Has Connected Spaces (Zweistein)
        SubmitQ->>ZsQ: sendResultsToZsService.addJob()<br/>{ flowId, formResults, aiResults }
        ZsQ->>Zweistein: Upload form attachments and<br/>AI result files to Zweistein Spaces<br/>via M2M token auth
    end

    alt Has Email Notifications
        SubmitQ->>MailQ: sendFlowSubmissionNotificationsService.addJob()<br/>{ flowId, formResults, aiResults,<br/>forwardEmails, attachments }
        MailQ->>MailQ: MailSendingProcessor renders email<br/>template and sends via SMTP<br/>(Mailer service)
    end

The Bull queue architecture: Studio API uses Bull (backed by Redis) for asynchronous job processing. When a submission arrives, it is immediately enqueued to flow-submission-queue and the HTTP response returns right away with a status token. The FlowSubmissionQueueProcessor then handles the heavy work: saving to the database, triggering integrations (Google Sheets, webhooks), sending notification emails, and forwarding results to Zweistein. This decouples the user experience from potentially slow downstream operations.

Flow 3: AI Chatbot Conversation (Cockpit)

This flow describes the "Cockpit" feature — an AI-powered chatbot that end-users can interact with after or during a flow. The chatbot uses form submission data and AI results as context, creates a conversation in Zweistein, and routes questions through the Query Engine to an LLM (GPT-4o, Claude, or Gemini).

Key source files
File Purpose
flow.controller.ts POST /houston/flows/cockpit-conversation
flow.service.ts createCockpitConversation() method
zweistein-common.service.ts createCockpitConversationUrl() — calls Zweistein
chat/ Chat services (conversations, chatbot, messages)
query_engine/ Python Query Engine (FastAPI, LlamaIndex)
settings.py LLM config (GPT-4o, Claude Opus, Gemini) 
sequenceDiagram
    actor EndUser
    participant Houston as Houston App<br/>(picasso-fe/houston)
    participant StudioAPI as Studio API<br/>(studio-api)
    participant PostgreSQL as PostgreSQL<br/>(Studio API DB)
    participant ZsServer as Zweistein Server<br/>(NestJS)
    participant QueryEngine as Query Engine<br/>(Python/FastAPI)
    participant GoogleSearch as Google File Search<br/>(Vector Store)
    participant LLM as LLM Provider<br/>(GPT-4o / Claude / Gemini)

    EndUser->>Houston: Starts AI chat in a flow<br/>(Cockpit component)
    Houston->>StudioAPI: POST /houston/flows/cockpit-conversation<br/>{ flowUuid, sessionId, trackingId,<br/>chatBotUuid, agentUuid }

    Note over StudioAPI: createCockpitConversation() builds<br/>a context message from form<br/>submissions + AI results

    StudioAPI->>PostgreSQL: Fetch FlowResult<br/>(formResults, aiResults)
    StudioAPI->>PostgreSQL: Fetch FormSubmissions<br/>for this session
    StudioAPI->>PostgreSQL: Fetch FlowHistory<br/>(form field definitions)
    Note over StudioAPI: Builds context message:<br/>"FormName. FieldLabel: value\n..."<br/>+ AI source URLs + transcripts

    alt No Auth Token Present
        StudioAPI->>ZsServer: POST /authz/create-anonymous-token
        ZsServer-->>StudioAPI: { accessToken }
    end

    StudioAPI->>ZsServer: POST /chatbot/conversation-from-context<br/>{ message (context), chatBotUuid,<br/>agentUuid, isHiddenMessage: true }<br/>(Bearer token in header)
    Note over ZsServer: Creates a new Conversation entity,<br/>attaches the hidden context message,<br/>and associates with the chatbot/agent

    ZsServer-->>StudioAPI: { conversationUrl }
    StudioAPI-->>Houston: { data: { token, conversation } }

    Note over Houston: Houston now opens the chat UI<br/>connected to Zweistein

    EndUser->>Houston: Types a question
    Houston->>ZsServer: Send message via chat API<br/>(REST or WebSocket)

    ZsServer->>QueryEngine: POST /chat<br/>{ messages[], concept_id,<br/>system_prompt, max_tokens }
    Note over QueryEngine: Query Engine uses LlamaIndex<br/>with configurable LLM providers

    QueryEngine->>GoogleSearch: Vector similarity search<br/>on concept's file store<br/>(Google File Search API)
    GoogleSearch-->>QueryEngine: Relevant document chunks<br/>(context for RAG)

    QueryEngine->>LLM: Send prompt with retrieved context<br/>+ conversation history + system prompt
    Note over LLM: LLM generates response<br/>(GPT-4o, Claude Opus 4.6,<br/>Gemini 2.5 Pro, etc.)
    LLM-->>QueryEngine: Generated response text

    QueryEngine-->>ZsServer: Response from LLM
    ZsServer-->>Houston: AI response displayed in chat
    Houston-->>EndUser: Shows AI answer

RAG (Retrieval-Augmented Generation) pattern: The Query Engine does not simply forward questions to an LLM. Instead, it first searches the relevant "concept" (knowledge base) using Google File Search to find document chunks that are semantically related to the user's question. These chunks are then included as context in the LLM prompt, ensuring the AI answers based on actual uploaded content rather than general knowledge.

Flow 4: File Ingestion & AI Processing Pipeline

This flow describes what happens when a user uploads a file in the Zweistein Admin panel. The file is stored in Google Cloud Storage, then processed asynchronously by the Python Ingestion Worker via Redis Streams. The worker extracts text, generates embeddings, and indexes content into Google File Search for RAG retrieval.

Key source files
File Purpose
files.service.ts File upload handling
data-processing.service.ts Event-driven processing orchestrator
data-processing.gateway.ts WebSocket status updates
jobs.service.ts Enqueues jobs to Redis Streams
jobs-listener.service.ts Listens for notification stream
redis_worker.py Redis Streams consumer (Python)
message_processor.py File type routing and processing
document.py PDF/text extraction and indexing
image_indexer.py Image OCR via VLM
google_file_search_indexer.py Google File Search indexing
  • zweistein-dev/python_server/ingestion_worker/settings.py — Worker configuration (streams, endpoints)
sequenceDiagram
    actor User
    participant AdminUI as Zweistein Admin<br/>(React SPA)
    participant ZsServer as Zweistein Server<br/>(NestJS)
    participant GCS as Google Cloud<br/>Storage (GCS)
    participant PostgreSQL as PostgreSQL<br/>(Zweistein DB)
    participant RedisJob as Redis Streams<br/>(stream:zweistein)
    participant Worker as Ingestion Worker<br/>(Python)
    participant VLM as IMAGE_EXPLAINER<br/>Endpoint (VLM/OCR)
    participant Whisper as Audio Transcriber<br/>Endpoint
    participant GoogleSearch as Google File Search<br/>(Vector Store)
    participant RedisNotify as Redis Streams<br/>(stream:zweistein:notifications)
    participant WS as WebSocket<br/>(Socket.io)

    User->>AdminUI: Uploads file in Zweistein Admin
    AdminUI->>ZsServer: POST /files/:conceptId/files<br/>(multipart upload)

    ZsServer->>GCS: Upload file to GCS bucket<br/>(path: tenantId/conceptfiles/...)
    GCS-->>ZsServer: Cloud file path stored

    ZsServer->>PostgreSQL: INSERT FileEntity<br/>(filename, concept, tenant,<br/>processingStatus: QUEUED)
    PostgreSQL-->>ZsServer: File entity created (fileId)

    ZsServer->>ZsServer: EventEmitter2.emit('file.uploaded')<br/>{ tenantId, userId, conceptId,<br/>fileId, filename, group }

    Note over ZsServer: DataProcessingService listens<br/>for 'file.uploaded' event<br/>(@OnEvent decorator)

    ZsServer->>RedisJob: XADD stream:zweistein<br/>{ type: 'file', tenantId, conceptId,<br/>entityId, cloudFilePath, filename, group }

    ZsServer-->>AdminUI: { fileId, status: QUEUED }

    Note over Worker: RedisWorker reads from<br/>stream:zweistein using XREADGROUP<br/>(consumer group: group:zweistein)

    Worker->>RedisJob: XREADGROUP GROUP group:zweistein<br/>consumer worker-1 BLOCK 5000
    RedisJob-->>Worker: Message received

    Worker->>RedisNotify: XADD notification:<br/>{ phase: "processing" }
    Note over ZsServer: JobsListenerService reads<br/>notification stream

    RedisNotify-->>ZsServer: Notification received
    ZsServer->>PostgreSQL: UPDATE FileEntity<br/>SET processingStatus = PROCESSING
    ZsServer->>WS: Emit 'jobStatusUpdate'<br/>{ entityId, status: processing }
    WS-->>AdminUI: Real-time status update

    alt PDF / Text / Document Files
        Note over Worker: DocumentIndexer handles:<br/>.pdf, .txt, .md, .docx, .xlsx, etc.
        Worker->>Worker: SimpleDirectoryReader extracts text<br/>(LlamaIndex) or uploads natively<br/>to Google File Search if supported
        Worker->>GoogleSearch: GoogleFileSearchIndexer.index_text()<br/>or index_file() — uploads to<br/>Google File Search vector store
    end

    alt Image Files (.jpg, .png, etc.)
        Note over Worker: ImageIndexer handles images
        Worker->>VLM: POST to IMAGE_EXPLAINER_ENDPOINT/v2<br/>(multipart: image + prompt)
        VLM-->>Worker: Image description text
        Worker->>Worker: VLM meta-prompt chain:<br/>1. Initial description<br/>2. Generate exhaustive prompt<br/>3. Final detailed explanation
        Worker->>GoogleSearch: GoogleFileSearchIndexer.index_text()<br/>(image explanation as text)
    end

    alt Audio / Video Files (.mp3, .mp4, etc.)
        Note over Worker: DocumentIndexer.index_file_containing_audio()
        Worker->>Whisper: POST to AUDIO_TRANSCRIBER_ENDPOINT<br/>{ audio_url: signed GCS URL }
        Whisper-->>Worker: Transcript text
        Note over Worker: Falls back to local Whisper model<br/>if remote transcription fails
        Worker->>GoogleSearch: GoogleFileSearchIndexer.index_text()<br/>(transcript)
    end

    alt Website URLs
        Note over Worker: MessageProcessor._process_website()
        Worker->>Worker: spider_rs scrapes website HTML
        Worker->>Worker: readabilipy strips to content<br/>html2text converts to markdown
        Worker->>GoogleSearch: GoogleFileSearchIndexer.index_text()<br/>(markdown content)
    end

    alt YouTube Videos
        Note over Worker: MessageProcessor._process_youtube()
        Worker->>Worker: Calls Query Engine /quick-actions/media/transcribe<br/>(SSE streaming transcription)
        Worker->>GoogleSearch: GoogleFileSearchIndexer.index_text()<br/>(transcript)
    end

    Worker->>GCS: Upload transcript as .md file<br/>(for user download)
    GCS-->>Worker: Transcript cloud path

    Worker->>RedisNotify: XADD notification:<br/>{ phase: "done",<br/>payload: { transcript_cloud_filepath } }

    RedisNotify-->>ZsServer: Notification received
    ZsServer->>PostgreSQL: UPDATE FileEntity<br/>SET processingStatus = DONE,<br/>link transcript file
    ZsServer->>WS: Emit 'jobStatusUpdate'<br/>{ entityId, status: done }
    WS-->>AdminUI: Real-time: file ready

    Worker->>RedisJob: XACK stream:zweistein<br/>(acknowledge processed message)

Redis Streams architecture: The Zweistein system uses Redis Streams (not Bull queues) for the ingestion pipeline. There are two streams: - stream:zweistein — The job stream. The NestJS server writes jobs here via XADD, and the Python Ingestion Worker reads from it using XREADGROUP with a consumer group (group:zweistein). - stream:zweistein:notifications — The notification stream. The Python worker writes status updates here (processing, done, error), and the NestJS JobsListenerService reads them to update the database and push WebSocket events to the Admin UI.

Indexing backend — Google File Search: The ingestion worker indexes content into Google File Search (a Google-hosted vector store and retrieval service). The GoogleFileSearchIndexer provides a unified interface for indexing text and files. For file types that Google File Search can parse natively (PDF, DOCX, TXT, MD, HTML, and many more), files are uploaded directly. For unsupported formats, text is extracted locally using LlamaIndex's SimpleDirectoryReader and then uploaded as text.

Flow 5: Authentication Flow

This flow shows how authentication works across the Blinkin platform. Both the Picasso Editor (creator-facing) and Houston (end-user-facing) use Auth0 for authentication, but they use different Auth0 applications with different configurations. The Studio API validates tokens using two separate Passport strategies.

Key source files
File Purpose
jwt-studio.strategy.ts JwtStudioStrategy — Picasso Editor users
jwt-houston.strategy.ts JwtHoustonStrategy — Houston users
Both use jwks-rsa to validate tokens against Auth0's JWKS endpoint 
sequenceDiagram
    actor User
    participant App as Picasso Editor<br/>or Houston App
    participant Auth0 as Auth0<br/>Universal Login
    participant StudioAPI as Studio API<br/>(studio-api)
    participant JWKS as Auth0 JWKS<br/>Endpoint

    User->>App: Clicks "Login"

    App->>Auth0: Redirect to Auth0 Universal Login<br/>(different Auth0 app per frontend)
    Note over Auth0: Auth0 app config differs:<br/>• Picasso: AUTH0_ISSUER_URL + AUTH0_AUDIENCE<br/>• Houston: HOUSTON_AUTH0_ISSUER_URL +<br/>  HOUSTON_AUTH0_AUDIENCE

    User->>Auth0: Enters credentials<br/>(email/password, SSO, social login)
    Auth0->>Auth0: Authenticates user
    Auth0-->>App: Returns JWT access token<br/>(contains user claims: sub, email, etc.)

    App->>App: Stores JWT in memory/localStorage

    Note over App: Every subsequent API request<br/>includes: Authorization: Bearer [JWT]

    User->>App: Performs any action
    App->>StudioAPI: API request with<br/>Authorization: Bearer [JWT]

    alt Request from Picasso Editor (picasso/* routes)
        Note over StudioAPI: JwtAuthGuard triggers<br/>JwtStudioStrategy
        StudioAPI->>JWKS: GET https://AUTH0_ISSUER_URL/<br/>.well-known/jwks.json
        Note over JWKS: Returns public keys<br/>to verify JWT signature
        JWKS-->>StudioAPI: JSON Web Key Set
        StudioAPI->>StudioAPI: Verify JWT signature,<br/>issuer, audience
        StudioAPI->>StudioAPI: PassportConfigurationService.validate()<br/>extracts user from token payload
    end

    alt Request from Houston App (houston/* routes)
        Note over StudioAPI: JwtAuthGuard triggers<br/>JwtHoustonStrategy
        StudioAPI->>JWKS: GET https://HOUSTON_AUTH0_ISSUER_URL/<br/>.well-known/jwks.json
        Note over JWKS: Different Auth0 tenant<br/>or application
        JWKS-->>StudioAPI: JSON Web Key Set
        StudioAPI->>StudioAPI: Verify JWT signature,<br/>issuer, audience<br/>(Houston-specific values)
        StudioAPI->>StudioAPI: PassportConfigurationService.validate()<br/>extracts user from token payload
    end

    StudioAPI-->>App: Authorized response<br/>(or 401 if token is invalid/expired)

Two strategies, one shared validation: Both JwtStudioStrategy and JwtHoustonStrategy extend Passport's JWT strategy and use jwks-rsa to dynamically fetch signing keys from Auth0. The key differences are:

Setting JwtStudioStrategy (Picasso) JwtHoustonStrategy (Houston)
Issuer URL AUTH0_ISSUER_URL HOUSTON_AUTH0_ISSUER_URL
Audience AUTH0_AUDIENCE HOUSTON_AUTH0_AUDIENCE
JWKS Endpoint AUTH0_ISSUER_URL/.well-known/jwks.json HOUSTON_AUTH0_ISSUER_URL/.well-known/jwks.json

Both strategies delegate to the same PassportConfigurationService.validate() method to extract and return the user object from the JWT claims.

WebSocket authentication: The collaboration gateway (CollaborationGateway) uses a WsRBACGuard that authenticates the WebSocket handshake using the JWT token, ensuring only authorized users can join editing rooms.

Anonymous tokens: For end-users interacting with cockpit conversations without logging in, the Studio API can request an anonymous token from Zweistein via POST /authz/create-anonymous-token.

Flow 6: Widget Embed Flow

This flow describes how Blinkin flows are embedded into external websites using a lightweight JavaScript widget. The widget scans the page for trigger elements, and when clicked, opens an iframe pointing to the Houston app.

Key source files
File Purpose
main.ts Widget entry point — scans for data-blinkin-slug attributes
iframe.ts Creates the iframe element
popup.ts Popup rendering mode
floating.ts Floating button rendering mode 
sequenceDiagram
    actor Visitor
    participant ExtSite as External Website
    participant Widget as Blinkin Widget<br/>(Embedded JS)
    participant Houston as Houston App<br/>(inside iframe)
    participant StudioAPI as Studio API<br/>(studio-api)
    participant PostgreSQL as PostgreSQL

    Note over ExtSite: Website includes widget script:<br/>script src="widget.js"<br/>and sets BLINKIN_HOST variable

    ExtSite->>Widget: Page loads → window 'load' event
    Widget->>Widget: Scans DOM for elements with<br/>data-blinkin-slug attribute<br/>(also legacy data-houston-slug)
    Widget->>Widget: For floating triggers:<br/>applies BLINKIN_EMBEDDING_DATA<br/>icon + text from global config

    Note over Widget: Widget attaches click handlers<br/>to all discovered trigger elements

    Visitor->>ExtSite: Clicks trigger element<br/>(button, link, floating icon)
    ExtSite->>Widget: Click event fires

    Widget->>Widget: Reads from element dataset:<br/>• blinkinSlug (flow identifier)<br/>• blinkinTrigger (popup | floating)<br/>• branding, share, navigation, closebtn

    alt Trigger type = "popup"
        Widget->>Widget: PopupRenderer creates overlay container
    end

    alt Trigger type = "floating"
        Widget->>Widget: FloatingRenderer creates<br/>floating container<br/>(toggles open/close on re-click)
    end

    Widget->>Widget: IframeRenderer creates iframe<br/>src = BLINKIN_HOST/b/{slug}?params
    Note over Widget: URL params include:<br/>branding, navigation, embed_type,<br/>autoplay, share

    Widget->>ExtSite: Appends iframe + container<br/>to document.body

    Houston->>StudioAPI: GET /houston/flows/:id-:slug<br/>(triggered by iframe page load)
    StudioAPI->>PostgreSQL: Fetch published FlowHistory snapshot
    PostgreSQL-->>StudioAPI: FlowHistory data
    StudioAPI-->>Houston: Flow snapshot returned

    Houston-->>Visitor: Flow renders inside iframe

    Visitor->>Houston: Interacts with flow inside iframe<br/>(fills forms, navigates steps)

    Visitor->>Houston: Submits flow
    Houston->>StudioAPI: POST /houston/flows/v2/submit<br/>(normal Houston submission path)
    Note over StudioAPI: Same submission flow as<br/>Flow 2 — enqueues to<br/>flow-submission-queue

    StudioAPI-->>Houston: Submission acknowledged

Widget trigger modes: The widget supports three trigger modes based on the data-blinkin-trigger attribute: - popup — Opens the flow in a centered overlay/modal - floating — Opens the flow in a floating panel (toggles on repeated clicks) - No trigger attribute — Element is still clickable but requires explicit renderer setup

iframe permissions: The iframe is configured with broad permissions to support camera, microphone, geolocation, autoplay, clipboard, and fullscreen — all features that flows may need.

Summary: Cross-System Communication Map

This diagram provides a bird's-eye view of all communication paths between the four major systems in the Blinkin platform. It shows which system talks to which, what protocol is used, and what kind of data moves between them.

graph TB
    subgraph "Frontend Applications (picasso-fe)"
        PicassoEditor["Picasso Editor<br/>(React SPA)"]
        HoustonApp["Houston App<br/>(React SPA)"]
        Widget["Widget<br/>(Vanilla JS)"]
    end

    subgraph "Studio API (studio-api / NestJS)"
        StudioPicasso["Picasso Routes<br/>picasso/*"]
        StudioHouston["Houston Routes<br/>houston/*"]
        CollabWS["Collaboration Gateway<br/>(Socket.io WebSocket)"]
        SubmitQueue["flow-submission-queue<br/>(Bull/Redis)"]
        MailQueue["mail-sending-queue<br/>(Bull/Redis)"]
        ZsQueue["send-results-to-zs-queue<br/>(Bull/Redis)"]
    end

    subgraph "Zweistein Server (NestJS)"
        ZsAPI["Zweistein REST API<br/>(NestJS Controllers)"]
        ZsChatbot["Chatbot / Chat<br/>Services"]
        ZsFiles["Files Service"]
        DataProc["DataProcessingService<br/>(Event-Driven)"]
        ZsWS["Data Processing Gateway<br/>(Socket.io WebSocket)"]
    end

    subgraph "Zweistein Python Services"
        QueryEngine["Query Engine<br/>(FastAPI + LlamaIndex)"]
        IngestionWorker["Ingestion Worker<br/>(Python + Redis Consumer)"]
    end

    subgraph "Data Stores"
        StudioPG[("PostgreSQL<br/>(Studio API)")]
        ZsPG[("PostgreSQL<br/>(Zweistein)")]
        Redis[("Redis<br/>(Bull Queues +<br/>Redis Streams)")]
        GoogleSearch[("Google File Search<br/>(Vector Store)")]
        GCS[("Google Cloud Storage<br/>(Files)")]
        Azure[("Azure Blob Storage<br/>(Media + Attachments)")]
    end

    subgraph "External Services"
        Auth0["Auth0"]
        LLMProviders["LLM Providers<br/>(OpenAI, Anthropic,<br/>Google AI)"]
        SMTP["SMTP / Mailer"]
        GoogleSheets["Google Sheets API"]
        Webhooks["Webhooks<br/>(Make.com, Zapier, etc.)"]
        VLM["VLM / OCR Service"]
        Whisper["Audio Transcriber"]
    end

    %% Frontend to Studio API
    PicassoEditor -->|"HTTP REST<br/>(JWT Bearer)"| StudioPicasso
    PicassoEditor -->|"WebSocket<br/>(Socket.io)"| CollabWS
    HoustonApp -->|"HTTP REST<br/>(JWT Bearer or public)"| StudioHouston
    Widget -->|"iframe loads<br/>Houston App"| HoustonApp

    %% Studio API to data stores
    StudioPicasso -->|"TypeORM"| StudioPG
    StudioHouston -->|"TypeORM"| StudioPG
    StudioPicasso -->|"File upload"| Azure
    StudioHouston -->|"File upload"| Azure

    %% Studio API queues
    StudioHouston -->|"Enqueue job"| SubmitQueue
    SubmitQueue -->|"Process"| StudioPG
    SubmitQueue -->|"Trigger"| MailQueue
    SubmitQueue -->|"Forward results"| ZsQueue
    SubmitQueue -->|"Google Sheets API"| GoogleSheets
    SubmitQueue -->|"HTTP POST"| Webhooks
    MailQueue -->|"Send email"| SMTP
    ZsQueue -->|"HTTP + M2M token"| ZsAPI

    %% Studio API to Zweistein
    StudioPicasso -->|"HTTP REST<br/>(AI features proxy)"| ZsAPI
    StudioHouston -->|"HTTP REST<br/>(cockpit conversation)"| ZsChatbot

    %% Zweistein internal
    ZsAPI --> ZsFiles
    ZsFiles -->|"Upload"| GCS
    ZsFiles -->|"TypeORM"| ZsPG
    ZsFiles -->|"EventEmitter2<br/>file.uploaded"| DataProc
    DataProc -->|"XADD<br/>stream:zweistein"| Redis
    ZsChatbot -->|"HTTP proxy"| QueryEngine

    %% Python services
    IngestionWorker -->|"XREADGROUP<br/>stream:zweistein"| Redis
    IngestionWorker -->|"XADD<br/>stream:zweistein:notifications"| Redis
    Redis -->|"Notifications<br/>XREADGROUP"| DataProc
    DataProc -->|"Update status"| ZsPG
    DataProc -->|"Emit event"| ZsWS

    %% Ingestion Worker to external
    IngestionWorker -->|"Index text/files"| GoogleSearch
    IngestionWorker -->|"Image analysis"| VLM
    IngestionWorker -->|"Audio transcription"| Whisper
    IngestionWorker -->|"Download files"| GCS
    IngestionWorker -->|"Upload transcripts"| GCS

    %% Query Engine
    QueryEngine -->|"Vector search"| GoogleSearch
    QueryEngine -->|"LLM inference"| LLMProviders

    %% Auth
    PicassoEditor -->|"OAuth2 redirect"| Auth0
    HoustonApp -->|"OAuth2 redirect"| Auth0
    StudioPicasso -->|"JWKS validation"| Auth0
    StudioHouston -->|"JWKS validation"| Auth0

    %% WebSocket to Admin
    ZsWS -->|"WebSocket<br/>(file processing status)"| PicassoEditor

    %% Styling
    classDef frontend fill:#4A90D9,stroke:#2C5F8A,color:#fff
    classDef backend fill:#7B68EE,stroke:#5A4ABF,color:#fff
    classDef python fill:#3CB371,stroke:#2E8B57,color:#fff
    classDef datastore fill:#FF8C00,stroke:#CC7000,color:#fff
    classDef external fill:#DC143C,stroke:#A00F2D,color:#fff

    class PicassoEditor,HoustonApp,Widget frontend
    class StudioPicasso,StudioHouston,CollabWS,SubmitQueue,MailQueue,ZsQueue,ZsAPI,ZsChatbot,ZsFiles,DataProc,ZsWS backend
    class QueryEngine,IngestionWorker python
    class StudioPG,ZsPG,Redis,GoogleSearch,GCS,Azure datastore
    class Auth0,LLMProviders,SMTP,GoogleSheets,Webhooks,VLM,Whisper external

Communication Protocol Summary

Source Destination Protocol Data/Purpose
Picasso Editor Studio API (picasso/*) HTTP REST + JWT Flow CRUD, media upload, AI feature requests
Picasso Editor Studio API (Collaboration) WebSocket (Socket.io) Real-time collaborative editing
Houston App Studio API (houston/*) HTTP REST (public or JWT) Fetch published flows, submit results, cockpit conversations
Widget Houston App iframe embed Loads Houston inside iframe on external sites
Studio API PostgreSQL (Studio) TypeORM / SQL Flow, FlowHistory, FormSubmission, FlowResult, Media entities
Studio API Azure Blob Storage Azure SDK / HTTP Media files, form attachments
Studio API Bull Queues (Redis) Bull (Redis-backed) Async job processing: submissions, emails, Zweistein forwarding
Studio API Zweistein Server HTTP REST AI features proxy, cockpit conversation creation, file upload to spaces
Studio API Google Sheets API OAuth2 + HTTP Integration: append form data to spreadsheets
Studio API Webhooks HTTP POST Integration: forward submission data to external services
Studio API SMTP Nodemailer Notification emails for form submissions
Zweistein Server PostgreSQL (Zweistein) TypeORM / SQL Files, concepts, tenants, conversations, chatbots
Zweistein Server Google Cloud Storage GCS SDK File storage for uploaded content
Zweistein Server Redis Streams ioredis XADD Enqueue file processing jobs
Zweistein Server Admin UI WebSocket (Socket.io) Real-time file processing status updates
Ingestion Worker Redis Streams redis-py XREADGROUP Consume processing jobs, send notifications
Ingestion Worker Google Cloud Storage GCS SDK Download files for processing, upload transcripts
Ingestion Worker Google File Search Google API Index extracted text and file content for RAG
Ingestion Worker VLM/OCR Service HTTP POST Image explanation / OCR
Ingestion Worker Audio Transcriber HTTP POST Audio/video transcription
Query Engine Google File Search Google API Vector similarity search for RAG context retrieval
Query Engine LLM Providers HTTP (OpenAI, Anthropic, Google) Generate AI responses with retrieved context
Both Frontends Auth0 OAuth2 / OIDC User authentication
Studio API Auth0 JWKS HTTP GET JWT token validation