conductor/docs/ARCHITECTURE.md

ARCHITECTURE.md

Conductor Architecture

1. Overview

Conductor is a web-based application builder for creating simple user interfaces backed by REST API endpoints.

The system consists of:

• React frontend
• Backend API service
• SQLite database
• Canonical JSON project definition
• Server-side REST API proxy
• External web server or reverse proxy

The MVP shall function without AI assistance.

---

2. Core Architectural Principle

The canonical source of truth for a Conductor application is a structured JSON project definition.

All editors and runtime views operate against this project definition.

This means:

• The Visual Editor modifies the JSON definition.
• The JSON Editor modifies the same JSON definition.
• Preview mode renders from the JSON definition.
• Saved projects persist the JSON definition.
• Future AI features will read and modify the JSON definition.

---

3. Recommended Stack

Frontend

• React
• TypeScript
• Drag-and-drop canvas library
• Monaco Editor or similar JSON editor
• JSON schema validation

Backend

• Node.js 20
• Express 4
• TypeScript 5

Database

• SQLite for MVP
• PostgreSQL as a future migration target

Web Server

Production deployments should run behind:

• NGINX
• Apache HTTP Server
• Caddy
• IBM-approved internal reverse proxy

The application should not implement its own production-grade web server.

---

4. High-Level Architecture

Browser
  |
  |-- Visual Editor
  |-- JSON Editor
  |-- Preview Runtime
  |
  v
Backend API Service
  |
  |-- Project API
  |-- Validation API
  |-- REST Proxy API
  |-- Secret Handling
  |
  v
SQLite Database
  |
  v
Stored Project Definitions

External API calls should flow through the backend proxy:

Browser
  |
  v
Conductor Backend
  |
  v
External REST API / Concert / RIA Endpoint

---

5. Major Components

5.1 Visual Editor

The Visual Editor provides the drag-and-drop interface.

Responsibilities:

• Render canvas from project JSON
• Add components
• Move components
• Resize components
• Edit properties
• Configure events
• Configure bindings
• Update canonical project JSON

5.2 JSON Editor

The JSON Editor provides direct access to the canonical project definition.

Responsibilities:

• Display project JSON
• Validate against schema
• Show syntax errors
• Apply edits to project state
• Keep Visual Editor synchronized

5.3 Preview Runtime

Preview mode renders the project as an end user would experience it.

Responsibilities:

• Render components from JSON
• Execute configured events
• Call backend proxy for REST actions
• Apply response mappings
• Display success/error states

5.4 Backend API Service

Responsibilities:

• Save projects
• Load projects
• Validate project definitions
• Execute REST proxy requests
• Store non-secret metadata
• Manage secret references
• Provide basic execution logs

5.5 REST API Proxy

The backend shall proxy external REST API calls.

Reasons:

• Avoid exposing secrets in the browser
• Centralize authentication handling
• Support API keys and bearer tokens safely
• Normalize errors
• Capture sanitized execution logs
• Enforce the default-deny destination policy and explicit internal-origin exceptions

5.6 Database Layer

SQLite stores:

• Project metadata
• Canonical project JSON
• Encrypted secret records and metadata; canonical JSON stores only opaque references
• Sanitized bounded execution history
• Application settings

The backend should use a data access layer so SQLite can later be replaced with PostgreSQL.

---

6. Data Flow

6.1 Editing Flow

User edits Visual Editor
  -> Project JSON updated
  -> Schema validation runs
  -> UI rerenders
  -> User saves project
  -> Backend persists JSON to SQLite

6.2 JSON Editing Flow

User edits JSON
  -> JSON parsed
  -> Schema validation runs
  -> If valid, project state updates
  -> Visual Editor rerenders
  -> User saves project
  -> Backend persists JSON to SQLite

6.3 API Execution Flow

User clicks button
  -> Event fires
  -> Binding resolves input values
  -> Backend REST proxy is called
  -> Backend calls external API
  -> Response returns to Preview Runtime
  -> Response mapping updates target component

---

7. Project Definition

The project definition should include:

• Project metadata
• Pages
• Components
• Layout
• Component properties
• Events
• Actions
• Bindings
• Variables
• Settings
• Schema version

Example:

{
  "schemaVersion": "0.1.0",
  "project": {
    "id": "example-project",
    "name": "Example Project",
    "pages": [],
    "actions": [],
    "variables": {},
    "settings": {}
  }
}

---

8. Security Model

The MVP security model should assume:

• Secrets must not be exposed to the frontend.
• API calls requiring secrets must be executed by the backend.
• Logs must mask sensitive values.
• Project exports should exclude secrets by default.
• Anonymous endpoints may be called without stored credentials.
• Future deployments may require endpoint allowlisting.

Potential sensitive values:

• Authorization headers
• API keys
• Bearer tokens
• Basic auth passwords
• Session cookies

---

9. Deployment Model

The MVP deployment model should support:

Reverse Proxy
  -> Single same-origin Conductor production service
       -> Compiled frontend and published SPA routes
       -> Backend API
  -> SQLite database file

Development retains separate React and backend processes. Production uses one non-root, read-only-root container with a dedicated writable /data volume, required persistent encryption/session keys, and a health check. Docker Compose is the supported self-hosted deployment contract; Caddy or NGINX terminates TLS and proxies all paths to the same internal port.

Live SQLite backups use the application's online backup command. A recovery archive includes the database and its matching encryption key plus manifest/checksums; copying only the live WAL database is unsupported.

Future deployment options may include:

• Kubernetes
• OpenShift
• IBM Cloud Code Engine
• Internal IBM hosting platform

---

10. Suggested Repository Structure

conductor/
  frontend/
    src/
  backend/
    src/
  docs/
    REQUIREMENTS.md
    NICE-TO-HAVE.md
    ARCHITECTURE.md
  examples/
    project-definitions/
  docker-compose.yml
  README.md

---

11. Initial API Surface

Potential backend endpoints:

GET    /api/projects
POST   /api/projects
GET    /api/projects/:id
PUT    /api/projects/:id
DELETE /api/projects/:id

POST   /api/projects/validate
POST   /api/proxy/execute
GET    /api/executions
DELETE /api/executions
GET    /api/secrets
POST   /api/secrets
PUT    /api/secrets/:id
DELETE /api/secrets/:id

---

12. Architectural Decisions

Initial decisions:

• Conductor is a web application.
• Conductor uses a React frontend.
• Conductor uses a backend API service.
• Conductor uses SQLite for MVP persistence.
• Conductor stores projects as canonical JSON documents.
• REST API calls flow through the backend proxy.
• Secrets are never intentionally exposed to the browser.
• Production deployments use an external reverse proxy.
• AI assistance is not required for MVP.
• Provider-neutral AI assistance is post-MVP.
• Conductor executes REST actions and simple bindings; general workflow orchestration is post-MVP.

Backend-Agnostic REST Integration

Conductor shall be designed as a backend-agnostic REST UI builder.

Although the initial target use case is IBM Concert Workflows / Rapid Infrastructure Automation, Conductor should not be tightly coupled to any single automation platform.

Any system that exposes reachable HTTP/REST endpoints may be used as an integration target.

Potential integration targets include:

• IBM Concert Workflows / Rapid Infrastructure Automation
• Node-RED HTTP endpoints
• Custom internal APIs
• FastAPI, Flask, Express, or similar backend services
• Other workflow or automation platforms with REST APIs

Conductor should treat external systems as REST action providers.

For the MVP, Conductor is responsible for:

• Rendering the user interface
• Collecting user input
• Calling configured REST endpoints through the backend proxy
• Passing request parameters
• Receiving responses
• Mapping responses back into UI components

External systems are responsible for:

• Workflow execution
• Automation logic
• Business logic
• External integrations
• Long-running task handling

Conductor should avoid implementing workflow orchestration internally unless required by a future enhancement.