conductor/docs/ARCHITECTURE.md

# ARCHITECTURE.md

# Conductor Architecture

## 1. Overview

Conductor is a web-based application builder for creating simple user interfaces backed by REST API endpoints.

The system consists of:

* React frontend
* Backend API service
* SQLite database
* Canonical JSON project definition
* Server-side REST API proxy
* External web server or reverse proxy

The MVP shall function without AI assistance.

---

## 2. Core Architectural Principle

The canonical source of truth for a Conductor application is a structured JSON project definition.

All editors and runtime views operate against this project definition.

This means:

* The Visual Editor modifies the JSON definition.
* The JSON Editor modifies the same JSON definition.
* Preview mode renders from the JSON definition.
* Saved projects persist the JSON definition.
* Future AI features will read and modify the JSON definition.

---

## 3. Recommended Stack

## Frontend

* React
* TypeScript
* Drag-and-drop canvas library
* Monaco Editor or similar JSON editor
* JSON schema validation

## Backend

* Node.js 20
* Express 4
* TypeScript 5

## Database

* SQLite for MVP
* PostgreSQL as a future migration target

## Web Server

Production deployments should run behind:

* NGINX
* Apache HTTP Server
* Caddy
* IBM-approved internal reverse proxy

The application should not implement its own production-grade web server.

---

## 4. High-Level Architecture

```text
Browser
  |
  |-- Visual Editor
  |-- JSON Editor
  |-- Preview Runtime
  |
  v
Backend API Service
  |
  |-- Project API
  |-- Validation API
  |-- REST Proxy API
  |-- Secret Handling
  |
  v
SQLite Database
  |
  v
Stored Project Definitions
```

External API calls should flow through the backend proxy:

```text
Browser
  |
  v
Conductor Backend
  |
  v
External REST API / Concert / RIA Endpoint
```

---

## 5. Major Components

## 5.1 Visual Editor

The Visual Editor provides the drag-and-drop interface.

Responsibilities:

* Render canvas from project JSON
* Add components
* Move components
* Resize components
* Edit properties
* Configure events
* Configure bindings
* Update canonical project JSON

## 5.2 JSON Editor

The JSON Editor provides direct access to the canonical project definition.

Responsibilities:

* Display project JSON
* Validate against schema
* Show syntax errors
* Apply edits to project state
* Keep Visual Editor synchronized

## 5.3 Preview Runtime

Preview mode renders the project as an end user would experience it.

Responsibilities:

* Render components from JSON
* Execute configured events
* Call backend proxy for REST actions
* Apply response mappings
* Display success/error states

## 5.4 Backend API Service

Responsibilities:

* Save projects
* Load projects
* Validate project definitions
* Execute REST proxy requests
* Store non-secret metadata
* Manage secret references
* Provide basic execution logs

## 5.5 REST API Proxy

The backend shall proxy external REST API calls.

Reasons:

* Avoid exposing secrets in the browser
* Centralize authentication handling
* Support API keys and bearer tokens safely
* Normalize errors
* Capture sanitized execution logs
* Enforce the default-deny destination policy and explicit internal-origin exceptions

## 5.6 Database Layer

SQLite stores:

* Project metadata
* Canonical project JSON
* Encrypted secret records and metadata; canonical JSON stores only opaque references
* Sanitized bounded execution history
* Application settings

The backend should use a data access layer so SQLite can later be replaced with PostgreSQL.

---

## 6. Data Flow

## 6.1 Editing Flow

```text
User edits Visual Editor
  -> Project JSON updated
  -> Schema validation runs
  -> UI rerenders
  -> User saves project
  -> Backend persists JSON to SQLite
```

## 6.2 JSON Editing Flow

```text
User edits JSON
  -> JSON parsed
  -> Schema validation runs
  -> If valid, project state updates
  -> Visual Editor rerenders
  -> User saves project
  -> Backend persists JSON to SQLite
```

## 6.3 API Execution Flow

```text
User clicks button
  -> Event fires
  -> Binding resolves input values
  -> Backend REST proxy is called
  -> Backend calls external API
  -> Response returns to Preview Runtime
  -> Response mapping updates target component
```

---

## 7. Project Definition

The project definition should include:

* Project metadata
* Pages
* Components
* Layout
* Component properties
* Events
* Actions
* Bindings
* Variables
* Settings
* Schema version

Example:

```json
{
  "schemaVersion": "0.1.0",
  "project": {
    "id": "example-project",
    "name": "Example Project",
    "pages": [],
    "actions": [],
    "variables": {},
    "settings": {}
  }
}
```

---

## 8. Security Model

The MVP security model should assume:

* Secrets must not be exposed to the frontend.
* API calls requiring secrets must be executed by the backend.
* Logs must mask sensitive values.
* Project exports should exclude secrets by default.
* Anonymous endpoints may be called without stored credentials.
* Future deployments may require endpoint allowlisting.

Potential sensitive values:

* Authorization headers
* API keys
* Bearer tokens
* Basic auth passwords
* Session cookies

---

## 9. Deployment Model

The MVP deployment model should support:

```text
Reverse Proxy
  -> Frontend static assets
  -> Backend API service
  -> SQLite database file
```

A simple Docker Compose deployment is recommended for local demos and early internal use.

Future deployment options may include:

* Kubernetes
* OpenShift
* IBM Cloud Code Engine
* Internal IBM hosting platform

---

## 10. Suggested Repository Structure

```text
conductor/
  frontend/
    src/
  backend/
    src/
  docs/
    REQUIREMENTS.md
    NICE-TO-HAVE.md
    ARCHITECTURE.md
  examples/
    project-definitions/
  docker-compose.yml
  README.md
```

---

## 11. Initial API Surface

Potential backend endpoints:

```text
GET    /api/projects
POST   /api/projects
GET    /api/projects/:id
PUT    /api/projects/:id
DELETE /api/projects/:id

POST   /api/projects/validate
POST   /api/proxy/execute
GET    /api/executions
DELETE /api/executions
GET    /api/secrets
POST   /api/secrets
PUT    /api/secrets/:id
DELETE /api/secrets/:id
```

---

## 12. Architectural Decisions

Initial decisions:

* Conductor is a web application.
* Conductor uses a React frontend.
* Conductor uses a backend API service.
* Conductor uses SQLite for MVP persistence.
* Conductor stores projects as canonical JSON documents.
* REST API calls flow through the backend proxy.
* Secrets are never intentionally exposed to the browser.
* Production deployments use an external reverse proxy.
* AI assistance is not required for MVP.
* Provider-neutral AI assistance is post-MVP.
* Conductor executes REST actions and simple bindings; general workflow orchestration is post-MVP.



## Backend-Agnostic REST Integration

Conductor shall be designed as a backend-agnostic REST UI builder.

Although the initial target use case is IBM Concert Workflows / Rapid Infrastructure Automation, Conductor should not be tightly coupled to any single automation platform.

Any system that exposes reachable HTTP/REST endpoints may be used as an integration target.

Potential integration targets include:

* IBM Concert Workflows / Rapid Infrastructure Automation
* Node-RED HTTP endpoints
* Custom internal APIs
* FastAPI, Flask, Express, or similar backend services
* Other workflow or automation platforms with REST APIs

Conductor should treat external systems as REST action providers.

For the MVP, Conductor is responsible for:

* Rendering the user interface
* Collecting user input
* Calling configured REST endpoints through the backend proxy
* Passing request parameters
* Receiving responses
* Mapping responses back into UI components

External systems are responsible for:

* Workflow execution
* Automation logic
* Business logic
* External integrations
* Long-running task handling

Conductor should avoid implementing workflow orchestration internally unless required by a future enhancement.