conductor/docs/REQUIREMENTS.md

# REQUIREMENTS.md

# Project: Conductor REST UI Builder

## Scope Authority

`MVP_SCOPE.md` is the approved v0.1.0 release boundary. `ROADMAP.md` maps that boundary to implementation slices, and `docs/MVP_TRACEABILITY.md` maps each release-critical requirement area to validation evidence. Statements labeled **Post-MVP** are intentionally excluded from v0.1.0 and do not block release.

## 1. Purpose

Build a lightweight UI builder that allows users to create simple frontend applications backed by REST API endpoints, with a focus on IBM Concert Workflows / Rapid Infrastructure Automation use cases.

The tool should allow users to drag and drop UI elements onto a canvas, configure those elements, connect them to REST API calls, and allow one UI element to update another based on API responses or user interaction.

The v0.1.0 application has no AI dependency. Provider-neutral AI assistance may generate, configure, document, or refine project definitions in a future post-MVP increment.

---

## 2. Goals

The system should:

* Allow users to visually build simple UI workflows.
* Support drag-and-drop placement of UI components on a canvas.
* Allow UI components to call REST API endpoints.
* Support multiple authentication types, including anonymous access.
* Allow API responses to update other UI components.
* Make it easier for CSMs, architects, and technical users to build working demos or operational tools without hand-coding every frontend.
* Preserve a canonical project model that future provider-neutral tooling can inspect and modify safely.

---

## 3. Non-Goals

The system is not intended to be a full Appsmith replacement.

The MVP will not include:

* Full database integrations.
* Per-application user assignments, custom roles, groups, or approval workflows beyond global admin/user roles.
* Marketplace-style widget plugins.
* Pixel-perfect design tooling.
* Full workflow orchestration.
* Public SaaS hosting.
* Complex version control or branching.
* Enterprise-grade audit logging.

---

## 4. Primary Users

### 4.1 Customer Success Managers

CSMs may use the tool to build demos, internal utilities, workflow launchers, and simple operational dashboards.

### 4.2 Technical Sellers / Architects

Technical users may use the tool to demonstrate how REST APIs, Concert Workflows, or Rapid Infrastructure Automation endpoints can be exposed through a simple UI.

### 4.3 Internal Operators

Internal IBM teams may use the tool to create small utilities for repeatable operational tasks.

---

## 5. Core Use Cases

### 5.1 Build a Simple Workflow Launcher

A user creates a page with:

* A dropdown to select an environment.
* A text input for parameters.
* A button to call a REST endpoint.
* A status/output panel showing the response.

### 5.2 Build a Dependent Dropdown UI

A user creates:

* A dropdown that calls an API to retrieve available environments.
* A second dropdown that updates based on the selected environment.
* A button that launches a workflow using both selected values.

### 5.3 Build a Read-Only API Dashboard

A user creates:

* A table connected to a REST endpoint.
* A refresh button.
* A details panel that updates when a table row is selected.

### 5.4 Build a Concert/RIA Workflow Form

A user creates:

* Input fields for required workflow parameters.
* Authentication configuration.
* A submit button that calls the workflow endpoint.
* A response display showing success, failure, or returned data.

---

## 6. Functional Requirements

## 6.1 Canvas Builder

The system shall provide a visual canvas where users can place UI elements.

The canvas shall support:

* Drag-and-drop widget placement.
* Selecting widgets.
* Moving widgets.
* Resizing widgets.
* Deleting widgets.
* Basic alignment or snapping.
* Saving the canvas layout.

## 6.2 UI Components

The MVP shall support the following UI components:

* Button
* Text input
* Text area
* Dropdown/select
* Checkbox
* Radio group
* Static text/label
* Table
* JSON response viewer
* Status/message panel
* Container/card

Future components may include:

* Tabs
* Modal dialog
* Date picker
* File upload
* Chart
* Progress indicator

## 6.3 Component Configuration

Each component shall have configurable properties.

Common properties should include:

* Component name / ID
* Label
* Placeholder text
* Default value
* Visibility
* Disabled state
* Required state
* Styling basics

Component-specific properties may include:

* Dropdown options
* Table columns
* Button action
* API binding
* Response mapping
* Validation rules

## 6.4 REST API Configuration

The system shall allow users to define REST API actions.

Each REST API action shall support:

* Name
* Description
* HTTP method: GET, POST, PUT, PATCH, DELETE
* Endpoint URL
* Headers
* Query parameters
* Path parameters
* Request body
* Authentication type
* JSON request bodies where the selected method sends a body
* JSON and safely represented non-JSON responses through the proxy

The backend applies the fixed v0.1.0 proxy-policy timeout. Per-action timeout and expected-response-format controls are post-MVP because schema version `0.1.0` has no such fields.

## 6.5 Authentication Support

The system shall support multiple authentication types for REST API calls.

MVP authentication types:

* Anonymous / no authentication
* Basic authentication
* Bearer token
* API key in header
* API key in query parameter

Future authentication types may include:

* OAuth 2.0
* mTLS
* IBM Cloud IAM token flow
* Custom authentication script or pre-request action

## 6.6 UI-to-API Binding

The system shall allow a UI component to trigger a REST API action.

Examples:

* Button click calls an API endpoint.
* Page load calls an API endpoint.
* Table row selection updates another component.

Dropdown selections may supply values to a later Button-triggered request. Direct Dropdown-triggered REST execution is post-MVP.

The system shall allow component values to be used in API requests.

Examples:

* Text input value used in a POST body.
* Dropdown value used as a query parameter.
* Table row value used in a path parameter.

## 6.7 Component-to-Component Updates

The system shall allow one component to update another component.

Examples:

* Dropdown A updates the options in Dropdown B.
* Button response updates a JSON viewer.
* Table selection updates a details panel.
* API response updates a status/message component.

The MVP should support simple response mapping using JSON paths.

Example:

```json
{
  "source": "apiResponse.environments",
  "target": "environmentDropdown.options"
}
```

## 6.8 AI Assistance — Post-MVP

AI assistance is not a v0.1.0 requirement. Any future integration must be provider-neutral and may assist users by:

* Suggesting UI layouts from a natural language prompt.
* Generating REST API action definitions.
* Explaining API responses.
* Suggesting response mappings.
* Helping generate sample request bodies.
* Helping troubleshoot API errors.
* Producing documentation for a generated UI.
* Suggesting improvements to usability.

Example prompt:

> Build a form that lets me select an environment, enter a hostname, and call a Rapid Infrastructure Automation workflow to provision monitoring.

A future AI assistant should produce a proposed page layout, components, API bindings, and configuration steps without becoming required for normal product operation.

## 6.9 Save and Load

The system shall allow users to save a project.

A saved project should include:

* Pages
* Canvas layout
* Components
* API action definitions
* Top-level component and response bindings
* Non-secret configuration

Secret values shall not be stored in or exported with project definitions. Credential-backed actions contain only opaque server-side references.

## 6.10 Preview Mode

The system shall provide a preview mode.

Preview mode shall allow users to:

* Interact with the UI as an end user.
* Trigger API calls.
* View API responses.
* Validate bindings.
* Test component-to-component updates.

## 6.11 Export

The system should support exporting the project definition as JSON.

Future export options may include:

* React application export
* Static frontend bundle
* Documentation export
* Markdown summary
* Deployment package

## 6.12 Project Editors

Conductor shall support multiple methods for creating and editing a project.

### Visual Editor

The Visual Editor shall be the primary user interface for creating applications.

The Visual Editor shall provide:

* Drag-and-drop placement of UI components
* Component selection
* Component movement and resizing
* Property editing
* Event configuration
* Component binding configuration
* Clear selected-component indication

Canvas zoom/pan and undo/redo are post-MVP enhancements.

Changes made through the Visual Editor shall immediately update the project's canonical JSON definition.

### JSON Editor

Conductor shall provide a JSON Editor for advanced users.

The JSON Editor shall allow users to directly edit the project's canonical JSON document.

The v0.1.0 JSON Editor provides:

* Schema validation
* Error reporting
* Editable canonical JSON with explicit Apply behavior

Syntax highlighting, automatic formatting, search/replace, and a separate read-only mode are post-MVP editor enhancements.

Changes made through the JSON Editor shall immediately update the Visual Editor.

The Visual Editor and JSON Editor shall always represent the same project state.

### Canonical Project Model

Both editors shall operate on the same underlying project definition.

Neither editor shall maintain an independent representation of the application.

All modifications shall update the project's canonical JSON document, which serves as the authoritative representation of the application.


---

## 7. Data Model

## 7.1 Project

A project represents a UI application.

A project contains:

* Project ID
* Name
* Description
* Pages
* API actions
* Global variables
* Metadata

## 7.2 Page

A page contains:

* Page ID
* Name
* Layout
* Components
* Page-level events

## 7.3 Component

A component contains:

* Component ID
* Type
* Name
* Position
* Size
* Properties
* Events

Canonical response and component bindings are stored in top-level `project.bindings`, not inside component records.

## 7.4 API Action

An API action contains:

* Action ID
* Name
* Method
* URL
* Headers
* Parameters
* Body template
* Authentication configuration

Canonical response mappings are top-level `project.bindings`. The legacy action-local `responseMapping` field is compatibility-only and is not executed.

## 7.5 Binding

A binding defines how data moves between components and actions.

A binding contains:

* Source
* Target
* Trigger event

The schema-retained `transform` field and configurable per-binding error policies are compatibility/future shapes and are not executed in v0.1.0.

## 7.6 Project Definition Document

The canonical representation of a Conductor project shall be a single structured JSON document.

This document represents the complete application definition and serves as the authoritative source of truth for the project.

The project definition shall include:

* Project metadata
* Pages
* Canvas layout
* UI components
* Component properties
* Events
* REST API action definitions
* Component bindings
* Global variables
* Application settings
* Theme information (future)
* Version information

The backend shall persist this project definition as JSON.

The backend may extract selected metadata into relational database fields for indexing, searching, reporting, or performance optimization, but the JSON project definition remains the canonical representation.

A project definition should be portable between Conductor installations and suitable for export, import, version control, and future tool-assisted modification.

Example capabilities enabled by this approach include:

* Exporting a complete application as a single file.
* Importing existing applications.
* Versioning projects in Git.
* Comparing changes between revisions.
* Future provider-neutral assisted editing of complete applications.
* Generating documentation from the project definition.
* Generating frontend code from the project definition.

Conductor publishes the v0.1.0 project schema at `shared/schemas/conductor-project.schema.json` for validation, tooling, and interoperability.


---

## 8. Example MVP Workflow

1. User creates a new project.
2. User drags a dropdown onto the canvas.
3. User configures the dropdown to call an API endpoint on page load.
4. API response populates the dropdown options.
5. User drags a text input onto the canvas.
6. User drags a button onto the canvas.
7. User configures the button to call a POST endpoint.
8. The POST body uses values from the dropdown and text input.
9. User drags a JSON viewer onto the canvas.
10. Button response updates the JSON viewer.
11. User tests the flow in preview mode.
12. User saves the project.

---

## 9. Technical Requirements

## 9.1 Frontend

Recommended frontend stack:

* React
* TypeScript
* Drag-and-drop library
* Component state management
* JSON schema-driven configuration panels

## 9.2 Backend

Recommended backend capabilities:

* REST API proxy
* Project persistence
* Secret handling
* Authentication configuration storage
* API execution logging for troubleshooting
* Provider-neutral AI assistant integration (post-MVP)

## 9.3 Security

The system must:

* Avoid exposing secrets in the frontend.
* Store secrets securely.
* Mask sensitive values in logs.
* Support anonymous API calls where appropriate.
* Prevent arbitrary unsafe code execution in user-defined mappings.
* Validate URLs and headers before execution.
* Enforce the default-deny destination policy and require explicit exceptions for approved internal origins.

## 9.4 Observability

The system should provide basic troubleshooting details:

* API request timestamp
* Method and endpoint
* Response status
* Response duration
* Error message
* Sanitized request/response preview

---

## 10. MVP Scope

The MVP should include:

* Multi-page project builder with flat navigation and refreshable published deep links.
* Basic canvas.
* Core widgets.
* REST action configuration.
* Anonymous, Basic, Bearer, and API key authentication.
* Component-to-API binding.
* API response-to-component binding.
* Preview mode.
* Save/load project as JSON.
* Local authentication with global admin/user roles and browser first-run administration.
* Public or authenticated immutable application publishing.
* Single-image production distribution with verified backup and restore.

The MVP does not include IBM Bob, watsonx, or any other AI dependency.

---

## 11. Future Enhancements

Potential future enhancements:

* Nested/dynamic pages and per-page authorization.
* Custom roles, groups, and per-application user assignments.
* OAuth support.
* IBM Cloud IAM integration.
* Advanced or enterprise execution-history retention and integrations.
* Generated React code export.
* Import from OpenAPI specification.
* Visual JSON path mapper.
* Charts and dashboards.
* Reusable templates.
* Version history.
* Approval workflows.
* Managed public SaaS hosting.
* Git integration.
* Team collaboration.

---

## 12. Resolved MVP Decisions

* Conductor v0.1.0 is a standalone web application deployed locally or behind an operator-managed reverse proxy.
* REST calls execute through the backend proxy; credential-backed calls never execute directly from the browser.
* Secrets use encrypted server-side storage and opaque project references as documented in `docs/SECRETS.md`.
* All five modeled authentication modes are required.
* OpenAPI import, generated React output, and provider-neutral AI assistance are post-MVP.
* The primary v0.1.0 output is a working hosted UI backed by a portable canonical project definition.
* Sanitized local execution history is required; enterprise audit retention and SIEM integration are post-MVP.
* Proxy destinations are governed by the default-deny policy and explicit exceptions in `docs/PROXY_SECURITY.md`; arbitrary unsafe URLs are not allowed.
* Production-endpoint approval workflows are an operator policy and post-MVP product capability.

---

## 13. Success Criteria

The MVP is successful if a user can:

* Create a simple UI without writing frontend code.
* Configure at least one REST API endpoint.
* Pass values from UI components into the API request.
* Display the API response in another UI component.
* Save and reload the project.
* Demonstrate a working Concert/RIA workflow launcher.
* Complete all six approved workflows in `MVP_SCOPE.md` without an AI dependency.

## Web Application Requirement

The system shall be delivered as a web-based application.

Users shall access the builder through a browser. No desktop client shall be required.

The application shall include:

* A browser-based frontend UI
* A backend API service
* Persistent project storage
* A secure server-side API proxy for calling external REST endpoints

## Web Server / Hosting Requirement

The system should not implement its own production-grade web server.

Instead, it should run behind a standard web server or reverse proxy such as:

* NGINX
* Apache HTTP Server
* Caddy
* IBM-approved internal hosting infrastructure

The application backend may include an embedded development server for local testing, but production deployment should use an external reverse proxy.

The reverse proxy should handle:

* HTTPS termination
* Routing requests to the same-origin Conductor production service
* Request size limits
* Basic security headers
* Optional access restrictions

Recommended deployment model:

```text
Browser
  ↓
NGINX / Apache / Caddy / IBM-approved reverse proxy
  ↓
Single Conductor service (compiled frontend + Backend API)
  ↓
REST API endpoints / Concert / RIA / other systems
```

The backend service should focus on application logic, project storage, authentication handling, secret management, REST API proxying, validation, and sanitized execution history.

## Backend Persistence Requirement

The backend shall include persistent storage for project definitions and runtime metadata.

For the MVP, SQLite is the preferred database.

SQLite should store:

* Projects
* Canonical project JSON, which contains pages, components, layout, actions, bindings, variables, settings, and non-secret configuration
* Encrypted secret records and metadata
* Sanitized bounded execution history
* Local users, sessions, and immutable publication snapshots

Secrets should not be stored directly in plain text in SQLite.

The database should be treated as local application state for the MVP, not as an enterprise shared data platform.

## Recommended MVP Database

The MVP should use SQLite because it is:

* Simple to deploy
* Easy to back up
* Suitable for single-instance usage
* Lightweight
* Good enough for project metadata and configuration storage
* Easier to package with a demo or proof-of-concept

## Future Database Options

If the project grows beyond MVP, the backend should be designed so SQLite can later be replaced by a full RDBMS such as PostgreSQL.

A full RDBMS may be required if the system needs:

* High-concurrency or horizontally scaled deployments
* Team collaboration
* Custom/per-application authorization models
* High availability
* Centralized deployment
* Enterprise backup/restore
* Larger execution history
* Reporting or analytics
* Strict audit retention