MidiPilot — Your AI Copilot

MidiPilot is the AI brain embedded directly in MidiEditor AI. Open the sidebar panel, type what you want in plain English, and watch it compose, edit, transform, and analyze your MIDI data automatically.

MidiPilot Agent mode composing a full arrangement from a single prompt

Key Features

Chat Panel

The MidiPilot panel features a clean chat interface with context-aware track info, mode selection (Agent / Simple), and model switching — all accessible without leaving the editor.

The input bar at the bottom provides quick access to everything you need:

Agent/Simple mode toggle · FFXIV checkbox · Model selector · Reasoning level

• Mode selector — Switch between Agent (multi-step) and Simple (single response)
• New Chat — Start a fresh conversation (previous one is auto-saved to history)
• History — Browse and resume past conversations
• FFXIV checkbox — Enable FFXIV Bard Performance constraints
• Model dropdown — Switch models on the fly without opening settings
• Reasoning level — Adjust thinking effort (when supported by the model)
• Token counter — Shows session usage vs. context window size
• Send / Stop — Send your prompt or cancel an in-progress Agent run
• Settings gear — Open AI settings or save per-file AI presets
• Status indicator — Shows Ready, Thinking, or error states in real time

Simple Mode (One-Shot)

Simple mode sends a single API call to the AI model containing your instruction, the current editor state, selected events, and surrounding musical context. The model responds with one complete answer — no follow-up calls, no iterative loop.

How It Works

1. Your prompt is bundled with a JSON snapshot of the editor (cursor position, active track, tempo, time signature, selection, and ±5 measures of surrounding events)
2. Everything is sent as one request to the model
3. The model returns a single JSON response with one or more actions (edit, delete, create_track, set_tempo, etc.)
4. MidiEditor executes all actions from that response at once

When to Use Simple Mode

• Quick edits — transpose a selection, quantize notes, change velocities
• Small segments — work on a specific track or a few measures
• Focused tasks — delete notes, change instruments, adjust tempo
• Fast turnaround — one call means instant results with no waiting for multi-step loops

Limitations

• Token limit risk — Complex requests (many tracks, long passages) can exceed the model’s output token limit. When this happens the response is truncated, which can result in incomplete or broken edits
• No tool access — The model cannot call tools to inspect the file or request additional context. It works only with whatever was included in the initial snapshot
• No self-correction — If something goes wrong, it cannot verify or retry. You have to undo and try again manually

If MidiPilot detects a truncated response (finish_reason: "length"), it will display a warning suggesting you switch to Agent mode for the task.

Agent Mode (Multi-Step)

Agent mode is the powerhouse for complex compositions and large-scale edits. Instead of squeezing everything into one response, the AI works iteratively — planning, executing, inspecting, and adjusting across multiple API calls until the task is complete.

How It Works

1. Your prompt is sent along with the system prompt and a set of 15 tools the model can call
2. The model responds with one or more tool calls (e.g., create_track, insert_events, get_editor_state)
3. MidiEditor executes each tool call and sends the results back to the model
4. The model reviews the results, decides the next step, and issues more tool calls
5. This loop continues until the model decides the task is complete (or the max steps limit is reached)

Why This Matters

• No token limit risk — Work is split across many smaller API calls, so each individual response stays well within token limits. No truncation, no data loss
• Self-monitoring — The model can call get_editor_state or query_events at any time to inspect what it has built so far, catch mistakes, and correct them
• Complex compositions — Multi-track arrangements, genre conversions, full orchestrations — tasks that would overwhelm a single response are handled step by step
• Parallel tool calls — The model can issue multiple tool calls in a single step (e.g., create three tracks at once), speeding up large jobs
• Granular undo — Each tool call from an Agent run gets its own undo step. Press Ctrl+Z to revert one action at a time, keeping the rest intact

Agent Steps Panel

During an Agent run, a collapsible Agent Steps panel appears below the chat showing real-time progress: each tool call, its parameters, and results. Step indicators use theme-aware colors that adapt to dark and light mode (⏳ pending, 🔄 active, ✅ done, ⚠ retrying, ❌ failed).

When to Use Agent Mode

• Composing from scratch — “Create a jazz waltz with piano, bass, and drums”
• Multi-track work — Adding harmonies, arranging for multiple instruments
• Genre conversion — “Make this classical piece into a metal version”
• FFXIV setups — Preparing an 8-track octet with channel mapping and drum conversion
• Analysis tasks — “Check all tracks for issues and fix them”

Configuration

FFXIV Bard Mode

When the FFXIV checkbox is enabled, MidiPilot appends additional constraints to the system prompt that enforce Final Fantasy XIV Bard Performance rules. This works with both Simple and Agent mode.

• Agent + FFXIV — Full FFXIV context (~3500 chars) with detailed guidance + 3 FFXIV-specific tools (validate_ffxiv, convert_drums_ffxiv, setup_channel_pattern)
• Simple + FFXIV — Compact FFXIV context (~1300 chars) optimized for token budget. Includes a warning that large compositions may exceed limits

Enforced Rules Overview

• ✅ Note range C3–C6 (MIDI 48–84) — strict boundaries, no notes outside
• ✅ Monophonic tracks — one note at a time (Lute/Harp/Piano allow limited chords)
• ✅ Track name = instrument — the track name determines which instrument the bard equips in-game (29 valid names)
• ✅ Instrument → GM program mapping — correct program_change at tick 0 per channel
• ✅ Channels are cosmetic — the channel number does not affect the instrument. Unique channels per track are recommended for clean editing, but not required
• ✅ Drums can stay on channel 9 — the track name (Bass Drum, Snare Drum, Cymbal, etc.) determines the percussion instrument, not the channel
• ✅ GM drum conversion — kick → Bass Drum C4, snare → Snare C5, hi-hat → Cymbal C6, crash → Cymbal C5, toms → Timpani
• ✅ Guitar switches — the only case where channels matter functionally. Each of the 5 variants (Clean, Muted, Overdriven, PowerChords, Special) needs its own channel with program_change at tick 0. Switch sound mid-track by changing the note’s channel
• ✅ No pitch bend / no velocity editing — velocity fixed at 100
• ✅ Instrument transposition — native ranges mapped per instrument (e.g., Piano C4–C7 = −1 octave, Tuba C1–C4 = +2 octaves)
• ✅ Track ordering — melodic instruments first, drums at end (highest track indices)

See FFXIV Prompt Examples for tested prompts.

Fix X|V Channels

The Fix X|V Channels tool provides a one-click, deterministic channel fixer that sets up the complete MidiBard2 channel mapping — no AI calls needed. Find it in the toolbar or via Tools → Fix X|V Channels.

👉 Full documentation: Fix X|V Channels — the 5-step algorithm, Rebuild vs Preserve modes, supported instruments, guitar variant switching, before/after screenshots, and tips.

Mode Comparison

Token Tracking & Context Window

MidiPilot tracks token usage per API call and per session, with automatic normalization across providers (OpenAI, Anthropic, Gemini). The token counter is displayed at the bottom of the chat panel:

<last call> | <session total>🔥 / <context window> [<limit>✂]

• Last call — Prompt + completion tokens from the most recent API call
• Session total — Cumulative tokens since the chat was opened (🔥 fire icon)
• Context window — The model’s known context limit (e.g., 128k for GPT-4o, 1M for GPT-5). Turns yellow when usage exceeds 80%
• Limit — Only shown when a max token limit is enabled (✂ scissors icon)

Context Window Management

When conversations grow long, MidiPilot automatically manages context to prevent exceeding the model’s limit:

• Sliding window — When estimated token usage exceeds 70% of the context window, older messages are dropped while keeping the system prompt, first 2 messages (task context), and most recent messages
• Truncation marker — A “[Context truncated]” message is inserted where older messages were removed
• Visual warning — The token label turns yellow when approaching 80% of the context window

Multi-Provider Token Normalization

Different providers report token usage in different formats. MidiPilot normalizes all of them:

• OpenAI Chat Completions — prompt_tokens / completion_tokens (native format)
• OpenAI Responses API — input_tokens / output_tokens → normalized
• Anthropic — input_tokens / output_tokens → normalized
• Google Gemini — usageMetadata.promptTokenCount / candidatesTokenCount → normalized
• Estimation fallback — When a provider returns no usage data, MidiPilot estimates ~4 characters per token

AI Settings

Configure your AI connection from Settings → MidiPilot AI. Select a provider, enter your API key, choose a model, and customize behavior.

Custom System Prompts

Click Edit System Prompts… in settings to open the built-in editor. Each mode (Simple, Agent, FFXIV, FFXIV Compact) has its own tab with fully customizable instructions.

Built-in System Prompt Editor with tab-based mode selection

• Load Default — Reset the current tab to the built-in default prompt
• Export to JSON — Save all prompts to a file for backup or sharing
• Import from JSON — Load prompts from a previously exported file
• Reset All to Defaults — Restore all modes to factory defaults

Prompts are saved as system_prompts.json in the application directory. If no custom file exists, MidiPilot uses the hardcoded defaults.

Auto-Save

MidiEditor AI automatically saves a backup copy of your work at regular intervals, so you never lose progress to a crash or accidental close. Your original file is never overwritten — the backup is stored as a separate .autosave sidecar file alongside your MIDI file.

How It Works

• Debounce timer — after each edit, a countdown resets. The backup is only written once you pause editing for the configured interval (default: 120 seconds).
• Named files — saved as YourSong.mid.autosave next to the original file.
• Untitled documents — saved to AppData/MidiEditor AI/autosave/untitled.autosave.mid.
• Cleanup — the .autosave backup is automatically deleted when you save normally or exit cleanly.
• Protocol entry — each auto-save appears as an “Auto-saved” marker in the undo history panel.

Crash Recovery

• When opening a file, if a newer .autosave backup exists, MidiEditor AI offers to recover it.
• On startup, orphaned untitled backups from previous crashes are detected and a recovery dialog is shown.
• Recovered files open as unsaved — use Save As to give them a permanent name.

Settings

Auto-save options are in Settings → System & Performance:

Auto-Save settings with enable toggle and interval configuration

AI Tools Reference

In Agent mode, the AI has access to 15 tools (12 base + 3 FFXIV-specific) for inspecting and modifying MIDI files:

Supported Providers

Getting Started

1. Open Settings (gear icon or Edit → Settings) and click the MidiPilot AI tab
2. Select your Provider (Google Gemini is a great free option)
3. Enter your API Key (get one from OpenAI, OpenRouter, or Google Gemini)
4. Choose a Model (e.g., gemini-2.5-flash)
5. Click Test Connection to verify everything works
6. Close settings and open the MidiPilot panel from the sidebar
7. Type a prompt and press Enter:

"Create an 8-bar jazz waltz in Bb major with piano, bass, and drums"

The AI will compose the requested music directly into the editor using its built-in tools. In Agent mode, it works iteratively — creating tracks, setting tempo, inserting notes, and validating the result step by step.

See Prompt Examples for more real-world prompts and a full demo.

Conversation History

MidiPilot automatically saves every conversation as a JSON file. You can browse, search, and resume past sessions at any time.

How It Works

• Auto-save — After every assistant response, the conversation is saved (debounced 2 seconds) to AppData/MidiPilotHistory/
• Associated with MIDI file — Each conversation records which MIDI file it was working on
• History button — Click the 📜 history icon in the toolbar (next to New Chat) to browse past conversations
• Resume — Click any past conversation to load it back into the chat and continue where you left off
• Persistent across sessions — Conversations survive app restarts

Conversation File Format

Each conversation is stored as a single JSON file containing the full message history, model/provider info, token usage, and the associated MIDI file path. Files are human-readable and can be exported or shared.

Response Streaming

In Simple mode, MidiPilot uses Server-Sent Events (SSE) to stream text responses in real time. Instead of waiting for the entire response to complete, you see text appear word by word as the model generates it.

How It Works

• Streaming bubble — A chat bubble appears immediately and grows as text arrives
• Final replacement — Once the response is complete, the streaming bubble is replaced with a properly styled chat bubble
• JSON actions — When the model returns a JSON action response (edits, deletes, etc.), streaming is suppressed and the actions are executed after the full response arrives
• Token capture — Usage statistics are captured from the final [DONE] SSE chunk

When Streaming Is Used

Per-File AI Presets

Different MIDI files may need different AI settings. A 16-track orchestral arrangement needs different guidance than a 3-track FFXIV bard song. Per-file presets let you save and auto-load settings for each file.

What’s Saved

• Provider — OpenAI, OpenRouter, Gemini, or Custom
• Model — e.g., gpt-5.4, gemini-2.5-flash
• Mode — Simple or Agent
• FFXIV mode — On or off
• Reasoning effort — None / Low / Medium / High / Extra High
• Custom instructions — Free-text notes appended to the system prompt (e.g., “This is a jazz arrangement, keep swing feel”)

How to Use

1. Click the ⚙ gear button in the MidiPilot footer
2. Select “Save AI preset for this file”
3. The current settings are saved as a .midipilot.json sidecar file next to your MIDI file
4. Next time you open that MIDI file, the preset is auto-loaded

Sidecar File

Presets are stored as <filename>.midipilot.json next to the MIDI file. For example:

Sweet Child O Mine.mid
Sweet Child O Mine.mid.midipilot.json   ← preset

The preset file is a simple JSON object. All fields are optional — any field not present falls back to the global default.

API Log

MidiPilot writes every API request and response to a log file for debugging and transparency. The log is saved as midipilot_api.log in the same directory as the MidiEditor AI executable.

If the AI produces unexpected results, open the log to inspect the raw JSON sent to and received from the provider. This is especially useful for debugging tool-call sequences in Agent mode.