PM Import Framework (import data into Mission Control boards)
Omniscio's universal import framework converts data from CSV, Excel, Google Sheets, Trello, Jira, Asana, Monday, Linear, Notion, ClickUp and Airtable into local PM boards through one seven-stage pipeline, with conflict strategies, error recovery, job tracking, connected-source refresh and third-party PII consent.
On this page
What it is
Omniscio's PM system includes a universal import framework that converts data from external sources (CSV, Excel, Google Sheets, Trello, Jira, Asana, Monday, Linear, Notion, ClickUp, and Airtable) into local PM boards. The framework includes an import wizard UI, CLI routes for programmatic access, and connected source refresh.
Where to find it
The import wizard is the user-facing entry point: a five-step flow that walks you through selecting a source type, configuring the connection or file, previewing the data (with column type overrides and reorder/rename/exclude controls), executing the import, and reading the report. Import is a shipped feature: it is on whenever Mission Control is on, it has no switch in Settings → Lab, and turning Mission Control off is how you turn it off.
A batch run of several sources at once shows its own progress screen — one row per platform with a progress bar and stage label — which transitions to a completion summary when every job reaches a terminal state.
Connected sources (Google Sheets and the API-based adapters) refresh on a schedule with no UI
at all, and the same pipeline is reachable over the CLI routes under /pm/import/ for agents.
How it behaves
The 8-stage pipeline
The import framework is an 8-stage pipeline: extract, map, validate, execute, report, landing
board, preset application, and upload to Mission Control. Every adapter plugs in by implementing
ImportSourcePlugin: it converts its native data into a canonical intermediate form, and the
pipeline handles column type mapping, validation, atomic writing, job tracking and entity mappings
for re-import dedup.
Stages:
- Extract — the adapter reads the source (a file, an API, etc.) and produces an
ImportManifestcontaining one or moreImportBoards, each with groups, columns, items, and column values. - Map — columns are mapped to PM column types. Known sources (Monday, Jira) have explicit type maps; unknown sources fall back to the column-type inference engine (the same one used by the existing data-transfer CSV import).
- Validate — the manifest is checked for: non-empty board names, no duplicate sourceIds, items reference valid groups, column values reference valid columns. Size is not a validation failure: a board over the 5,000-item limit is not rejected.
- Execute — a board over the 5,000-item limit imports its first 5,000 items, and the report
carries one warning naming the project, how many items came in and how many were left out. A
source that stopped fetching at the limit still warns when it lands at exactly 5,000. Each board
is written in its own transaction, with items batched 50 at a time and the event loop yielding
between batches so progress pushes reach the screen. Boards, groups, columns and items are created
through the standard upsert functions. A cell in a text or long-text column is stored as a JSON
string, the same form the cloud sync and every other writer use, so a later sync carrying the same
text records no history change; every other column keeps the adapter's value as-is, and the merge
comparison encodes the same way. Entity mappings are recorded for dedup. A board-level failure
isolates to that board, so previously committed boards survive. An imported board is written as
local_pending, never as synced: a board claiming a sync that never happened could never be pushed, so it goes out through the normal push like any other local board and appears on other devices. Boards imported by an older build are repaired once by a migration that flips only boards marked local-only with no sync-state row at all. - Report — an
ImportReportis returned with counts (boards/groups/columns/items created, skipped, failed), warnings, errors, the entity source-to-local ID map, and an optionalcontentSummaryarray describing what content types were found in the source and which were imported vs. skipped (e.g., "Databases: 3 imported; Pages: 2 imported"). - Landing board — selects the board to open after import (with several, the busiest: the most items; ties go to the most recently viewed, then most recently modified) and ensures it has a table view. On a clean finish the import screen shows its summary for about three seconds and then opens that board by itself; it stays put when there were errors, when a project did not reach Mission Control, or when the person kept the import on this computer and is offered "Add to Mission Control". "View imported project" still opens it right away.
- Preset application (non-fatal) — for Linear and Jira imports, auto-applies the
matching engineering preset (
engineering-linear/engineering-jira), sets status column settings to the engineering workflow, and remaps imported status values from text to indexed format using state type metadata from the adapter. - Upload to Mission Control (on by default, the person can turn it off) — when chosen, each staged project is created in the chosen workspace in one backend transaction carrying its groups, columns, views and items with their cell values, mirrored locally under the cloud id, and the import's board mapping is re-pointed to that id. See the upload sections below.
Reading a spreadsheet file (CSV, TSV, TXT, JSON, XLSX)
The two file adapters read a person's file and hand the pipeline the same intermediate form every other source produces. Four rules hold on both paths, so a file behaves the same whichever format it arrives in:
- The 5,000-row cap never truncates silently. A file or sheet past the cap imports its first 5,000 rows and carries one warning, in the response and on the job's log, naming how many rows were left out.
- A date column is stored as
YYYY-MM-DD, never as the text the file held. Month-first or day-first is settled by the column's own values (a day field over 12 decides it for the whole column); when nothing proves the order, a consistent separator decides it and the import warns which column and which order it assumed, so a wrong guess is visible. A column whose values conflict keeps its original text and warns. Values become ISO; a value that cannot be read is left empty and named in a warning. - A
Group(orSection) column becomes real groups, and is not kept as a board column. This is what makes an export → import round trip restore the groups. - A formula cell imports its stored result. A formula saved with no result imports as an empty
cell, with one warning per column naming the count — never as
[object Object].
A workbook with nothing to import is refused with the same message a file with no rows gets
(The file contains no data to import.), and a multi-sheet workbook lists every board it created,
each with its own item count.
A board exported and imported back keeps its real shape (xlsx only)
A plain table has nowhere to put a column's type, its settings, or a cell's stored value, so an import used to re-guess all three from the header word and the shape of the cells: an owner came back as text instead of a person, and a Status column collapsed to the one label it happened to use.
The xlsx export now writes a hidden second worksheet named amc-board-metadata immediately
after the visible one, and the import reads it. The visible sheet is unchanged and is still what a
person edits.
- What travels: one JSON object per row — a format version, each column's exact stored type and settings, each cell's exact stored value and text, and the text the export actually rendered into the visible sheet.
- The visible sheet stays the authority on what exists. The metadata never decides which items or columns are present; both its column list and its cell list are built from the same ordered array the visible header is built from, so it cannot describe a column the sheet does not show.
- An edit wins, every time. A cell takes its stored value only while it still says exactly what
the export wrote into it; edit a cell and the edit is imported, while the column keeps its real
type and settings. The comparison also accepts the formula-neutralized form, because a value a
spreadsheet treats as a formula (a phone number starting with
+) is written with a leading apostrophe and must still be recognized as unedited. - Absent, unreadable or wrong-versioned metadata falls back to today's behaviour, silently and without throwing. A row too large for a spreadsheet's cell limit is dropped rather than written, so one oversized cell costs one cell, not the export.
- Renaming an item or a group loses that row's stored values, because the importer identifies a row by its group and name. The rename still imports; only the extra detail is lost.
CSV keeps no such channel, deliberately. It cannot hide data — anything it carried would be a row or a column a person would see — so CSV stays a plain spreadsheet and keeps its losses: a column's type survives only where its header word and cells imply it, its settings cannot come back at all, and a cell's structured value follows its type. That is a property of the format, not a gap in the code. Use xlsx for a backup or a move.
What could not be tested. A hidden worksheet survives a write and read with its hidden state intact, which is proven by test. It has NOT been verified against Excel, Google Sheets or LibreOffice — a tool that strips hidden sheets on re-save would silently drop the metadata and give back today's lossy import.
Conflict strategies
When re-importing data that overlaps with previously imported entities:
- Skip (default) — if the entity was already imported (found via entity mapping + liveness check), skip it entirely.
- Overwrite — reuse the existing local ID and upsert over it. Changed items count
as
itemsUpdated. - Merge — reuse the existing local ID but compare name, group, and all column
values against existing rows before writing. Unchanged items are skipped
(
itemsUnchanged); changed items are upserted (itemsUpdated). The import wizard shows a SegmentedControl strategy picker when existing items are detected, with a per-board preview of new vs. existing item counts.
A re-import never writes over a project you cannot edit. If you can only view the existing project, the import skips it and says why, and the project stays exactly as it was. A re-import you are allowed to run keeps the project's read-only and private-column settings.
A task deleted at the source goes away on re-import. Under overwrite or merge (a scheduled refresh always uses overwrite), a task this source imported onto a project, which is missing from a fresh download of that project, is soft-deleted, with import history keyed to the run so it can be restored. The result says how many were removed from which project. Nothing is removed when the download carried any warning (a cut-short list, a failed board, an unread subtask list), came back empty, or reached the 5,000-task limit; a task added by hand in Mission Control, or one with an offline edit still waiting to sync, is never touched. Skip leaves an already-imported project alone entirely.
A stopped import keeps what arrived and says so. Each project's download records how far it got. Only a download known to be complete may remove anything, so a source that records nothing removes nothing. When a download stops early (it took too long, a later page failed after its retries, or the import was canceled), every task that arrived is kept, the result reads "partial" instead of "completed", and the warnings say "Import incomplete: <project> ... Nothing was removed." Canceling stops the download between pages; it never throws away what already arrived. Monday.com records every download today; the other connected sources do not record one yet, so they never remove tasks at the source.
Error recovery
Five mechanisms cover the failure paths:
- Retry with backoff — extraction retries 3 times (1s base, 16s max, full jitter). Permanent errors (400/401/403/404/422) fail immediately; transient errors (5xx, 429, network) are retried. Per-board extraction retries each board independently.
- Per-board resilience — a board that fails extraction after retries does not stop the others; failed boards appear as warnings on the manifest and in the report, and if every board fails with the same error class the error surfaces as one primary message.
- Checkpointing — after successful extraction the manifest is saved on the job row. A later import from the same source by the same signed-in user reuses it and skips extraction, but only when the input fingerprint matches (adapter, user, credentials, selection, options, file). A checkpoint taken for a different selection, sheet, key or user, or a legacy one with no fingerprint, is ignored. Checkpoints are cleared on success and cleaned up after 7 days.
- Preview reuse — the preview keeps its manifest (at most 2 previews, 10 minutes, 50,000 items) and returns a single-use id; executing with that id and an identical fingerprint skips the second download. Mapping, validation and de-duplication always run on the reused manifest. An Asana import with section-to-status mappings downloads again, because those mappings change what is downloaded.
- Humanized errors — raw API and HTTP error strings are mapped to friendly messages with
recovery hints, and the raw errors stay in the job's error log. A per-item failure keeps the shape
Item "<name>": <reason>, so a listed failure still says WHICH item it was.
The batch progress screen shows a rate-limited stage when the pipeline retries after a transient error, so the user knows the import is waiting, not stalled.
Job tracking
Every pipeline run creates a row in pm_import_jobs tracking source, status (pending /
extracting / mapping / validating / importing / completed / partial / failed / cancelled),
conflict strategy, entity counts, error log, and timestamps. Entity mappings
(pm_import_entity_map) persist across jobs so re-imports of the same source can
detect previously imported entities.
Connected sources and scheduled refresh
Google Sheets and API-based adapters can be saved as connected sources that
refresh automatically. Sources are stored in pm_import_sources with connection config,
refresh schedule (hourly/daily/weekly/manual), and sync timestamps. The refresh scheduler
checks every 5 minutes and re-runs the pipeline with overwrite strategy for due sources.
A refresh runs unattended, so it inherits the third-party PII consent recorded on the source
row rather than asking again — see below.
A refresh re-uses the projects you picked. The selection an import was run with is saved on the
source row (selected_entities), so a scheduled or manual refresh re-imports exactly those boards
and projects and updates them in place. A source created before this was recorded falls back to
whatever that adapter already imported, so an existing connection keeps working. If one of those
projects has since been deleted at the source, or is no longer shared with your connection, it is
skipped and the reason is written to the source's error log — the rest of the refresh still runs.
A refresh whose credentials are refused still fails and says so, so a genuine outage is never
mistaken for "nothing to update".
Third-party PII consent
Importing from a platform — Notion, Trello, Asana, Jira, Monday, ClickUp, Linear,
Airtable — copies other people's names, email addresses and phone numbers out of that
account and into the local database, alongside the tasks. The executor refuses such an
import unless the caller passes thirdPartyPiiConsent.
Importing a document the user already owns is exempt: csv, excel and
google-sheets (their own sheet, read with their own credentials) carry no account directory. The
exempt set is an allow-list, so a newly added adapter needs consent by default, and both the executor
and the wizard check it through the same function, so the UI and the gate always agree.
Where the acknowledgment comes from:
| Caller | Source of consent |
|---|---|
| Import wizard | A toggle on the preview step, shown only for a non-exempt source. Start Import stays disabled until it is on. |
POST /pm/import/execute, /pm/import/batch, the IPC handlers |
thirdPartyPiiConsent in the request — an API caller opts in explicitly. |
Scheduled refresh + /pm/import/sources/:id/refresh |
Inherited from pm_import_sources.third_party_pii_consent_at, recorded when the source was created with thirdPartyPiiConsent: true. A source nobody acknowledged stays blocked. |
A blocked import returns a failed report naming consent as the reason — not a thrown
error. The refusal has to carry the required warnings and entityMap fields, because the
pipeline spreads ...report.warnings when it merges warning sources; omitting them threw
report.warnings is not iterable and replaced the real reason with that raw text.
Batch import and progress screen
Multiple sources can be imported at once. Adapters are processed three at a time, with a 5-second cooldown between batch calls. Two push channels keep the screen live: one carries stage updates per adapter (connecting, fetching, rate-limited, mapping, importing, completed, failed), the other asks the user to resolve a column-mapping ambiguity.
The progress screen renders a row per platform with a progress bar and stage label, and auto-transitions to a completion summary when every job reaches a terminal state (after three seconds, skipped when all failed). Disambiguation pauses only the affected adapter and waits up to five minutes for the answer; the other adapters continue unblocked.
Content-type detection
API-based adapters report what content types exist in the source workspace, each with a type, label, count and whether it is supported. The wizard uses that to show an info banner when unsupported content exists ("3 items can be imported. Some content is not available for import."), dim unsupported entities so they cannot be selected, leave them out of select all/none, and show a content summary on the completion screen. Notion detects live, listing every page and database as an individually selectable supported entity; other adapters declare known unsupported types statically (Jira: Confluence pages, roadmaps, sprints). File-based adapters report no content summary.
Column mapping (reorder, rename, exclude)
The preview step exposes per-column controls beyond type overrides: reorder by drag-and-drop, rename through an inline text input, and exclude through a checkbox. Excluded columns are filtered out before import, and Start Import is disabled when every column is excluded. On a successful import from a connected source the mappings are stored with the source and restored on re-import. The backend applies them in the pipeline's map stage: filter excluded, apply overrides, sort by position, then normalise to sequential positions.
Importing a file (CSV or Excel)
The import screen shows one file tile, "A file" with a "CSV or Excel" chip, instead of separate CSV and Excel tiles. You pick or drop a file and Omniscio works out the rest:
- The file decides the reader, not its name. A real Excel workbook is read as Excel even if it is
called
.csv, and a text file called.xlsxis read as text. Text files are CSV, TSV, TXT or JSON (by extension). An old.xlsworkbook gets a plain message asking you to open it in Excel and save it as.xlsx. - Text is read the way it was written. UTF-8 (with or without a byte-order mark), UTF-16 (with a mark) and Windows-1252 files all come in with their accents intact, and the separator is found for you: comma, semicolon, tab or pipe, with quoted cells respected.
- The file names its project. The project is called after the file (extension dropped; "Imported file" if nothing is left). Each file is remembered by its name and a short fingerprint of its content, so a second, different file always becomes its own project. An updated export with the same name is a different file and also becomes its own project. Importing the very same file again adds nothing, says "Already imported — this is the project from <file>", and opens the project it made before.
- Warnings come before the import. A CSV with more than 5,000 rows is cut at 5,000, and the warning appears before you import, not only afterwards.
- One click when nothing is unclear. If every column is confidently understood and there are no warnings, the import starts straight away. Otherwise a "columns need a quick look" card lists only the unclear columns by name, with the best guess already chosen (an AI guess from the column name and sample values when an AI key is set up; personal data is held back). Looks right, import goes ahead; Review all columns opens the full grid. The answers travel with the import, so it never asks twice. Other platforms keep their mapping step and consent tick-box.
- Landing. When the import makes one project and nothing went wrong, Mission Control opens that project with a banner, "Imported 15 tasks from team-tasks.csv", and an Undo button that deletes the project it just made. The banner closes by itself after 15 seconds.
- Drop a file anywhere. Dropping a CSV, TSV, TXT, JSON or Excel file anywhere on Mission Control shows "Drop to import" and opens the import with that file already chosen. A drop that lands on a drop zone of its own (a board's Files tab, for example), a file of another kind, a drop while import is switched off, and a drop on the import screen itself are left alone.
- Settings has the same door, and its own. Mission Control Settings → Document Import has a One-click import button that opens this import screen on the file tile, and an Import file button that picks a CSV, JSON or Markdown file and asks for the project name (prefilled from the file name) before importing it. The section is hidden while import is switched off. There is no header setting: the first row of a file is always read as the column names.
A file picked or dropped on the screen can be up to 7 MB.
Import wizard UI
The wizard is the Mission Control plugin's own, bridged to the main process. It walks users through: select source type, configure connection or file, preview data with column type overrides and reorder/rename/exclude controls, execute import, and view report. For a third-party platform the preview step also carries the PII consent toggle, which blocks Start Import until the user acknowledges it. It goes through the shipped import entry, so it is available whenever Mission Control is on; the old import setting is no longer read.
Opening it from Mission Control setup. The first-run "where are you coming from?" screen leads straight into this wizard:
- One tool picked (anything but Asana alone) — the wizard opens on that tool's connect screen, skipping the tool list.
- Several tools picked — the wizard opens on the tool list with the picked tools first, in the order they were picked. One import runs at a time.
- "I have a file" — the wizard opens on the file tile, which takes CSV and Excel files alike. A picked file is read by its first bytes, not its name: an Excel workbook goes to the Excel reader, anything else to the CSV reader. Files are capped at 7 MB, because the file travels base64-encoded (about a third larger) and must fit under the 10-million-character IPC limit.
- Asana alone — keeps its own Asana import screen.
- A tool the wizard cannot offer, or a failed lookup, shows the normal tool list instead of a dead end, and with import turned off the page returns home as it always does.
The Asana finish step tells the truth about a partial run. When some items failed it says Import finished with problems, states how many items could not be imported, and lists every per-item reason with a Copy button; when every item failed it says Nothing could be imported. Only a run with no failed items reads Import complete. The per-item reasons come from the job's report, the same list the job's error log holds.
The connect screen for a key-based platform works like this:
- Saved connection. When that platform's key is already saved in Settings, the screen shows "Use your saved <Platform> connection". One click connects and lists boards with nothing typed; the typed-key fields stay underneath. The key itself never reaches the screen — only a yes/no that one is saved.
- Where to get the key. Every key-based platform shows one plain sentence on where its key lives and a link that opens that platform's key page in the browser.
- Clear errors. A failed connect says what went wrong: the key was not accepted, a field is missing, too many requests, the platform took too long, no internet, the site or account was not found, or the platform is having trouble. A saved key that the platform rejects says so and points to Settings.
- Asana. With an Asana token saved, the Asana import skips the token step and goes straight to picking projects.
- Google Sheets without Google connected. The Sheets screen says Google isn't connected yet and offers Open Google settings, which goes to Settings → Google — the one Google connection mail, calendar and sheets share. Nothing on the import screen signs in to Google. Coming back to the window (or clicking Check again) shows the sheet box once Google is connected.
- Big accounts. Monday.com lists every board up to 1,000, not just the first 100. Up to 1,000 boards or projects can be selected in one import; with more than 100 listed, the 100 most recently used start selected and Select all takes the rest. Bigger selections get proportionally more time to preview and import.
Related
The import framework lands everything into Mission Control boards, so mission-control.md is the parent page for what happens to the data once it arrives, and data-transfer.md covers the older CSV import path whose column-type inference engine this pipeline reuses. When the data is coming from a platform someone is migrating off, pm-onboarding-features.md describes the first-run import entry screen and the presets it applies to each imported board. The full list of library pages is in INDEX.md.
Last verified 2026-10-24