Define flat-file feeds
Building a connector, step 7 (Flat-file track, 1 of 2). Previous: Choose a kind.
A flat-file connector describes files that are uploaded to Owlie, not a system Owlie calls. Its one
file, definition.json, lists feeds: which uploaded file, recognized by its name, holds which
entity or relationship, how to parse it, and which column is which. Owlie reads each uploaded file as
the complete, current contents of its feed. A flat-file connector syncs and never provisions: it has
no credentials, reaches no network and writes nothing anywhere.
When flat file is the right kind
Choose flat file when the source delivers its people as files and has no usable API: a scheduled CSV export from an HR system, a user extract from a legacy application, a spreadsheet someone maintains. Everything it provides is read-only to Owlie, so use it for identity sources and for observing accounts and access, not for anything Owlie must change. If the system has an API, even a limited one, an HTTP connector is usually the better choice.
A flat-file connector declares no sync settings, so the account feed's Sync job starts in Identity mode: Sync creates or updates an Owlie identity from every row. That is right for an HR export. For a file of an application's accounts, the integration's admin opens that Sync job and, on its Behavior tab, sets Sync mode to Resource, so rows become accounts of existing identities.
Create the connector
The kind is chosen when the connector is created and cannot change later.
In the Connector Builder
- Go to Admin > Integrations > Connectors, open the Custom tab and select New connector.
- In the New Connector dialog, keep Start from on Blank and choose Flat file as the Kind.
- Enter a Name and a Key (lowercase letters, digits,
-and_, starting with a letter or digit), and optionally a Description. Select Create connector.
The flat-file editor opens with a starter definition: one feed, users, that takes files named
users*.csv, comma-separated with a header row, with an employee_id identifier column and an
email column; its user entity is the account. The editor's tabs are Basics, Feeds,
Install values, Derived manifest, Raw JSON and Files. There is no Handlers tab:
the only operations are the two connection checks, which the engine supplies. The builder saves as
you type.
Infer feeds from a sample file
The fastest start is a real export. On the Feeds tab, select From a sample file, then Choose sample files. You can pick several files at once; each becomes one entity feed. Owlie reads the start of each file in your browser: at most its first 256 KiB and at most 1,000 rows. The sample is never uploaded or saved: only the column names, types and positions, the parse options and the file name pattern reach the definition.
For each file, Owlie infers:
| What | How |
|---|---|
| Delimiter | Comma, semicolon or tab: whichever splits the first 20 lines into the most consistent number of fields. |
| Header row | On only when row 1 is clearly names: its cells are distinct, non-blank and not numbers, and at least one column below it holds numbers or booleans. When Owlie cannot tell, the switch starts off and asks you to check. |
| Column names | The header text exactly, or column_1, column_2... without a header. |
| Types | boolean when every value is true or false; number when every value is a decimal number with no leading zero and at most 15 digits, so ids such as 00123 stay strings; string otherwise. Blank cells are ignored. |
| Feed name and file pattern | From the file name, with a trailing date or number replaced by *: users_2026-09-24.csv becomes the feed users and the pattern users*.csv. |
| Identifier | Suggested, never chosen for you: a column whose values are all present and unique, preferring id, then a name ending in an id word such as employee_id, then email, then username. |
Check Delimiter and Header row, choose the column under Identifier, adjust the File name
pattern if needed, and select Add feed (or Add 2 feeds and so on). Review each inferred type
afterwards. Choose as the identifier the value the source never changes or reuses, such as an employee
number, never an email address or a name. When a sample has no id-like column, the suggestion can be
email or username: choose another column.
To update one feed's columns from a newer export, open the feed and select Infer columns from a sample. The dialog lists what would change before you apply it, and warns if the change would touch an identifier column.
Through the API
Create the connector with createCustomConnector, passing kind: "flat-file" with the key and
name. Its draft starts from the same starter definition. Write your definition.json into the draft
with updateCustomConnectorDraft; Owlie derives the declaration from it.
Build with the API and agents lists the operations and their inputs. Inference
from a sample happens only in the builder, so through the API you write the columns yourself.
The definition
{
"engine": 1,
"description": "Employees from the Acme HR nightly CSV export.",
"accountEntityKey": "employee",
"config": { "properties": {} },
"feeds": {
"employees": {
"kind": "entity",
"entityKey": "employee",
"fileMatch": "employees*.csv",
"parse": { "delimiter": ",", "header": true, "encoding": "utf-8" },
"columns": {
"employee_id": { "position": 0, "type": "string", "identifier": true },
"email": { "position": 1, "type": "string" },
"employment_status": { "position": 2, "type": "string" }
},
"isActiveExpr": "employment_status == 'active'"
}
}
}
| Field | What it is |
|---|---|
engine | Always 1, the version of the flat-file definition format. |
description, icon | What the catalog shows. |
accountEntityKey | The entity whose records are accounts. |
config.properties | The fields an install fills in, as JSON Schema: in practice the seeds of install-editable file patterns and columns. |
feeds | Required. The feeds, keyed by feed name. |
A complete example with a relationship feed is in CSV feed, and every field is in the flat-file definition reference.
The account entity
accountEntityKey names an entity, that is, the entityKey of one of the entity feeds, not the
name of the feed. That entity's records are the accounts Owlie links to identities. Only its feed may
carry lifecycle expressions. Leave accountEntityKey out and the connector has no account entity,
only other entities.
On the Basics tab, choose it under Account entity.
Entity feeds
An entity feed ("kind": "entity") holds the records of one entity, one row per record.
| Field | What it is |
|---|---|
entityKey | The entity the rows are records of, such as employee or department. One feed per entity. |
fileMatch | The file name pattern that routes an uploaded file to this feed. |
parse | How the file is read. |
columns | The columns, keyed by column name. |
isActiveExpr, isDeletedExpr | The account's lifecycle, on the account entity's feed only. |
File name patterns
fileMatch is a pattern for the uploaded file's name: * matches any run of characters, ? matches
one character, and case is ignored. employees*.csv takes employees.csv and
Employees-2026-09-27.CSV. A file is routed to the first feed, in the order the feeds appear in the
definition, whose pattern matches, unless the upload names its feed. Put specific patterns before
general ones. Two feeds cannot have the same pattern, and a feed name cannot contain /. That check
compares patterns exactly, while routing ignores case, so Users*.csv and users*.csv are both
accepted and the first one takes every file.
Parsing
parse has three fields, all required:
| Field | Values |
|---|---|
delimiter | ",", ";" or "\t" (tab) |
header | true when the first row names the columns |
encoding | "utf-8", the only encoding |
Quoted cells may contain the delimiter, doubled quotes ("") and line breaks. Blank lines are
skipped.
Columns
Each column has a position (0 for the first column; two columns cannot share one), a type
(string, number or boolean) and optionally identifier: true. An entity feed needs at least one
identifier column, and may have several. The identifier with the lowest position is the record's
external id: Owlie keys the record by its value, and relationships name the record by it. Later
identifiers are for lookups and matching.
Owlie takes the external id from the first identifier column that has a value in the row. A row whose first identifier cell is blank is therefore keyed by the next identifier, such as an email address, and becomes a different record from the one the same person had before. Never leave the first identifier blank; if your export can, declare a single identifier column. A row with every identifier blank is a failed record. When two rows share an external id, the later row wins.
How a row becomes a record depends on the header:
- With a header row, each cell goes to the column whose name is exactly its header text. Name every column exactly as the export's header spells it. A header column you did not declare is kept on the record as text. A declared column the file lacks is absent from the record.
- Without a header row, cells are read in the order of the columns' positions: the first cell to the column with the lowest position, and so on. Number the positions from 0 without gaps.
Each cell is converted by its column's type:
| Type | A cell becomes |
|---|---|
string | The text as it is. |
number | The number, for a cell that holds one. A cell that does not is kept as text. |
boolean | true for true or 1, false for false or 0, in any case, ignoring spaces. Any other value makes the row unreadable. |
A blank cell in a number or boolean column becomes an empty value (null), never 0 or false.
An unreadable row is left out of the run and counted as a failed record: see
what a failed row means.
Lifecycle
isActiveExpr and isDeletedExpr are OXL expressions over the account record, with its column names
as variables, such as employment_status == 'active'. For each account record, Owlie evaluates
isDeletedExpr first: true makes the account deleted. Otherwise isActiveExpr decides: true is
active, false is disabled. An expression that gives neither true nor false sets no state. Give
the account feed an isActiveExpr, or the file tells Owlie nothing about account state.
On the Feeds tab, these are Active when and Deleted when, shown for the account entity's feed.
Relationship feeds
A relationship feed ("kind": "relationship") holds links between records of two entities, one row
per link, such as which employee belongs to which department.
{
"department_members": {
"kind": "relationship",
"fileMatch": "department_members*.csv",
"parse": { "delimiter": ",", "header": true, "encoding": "utf-8" },
"edge": {
"edgeKey": "member_of_department",
"sourceEntityKey": "employee",
"targetEntityKey": "department",
"sourceColumn": "employee_id",
"targetColumn": "department_id"
}
}
}
| Field | What it is |
|---|---|
edge.edgeKey | The relationship's name, unique among the relationship feeds. |
edge.sourceEntityKey, edge.targetEntityKey | The two entities it links. Each must be an entity one of the entity feeds provides. |
edge.sourceColumn, edge.targetColumn | The header names of the columns holding each end's value. |
A relationship file must have a header row ("header": true); Owlie reads its columns by name, and
refuses a relationship feed without one. Each cell under sourceColumn and targetColumn holds the
external id of a record, which is its lowest-position identifier value. An id that no record has yet
does not fail the row: Owlie creates a placeholder record for it, which the entity file fills in when
its record arrives. Files can therefore be uploaded in any order, and a mistyped id leaves a
placeholder that no entity file fills. Name sourceColumn after
the source entity feed's identifier column and targetColumn after the target's: when an install's
sync jobs are set up, Owlie refuses a relationship feed whose sourceColumn is not a column of the
source entity's feed, or whose targetColumn is not a column of the target's. A row with a blank end,
or a file whose header lacks either column, fails the run.
A relationship feed has no entityKey, no install-editable columns and no lifecycle expressions.
Let installs edit the file pattern and columns
Exports differ between customers: a renamed file, an extra column, columns in another order. Two
values can be install-editable, so one connector serves many exports: a feed's fileMatch, and an
entity feed's columns. Replace the value with { "$oxl": "config.<name>" } and declare that config
field, with the value as its default, which is the seed each install starts from:
{
"config": {
"properties": {
"employeeFileMatch": {
"type": "string",
"title": "Employees file name",
"description": "An uploaded file whose name matches this pattern is the employees snapshot.",
"default": "employees*.csv"
},
"employeeColumns": {
"type": "object",
"title": "Employee columns",
"description": "The columns of your employees file, by header name, with their position.",
"default": {
"employee_id": { "position": 0, "type": "string", "identifier": true },
"email": { "position": 1, "type": "string" }
}
}
}
}
}
The feed then reads "fileMatch": { "$oxl": "config.employeeFileMatch" } and
"columns": { "$oxl": "config.employeeColumns" }. A file pattern needs a string config field and a
column list an object one. Everything else, including feed kinds, entity keys, edges, parse options
and accountEntityKey, stays fixed by the definition.
Identifier columns stay exactly as seeded. An install can rename, add, remove, retype and move ordinary columns. It cannot change an identifier column's name, type or position, remove it, or mark another column as an identifier: Owlie refuses the configuration. Moving an identifier column would read another column's values as identities.
Each install gets its own copy of every seed when it is installed. When a later version of your connector changes a seed, the install keeps its copy, and the integration's page lists the change for an admin to apply or ignore.
In the Connector Builder, the Install values tab's What installs may edit has a switch per feed for its file name and, for an entity feed, its columns. Turning one on creates the config property with the current value as its default.
What Owlie derives
Owlie derives a small declaration from the definition. It provides Sync only: no provisioning, no
authentication (installs enter no credentials), and no Grant kinds. Each entity feed becomes one
entity, keyed by its entityKey, with a record schema built from its columns. Its identifiers are the
identifier columns in position order, and it never supports incremental Sync, because every file is a
complete snapshot. Relationship feeds do not appear in the declaration; Owlie reads them when their
files are ingested. The install form shows the declared config.properties, each required unless its
default is "" or null.
The connector's only operations are the two connection checks every connector has: validating an install's configuration, which checks that its file patterns and columns derive, and the connection test, which always succeeds because the drop is Owlie's.