`civex.match_files_to_records`

Match each file to an existing child record by extracting a key value from the filename. If a matching record is found, its file field is updated; if not, a new record is created with the key and file field set. Reach for this when files arrive that may correspond to records created by an earlier step (e.g. civex.rows_to_records from a selection table), rather than always creating new records.

Plugin ID: civex.match_files_to_records
Category: outputs
Capabilities: create_record, update_record, find_records

Config

Field	Type	Required	Default	Description
`schema`	string	yes		Name of the child schema to match/create records under. Workflow YAML sets this via the `schema:` key (the Python field is `schema_name`).
`key_field`	string	yes		Field on child records to match the extracted key against, e.g. 'selection_number'.
`file_field`	string	yes		Field on child records to set with the matched file reference, e.g. 'contour_file'.
`pattern`	string	yes		Regex with one capture group that extracts the key value from each filename.
`dataset`	string	no	""	Dataset to search and create records in. Empty string falls back to the trigger record's own dataset.
`parent_record_id`	string	no	""	Parent record ID to scope matching and creation to. Empty string falls back to the trigger record's ID.

Inputs

Name	Type	Required	Description
`files`	files	yes	FileRef dicts to match.

Outputs

Name	Type	Required	Description
`created`	number	yes	Records created because no match was found.
`updated`	number	yes	Existing records given a file.
`unmatched`	list	yes	Filenames the pattern missed, or whose record could not be created.
`ambiguous`	list	yes	Files left alone because another file in the same run has the same key (one record holds one file), each with the files it clashes with.

Make the pattern read the whole number

A pattern like sel_([0-9]{2}) reads exactly two digits, so selections 14, 142 and 149 all give the key 14. Use sel_([0-9]+) (any number of digits). When two or more files in one run give the same key, none of them is attached, because a record holds one file and there is no telling which is meant. They are listed in the ambiguous output, each with the files it clashes with, and the run says how many there were.

Note

Numeric captures are normalised (e.g. "042" → "42") before matching so that integer fields match correctly.

Match contour files like sel_042_contour.csv to Selection records with selection_number = 42:

- id: match_contours
  plugin: civex.match_files_to_records
  config:
    schema: Selection
    key_field: selection_number
    file_field: contour_file
    pattern: 'sel_(\d+)'
  inputs:
    files: __input__.files