filter_manifest_files#

filter_manifest_files(manifest: DataFrame, extensions: list[str], include_filters: list[str], exclude_filters: list[str]) → dict[source]#

Filter manifest file keys by incomplete extensions and given filters.

Parameters:

  • manifest – Manifest of file keys, extensions, and locations.

  • extensions – List of single simulation output extensions.

  • include_filters – List of Unix filename pattern matching filter for included file keys.

  • exclude_filters – List of Unix filename pattern matching filter for excluded file keys.

Returns:

Map of filtered manifest file keys to extensions and locations.

filter_incomplete_extensions(manifest: DataFrame, extensions: list[str]) → DataFrame[source]#

Filter manifest for files with incomplete set of extensions.

Parameters:

  • manifest – Manifest of file keys, extensions, and locations.

  • extensions – List of single simulation output extensions.

Returns:

Filtered manifest of file keys, extensions, and locations.

convert_to_dictionary(manifest: DataFrame) → dict[source]#

Convert manifest dataframe to map of file key to extensions and locations.

Parameters:

manifest – Manifest of file keys, extensions, and locations.

Returns:

Map of file key to extensions and locations.

filter_file_keys(files: list[str], include_filters: list[str], exclude_filters: list[str]) → set[source]#

Filter keys using include and exclude Unix filename pattern matching filters.

Parameters:

  • files – List of file keys.

  • include_filters – List of Unix filename pattern matching filter for included file keys.

  • exclude_filters – List of Unix filename pattern matching filter for excluded file keys.

Returns:

Filtered set of file keys.