opengate_data.file_connector.file_connector

FileConnectorBuilder Objects

class FileConnectorBuilder()

File Connector Builder

with_organization_name

def with_organization_name(organization_name: str) -> "FileConnectorBuilder"

Set organization name

Arguments:

  • organization_name str - The name of the organization the files belong to.

Returns:

  • FileConnectorBuilder - Returns itself to allow for method chaining.

Example:

builder.with_organization_name("organization_name")

with_overwrite_files

def with_overwrite_files(overwrite_files: bool) -> "FileConnectorBuilder"

Set the overwrite file.

Arguments:

  • overwrite_files bool - Whether an upload replaces a file that already exists.

Returns:

  • FileConnectorBuilder - Returns itself to allow for method chaining.

add_local_file

def add_local_file(local_file: str) -> "FileConnectorBuilder"

Add a single local file to be uploaded.

You can call this method multiple times or combine it with add_local_multiple_files().

Arguments:

  • local_file str - Local file path to be uploaded.

Returns:

  • FileConnectorBuilder - Returns itself to allow for method chaining.

Example:

builder.add_local_file("data/file1.csv")

add_local_multiple_files

def add_local_multiple_files(local_files: list[str]) -> "FileConnectorBuilder"

Add multiple local files to be uploaded.

The provided list can include one or more filesystem paths. They will be appended to the same internal collection used by add_local_file, so you can freely combine both methods:

builder.add_local_file(“foo.zip”).add_local_multiple_files([“bar.zip”, “baz.tar”])

Arguments:

  • local_files list[str] - List of file paths to upload.

Returns:

  • FileConnectorBuilder - Returns itself to allow for method chaining.

with_format

def with_format(format_data: str) -> "FileConnectorBuilder"

Formats the flat entities data based on the specified format (‘dict’, or ‘pandas’). By default, the data is returned as a dictionary.

Arguments:

  • format_data str - The format to use for the data.

Example:

builder.with_format(‘dict’) builder.with_format(‘pandas’)

Returns:

  • FileConnectorBuilder - Returns itself to allow for method chaining.

with_destiny_path

def with_destiny_path(path: str) -> "FileConnectorBuilder"

Set destiny path

Arguments:

  • path str - The remote directory the operation works on. A trailing slash is added if missing.

Returns:

  • FileConnectorBuilder - Returns itself to allow for method chaining.

Example:

builder.with_destiny_path("path")

with_find_name

def with_find_name(find_file_name: str) -> "FileConnectorBuilder"

Set find by name

Arguments:

  • find_file_name str - The name of the file to look for.

Returns:

  • FileConnectorBuilder - Returns itself to allow for method chaining.

Example:

builder.with_find_name("file.csv")

with_output_path

def with_output_path(output_path: str) -> "FileConnectorBuilder"

Set output path so that when downloading the file it is saved in that path

Arguments:

  • output_path str - The local directory a download is saved into. A trailing slash is added if missing.

Returns:

  • FileConnectorBuilder - Returns itself to allow for method chaining.

Example:

builder.with_output_path("output_path")

add_remote_file

def add_remote_file(file: str) -> "FileConnectorBuilder"

Adds one remote file to the operation.

Call it as many times as needed, or combine it with add_remote_multiple_files().

Arguments:

  • file str - The name of the remote file.

Returns:

  • FileConnectorBuilder - Returns itself to allow for method chaining.

Example:

builder.add_remote_file("file.csv")

add_remote_multiple_files

def add_remote_multiple_files(files: list) -> "FileConnectorBuilder"

Adds several remote files to the operation at once.

The list is appended to the same collection add_remote_file() fills, so both can be combined.

Arguments:

  • files list - The names of the remote files.

Returns:

  • FileConnectorBuilder - Returns itself to allow for method chaining.

Example:

builder.add_remote_multiple_files(["file1.csv", "file2.csv"])

from_dataframe

def from_dataframe(df: "pd.DataFrame",
                   *,
                   defaults: dict | None = None) -> "FileConnectorBuilder"

Assign a DataFrame that describes operations per row. Expected columns (depending on method):

  • upload: local_file [req], destiny_path [opt], ​​overwrite [opt]
  • If there is no destiny_path, use defaults[‘destiny_path’] or defaults[‘path’] or with_destiny_path(…) or “/”
  • default overwrite = False
  • download: path [req], filename [req], output_path [opt]
  • path can come from column, defaults[‘path’], with_destiny_path(…)
  • delete: path [req], filename [opt] (empty or absent -> delete the entire path)

Supported defaults: {“destiny_path”: str, “path”: str, “overwrite”: bool, “output_path”: str}

NOTE: if you mix from_dataframe with setters in download (path/filenames/output_path), then you must use all THREE setters; if not, use only from_dataframe.

Arguments:

  • df pandas.DataFrame - The DataFrame holding the files to upload.
  • defaults dict | None - Values to fall back on when a row leaves a column out: “destiny_path”, “path”, “overwrite” and “output_path”.

upload

def upload() -> "FileConnectorBuilder"

Configures the builder to upload a file to the specified organization.

This method sets the internal state of the builder to prepare for a file upload operation. It does not execute the operation immediately but prepares the necessary configurations for when the execute method is called.

Returns:

  • FileConnectorBuilder - Returns itself to allow for method chaining.

Example:

builder.upload()

list_all

def list_all() -> "FileConnectorBuilder"

Configures the builder to list files available in the specified organization.

This method sets the internal state of the builder to prepare for a file listing operation. It does not execute the operation immediately but prepares the necessary configurations for when the execute method is called.

Returns:

  • FileConnectorBuilder - Returns itself to allow for method chaining.

Example:

builder.list_all()

list_one

def list_one() -> "FileConnectorBuilder"

Configures the builder to list a single file from the specified organization.

This method sets the internal state of the builder to prepare for a single file listing operation. It does not execute the operation immediately but prepares the necessary configurations for when the execute method is called.

Returns:

  • FileConnectorBuilder - Returns itself to allow for method chaining.

Example:

builder.list_one()

download

def download() -> "FileConnectorBuilder"

Configures the builder to download a file from the specified organization.

This method sets the internal state of the builder to prepare for a file download operation. It does not execute the operation immediately but prepares the necessary configurations for when the execute method is called.

Returns:

  • FileConnectorBuilder - Returns itself to allow for method chaining.

Example:

builder.download()

delete

def delete() -> "FileConnectorBuilder"

Selects the operation that deletes files from the file connector.

Requires with_destiny_path(). Name the files to remove with add_remote_file() or add_remote_multiple_files(); with none named, the whole path is deleted.

Returns:

  • FileConnectorBuilder - Returns itself to allow for method chaining.

Example:

builder.with_organization_name("organization").with_destiny_path(
"/to/delete"
).add_remote_file("file.csv").delete().build().execute()

build

def build() -> "FileConnectorBuilder"

Closes the chain and checks it, without sending the request.

Call it last, immediately before execute(). Use build_execute() instead to do both in one step; mixing the two raises.

Returns:

  • FileConnectorBuilder - Returns itself to allow for method chaining.

Raises:

  • RuntimeError - If called more than once.
  • Exception - If combined with build_execute().
  • ValueError - If the chain is missing a method the selected operation requires, or carries one it forbids.

Example:

builder.upload().build()

build_execute

def build_execute()

Closes the chain, checks it and sends the request in one step.

Returns:

requests.Response | dict | pandas.DataFrame: The same result as execute().

Raises:

  • RuntimeError - If combined with build() or with execute().
  • ValueError - If the chain is missing a method the selected operation requires, or carries one it forbids.

Example:

builder.with_organization_name("organization").with_destiny_path(
"/path"
).add_local_file("file.csv").upload().build_execute()

execute

def execute() -> Response

Sends the selected operation to the file connector.

Which request goes out depends on the operation chosen in the chain: upload(), list_all(), list_one(), download() or delete(). The two listing operations default to the “dict” format when with_format() was not called.

Returns:

requests.Response | dict | str | pandas.DataFrame: The listing in the requested format, the contents of a download, or a dict carrying status_code and either data or error.

Raises:

  • RuntimeError - If build() was not the last call before execute(), or if neither build() nor build_execute() was called.
  • ValueError - If no operation was selected in the chain.

Example:

builder.with_organization_name("organization").with_destiny_path(
"/path"
).list_all().build().execute()

opengate_data.file_connector