Configuration

In this section, we'll take a closer look at how you can quickly and easily set up a configuration file to start implementing any of your tasks. You can use absolutely any format of configuration file:

• JSON
• YAML
• TOML
• HCL (Terraform)

The main condition is that your config file must contain the sections necessary for csv2api to work.

A configured config.yml file usually looks like this:

save_filtered_pk_to_csv: truecolumns_order:
- id
- order_id
- is_paidcsv_separator: ','api:
base_url: my-api-server.combase_url_schema: httpsauth_method: Bearertoken: '{{ CONFIG_API_TOKEN }}'request_timeout: 0update_endpoint:
endpoint_name: /api/v1/order/%scontent_type: application/jsonadd_pk_to_endpoint_name: truehttp_method: PATCHendpoint_body:
data:
order: idattributes:
tags: tagsfilter_columns:
- column_name: order_idcondition: NEQvalue: ''update_fields:
- field_name: tagsvalues:
- paidconditions:
- column_name: is_paidcondition: EQvalue: '1'

Let's look at each section in more detail.

save_filtered_pk_to_csv

This section accepts a boolean (true or false) value. Allows you to specify whether to save the filtered primary keys (PK) to a CSV file.

This is useful when you need to keep track of which PK (IDs) have been filtered, to control how they change in the database or elsewhere.

columns_order

This section accepts a list of strings. Allows you to specify the order of the fields for filtering and updating.

❗️ Please note: the first element in this list is the primary key (PK), which will be used in other configuration sections.

csv_separator

This section accepts a string. Allows you to specify a separator between columns in a CSV data file.

❗️ Please note: the setting must match the parameters of the Go standard library: https://pkg.go.dev/encoding/csv#Writer

api

This section accepts many parameters and objects. Allows you to specify settings for your REST API instance.

base_url

This section accepts a string. Allows you to specify a base URL address (host) for your REST API.

base_url_schema

This section accepts the one of the values: http or https. Allows you to specify an HTTP schema for the base URL address (host) for your REST API.

auth_method

This section accepts a string. Allows you to specify an auth method for making requests to your REST API.

For example, Bearer.

token

This section accepts a string. Allows you to specify an auth token for making requests to your REST API.

WARNING: do not forget to add your config file to a .gitignore, if you place API token to this section!

It is recommended to keep such data under protection, for example by using environment variables. For example, you can specify a token like this when you start the parser:

MYVAR_API_TOKEN=secret-1234567 csv2api -c /path/to/config.yml -d /path/to/data.csv -e MYVAR

During initialization, the parser will insert your token from env into this configuration section by itself.

❗️ Please note: to be built into the configuration correctly, you must specify environment variables in object style, but use underscores (_) instead of dots (.).

request_timeout

This section accepts an int. Allows you to specify a request timeout for HTTP client.

update_endpoint

This section accepts object. Allows you to specify endpoint to update actions from your REST API.

endpoint_name

This section accepts a string. Allows you to specify an endpoint name (path).

For example, if you need to set a primary key (PK) to endpoint path, this section support Go markers (like you use in fmt.Sprintf function):

endpoint_name: /api/v1/order/%s # <-- '%s' is a stub where PK will be inserted

content_type

This section accepts a string. Allows you to specify an endpoint's Content-Type.

add_pk_to_endpoint_name

This section accepts a boolean (true or false) value. Allows you to set a primary key (PK) to the endpoint name (path).

❗️ Please note: if you set this setting to true, then you must necessarily specify a stub in the endpoint_name option.

http_method

This section accepts a string. Allows you to specify an endpoint's HTTP method.

endpoint_body

This section accepts an object. Allows you to describe in a simple format the structure of the body of a request.

For example, if your API endpoint has a body structure like this:

{
"data": {
"order": "1",
"attributes": {
"tags": [
"new",
"paid"
]
}
}
}

Then the configuration for this subsection will be like this:

api:
# ...update_endpoint:
# ...endpoint_body:
data:
order: idattributes:
tags: tags

❗️ Please note: you must use exactly the field/column names, that you specified in the columns_order and update_fields sections.

filter_columns

This section accepts a list of objects. Allows you to describe in a simple format the structure of the filter for input data.

❗️ Please note: there is a logical AND between all filters.

column_name

This section accepts a string. Allows you to specify a column name for filtering.

condition

This section accepts the one of the values: EQ, NEQ, GT, LT, GTE, LTE. Allows you to specify a condition for filtering.

For string or int values:

• EQ – equals to the given value (==)
• NEQ – not equals to the given value (!=)

Only for int values:

• GT – greater than the given value (>)
• LT – less than the given value (<)
• GTE – greater than or equal to the given value (>=)
• LTE – less than or equal to the given value (<=)

value

This section accepts a string. Allows you to specify a column value for filtering.

❗️ Please note: if the value should be empty, specify it as ''.

update_fields

This section accepts a list of objects. Allows you to specify fields to be updated in your REST API.

field_name

This section accepts a string. Allows you to specify a field name for updating in your REST API.

values

This section accepts a list of strings. Allows you to specify field value(s) for updating in your REST API.

conditions

This section accepts a list of objects. Allows you to specify conditions for the field.

❗️ Please note: there is a logical AND between all conditions.

column_name

This section accepts a string. Allows you to specify a column name for filtering.

condition

This section accepts the one of the values: EQ, NEQ, GT, LT, GTE, LTE. Allows you to specify a condition for filtering.

For string or int values:

• EQ – equals to the given value (==)
• NEQ – not equals to the given value (!=)

Only for int values:

• GT – greater than the given value (>)
• LT – less than the given value (<)
• GTE – greater than or equal to the given value (>=)
• LTE – less than or equal to the given value (<=)

value

This section accepts a string. Allows you to specify a column value for filtering.

❗️ Please note: if the value should be empty, specify it as ''.