VDI Plugin Configuration Schema

Type: object

VDI Configuration Schema

Type: object
No Additional Properties

Type: object


Host Address String

Type: string
Must match regular expression: ^(?=[\da-zA-Z.-]{1,253}(?::|$))([\da-zA-Z][\da-zA-Z-]{0,62}(?:\.[\da-zA-Z][\da-zA-Z-]{0,62})*)(?::([1-9]\d{0,3}|[1-5]\d{4}|6(?:[0-4]\d{3}|5(?:[0-4]\d{2}|5(?:[0-2]\d|3[0-5])))))?$
Examples:

something:8080
foo.com:1234
portless.host.string

Address Config Object

Type: object

Examples:

host: abc.def.com
port: 1234
host: abc.def.com

Hostname String

Type: string
Must match regular expression: ^(?=[\da-zA-Z.-]{1,253}$)([\da-zA-Z][\da-zA-Z-]{0,62}(?:\.[\da-zA-Z][\da-zA-Z-]{0,62})*)$

Must be at least 1 characters long

Must be at most 253 characters long

Connection Port

Type: integer

Value must be greater or equal to 1 and lesser or equal to 65535

Type: string

Must be at least 3 characters long

Type: string

Must be at least 3 characters long

Type: string Default: "vdi"

Must be at least 3 characters long

Type: integer Default: 5

Value must be greater or equal to 1 and lesser or equal to 32

VDI Rest Service Config

Type: object

Max Dataset Upload Size

Type: integer Default: 1073741824

Maximum allowed size, in bytes, for an individual dataset data upload request.

Defaults to 1GiB.

Value must be greater or equal to 1048576

Max User Storage Quota

Type: integer Default: 10737418240

Maximum allowed total storage usage, in bytes, for raw user upload files.

Defaults to 10GiB

Value must be greater or equal to 1073741824

Enable Trace Response Headers

Type: boolean Default: false

Type: object
No Additional Properties

Object Store Event Demuxer Config

Type: object
No Additional Properties

Kafka Producer ID

Type: string Default: "event-router"

Identifier the event router will announce itself with when publishing events to Kafka.

Must be at least 3 characters long

Must be at most 32 characters long

Dataset Pruner Config

Type: object
No Additional Properties

Deleted Dataset Retention Window

Type: string Default: "30d"
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Pruning Interval

Type: string Default: "6h"

Interval at which the dataset pruner will prune deleted datasets that have been marked as deleted for longer than the configured retention window.

Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Wakeup Interval

Type: string Default: "2s"

Interval at which the pruner will wake up and check for a service shutdown signal.

Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Dataset Reconciler Config

Type: object
No Additional Properties

Enable Automatic Reconciliation

Type: boolean Default: true

Whether the reconciler daemon should be active.
For developing or running locally against a shared install-target, this should likely be set to false to avoid accidentally breaking other developers' test data.

Full Reconciliation Interval

Type: string Default: "6h"

Interval at which a full synchronization of all dataset install targets will be performed.

Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Slim Reconciliation Interval

Type: string Default: "5m"

Interval at which a synchronization of the internal cache database will be executed. This is needed to accommodate for MinIO lacking the S3 event-on-replication feature.

Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Uninstall Missing Datasets

Type: boolean Default: true

Whether the reconciler should attempt to uninstall datasets that are absent from the object store.
When testing or running locally against a shared install-target, this should be set to false to avoid breaking other developers' test data.

Kafka Configuration

Type: object
No Additional Properties

Type: array
No Additional Items

Each item of this array must be:


Host Address String

Type: string
Must match regular expression: ^(?=[\da-zA-Z.-]{1,253}(?::|$))([\da-zA-Z][\da-zA-Z-]{0,62}(?:\.[\da-zA-Z][\da-zA-Z-]{0,62})*)(?::([1-9]\d{0,3}|[1-5]\d{4}|6(?:[0-4]\d{3}|5(?:[0-4]\d{2}|5(?:[0-2]\d|3[0-5])))))?$
Examples:

something:8080
foo.com:1234
portless.host.string

Address Config Object

Type: object

Examples:

host: abc.def.com
port: 1234
host: abc.def.com

Hostname String

Type: string
Must match regular expression: ^(?=[\da-zA-Z.-]{1,253}$)([\da-zA-Z][\da-zA-Z-]{0,62}(?:\.[\da-zA-Z][\da-zA-Z-]{0,62})*)$

Must be at least 1 characters long

Must be at most 253 characters long

Connection Port

Type: integer

Value must be greater or equal to 1 and lesser or equal to 65535

Type: object
No Additional Properties

Duration String

Type: string Default: "5s"
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Type: enum (of string) Default: "earliest"

Must be one of:

  • "earliest"
  • "latest"
  • "none"

Duration String

Type: string Default: "9m"
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Type: string Default: "vdi-kafka-consumers"

Type: string Default: null

Duration String

Type: string Default: "1m"
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Type: boolean Default: true

Type: integer Default: 52428800

Value must be greater or equal to 0

Type: integer Default: 1

Value must be greater or equal to 0

Duration String

Type: string Default: "3s"
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Duration String

Type: string Default: "5m"
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Type: integer Default: 500

Value must be greater or equal to 0

Duration String

Type: string Default: "2s"
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Type: integer Default: 65536

Value must be greater or equal to -1

Duration String

Type: string Default: "1s"
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Duration String

Type: string Default: "50ms"
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Duration String

Type: string Default: "30s"
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Duration String

Type: string Default: "100ms"
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Type: integer Default: 131072

Value must be greater or equal to -1

Duration String

Type: string Default: "45s"
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Type: object
No Additional Properties

Type: integer Default: 33554432

Value must be greater or equal to 0

Type: enum (of string) Default: "none"

Must be one of:

  • "none"
  • "gzip"
  • "snappy"
  • "lz4"
  • "zstd"

Type: integer Default: 2147483647

Value must be greater or equal to 0 and lesser or equal to 2147483647

Type: integer Default: 16384

Value must be greater or equal to 0

Duration String

Type: string Default: "9m"
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Duration String

Type: string Default: "2m"
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Duration String

Type: string Default: "5ms"
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Duration String

Type: string Default: "1m"
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Type: integer Default: 1048576

Value must be greater or equal to 0

Type: integer Default: 32768

Value must be greater or equal to -1

Duration String

Type: string Default: "30s"
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Type: integer Default: 131072

Value must be greater or equal to -1

Duration String

Type: string Default: "1s"
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Duration String

Type: string Default: "50ms"
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Duration String

Type: string Default: "100ms"
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Type: object

Hard Delete Lane Config

Type: object

Kafka Consumer ID

Type: string Default: "hard-delete-lane-receive"

Name the lane will identify itself with when fetching events from Kafka.

Must be at least 3 characters long

Must be at most 32 characters long

In-Memory Job Queue Size

Type: integer Default: 10

Number of jobs that should be pulled from the event channel and kept in memory for processing. If there is a service shutdown, the in-memory jobs may be lost and need to be re-queued by the reconciler.

Value must be greater or equal to 1 and lesser or equal to 100

Worker Count

Type: integer Default: 10

Number of worker coroutines that should be actively processing events.

Value must be greater or equal to 1 and lesser or equal to 20

Event Channel Name

Type: string Default: "hard-delete-triggers"

Name of the event channel (topic) the lane will subscribe to.

Must be at least 3 characters long

Must be at most 32 characters long

Event Message Key

Type: string Default: "hard-delete-trigger"

Message key used to filter events.

Must be at least 3 characters long

Must be at most 32 characters long

Data Preprocessing Lane Config

Type: object

Kafka Consumer ID

Type: string Default: "import-lane-receive"

Name the lane will identify itself with when fetching events from Kafka.

Must be at least 3 characters long

Must be at most 32 characters long

In-Memory Job Queue Size

Type: integer Default: 10

Number of jobs that should be pulled from the event channel and kept in memory for processing. If there is a service shutdown, the in-memory jobs may be lost and need to be re-queued by the reconciler.

Value must be greater or equal to 1 and lesser or equal to 100

Worker Count

Type: integer Default: 10

Number of worker coroutines that should be actively processing events.

Value must be greater or equal to 1 and lesser or equal to 20

Event Channel Name

Type: string Default: "import-triggers"

Name of the event channel (topic) the lane will subscribe to.

Must be at least 3 characters long

Must be at most 32 characters long

Event Message Key

Type: string Default: "import-trigger"

Message key used to filter events.

Must be at least 3 characters long

Must be at most 32 characters long

Data Installation Lane Config

Type: object

Kafka Consumer ID

Type: string Default: "install-data-lane-receive"

Name the lane will identify itself with when fetching events from Kafka.

Must be at least 3 characters long

Must be at most 32 characters long

In-Memory Job Queue Size

Type: integer Default: 5

Number of jobs that should be pulled from the event channel and kept in memory for processing. If there is a service shutdown, the in-memory jobs may be lost and need to be re-queued by the reconciler.

Value must be greater or equal to 1 and lesser or equal to 100

Worker Count

Type: integer Default: 5

Number of worker coroutines that should be actively processing events.

Value must be greater or equal to 1 and lesser or equal to 20

Event Channel Name

Type: string Default: "install-triggers"

Name of the event channel (topic) the lane will subscribe to.

Must be at least 3 characters long

Must be at most 32 characters long

Event Message Key

Type: string Default: "install-trigger"

Message key used to filter events.

Must be at least 3 characters long

Must be at most 32 characters long

Dataset Reconciliation Lane Config

Type: object

Kafka Consumer ID

Type: string Default: "reconciliation-lane-receive"

Name the lane will identify itself with when fetching events from Kafka.

Must be at least 3 characters long

Must be at most 32 characters long

Kafka Producer ID

Type: string Default: "reconciliation-lane-send"

Name the lane will identify itself with when publishing events to Kafka.

Must be at least 3 characters long

Must be at most 32 characters long

In-Memory Job Queue Size

Type: integer Default: 10

Number of jobs that should be pulled from the event channel and kept in memory for processing. If there is a service shutdown, the in-memory jobs may be lost and need to be re-queued by the reconciler.

Value must be greater or equal to 1 and lesser or equal to 100

Worker Count

Type: integer Default: 10

Number of worker coroutines that should be actively processing events.

Value must be greater or equal to 1 and lesser or equal to 20

Event Channel Name

Type: string Default: "reconciliation-triggers"

Name of the event channel (topic) the lane will subscribe to.

Must be at least 3 characters long

Must be at most 32 characters long

Event Message Key

Type: string Default: "reconciliation-trigger"

Message key used to filter events.

Must be at least 3 characters long

Must be at most 32 characters long

Dataset Share Lane Config

Type: object

Kafka Consumer ID

Type: string Default: "share-lane-receive"

Name the lane will identify itself with when fetching events from Kafka.

Must be at least 3 characters long

Must be at most 32 characters long

In-Memory Job Queue Size

Type: integer Default: 10

Number of jobs that should be pulled from the event channel and kept in memory for processing. If there is a service shutdown, the in-memory jobs may be lost and need to be re-queued by the reconciler.

Value must be greater or equal to 1 and lesser or equal to 100

Worker Count

Type: integer Default: 10

Number of worker coroutines that should be actively processing events.

Value must be greater or equal to 1 and lesser or equal to 20

Event Channel Name

Type: string Default: "share-triggers"

Name of the event channel (topic) the lane will subscribe to.

Must be at least 3 characters long

Must be at most 32 characters long

Event Message Key

Type: string Default: "share-trigger"

Message key used to filter events.

Must be at least 3 characters long

Must be at most 32 characters long

Soft Delete Lane Config

Type: object

Kafka Consumer ID

Type: string Default: "soft-delete-lane-receive"

Name the lane will identify itself with when fetching events from Kafka.

Must be at least 3 characters long

Must be at most 32 characters long

In-Memory Job Queue Size

Type: integer Default: 5

Number of jobs that should be pulled from the event channel and kept in memory for processing. If there is a service shutdown, the in-memory jobs may be lost and need to be re-queued by the reconciler.

Value must be greater or equal to 1 and lesser or equal to 100

Worker Count

Type: integer Default: 5

Number of worker coroutines that should be actively processing events.

Value must be greater or equal to 1 and lesser or equal to 20

Event Channel Name

Type: string Default: "soft-delete-triggers"

Name of the event channel (topic) the lane will subscribe to.

Must be at least 3 characters long

Must be at most 32 characters long

Event Message Key

Type: string Default: "soft-delete-trigger"

Message key used to filter events.

Must be at least 3 characters long

Must be at most 32 characters long

Dataset Metadata Upsert Lane Config

Type: object

Kafka Consumer ID

Type: string Default: "update-meta-lane-receive"

Name the lane will identify itself with when fetching events from Kafka.

Must be at least 3 characters long

Must be at most 32 characters long

Kafka Producer ID

Type: string Default: "update-meta-lane-send"

Name the lane will identify itself with when publishing events to Kafka.

Must be at least 3 characters long

Must be at most 32 characters long

In-Memory Job Queue Size

Type: integer Default: 5

Number of jobs that should be pulled from the event channel and kept in memory for processing. If there is a service shutdown, the in-memory jobs may be lost and need to be re-queued by the reconciler.

Value must be greater or equal to 1 and lesser or equal to 100

Worker Count

Type: integer Default: 5

Number of worker coroutines that should be actively processing events.

Value must be greater or equal to 1 and lesser or equal to 20

Event Channel Name

Type: string Default: "update-meta-triggers"

Name of the event channel (topic) the lane will subscribe to.

Must be at least 3 characters long

Must be at most 32 characters long

Event Message Key

Type: string Default: "update-meta-trigger"

Message key used to filter events.

Must be at least 3 characters long

Must be at most 32 characters long

LDAP Connection Configuration

Type: object

LDAP Server List


List of servers that container core will use to lookup database details. Servers will be tried in the order they are defined.

Host Strings

Type: array
No Additional Items

Each item of this array must be:

Host Address String

Type: string
Must match regular expression: ^(?=[\da-zA-Z.-]{1,253}(?::|$))([\da-zA-Z][\da-zA-Z-]{0,62}(?:\.[\da-zA-Z][\da-zA-Z-]{0,62})*)(?::([1-9]\d{0,3}|[1-5]\d{4}|6(?:[0-4]\d{3}|5(?:[0-4]\d{2}|5(?:[0-2]\d|3[0-5])))))?$
Examples:

something:8080
foo.com:1234
portless.host.string

Host Details

Type: array
No Additional Items

Each item of this array must be:

Address Config Object

Type: object

Examples:

host: abc.def.com
port: 1234
host: abc.def.com

Hostname String

Type: string
Must match regular expression: ^(?=[\da-zA-Z.-]{1,253}$)([\da-zA-Z][\da-zA-Z-]{0,62}(?:\.[\da-zA-Z][\da-zA-Z-]{0,62})*)$

Must be at least 1 characters long

Must be at most 253 characters long

Connection Port

Type: integer

Value must be greater or equal to 1 and lesser or equal to 65535

Comma Separated Address List

Type: string
Must match regular expression: ^(?:(?=[\da-zA-Z.-]{1,253}(?::|,|$))([\da-zA-Z][\da-zA-Z-]{0,62}(?:\.[\da-zA-Z][\da-zA-Z-]{0,62})*)(?::([1-9]\d{0,3}|[1-5]\d{4}|6(?:[0-4]\d{3}|5(?:[0-4]\d{2}|5(?:[0-2]\d|3[0-5])))))?(?:,(?=.+)|$))+$
Examples:

something:8080,foo.com
foo.com,apple-service:9001

Type: string

Object Store Connection Config

Type: object
No Additional Properties

Type: string


Host Address String

Type: string
Must match regular expression: ^(?=[\da-zA-Z.-]{1,253}(?::|$))([\da-zA-Z][\da-zA-Z-]{0,62}(?:\.[\da-zA-Z][\da-zA-Z-]{0,62})*)(?::([1-9]\d{0,3}|[1-5]\d{4}|6(?:[0-4]\d{3}|5(?:[0-4]\d{2}|5(?:[0-2]\d|3[0-5])))))?$
Examples:

something:8080
foo.com:1234
portless.host.string

Address Config Object

Type: object

Examples:

host: abc.def.com
port: 1234
host: abc.def.com

Hostname String

Type: string
Must match regular expression: ^(?=[\da-zA-Z.-]{1,253}$)([\da-zA-Z][\da-zA-Z-]{0,62}(?:\.[\da-zA-Z][\da-zA-Z-]{0,62})*)$

Must be at least 1 characters long

Must be at most 253 characters long

Connection Port

Type: integer

Value must be greater or equal to 1 and lesser or equal to 65535

Type: boolean Default: true

Type: string

Must be at least 3 characters long

Type: string

Must be at least 8 characters long

Type: object

Each additional property must conform to the following schema

VDI Plugin Config

Type: object
No Additional Properties


Host Address String

Type: string
Must match regular expression: ^(?=[\da-zA-Z.-]{1,253}(?::|$))([\da-zA-Z][\da-zA-Z-]{0,62}(?:\.[\da-zA-Z][\da-zA-Z-]{0,62})*)(?::([1-9]\d{0,3}|[1-5]\d{4}|6(?:[0-4]\d{3}|5(?:[0-4]\d{2}|5(?:[0-2]\d|3[0-5])))))?$
Examples:

something:8080
foo.com:1234
portless.host.string

Address Config Object

Type: object

Examples:

host: abc.def.com
port: 1234
host: abc.def.com

Hostname String

Type: string
Must match regular expression: ^(?=[\da-zA-Z.-]{1,253}$)([\da-zA-Z][\da-zA-Z-]{0,62}(?:\.[\da-zA-Z][\da-zA-Z-]{0,62})*)$

Must be at least 1 characters long

Must be at most 253 characters long

Connection Port

Type: integer

Value must be greater or equal to 1 and lesser or equal to 65535

Data Types

Type: array

Types and versions of dataset data handled by the plugin being defined.

Must contain a minimum of 1 items

All items must be unique

No Additional Items

Each item of this array must be:

Type: boolean Default: false

Whether datasets can be revised from this data type to another.

PATH Extension

Type: string

Allows adding additional paths to the PATH environment variable passed to plugin scripts on execution in the typical colon (:) divided format.
Leading or trailing colon characters are not required.


Example:

/opt/something/bin:/opt/other/bin

Dataset File Install Root

Type: stringFormat: uri-reference Default: "/datasets"

The root mount path INSIDE THE CONTAINER which will contain the site build directory hierarchy into which dataset files will be installed. The build directory itself MUST NOT be part of this path.


Example:

/datasets

Type: array
No Additional Items

Each item of this array must be:

Dataset Installation Target

Type: enum (of string)

Must be one of:

  • "AmoebaDB"
  • "ClinEpiDB"
  • "CryptoDB"
  • "FungiDB"
  • "GiardiaDB"
  • "HostDB"
  • "MicrobiomeDB"
  • "MicrosporidiaDB"
  • "OrthoMCL"
  • "PiroplasmaDB"
  • "PlasmoDB"
  • "ToxoDB"
  • "TrichDB"
  • "TriTrypDB"
  • "VectorBase"
  • "VEuPathDB"

Type: object
No Additional Properties

Data Compatibility Test

Type: object
No Additional Properties

Maximum Script Execution Time

Type: string
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Script Path Override

Type: string

Example:

/opt/vpdb/bin/import.sh

Data Import/Cleanup

Type: object
No Additional Properties

Maximum Script Execution Time

Type: string
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Script Path Override

Type: string

Example:

/opt/vpdb/bin/import.sh

Data Installation

Type: object
No Additional Properties

Maximum Script Execution Time

Type: string
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Script Path Override

Type: string

Example:

/opt/vpdb/bin/import.sh

Dataset Metadata Upsert

Type: object
No Additional Properties

Maximum Script Execution Time

Type: string
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Script Path Override

Type: string

Example:

/opt/vpdb/bin/import.sh

Dataset Uninstallation

Type: object
No Additional Properties

Maximum Script Execution Time

Type: string
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Script Path Override

Type: string

Example:

/opt/vpdb/bin/import.sh

Type: object
No Additional Properties

RabbitMQ Configuration

Type: object
No Additional Properties

Duration String

Type: string Default: "1s"
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Type: object
No Additional Properties

Connection Name

Type: string Default: "vdi"

Used in RabbitMQ error logging in the RabbitMQ instance as well as the RabbitMQ Java library.

RabbitMQ Server Details


Port defaults to 5671 with TLS enabled, or 5672 with TLS disabled.

Host Address String

Type: string
Must match regular expression: ^(?=[\da-zA-Z.-]{1,253}(?::|$))([\da-zA-Z][\da-zA-Z-]{0,62}(?:\.[\da-zA-Z][\da-zA-Z-]{0,62})*)(?::([1-9]\d{0,3}|[1-5]\d{4}|6(?:[0-4]\d{3}|5(?:[0-4]\d{2}|5(?:[0-2]\d|3[0-5])))))?$
Examples:

something:8080
foo.com:1234
portless.host.string

Address Config Object

Type: object

Examples:

host: abc.def.com
port: 1234
host: abc.def.com

Hostname String

Type: string
Must match regular expression: ^(?=[\da-zA-Z.-]{1,253}$)([\da-zA-Z][\da-zA-Z-]{0,62}(?:\.[\da-zA-Z][\da-zA-Z-]{0,62})*)$

Must be at least 1 characters long

Must be at most 253 characters long

Connection Port

Type: integer

Value must be greater or equal to 1 and lesser or equal to 65535


Examples:

rabbit.site.com
rabbit.site.com:5671
host: rabbit.site.com
host: rabbit.site.com
port: '5672'

Type: string

Must be at least 3 characters long

Type: string

Must be at least 3 characters long

Type: boolean Default: true

Duration String

Type: string Default: "10s"
Must match regular expression: ^(?=.{2,})((?:\d+d)? ?(?:(?<!\S)\d+h)? ?(?:(?<!\S)\d+m)? ?(?:(?<!\S)\d+(?:\.\d+s$|s))? ?(?:(?<!\S)\d+(?:\.\d+ms$|ms))? ?(?:(?<!\S)\d+(?:\.\d+us$|us))? ?(?:(?<!\S)\d+ns)?)$
Examples:

1d 12h 32m 30.340s
12s
1h 22us 5ns

Type: object
No Additional Properties

Type: string

Must be at least 3 characters long

Type: enum (of string) Default: "direct"

Must be one of:

  • "direct"
  • "fanout"
  • "topic"

Type: boolean Default: false

Type: boolean Default: true

Type: object

Each additional property must conform to the following schema

Type: string

Type: object
No Additional Properties

Type: string

Must be at least 3 characters long

Type: boolean Default: false

Type: boolean Default: true

Type: boolean Default: false

Type: object

Each additional property must conform to the following schema

Type: string

Type: object
No Additional Properties

Type: string

Must be at least 3 characters long

Type: object

Each additional property must conform to the following schema

Type: string

Site Build Directory Name

Type: string

Name of the parent directory containing the per-project dataset files subdirectories.

This directory name will be used to form paths such as {siteBuild}/{installTarget}/{datasetId}.


Example:

build-65

Dataset Installation Targets

Type: array

Must contain a minimum of 1 items

All items must be unique

No Additional Items

Each item of this array must be:

Dataset Installation Target

Type: object

If the conditions in the "If" tab are respected, then the conditions in the "Then" tab should be respected. Otherwise, the conditions in the "Else" tab should be respected.


Explicitly Enabled

Type: object

The enabled field is explicitly set to true.

Enabled by Default


The enabled field is omitted, using the default value of true.

Type: object

The following properties are required:

  • targetName

Database Connection Config

Type: object

Basic Options

Type: object

Type: string

Must be at least 3 characters long

Type: string

Must be at least 8 characters long

Type: integer Default: 5

Value must be greater or equal to 5 and lesser or equal to 32

Connection Details


Direct Connection Config

Type: object

Database Host Details


Host Address String

Type: string
Must match regular expression: ^(?=[\da-zA-Z.-]{1,253}(?::|$))([\da-zA-Z][\da-zA-Z-]{0,62}(?:\.[\da-zA-Z][\da-zA-Z-]{0,62})*)(?::([1-9]\d{0,3}|[1-5]\d{4}|6(?:[0-4]\d{3}|5(?:[0-4]\d{2}|5(?:[0-2]\d|3[0-5])))))?$
Examples:

something:8080
foo.com:1234
portless.host.string

Address Config Object

Type: object

Examples:

host: abc.def.com
port: 1234
host: abc.def.com

Hostname String

Type: string
Must match regular expression: ^(?=[\da-zA-Z.-]{1,253}$)([\da-zA-Z][\da-zA-Z-]{0,62}(?:\.[\da-zA-Z][\da-zA-Z-]{0,62})*)$

Must be at least 1 characters long

Must be at most 253 characters long

Connection Port

Type: integer

Value must be greater or equal to 1 and lesser or equal to 65535

Type: enum (of string) Default: null

Must be one of:

  • "oracle"
  • "postgres"

Database Connection Config

Type: object

Basic Options

Type: object

Type: string

Must be at least 3 characters long

Type: string

Must be at least 8 characters long

Type: integer Default: 5

Value must be greater or equal to 5 and lesser or equal to 32

Connection Details


Direct Connection Config

Type: object

Database Host Details


Host Address String

Type: string
Must match regular expression: ^(?=[\da-zA-Z.-]{1,253}(?::|$))([\da-zA-Z][\da-zA-Z-]{0,62}(?:\.[\da-zA-Z][\da-zA-Z-]{0,62})*)(?::([1-9]\d{0,3}|[1-5]\d{4}|6(?:[0-4]\d{3}|5(?:[0-4]\d{2}|5(?:[0-2]\d|3[0-5])))))?$
Examples:

something:8080
foo.com:1234
portless.host.string

Address Config Object

Type: object

Examples:

host: abc.def.com
port: 1234
host: abc.def.com

Hostname String

Type: string
Must match regular expression: ^(?=[\da-zA-Z.-]{1,253}$)([\da-zA-Z][\da-zA-Z-]{0,62}(?:\.[\da-zA-Z][\da-zA-Z-]{0,62})*)$

Must be at least 1 characters long

Must be at most 253 characters long

Connection Port

Type: integer

Value must be greater or equal to 1 and lesser or equal to 65535

Type: enum (of string) Default: null

Must be one of:

  • "oracle"
  • "postgres"

Enabled

Type: boolean Default: true

Whether the install target should be enabled for use in the VDI service.

Dataset Installation Target

Type: enum (of string)

Must be one of:

  • "AmoebaDB"
  • "ClinEpiDB"
  • "CryptoDB"
  • "FungiDB"
  • "GiardiaDB"
  • "HostDB"
  • "MicrobiomeDB"
  • "MicrosporidiaDB"
  • "OrthoMCL"
  • "PiroplasmaDB"
  • "PlasmoDB"
  • "ToxoDB"
  • "TrichDB"
  • "TriTrypDB"
  • "VectorBase"
  • "VEuPathDB"

Supported Data Types

Default: ["*"]

Optional list of data types that the install target supports.

Defaults to supporting all data types from any plugins enabled for the install target.

Dataset Control Database/Schema

Type: object

Dataset Data Database/Schema

Type: object

Dataset Characteristics Validation Schema


JSON schema used to validate dataset characteristics in the metadata for datasets being installed into this configured target.

If omitted, datasets for this install target are not permitted to have characteristics fields.

Inline Schema Definition

Type: object or boolean

Core vocabulary meta-schema

Type: object or boolean

Applicator vocabulary meta-schema

Type: object or boolean

Validation vocabulary meta-schema

Type: object or boolean

Meta-data vocabulary meta-schema

Type: object or boolean

Example:

type: object
properties:
  field:
    type: string
required:
- field

Remote Schema File

Type: string
Must match regular expression: ^(https?)?://([0-9a-zA-Z][\w-]+(?:\.[0-9a-zA-Z][\w-]*)*)((?:/[\w%.~-]+)+)(\?.+)?$

Local Schema File

Type: stringFormat: uri-reference

Path to a local schema file. If the path is absolute, and does not exist in the host filesystem, the service jar contents will be checked as well.

Must be at least 6 characters long


Examples:

/etc/vdi/mounted-filesystem-schema.json
/schema/data/jar-resource-schema.json

Shared Value Definitions

Type: object

Object that may be used to hold common $defs that may be referenced elsewhere in the config file using YAML anchors and aliases.

Type: stringFormat: uri