Configuring apps
The following table shows the config variables used to set individual apps inside nebula (via the API):
field | type | example value | description |
---|---|---|---|
starting_ports | list of int and\or dicts | [{"85": "80"}, 443, 5555] | a list of starting ports to bind the work containers, if ints the host port & the container port will be the same value, if dict it will follow the {"host_port":"container_port"} format, note that due to JSON format if using a dict values must be passed as strings (with ""), if using ints values must not be passed as strings, mixing and matching is allowed, for example a worker server with 8 CPU's & containers_per={"cpu": 3} and starting_ports=[{"81": "80"}] will have 24 containers binding to port 81 to port 104, so your worker-node LB can bind to port 80 and redirect traffic among the container on ports 81...104 which are bounded on the host & redirect to port 80 inside the container, an HAProxy config example can be found at example-config, use [] for non |
containers_per | dict | {"server": 3} or {"cpu": 0.5} | the number of containers to run on each worker node, possible keys are {"server": int}, {instance": int} & {cpu": int/float}, server and instance are both the same and will load x number of containers on each node, cpu will load x containers per cpu on each node |
env_vars | dict | {"test1": "test123", "example": "example123"} | a dict of envvars that will be passed to each work containers, use {} for non |
docker_image | string | "registry.vidazoo.com:5000/nginx" | what docker image to run, note that it's currently not possible to set a different starting command then the one set in the container Dockerfile |
running | bool | true | true - app will run, false - stops said app |
networks | list | ["nebula", "bridge"] | the networks containers will be part of, if you wish to use "host" or "none" networks make sure that they are alone in the list (["host"] or ["none"]), a default bridge user network named "nebula" is also added in any case which is not "host" or "none", the "nebula" network can either be added explicitly (["nebula", "example-net"]) or implicitly (["example-net"]) and will also be added if an empty list is set ([]), note that aside from the default "nebula" network (& Docker default networks of "bridge", "none", "host") you will have to create what networks you wish to use manually on the worker prior to using them, the main advantage of the default "nebula" user network over the "bridge" network is that it allows container name based discovery so if you have a server with 2 containers named "con1" and "con2" you can ping from "con1" to "con2" just by using "ping con2" from inside "con1" |
volumes | list | ["/tmp:/tmp/1", "/var/tmp/:/var/tmp/1:ro"] | what volumes to mount inside the containers ,follows docker run -v syntax of host_path:container_path:ro/rw, use [] for non |
privileged | bool | false | true - app gets privileged permissions, false - no privileged permissions |
devices | list | ["/dev/usb/hiddev0:/dev/usb/hiddev0:rwm"] | what devices to grant the containers access ,follows docker run --device of host_path:container_path:ro/rwm, use [] for non |
rolling_restart | bool | false | if the apps should be updated using a rolling restart or not, unless you configure the "containers_per" param to be higher then 1 container of the app on each device will have no affect |
Configuring device_groups
The following table shows the config variables used to set individual device_groups inside nebula (via the API):
field | type | example value | description |
---|---|---|---|
apps | list | ["app1", "app2"] | a list of apps that will run on each device that is part of the device_group |
Configuring Nebula
The following table shows the config variables used to set nebula worker & manager at start time:
config/conf.json variable name | envvar variable name | used in | default value | example value | type | description | required |
---|---|---|---|---|---|---|---|
basic_auth_user | BASIC_AUTH_USER | manager | admin | string | the basic auth user used to secure the manager - unless you set basic_auth_enabled=false you must configure it | no | |
basic_auth_password | BASIC_AUTH_PASSWORD | manager | P@ssw0rd | string | the basic auth password used to secure the api-manger - unless you set basic_auth_enabled=false you must configure it | no | |
basic_auth_enabled | BASIC_AUTH_ENABLED | manager | true | false | bool | defaults to true, if set to false will disable basic auth on nebula-api, setting to false is only recommended to be used if you are using an upstream LB\web server that handles the basic-auth for you instead | yes |
ENV | manager | prod | dev | string | envvar only, defaults to "prod", if set to "dev" will use flask built-in web server on port 5000 to make devs life a tad easier, unless your developing new features to nebula or hunting down a bug in it best to not include it at all | no | |
mongo_url | MONGO_URL | manager | mongodb://mongo_user:mongo_pass@mongo_host:27017/?ssl=true&replicaSet=mongo_replica_set&authSource=mongo_auth_schema | string | mongo URI string | yes | |
schema_name | SCHEMA_NAME | manager | nebula | mongo_schema | string | mongo schema name | yes |
cache_time | CACHE_TIME | manager | 10 | 30 | int | the amount of time (in seconds) the cache will be kept before the manager will check in with the backend DB to grab any changes, the higher this number the less frequent changes will take affect on the workers & the less load there will be on the DB | yes |
cache_max_size | CACHE_MAX_SIZE | manager | 1024 | 2048 | int | the maximum number of device_groups that will be kept in cache, if you have more device_groups then this number the LRU device_group will be evicted from the cache so for best performance best to make sure this number is higher then the maximum amount of device_group | yes |
mongo_max_pool_size | MONGO_MAX_POOL_SIZE | manager | 25 | 100 | int | the size of the connection pool between the manager and the backend MongoDB - a good rule of thumb is to have 3 for each device_group in the cluster but no more then 100 at most | yes |
registry_auth_user | REGISTRY_AUTH_USER | worker | None | user | string | the docker registry basic auth user, if not set or set to None (the default value) Nebula will attempt to use the standard registry config file that's located inside the container "~/.docker/config.json" file path & if that's not configured will carry using no registry auth | no |
registry_auth_password | REGISTRY_AUTH_PASSWORD | worker | None | P@ssw0rd | string | the docker registry basic auth password, if not set or set to None (the default value) Nebula will attempt to use the standard registry config file that's located inside the container "~/.docker/config.json" file path & if that's not configured will carry using no registry auth | no |
registry_host | REGISTRY_HOST | worker | https://index.docker.io/v1/ | https://my_registry.example.com:5000 | string | the docker registry FQDN or IP | yes |
max_restart_wait_in_seconds | MAX_RESTART_WAIT_IN_SECONDS | worker | 0 | 5 | int | maximum time (in seconds) each worker server will wait before reloading the containers in case of restart\update (minimum time hard coded to 0 and each worker server will randomly choose between the that range to spread load on on the managers to avoid a roaring hoard) | yes |
nebula_manager_auth_user | NEBULA_MANAGER_AUTH_USER | worker | admin | string | the basic auth user used to secure the manager - unless you set basic_auth_enabled=false you must configure it | no | |
nebula_manager_auth_password | NEBULA_MANAGER_AUTH_PASSWORD | worker | P@ssw0rd | string | the basic auth password used to secure manager - unless you set basic_auth_enabled=false you must configure it | no | |
nebula_manager_host | NEBULA_MANAGER_HOST | worker | 127.0.0.1 | mynebulaapi.exampledomain.com | string | the FQDN\IP of the Nebula manager | yes |
nebula_manager_port | NEBULA_MANAGER_PORT | worker | 80 | 80 | int | the port of the Nebula manager | yes |
nebula_manager_protocol | NEBULA_MANAGER_PROTOCOL | worker | http | http | string | protocol used to contact the Nebula manager - http or https | yes |
nebula_manager_request_timeout | NEBULA_MANAGER_REQUEST_TIMEOUT | worker | 60 | 60 | int | the time (in seconds) to wait for a response from the Nebula manager before timing out | yes |
nebula_manager_check_in_time | NEBULA_MANAGER_CHECK_IN_TIME | worker | 30 | 30 | int | the time (in seconds) the device will check in with the Nebula manager to see if any changes to the application happens, the lower this is the faster changes to any app or device_group will propagate to the device but the higher the load to the manager will be so more mangers will be needed | yes |
device_group | DEVICE_GROUP | worker | smart_fridge_device_group | string | the device group to attach a device to - each device can only exist in a single device_group but each device_group can have unlimited apps in it | yes |
envvars take priority over variables set in the config/conf.json file in case both are set (the registry user & pass values can also be set by using the standard "~/.docker/config.json" file and not setting them as envvars and the Nebula config file ), it's suggested to always wrap envvars values in Quotation marks but it's only required if the envvar uses special characters (for example "mongodb://mongo_user:mongo_pass@mongo_host:27017/?ssl=true&replicaSet=mongo_replica_set&authSource=mongo_auth_schema"), some variables have defaults that will be used if they are not set as envvars and in the conf.json file.
An example config file is located at "example_conf.json" at the /config/ folder of either the worker and the manager github repo (and inside the containers of them by extension).
The following table shows the path of each config file inside the docker containers:
container | config path inside container | example Dockerfile COPY command overwrite |
---|---|---|
manager | /www/config/conf.json | COPY config/conf.json /www/config/conf.json |
worker | /worker/config/conf.json | COPY config/conf.json /worker/config/conf.json |
As the manager uses gunicorn as the main application web-server inside the container there is a config.py at the container root folder which is pulled from the repo /config/config.py file, consult gunicron guide if you wish to change it's parameters.