Before installing EAC (ETL Administration Console), make sure your environment meets the following minimum requirements:

Configuration Web Server

For your production server, we recommend that you do not expose the server and PHP versions.

L’article Prerequisites est apparu en premier sur ETL administration console.

• The Settings page writes changes to .env.local (never committed).
• .env holds defaults that are committed.
• Real OS env vars (exported in the service/host) override both.
• Never commit secrets. Keep secrets in real env vars or .env.local only.
• In prod, changes to env may require php bin/console cache:clear (or PHP-FPM reload) to take effect.

Precedence (highest → lowest)

1. Real environment variables (e.g., export MAILER_DSN=…)
2. .env.local (your Settings form writes here)
3. .env.$APP_ENV
4. .env

What you can change from Settings (writes to .env.local)

Below are the main variables exposed in the UI. When you save, EAC writes/updates the corresponding key in .env.local. The “.env (Present value)” column in the UI is read-only context.

Scheduling & System

• SCHEDULE_ENABLED
• SCHEDULE_RESCHEDULE_OLD_TASK_AFTER_X_MIN
• EAC_CHECK_STATUS_WITH_GIT
• EAC_CRON_USER
• EXEC_TASK_AVG_COUNT
• EAC_DISPLAY_ERROR_ON_HOME
• PHP_PATH_COMMAND
• APP_DEFAULT_URI
• APP_TASK_LOG_TYPE (file or db)
• APP_JAVA_HOME
• Paths: APP_CONSOLE, APP_LDIR, APP_DEPLOY_DIR, APP_EXEC_EXTENSION, APP_SCANNED_PARTITIONS, APP_TMP_DIR, APP_UPLOAD_DIR

Mail

• APP_MAILER_DEFAULT_SENDER
• APP_MAILER_DEFAULT_RECIPIENTS
• APP_ENABLE_TASK_MAILER
• Transport: MAILER_PROTOCOL, MAILER_USER, MAILER_PASSWORD, MAILER_HOST, MAILER_PORT, MAILER_OPTIONS
  • Note: UI masks passwords; saved value goes to .env.local.

Database (core app)

• DB_DRIVER, DB_HOST, DB_PORT, DB_NAME, DB_USER, DB_PASSWORD, DB_OPTIONS
  • EAC computes DATABASE_URL from the above.

ETL log tables & ETL logging DB

• Flags & table names: APP_ENABLE_ETL_LOGS, ETL_STAT_TABLE_NAME, ETL_MET_TABLE_NAME, ETL_LOG_TABLE_NAME
• Connection (if different from core DB):
  DB_DRIVER_URL_ETL_LOG, DB_HOST_URL_ETL_LOG, DB_PORT_URL_ETL_LOG,
  DB_NAME_URL_ETL_LOG, DB_USER_URL_ETL_LOG, DB_PASSWORD_URL_ETL_LOG, DB_OPTIONS_URL_ETL_LOG
  • EAC computes DATABASE_URL_ETL_LOG from the above.

Security

• JWT_PASSPHRASE (passphrase only; keys themselves are files, see below)

What stays in files (not editable in Settings)

These are structural or build-time values. Set them in real env vars or in .env / .env.local:

• APP_ENV (e.g., prod, dev)
• APP_SECRET (never commit a real secret in .env; use .env.local or OS env)
• EAC_SERVER_CODE (instance identifier)
• APP_SITE_ENV (UI banner/env display)
• COMPOSE_PROJECT_NAME (only relevant if you use Docker)
• MESSENGER_TRANSPORT_DSN
• JWT key paths:
  JWT_SECRET_KEY, JWT_PUBLIC_KEY (they point to key files on disk; manage the files at deploy time)
• Any variable your organization treats as infrastructure (e.g., reverse proxy, PHP-FPM pool names, etc.)

Safe examples

.env (committed defaults — no secrets)

APP_ENV=prod
# Do NOT put the real secret here in git
APP_SECRET=change-me-in-env-local-or-OS

DB_DRIVER=postgresql
DB_HOST=db.example.internal
DB_PORT=5432
DB_NAME=eac_db
DB_USER=eac_usr
DB_OPTIONS=serverVersion=16&charset=utf8

APP_TASK_LOG_TYPE=file
SCHEDULE_ENABLED=true

.env.local (not committed — overrides & secrets)

# Real secrets and instance overrides
APP_SECRET=********REDACTED********
DB_PASSWORD=<strong-db-password>
MAILER_PASSWORD=<smtp-password>

APP_DEFAULT_URI=https://eac.example.com/
PHP_PATH_COMMAND=/usr/bin/php
EAC_CRON_USER=eac

Tip (prod): after changing env in production, run:

php bin/console cache:clear --no-warmup
php bin/console cache:warmup
# or reload PHP-FPM if your stack caches env at master process start

How the Settings form interacts with files

• On Save, the app writes only the keys it manages to .env.local.
• The UI shows:
  • “.env.local (updatable)” → the value that will be saved/overridden.
  • “.env (Present value)” → baseline default (read-only).
• If a key is present in real OS env, that value still wins at runtime.

Sensitive data guidelines

Audit who can access the Settings page (Admin-only).

Put passwords, tokens, secrets in .env.local or real environment variables, never in .env.

Keep JWT keys as files outside of git; reference them via JWT_SECRET_KEY / JWT_PUBLIC_KEY.

Limit file and directory permissions for:

var/eac/log/, var/eac/tmp/, var/eac/uploads/, var/eac/deploy/

L’article Configuration files est apparu en premier sur ETL administration console.

1. Supported Databases

EAC is compatible with the following database engines (for both App DB and ETL Log DB):

• MySQL ≥ 8.0
• PostgreSQL ≥ 15.0
• SQL Server ≥ 17.0
• MariaDB ≥ 10.5
• Oracle (basic support, logging only)
• SQLite (for development/testing only, not recommended for production)

You are free to use the same database engine for both App DB and EtlLog DB, or different ones depending on your infrastructure.

2. Database Roles

• Application Database (App DB)
  Stores metadata such as users, jobs, tasks, projects, schedules, and configuration.
• ETL Log Database (EtlLog DB)
  Stores execution history, logs, statuses, and performance metrics.

Best practice: separate the two databases to avoid heavy logging operations impacting application responsiveness.

3. Configuration

All database connections are defined in the .env or .env.local file:

### Application database
APP_DATABASE_URL="mysql://eac_user:password@127.0.0.1:3306/eac_app"

### ETL Log database
ETLLOG_DATABASE_URL="pgsql://etl_user:password@127.0.0.1:5432/eac_logs"

Replace the prefix (mysql://, pgsql://, sqlsrv://, oci://, sqlite://) depending on your database type.

4. Creating Databases and Users

MySQL Example

CREATE DATABASE eac_app CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'eac_user'@'%' IDENTIFIED BY 'StrongPassword123!';
GRANT ALL PRIVILEGES ON eac_app.* TO 'eac_user'@'%';
FLUSH PRIVILEGES;

PostgreSQL Example

CREATE DATABASE eac_logs;
CREATE USER etl_user WITH ENCRYPTED PASSWORD 'StrongPassword123!';
GRANT ALL PRIVILEGES ON DATABASE eac_logs TO etl_user;

SQL Server Example

CREATE DATABASE eac_app;
CREATE LOGIN eac_user WITH PASSWORD = 'StrongPassword123!';
CREATE USER eac_user FOR LOGIN eac_user;
ALTER ROLE db_owner ADD MEMBER eac_user;

5. Doctrine Integration

• The App DB schema is managed by Doctrine ORM.
  Run migrations after installation or upgrade: php bin/console doctrine:migrations:migrate
• The ETL Log DB schema is maintained by EAC and not exposed through Doctrine migrations.

6. Best Practices

• Use separate databases (App DB vs EtlLog DB) for scalability.
• Avoid using SQLite in production.
• Monitor storage usage regularly, especially for ETL logs.

L’article Databases est apparu en premier sur ETL administration console.

sudo useradd -m -g www-data -s /bin/bash -d /var/www/html eac

Define new password for « eac » user:

sudo passwd eac

You can check user creation with this command:

id eac

Define the default beavior for folder and file creation (change /var/www/html according to your installation folder) :

sudo chown -Rh eac:www-data /var/www/html

sudo find -L /var/www/html -type d -exec chmod 2775 {} \;
sudo find -L /var/www/html -type f -exec chmod 0664 {} \;
sudo find -L /var/www/html -type f -name "*.sh" -exec chmod 0770 {} \;

Change the default umask of eac user by editing ~/.bashrc file

sudo nano ~/.bashrc

Add at the end the following line:

umask 0007

Overwritte umask of php-fpm service by editing php-fpm service file. Example for php 8.3 version:

sudo systemctl edit php8.3-fpm.service

### Editing /etc/systemd/system/php8.3-fpm.service.d/override.conf
### Anything between here and the comment below will become the new contents of the file

[Service]
UMask=0007

### Lines below this comment will be discarded

and apache of nginx services:

Overwritte umask of apache service by editing apache2 service file:

sudo systemctl edit apache2

### Editing /etc/systemd/system/php8.3-fpm.service.d/override.conf
### Anything between here and the comment below will become the new contents of the file

[Service]
UMask=0007

### Lines below this comment will be discarded

Reload services and restart them:

sudo systemctl daemon-reload
sudo systemctl restart php8.3-fpm.service
sudo systemctl restart apache2

You can also manage folder rights by ACL but becarefull if you are in multi-server mode to the mount type you use between servers: so of them doesn’t support ACLs:

sudo setfacl -R -m u:eac:rwx,g:www-data:rwx /var/www/html
sudo setfacl -R -m d:u:eac:rwx,d:g:www-data:rwx,d:mask:rwx /var/www/html

You can re-execute the previous commands in the future if you made a mistake.

L’article Files rights management est apparu en premier sur ETL administration console.

• login: eac@cidwe.com
• password: eac

It is strongly recommended to remove or disable this user or at least to change its password for security reason.

L’article Default user est apparu en premier sur ETL administration console.

For mono-server instance, you can create only one server. It should be setup with « Is Main server » and « is Execution server » options.

For multi-server instance, create only one main server (normaly the one where you are actually). This server can also be an execution server. Do the same for execution servers but without option « Is Main Server »:

L’article Servers setup est apparu en premier sur ETL administration console.

crontab -e -u eac

1. Status Purge – on main server only

To clean up old memory and cpu usage on servers. We recommended retention of 4 days:

0 0 * * * {path_to_php} /var/www/html/bin/console eac:server-status:purge --days=1 >> /var/www/html/var/log/eac-master-server-status-purge.log 2>&1

Replace {path_to_php} by the full path to php. For example: /usr/bin/php

2. Deployment Verification – on all servers

When a job version is deployed in EAC, each execution servers must retrieve and keep a local copy of that version.
This scheduled verify that every execution server holds the correct copy; if not, EAC shows a visual alert and avoids launching an execution on any out-of-sync server.

0 1 * * * {path_to_php} /var/www/html/bin/console eac:check-jobversion-deployment --force >> /var/www/html/var/log/eac-master-check-jobversion-deployment.log 2>&1

Replace {path_to_php} by the full path to php. For example: /usr/bin/php

3. Check executions are still alive – on all servers

This schedule allow to kill all executions with status « IN PROGRESS » which no longer have any os pid running on the system.

* * * * * {path_to_php} /var/www/html/bin/console eac:check-background-task  --secondsToConsiderOutdatedAndKill=90 --secondsToConsiderOutdatedAndReaffectServer=10 >> /var/www/html/var/log/eac-master-check-background-task.log 2>&1

Replace {path_to_php} by the full path to php. For example: /usr/bin/php

You can adjust options

• secondsToConsiderOutdatedAndKill: reduce or increase the number of seconds after what an execution with the status « SENT TO QUEUE » is considering oudated because on server is available to launch it.
• secondsToConsiderOutdatedAndReaffectServer: reduce or increase the number of seconds to reaffect a task to an available server

4. Enable scheduler – on main server only

If you schedule tasks on EAC you need this to launch task on time.

* * * * * {path_to_php} /var/www/html/bin/console eac:scheduler --planForTheNextInSeconds=90 >> /var/www/html/var/log/eac-master-scheduler.log 2>&1

Replace {path_to_php} by the full path to php. For example: /usr/bin/php

You can adjust option « planForTheNextInSeconds » to provision tasks to be launched more or less than the next 90 seconds.

L’article Planned tasks est apparu en premier sur ETL administration console.

Server status

This service must be install on all servers. It retrives memory and cpu usage in order to determine what will be the best server to launch a task.

Copy /var/www/html/current/deployment/systemd/eac-server-status.service file into /etc/systemd/system/ with the following command

sudo cp /var/www/html/current/deployment/systemd/eac-server-status.service /etc/systemd/system/

Edit the file and change ExecStart parameter in order to specify the full path to php according to your installation.

ExecStart=/usr/bin/php /var/www/html/current/bin/console eac:server-status --every=3

You can change option « every« . This value must be set between 1 to 10 (seconds to save memory usage and cpu on database).

Assignment server

This service determines the best servers available for tasks with status « SEND TO QUEUE ». It must be installed only on the main server.

Copy /var/www/html/current/deployment/systemd/eac-execution-assign-server.service file into /etc/systemd/system/ with the following command

sudo cp /var/www/html/current/deployment/systemd/eac-execution-assign-server.service /etc/systemd/system/

Edit the file and change ExecStart parameter in order to specify the full path to php according to your installation.

ExecStart=/usr/bin/php  /var/www/html/current/bin/console eac:execution-assign-server --nextSecondsToAssign=90 --pastSecondsToAssign=60 --frequence=10

You can change options:

• nextSecondsToAssign: assign server to tasks which will executed in the next 90 seconds
• pastSecondsToAssign: assign server to tasks sent to queue and the start date is in the past of 60 seconds
• frequence: to this action all 10 seconds

Consume task

This services consumes / execute the task with status « SEND TO QUEUE ». It must be installed on all execution servers.

Copy /var/www/html/current/deployment/systemd/eac-consume.service file into /etc/systemd/system/ with the following command

sudo cp /var/www/html/current/deployment/systemd/eac-consume.service /etc/systemd/system/

Edit the file and change ExecStart parameter in order to specify the full path to php according to your installation.

ExecStart=/usr/bin/php  /var/www/html/current/bin/console eac:consume --launchForTheNextInSeconds=0 --launchForThePastInSeconds=60

You can change options:

• launchForTheNextInSeconds: execute tasks exactly on time : at 0 second. You can change to 1 if you prefer to execute task one second before
• launchForThePastInSeconds: execute tasks sent to queue and the start date is in the past of 60 seconds

L’article System services est apparu en premier sur ETL administration console.

Configuration

Edit ./shared/.env.local and set the following variables.

1. SMTP DSN

Use the DSN your app expects (usually MAILER_DSN; in some setups it may be MAIL_DSN). Keep the same key consistently everywhere.

Format (recommended):

MAILER_DSN="smtp://<LOGIN>:<PASSWORD>@<HOST>:<PORT>?encryption=<tls|ssl>&auth_mode=<login|plain>"

URL-encode special characters in <LOGIN> and <PASSWORD>.
Common encodings: @ → %40, : → %3A, / → %2F, ? → %3F, # → %23, & → %26.

Examples:

# Port 25, no encryption
MAILER_DSN="smtp://mylogin%40mycompany.com:My%3APassword@smtp.mycompany.com:25"

# Port 587 with STARTTLS
MAILER_DSN="smtp://mylogin%40mycompany.com:MyPassword@smtp.mycompany.com:587?encryption=tls"

# Port 465 with implicit TLS (alternative)
MAILER_DSN="smtp://mylogin%40mycompany.com:MyPassword@smtp.mycompany.com:465?encryption=ssl"

Tip: wrap the value in quotes if it contains ?, &, or other special characters (safer in .env files).

2. Enable/disable the feature

APP_ENABLE_TASK_MAILER=true

Set to false anytime to stop sending emails.

3. Default recipients

# Comma-separated list; no spaces recommended APP_MAILER_DEFAULT_RECIPIENTS= »ops@mycompany.com,data-team@mycompany.com »

L’article Mailer setup est apparu en premier sur ETL administration console.

L’article Project tree est apparu en premier sur ETL administration console.