The MySQL governor

The governor keeps one tenant's database workload from degrading everyone else's. It caps query rate and connections, kills runaway queries, and records per-tenant statistics.

It is free, like every other tenantd feature.

How it is built

The governor is an independent implementation that works entirely over MariaDB's public SQL and performance_schema surface. tenantd does not patch MariaDB, does not link against it, and does not ship a modified server. You run stock MariaDB from your distribution.

This matters for the same reason the stock kernel matters: your database upgrade path stays identical to every other host you operate.

Setup

Once per host:

tenantctl mysql-bootstrap

This creates the tenantd_governor MariaDB user and grants it the reads it needs. The password is stored at /etc/tenantd/governor.pw.

MariaDB ships with performance_schema=OFF. tenantd installs a drop-in at /etc/systemd/system/mariadb.service.d/tenantd.conf that enables it. MariaDB must be restarted for this to take effect. Without performance_schema the governor cannot see per-thread activity and will not enforce.

Start it:

systemctl enable --now tenantd-mysql-governor

What it enforces

Hard caps, enforced by MariaDB

Set per package profile, applied as grants:

MYSQL_MAX_QUERIES_PER_HOUR
MYSQL_MAX_USER_CONNECTIONS

Because these are MariaDB grants rather than daemon-side policy, they hold even if the governor is stopped or was never started.

Runaway kills, enforced by the daemon

Configured in /etc/tenantd/governor.conf:

Setting	Default	Behavior
`GOVERNOR_KILL_CPU_THRESHOLD_SECONDS`	`10`	A single statement burning more than this much CPU time is killed with `KILL QUERY`.
`GOVERNOR_LONG_TXN_THRESHOLD_SECONDS`	`300`	A single statement running this long with no progress is killed, rolling back its transaction. Higher than the CPU threshold because nightly cron jobs legitimately run long.
`GOVERNOR_SLOW_QUERY_THRESHOLD_SECONDS`	`2`	Recorded as a slow query for operator visibility. Not killed.
`GOVERNOR_KILL_DEBOUNCE_SECONDS`	`2`	Do not re-issue `KILL` against the same thread within this window, so a slow teardown does not cause kill thrash.
`GOVERNOR_POLL_INTERVAL_SECONDS`	`1`	Polling cadence. 1 Hz targets sub-2-second p99 kill latency. Halving it halves p99 at roughly double the (still under 1%) CPU cost on a 100-tenant host.

After editing the config:

systemctl restart tenantd-mysql-governor

Statistics

The governor writes per-tenant rollups every GOVERNOR_METRICS_INTERVAL_SECONDS (default 60) into the shared stats database at /var/lib/tenantd/stats.db, which holds a rolling 24 hours. The WHM plugin reads from there.

The IO gap, stated plainly

The governor cannot throttle block IO per tenant inside a shared mysqld.

This is not an oversight and not a roadmap item. cgroup v2's io controller is domain-only: it applies to a cgroup as a whole, not to individual threads within a process. A shared mysqld is one process serving every tenant, so there is no cgroup boundary to place between two tenants' queries. Solving it inside a shared mysqld requires patching the kernel, and refusing to patch the kernel is the entire premise of this product.

What the governor does instead is catch the symptoms. A tenant hammering the disk is usually also running long queries, flooding connections, or holding locks, and those are all visible and killable over the public interface.

For genuinely IO-heavy tenants: promote them

When symptom-catching is not enough, move the tenant off the shared mysqld entirely:

tenantctl mysql-promote-suggest    # rank tenants by promotion benefit
tenantctl mysql-promote alice      # dedicated mariadb@alice instance
tenantctl mysql-demote alice       # move back onto the shared mysqld

A promoted tenant gets their own mariadb@ systemd instance inside their own slice. At that point the mysqld is no longer shared, the cgroup boundary exists, and per-tenant IO enforcement works properly.

Promotion is free. There is no per-instance charge and no cap on how many tenants you promote.

You can let tenantd handle this for you:

tenantctl auto-promote show
tenantctl auto-promote enable --max=5

If this gap is a dealbreaker

If you run shared mysqlds where specific tenants must have hard per-thread IO caps and cannot be promoted onto dedicated instances, CloudLinux does that and tenantd does not. We would rather you know now than discover it in production.

Observability

With tenantd-ebpf installed, per-tenant IO attribution is visible even where it is not enforceable:

dnf install tenantd-ebpf
systemctl enable --now tenantd-mysql-ebpf

It attributes database IO to tenants using the USDT marker mysql:connection__start. This tells you who is causing IO pressure on a shared mysqld, which is what you need in order to decide whom to promote.