Skip to content
Skip to the article
In Databases: 3 articles
Databases

Recover Data from a MariaDB Server That Won't Start

Start MariaDB by hand in InnoDB forced-recovery mode, dump what you can, rebuild the data directory, and restore.

Updated
Applies to
  • MariaDB 10.11
  • MariaDB 11.4
  • MariaDB 11.8
  • Ubuntu 24.04
Tags
  • mariadb
  • innodb
  • recovery
  • backup
Reading time
4 min

Use this procedure when InnoDB corruption keeps MariaDB from starting and you need to get the data out. You start the server by hand with innodb_force_recovery, export every database, rebuild a clean data directory, and import the dump.

Warning

This is a last resort for a server that won't start. Before anything else, stop the service and copy the entire data directory (/var/lib/mysql) somewhere safe. Higher recovery levels can make corruption worse, and the later steps delete the data directory.

1. Stop the Service and Back Up the Data Directory

As root:

systemctl stop mariadb
cp -a /var/lib/mysql /root/mysql-datadir-backup-$(date +%F)

2. Start MariaDB in Recovery Mode

Start the server in the foreground. It skips the privilege tables (so you don't need passwords), doesn't listen on the network, and runs InnoDB in forced recovery:

mkdir -p /run/mysqld && chown mysql: /run/mysqld
mariadbd --user=mysql \
  --skip-grant-tables \
  --skip-networking \
  --skip-external-locking \
  --innodb-force-recovery=1 \
  --socket=/run/mysqld/mysqld.sock

If it won't start, stop it and try again with the next level up. Use the lowest level that lets the server start:

LevelEffect
1Keep running when a corrupt page is found. Safe.
2Don't run background threads (purge).
3Don't roll back incomplete transactions.
4Don't compute statistics or apply the change buffer. Can corrupt secondary indexes.
5Ignore undo logs. Queries can return inconsistent data.
6Don't apply the redo log. Can permanently corrupt data files.

Warning

Levels 4 and above can damage data permanently. Only use them on the backup copy you made in step 1, and only to extract data, never to keep running in production. The server is read-only at the higher levels.

--skip-grant-tables disables all authentication. --skip-networking keeps that limited to local socket connections, so don't remove it.

3. Dump the Data

From a second shell, export what you can. Dump application databases individually so one bad table doesn't stop the whole export:

mariadb-dump --routines --events --triggers --databases <DB_NAME> > <DB_NAME>.sql

Or export everything at once, including user accounts stored in the mysql schema:

mariadb-dump --all-databases --routines --events --triggers > all-databases.sql

For a WordPress site, wp db export dump.sql from the site directory works too.

If a table fails to dump, try the next recovery level, or skip that table with --ignore-table=<DB_NAME>.<TABLE_NAME> and handle it separately.

4. Shut Down and Rebuild the Data Directory

Stop the recovery-mode server cleanly, then move the damaged data directory aside and create a fresh one:

mariadb-admin shutdown
mv /var/lib/mysql /var/lib/mysql.broken
mkdir /var/lib/mysql && chown mysql: /var/lib/mysql
mariadb-install-db --user=mysql --datadir=/var/lib/mysql
systemctl start mariadb

Warning

Moving the directory aside is safer than deleting it. Remove /var/lib/mysql.broken and the step 1 backup only after the restore is verified. If mariadb-install-db isn't available, apt install --reinstall mariadb-server re-runs the package's setup on Debian and Ubuntu.

5. Restore and Recreate Users

Import the dump:

mariadb < <DB_NAME>.sql

If you dumped only application databases, recreate their users. See Create a Database and User.

CREATE USER '<DB_USER>'@'localhost' IDENTIFIED BY '<DB_PASSWORD>';
GRANT ALL PRIVILEGES ON <DB_NAME>.* TO '<DB_USER>'@'localhost';

6. Check and Optimize

Finally, verify every table and rebuild them for a clean start:

mariadb-check --check --auto-repair --all-databases
mariadb-check --optimize --all-databases

Confirm the application works before you delete the old data directory and the backup.

Sources

This article is in the public domain (CC0 1.0), code samples included. Use it however helps you.