This post goes over how to deploy Apache Airflow to Heroku. See the repository.
Stack
Env
Ensure the config vars from app.json are set in your Heroku app:
AIRFLOW__CELERY__WORKER_CONCURRENCY=2
AIRFLOW__CORE__FERNET_KEY=
AIRFLOW_HOME=/app
AIRFLOW__WEBSERVER__BASE_URL=
AIRFLOW__WEBSERVER__COOKIE_SAMESITE=Strict
AIRFLOW__WEBSERVER__COOKIE_SECURE=True
AIRFLOW__WEBSERVER__SECRET_KEY=
AIRFLOW__WEBSERVER__WORKERS=2
Install cryptography:
pip3 install cryptography
Generate a AIRFLOW__CORE__FERNET_KEY:
python3 -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"
Set AIRFLOW__CORE__FERNET_KEY in your Heroku app:
heroku config:set AIRFLOW__CORE__FERNET_KEY=$(python3 -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())")
Generate a AIRFLOW__WEBSERVER__SECRET_KEY:
openssl rand -base64 32 # head -c 32 /dev/random | base64
Set AIRFLOW__WEBSERVER__SECRET_KEY in your Heroku app:
heroku config:set AIRFLOW__WEBSERVER__SECRET_KEY=$(openssl rand -base64 32)
Database
If you’re seeing the error in your Heroku logs:
ERROR: You need to initialize the database. Please run `airflow db init`.
Then verify your Airflow config looks good before initializing the database in your Heroku app:
heroku run bash
airflow info
airflow db init
User
Create a user in your Heroku app:
heroku run bash
airflow users create -e EMAIL -f FIRSTNAME -l LASTNAME [-p PASSWORD] -r ROLE [--use-random-password] -u USERNAME
Here’s an example of how to create an admin user:
airflow users create \
--username admin \
--firstname FIRST_NAME \
--lastname LAST_NAME \
--role Admin \
--email [email protected]