How to monitor edu
You can monitor the Airflow infrastructure (AWS instances) using your favorite AWS monitoring tool (e.g., AWS CloudWatch)
Monitoring ELT processes#
Monitoring the for ELT process issues primarily involves logging into Airflow, checking for DAG errors, and viewing logs. All Airflow logs are stored in AWS CloudWatch Logs, which can be monitored directly if desired.
You can also set up Slack alerts to be notified of Airflow task errors as follows:
- create a Slack app; copy its webhook URL
- add an Airflow connection called
slackwith the webhook URL