How to monitor edu
Monitoring infrastructure#
You can monitor the Airflow infrastructure (AWS instances) using your favorite AWS monitoring tool (e.g., AWS CloudWatch)
Monitoring ELT processes#
Monitoring the for ELT process issues primarily involves logging into Airflow, checking for DAG errors, and viewing logs. All Airflow logs are stored in AWS CloudWatch Logs, which can be monitored directly if desired.
You can also set up Slack alerts to be notified of Airflow task errors as follows:
- create a Slack app; copy its webhook URL
- add an Airflow connection called
slack
with the webhook URL