How to monitor edu

Monitoring infrastructure

You can monitor the Airflow infrastructure (AWS instances) using your favorite AWS monitoring tool (e.g., AWS CloudWatch)

Monitoring ELT processes

Monitoring the for ELT process issues primarily involves logging into Airflow, checking for DAG errors, and viewing logs. All Airflow logs are stored in AWS CloudWatch Logs, which can be monitored directly if desired.

You can also set up Slack alerts to be notified of Airflow task errors as follows:

1. create a Slack app; copy its webhook URL
2. add an Airflow connection called slack with the webhook URL