Airflow Interview Questions (2026)
Practice 45 Airflow interview questions on DAGs, tasks, scheduling, operators, sensors, retries, backfills, XCom, datasets, executors, and monitoring.
45 questions with answersWhat Is Apache Airflow?
Key Takeaways
- ✓Apache Airflow is an orchestration platform for authoring, scheduling, and monitoring workflows as Python DAGs.
- ✓Interviews test DAG design, task dependencies, scheduling, retries, backfills, sensors, executors, XCom, variables, and monitoring.
- ✓Strong answers separate orchestration from processing. Airflow coordinates work, it should not hold heavy data processing inside the scheduler.
- ✓Production Airflow answers need idempotent tasks, clear ownership, alerts, retries, and safe backfill behavior.
Apache Airflow is an open-source workflow orchestration platform. Teams define workflows as Python DAGs, schedule them, and monitor task execution through the Airflow scheduler, workers, metadata database, and UI. In interviews, Airflow questions test whether you can design reliable data pipelines, handle failures, backfill safely, and keep orchestration logic separate from heavy processing.
Watch: Airflow 101: Essential tips for beginners
Video: Airflow 101: Essential tips for beginners (Astronomer, YouTube)
Test yourself and earn a certificate
6 quick questions. Score 70%+ to download your Airflow certificate.
All Questions on This Page
45 questions- 1. What is a DAG in Airflow?
- 2. What is a task?
- 3. What is an Airflow operator?
- 4. What is a sensor?
- 5. What does the scheduler do?
- 6. What is an executor in Airflow?
- 7. What is stored in the metadata database?
- 8. What is a data interval?
- 9. What does catchup do?
- 10. What is a backfill?
- 11. How do retries work?
- 12. What is a trigger rule?
- 13. What is XCom?
- 14. What is an Airflow pool?
- 15. What is dataset scheduling?
- 16. Design a daily orders pipeline in Airflow.
- 17. How do you set task dependencies?
- 18. How do you make an Airflow task idempotent?
- 19. How do you configure retries?
- 20. How do you wait for an external file?
- 21. How do you run a six-month backfill safely?
- 22. How do you branch in Airflow?
- 23. How do you pass values between tasks?
- 24. How do you stop Airflow from overloading a warehouse?
- 25. How do you alert on DAG failures?
- 26. How do you organize Airflow DAG code?
- 27. How do you test Airflow DAGs?
- 28. How do you choose an Airflow executor?
- 29. How do you manage Airflow secrets?
- 30. When would you use dataset scheduling?
- 31. A new DAG does not appear in Airflow. What do you check?
- 32. Tasks stay queued. What is likely?
- 33. A sensor blocks worker capacity all day. What do you change?
- 34. A retry duplicates rows. What was missing?
- 35. A DAG processes the wrong date. What do you inspect?
- 36. The scheduler is slow. What do you inspect?
- 37. A backfill overloads the warehouse. What should have been done?
- 38. A DAG stores large payloads in XCom. Why is that bad?
- 39. A DAG misses its daily deadline. What do you check?
- 40. A join task is skipped after branching. What is wrong?
- 41. How do you rerun one failed task safely?
- 42. Airflow UI and scheduling are slow. What else do you check?
- 43. A task hangs forever. What control should exist?
- 44. A DAG fails and nobody responds. What process is missing?
- 45. What do you ask in an Airflow design review?
Airflow Fundamentals
Foundational15 questionsStart here. These are the definitions and first-principle checks that open most rounds.
Q1. What is a DAG in Airflow?
A DAG is a directed acyclic graph that defines tasks and their dependencies.
It describes workflow structure, not a place to run heavy processing at parse time.
DAG changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Airflow run path
The scheduler decides what should run next.
Watch a deeper explanation
Video: Airflow 101: Essential tips for beginners (Astronomer, YouTube)
Q2. What is a task?
A task is one unit of work created from an operator or task-decorated function.
Good DAGs split work into clear, retryable tasks.
Task changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q3. What is an Airflow operator?
An operator is a reusable template that defines how a task runs.
Examples include PythonOperator, BashOperator, KubernetesPodOperator, and database operators.
Operator changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q4. What is a sensor?
A sensor waits for a condition, such as a file, partition, API result, or external task.
Use deferrable sensors or sensible modes to avoid wasting worker slots.
Sensor changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
| Answer part | What to say | Evidence to mention |
|---|---|---|
| Definition | Sensor in one direct sentence. | Official docs or course material |
| Use case | The work where it changes a decision. | Dataset, model, query, dashboard, or pipeline |
| Risk | What breaks when it is misunderstood. | Metric, log, test result, or review note |
Q5. What does the scheduler do?
The scheduler parses DAGs, creates DAG runs, checks dependencies, and queues ready tasks.
A slow scheduler can delay many workflows.
Scheduler changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Watch a deeper explanation
Video: How to write data pipelines with Apache Airflow 3.0 (Astronomer, YouTube)
Q6. What is an executor in Airflow?
An executor decides how tasks are run, such as locally, on Celery workers, or on Kubernetes.
Executor choice affects scale and isolation.
Executor changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q7. What is stored in the metadata database?
Airflow stores DAG runs, task instances, states, connections metadata, variables metadata, logs pointers, and scheduler state.
Metadata database health is critical.
Metadata database changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q8. What is a data interval?
A data interval is the time range a scheduled DAG run is meant to process.
It helps avoid confusion between run time and data time.
Data interval changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q9. What does catchup do?
Catchup tells Airflow whether to create past scheduled runs that were not executed.
It is useful for historical processing but risky if tasks are not idempotent.
Catchup changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q10. What is a backfill?
A backfill runs a DAG for historical intervals to fill or correct past data.
Backfills should be scoped, monitored, and validated.
Backfill changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q11. How do retries work?
Retries rerun a failed task based on retry count and delay settings.
Retries help transient failures, not deterministic bad logic.
Retry changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q12. What is a trigger rule?
A trigger rule controls when a downstream task runs based on upstream task states.
It matters for cleanup, branching, and partial-failure workflows.
Trigger rule changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q13. What is XCom?
XCom lets tasks exchange small pieces of metadata.
Do not pass large DataFrames or files through XCom.
XCom changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Watch a deeper explanation
Video: Introducing Apache Airflow 3 (Apache Airflow, YouTube)
Q14. What is an Airflow pool?
A pool limits concurrent tasks that use a shared resource.
Pools protect databases, APIs, and warehouses from overload.
Pool changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q15. What is dataset scheduling?
Dataset scheduling lets DAGs run when upstream datasets are updated.
It helps model data dependencies more directly than time-only schedules.
Dataset scheduling changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Airflow Practical Interview Questions
Intermediate15 questionsThese questions test whether you can apply the topic to real data, real code, and messy constraints.
Q16. Design a daily orders pipeline in Airflow.
Create extract, validate, transform, publish, and notify tasks with a daily schedule, retries, alerts, and date-partitioned outputs.
Make each task rerunnable for its data interval.
design a daily DAG changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
# Interview check: split data, run baseline, inspect metric
from sklearn.metrics import accuracy_score
print(accuracy_score(y_true, y_pred))Q17. How do you set task dependencies?
Use bitshift operators or dependency methods so Airflow can schedule tasks in the required order.
Keep dependency graphs readable.
set dependencies changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q18. How do you make an Airflow task idempotent?
Use partitioned writes, replace or merge by key, check outputs, and avoid hidden side effects.
Retries and backfills depend on this.
write idempotent task changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q19. How do you configure retries?
Set retry count, delay, exponential backoff where useful, and alerting after final failure.
Too many retries can hide real defects.
configure retries changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q20. How do you wait for an external file?
Use a sensor, deferrable sensor, dataset, or upstream signal, then timeout and alert if the file never arrives.
Do not let a sensor block a worker slot forever.
handle external file changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q21. How do you run a six-month backfill safely?
Scope date ranges, limit concurrency, validate each interval, monitor cost, and publish after checks.
Backfills can overwrite trusted data.
run backfill safely changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q22. How do you branch in Airflow?
Use branching operators or task flow branching and set trigger rules for downstream joins.
Plan skipped states clearly.
branch workflow changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q23. How do you pass values between tasks?
Use XCom for small metadata such as partition paths or counts, and use storage for large data.
Large payloads belong in object storage or tables.
pass values changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q24. How do you stop Airflow from overloading a warehouse?
Use pools, task concurrency, DAG concurrency, queues, and rate limits.
The orchestrator must respect shared resources.
protect warehouse changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q25. How do you alert on DAG failures?
Use callbacks, email or chat integrations, owner routing, and alerts that include DAG, task, run, logs, and run interval.
Alerts should be actionable.
alert failure changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Watch a deeper explanation
Video: Using Airflow for machine learning orchestration (Astronomer, YouTube)
Q26. How do you organize Airflow DAG code?
Keep DAG definitions thin, move reusable logic to modules, version configs, and avoid network calls at parse time.
Slow parsing hurts scheduler performance.
organize DAG code changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q27. How do you test Airflow DAGs?
Validate import, dependencies, task functions, parameter rendering, and representative task runs in a test environment.
A DAG that cannot parse will not schedule.
test a DAG changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q28. How do you choose an Airflow executor?
Choose based on scale, isolation, operations skill, and infrastructure: local for small use, Celery or Kubernetes for distributed task execution.
The executor is an operations choice, not only a code choice.
choose executor changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q29. How do you manage Airflow secrets?
Use connections, secret backends, environment integration, and restricted access instead of hardcoded credentials.
Never commit secrets in DAG files.
handle secrets changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q30. When would you use dataset scheduling?
Use it when a downstream DAG should run after a dataset is updated rather than only at a clock time.
It helps data dependency clarity.
schedule by dataset changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Airflow Advanced Scenarios
Advanced15 questionsAdvanced rounds test trade-offs, failure modes, and whether the decision can hold up under production pressure.
Q31. A new DAG does not appear in Airflow. What do you check?
Check DAG parse errors, file location, DAG id, start date, schedule, import dependencies, and scheduler logs.
A syntax or import error can hide the DAG.
DAG not appearing changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q32. Tasks stay queued. What is likely?
Executor capacity, pool slots, worker health, queue mismatch, scheduler delay, or concurrency limits may block execution.
Queued does not always mean failed.
task stuck queued changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q33. A sensor blocks worker capacity all day. What do you change?
Use deferrable sensors, reschedule mode, a timeout, or event-based scheduling.
Waiting should be cheap.
sensor blocking changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q34. A retry duplicates rows. What was missing?
The task was not idempotent or did not write with a safe key or partition policy.
Retries expose unsafe writes.
duplicate data after retry changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q35. A DAG processes the wrong date. What do you inspect?
Check data interval, logical date, templates, timezone, catchup, and manual run parameters.
Airflow date naming often trips candidates.
wrong date processed changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q36. The scheduler is slow. What do you inspect?
Check DAG parse time, number of DAGs, top-level code, metadata database, scheduler resources, and file processor logs.
Avoid heavy top-level code in DAG files.
slow scheduler changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q37. A backfill overloads the warehouse. What should have been done?
Limit concurrency, use pools, batch date ranges, and coordinate with warehouse owners.
Historical runs can multiply load quickly.
backfill overload changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q38. A DAG stores large payloads in XCom. Why is that bad?
It bloats metadata storage and slows Airflow operations.
Store large data externally and pass a pointer.
XCom misuse changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q39. A DAG misses its daily deadline. What do you check?
Check upstream wait time, task duration, retries, resource contention, schedule time, and external systems.
Measure the critical path.
missed SLA changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q40. A join task is skipped after branching. What is wrong?
The trigger rule likely does not account for skipped upstream tasks.
Use a trigger rule that matches branch semantics.
branch join skipped changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q41. How do you rerun one failed task safely?
Clear the task instance for the correct run after confirming the task is idempotent and dependencies are valid.
Do not blindly clear a whole DAG.
manual rerun changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q42. Airflow UI and scheduling are slow. What else do you check?
Check metadata database health, connection pool, cleanup, indexes, and scheduler load.
Airflow depends heavily on metadata storage.
metadata database issue changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q43. A task hangs forever. What control should exist?
Set execution timeout, sensor timeout, alerts, and retry policy.
Every external call needs a failure path.
task timeout changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q44. A DAG fails and nobody responds. What process is missing?
Owner metadata, alert routing, runbook, severity, and escalation path are missing.
Operations ownership belongs in the workflow design.
bad DAG ownership changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Q45. What do you ask in an Airflow design review?
Ask about idempotency, data intervals, retries, backfills, secrets, pools, alerts, XCom use, executor fit, and ownership.
These decide whether the DAG survives production.
senior review changes Airflow decisions by affecting the chosen method, the failure mode, or the validation step. Reliable evidence comes from a metric, test result, query output, log, or review note.
Airflow DAG vs Task vs Operator
Airflow interviews often vocabulary comes first. A DAG defines dependency structure, a task is a unit of work, and an operator is a template for creating a task.
| Term | Meaning | Example | Common mistake |
|---|---|---|---|
| DAG | Directed acyclic graph of tasks | daily_orders_pipeline | Putting runtime data loops into DAG parse time |
| Task | One scheduled unit of work | load_orders | Making one task do too much |
| Operator | Reusable task template | PythonOperator, BashOperator | Using the wrong operator for heavy work |
| Sensor | Task that waits for a condition | file or dataset arrival | Using long blocking sensors carelessly |
Airflow answer signals
Reliability and operational clarity matter more than memorizing operator names.
- DAG design: small tasks, clear dependencies
- Scheduling: data interval, catchup, backfill
- Failure handling: retries, idempotency, alerts
- Operations: logs, pools, executors, SLAs
How to Prepare for a Airflow Interview
Prepare by sketching a pipeline and explaining what runs when, what happens after failure, and how a backfill stays safe.
- Review DAGs, tasks, operators, sensors, dependencies, schedules, catchup, data intervals, and backfills.
- Know retries, trigger rules, XCom, variables, connections, pools, queues, and executors.
- idempotent tasks and why Airflow should coordinate processing rather than process huge datasets in scheduler code is the explanation path.
- One incident story involving a failed DAG, missed schedule, stuck sensor, or bad backfill is useful.
Airflow interview prep flow
Most rounds reward clear reasoning more than memorized phrasing.
How Strong Airflow Answers Sound
A strong Airflow answer explains workflow ownership and recovery. Say what each task does, how dependencies are defined, whether catchup is enabled, how retries are configured, how data intervals map to partitions, and how a failed run can be rerun safely. production-ready answers mention pools, executor choice, XCom limits, sensors, datasets, metadata database health, and alert routing.
Keep tasks idempotent
A retry or backfill should not duplicate data or corrupt outputs.
Understand data intervals
Airflow schedules work for a data interval. The run date is not always the wall-clock date people expect.
Use XCom lightly
XCom is for small metadata, not large datasets.
Treat backfills as production changes
Backfills need scoped dates, validation, and communication when they affect published metrics.
Test Yourself: Airflow Quiz
Ready to test your Airflow knowledge?
6 questions, about 4 minutes. Score 70% or higher to earn a shareable certificate.
Frequently Asked Questions
What do Airflow interviews usually ask?
Is Airflow a data processing engine?
What Airflow project should I discuss?
What separates experienced Airflow candidates?
Practice Airflow answers for data pipeline rounds
Use this Airflow bank for orchestration and incident answers, then practice Python and SQL tasks with Hyring's AI Coding Interviewer.
See Hyring AI Coding InterviewerSources
Written by Adithyan RK