We released packages that will allow us to use operators/hooks from Airflow 2.0 in Airflow 1.10. In all steps, the user is assisted by migration guides and runtime warnings. Once the migration is successful the user can happily use Airflow 2.0.
#AIRFLOW 2.0 UPGRADE#
In the next step, the user installs the latest version of Airflow and will migrate the database - airflow db upgrade.This command checks for common incompatibility problems. When the user is on the latest Airflow 1.10 release, they can use the airflow upgrade-check command to see if they can migrate to the new Airflow version.When the user is using Python 3.6+, they can start to use new operators and hooks from Python 3.6 thanks to backport packages.When the user is using Python 2.7, they need to migrate to Python 3.6+.We discussed the migration path that a user should do in order to start using Airflow 2.0 easily The 2.0 release of the Airflow is a significant upgrade and includes substantial major changes, For this reason, we must take additional steps to facilitate migration. Improve Scheduler performance and reliabilityĪIP-8 Split Prociders into Separate Pacjages for Airflow 2.0ĪIP-10: Multi-layered and multi-stage official Airflow CI imageĪIP-15 Support Multiple-Schedulers for HA & Better Scheduling PerformanceĪIP-31 Task Flow API for nicer DAG definitionĪIP-34 Improvements to SubDags / new concept "TaskGroup"ĪIP-26 Production-ready Airflow Docker Image t More information is available on the wiki:Īll tasks are collected in "Airflow 2.0rc1" milestone for RC1 and all remaining "administrative" tasks that do not require release are in "Airflow 2.0" milestone RoadmapĪpart from defining the scope of work, we should also complete these tasks. This will make it more transparent and friendly.
#AIRFLOW 2.0 CODE#
This ticket should not describe individual changes to the code but summarize information that active community members know, but new community members don't know but want to know. This is a meta-ticket, so I would ask committers and PMC members to keep it up-to-date so that everyone has easy access to up-to-date information. For example: from import PythonOperator from airflow.models import DAG from am creating this ticket to help follow this process and coordinate our activities. It’s an easy way to access the reference to XCom returned by a task and use it in other tasks. Understanding this, in each operator, we implemented the output attribute that returns an instance of XComArg. Indeed, it’s unnecessary! If there’s a data relationship between tasks, then there also has to be an order relationship.
And remembering about two relationships sounds like something unnecessary. If you’ve been using Airflow long enough, then you may have experienced that writing jinja templates by hand can be error-prone. This was commonly done by using jinja templates.