Search fresh public links, source activity, and ready-to-use post angles for Apache Airflow.
Fresh curated links around Apache Airflow are collected here so marketers can spot useful updates and turn timely ideas into posts faster.
Recent items include:
Recent curated links from global sources. Generate one free draft from any story, then use SocialBu to schedule and refine your content calendar.
Modern data engineering rarely lives on a single machine. As datasets grow from gigabytes into terabytes — and sometimes into petabytes — teams need orchestration tools that can sc...
The System Was Broken, and Everyone Knew It Our dashboards refreshed overnight. That was the expectation. Then, one week, they started taking six hours. Then eight. On a bad day, t...
Apache Airflow is widely used to orchestrate ETL pipelines, but failure handling in large-scale environments remains largely reactive. While Airflow provides strong scheduling and...
Part 2 of an MLOps End-to-End series — 60 models, fully automated, one Airflow DAGContinue reading on Medium »
Core Features pg-cdc is not just replication. pg-cdc streams Postgres Write Ahead Logs(WAL) out of production Postgres into typed, immutable, time-travelable Iceberg tables on S3...
--- title: "Stop Playing Data Detective: Automated Lineage Tracing Across Your Entire Pipeline Stack" published: false tags: [dataengineering, python, dbt, tutorial] --- # Stop Pl...
Discover new AI tools for data engineering announced at Snowflake Summit 2026. Learn how Snowflake CoCo and smart pipelines accelerate your workflows.
Six-hour data pipeline. Spot termination. Job crashes. 45 minutes of compute lost. Engineer paged at 2 AM. This isn't a tooling problem — it's a decision-making problem. And humans...
Я Игорь Юрченко, backend-разработчик Сбера, в этой статье расскажу о нашем опыте автоматизации деплоя потоков Apache NiFi.Apache NiFi — инструмент для управления потоками данных ме...
Discover how modern data engineering is shifting to meet AI demands. Learn to build resilient, declarative data pipelines and scale with coding agents.
If you're running a cookiecutter-django project on a droplet and still ssh-ing in every time you need a data snapshot, here's how I collapsed that into a single local command: dump...
If you've worked on a data platform for more than a few years, you've almost certainly built the same pipeline twice. First, the way the team wrote pipelines in 2019: a notebook he...
This post shows how developers, data engineers, and analysts can connect to a secure Spark Connect endpoint in Athena with Apache Spark. You can use your preferred tools, such as J...
Helmfile может координировать несколько Helm релизов. Argo CD и Flux могут синхронизировать Kubernetes объекты с Git. Terraform и Pulumi могут создать инфраструктуру. Argo Workflow...
Discover how Snowflake CoCo helps developers deploy Snowpark Python pipelines to production with a single prompt. Start automating your workflow today.
In this post, we show you how to build a CDC pipeline that delivers query-ready Iceberg tables directly. The pipeline captures inserts, updates, and deletes from Aurora PostgreSQL...
The Pipeline Did Not Fail Cleanly Most pipeline failures don't look like "the job failed." Consider a common scenario. A Glue job reads overnight event files, applies business rule...
Purpose and Core ComponentsA data pipeline is the automated system that moves raw data from source systems...
Explore Snowflake for AI and see how Snowflake CoWork (formerly Snowflake Intelligence) helps enterprises deploy AI agents, train models, automate workflows, and govern AI at scale...
Agentic data engineering is changing how pipelines are builtDaikin Applied Americas...
Discover how data engineers ship real-time data pipelines affordably using Snowpipe Streaming and Snowflake CoCo. Start streaming in minutes.
Analytics pipelines tend to scale in both cost and the age of their data sources: costs increase with data volume growth, while data freshness decreases due to longer batch jobs. T...
Discover Snowflake AIM, an AI-powered platform for enterprise migration, modernization, testing and validation on Snowflake.
What I thought was a scheduling problem turned out to be a portability problem first The post I Tried to Schedule My ETL Pipeline. Here’s What I Didn’t Expect. appeared first on To...
Use SocialBu to discover ideas, generate post drafts, and schedule them across your social channels.