Latest updates for Amazon Emr On Eks

Fresh curated links around Amazon EMR on EKS are collected here so marketers can spot useful updates and turn timely ideas into posts faster.

Recent items include:

  • Announcing Spark Connect on Amazon EMR on EKS: Interactive PySpark development, anywhere
  • Announcing Spark Connect on Amazon EMR on EC2: Interactive PySpark anywhere
  • Accelerate Apache Spark debugging on Amazon EMR with AWS DevOps Agent

Post angles to try

Share the most useful takeaway for your audience.
Turn one article into a quick practical checklist.
Ask your audience how this shift affects their work.
Turn angles into scheduled posts

Fresh articles and ideas

Recent curated links from global sources. Generate one free draft from any story, then use SocialBu to schedule and refine your content calendar.

aws.amazon.com /3 weeks ago

Announcing Spark Connect on Amazon EMR on EKS: Interactive PySpark development, anywhere

Announcing Spark Connect on Amazon EMR on EKS: build, test, and debug Spark applications from VS Code, PyCharm, Jupyter notebooks, Amazon SageMaker Unified Studio, or dbt, while ru...

Read source
aws.amazon.com /1 month ago

Announcing Spark Connect on Amazon EMR on EC2: Interactive PySpark anywhere

Amazon EMR on EC2 now supports Spark Connect, so you can develop and debug PySpark interactively from Amazon SageMaker Unified Studio Data Notebooks or your own IDE while Spark run...

Read source
aws.amazon.com /1 month ago

Accelerate Apache Spark debugging on Amazon EMR with AWS DevOps Agent

Extend AWS DevOps Agent to investigate Apache Spark failures on Amazon EMR. This post shows how to register the Apache Spark Troubleshooting Agent for Amazon EMR as a custom MCP ca...

Read source
aws.amazon.com /1 month ago

Query Amazon S3 Tables from Amazon EMR Trino using the Iceberg REST endpoint

Learn how to query Amazon S3 Tables from Trino on Amazon EMR using the Apache Iceberg REST catalog endpoint. This post shows how to deploy the integration with AWS CloudFormation,...

Read source
aws.amazon.com /1 week ago

Bring data quality to your Amazon EMR workloads with Amazon EMR Data Quality built on AWS Glue Data Quality

AWS Glue Data Quality is now available natively on Amazon EMR, bundled in AWS runtime for Apache Spark (emr-spark-8.1.0). Write the same DQDL rulesets you use in AWS Glue directly...

Read source
aws.amazon.com /4 weeks ago

Scaling MoE reinforcement learning on Amazon EKS with EFA and DeepEP with 40% more throughput

Learn how to scale Mixture-of-Experts (MoE) reinforcement learning on Amazon EKS using Elastic Fabric Adapter (EFA) and DeepEP. This post presents an architecture that combines Ama...

Read source
aws.amazon.com /3 weeks ago

Building an LLM-powered DAG failure analysis plugin for Amazon MWAA

Debugging Apache Airflow DAG failures across services like AWS Glue, Amazon EMR, and Amazon Athena is slow and manual. In this post, we show you how to build a custom Airflow plugi...

Read source
aws.amazon.com /3 weeks ago

Build agent memory with NVIDIA NeMo Agent Toolkit and Amazon S3 Vectors

Learn how to use Amazon S3 Vectors as the persistent memory layer within the NVIDIA NeMo Agent Toolkit (NAT), deployed on Amazon Elastic Kubernetes Service (Amazon EKS). This post...

Read source
aws.amazon.com /1 month ago

Run Positron on Amazon SageMaker AI for data science workflows

Positron, Posit's IDE for data science, now runs on Amazon SageMaker AI. This post shows how a data scientist explores an Amazon Athena table, validates features in R, trains an XG...

Read source
aws.amazon.com /2 weeks ago

Monitoring MWAA-orchestrated ETL pipelines with Amazon OpenSearch Service

Troubleshooting a multi-service, MWAA-orchestrated ETL pipeline often means hunting across scattered Amazon CloudWatch log groups. This post shows how to centralize your ETL logs i...

Read source
aws.amazon.com /1 month ago

Fault tolerant distributed training on Amazon EKS using NVRx

Integrate NVIDIA Resiliency Extension (NVRx) into PyTorch FSDP training on Amazon EKS to overlap checkpoint I/O with training and recover from GPU faults in seconds. This post cove...

Read source
aws.amazon.com /1 month ago

Run agent-driven Amazon SageMaker HyperPod operations with InstantStart

HyperPod InstantStart is an open source control plane that composes Amazon EKS orchestration with the managed capabilities of Amazon SageMaker HyperPod. It drives the same guarded...

Read source
aws.amazon.com /1 week ago

Run multi-node inference workloads using Ray LLM Deep Learning Containers

Learn how to deploy one large language model across two GPU nodes on Amazon EKS using the multi-node Ray LLM Deep Learning Containers, with separate prefill and decode nodes behind...

Read source
aws.amazon.com /2 weeks ago

Query across accounts and table formats with multi-catalog in Amazon EMR

Amazon EMR 8.1.0 adds multi-catalog support through the RedirectingSessionCatalog, so you can query Iceberg, Delta Lake, Hudi, and Hive tables through a single catalog and join dat...

Read source
aws.amazon.com /2 weeks ago

Share GPU clusters across teams with isolation and fairness using Amazon SageMaker HyperPod

A reference architecture for securely sharing one Amazon SageMaker HyperPod EKS cluster across multiple teams, using AWS IAM Identity Center for authentication, per-team SageMaker...

Read source
aws.amazon.com /2 weeks ago

Manage Amazon SageMaker HyperPod Spaces directly from SageMaker Studio

Data scientists and ML engineers can now create, configure, start, stop, and open Amazon SageMaker Spaces on SageMaker HyperPod EKS clusters directly from SageMaker Studio. Launch...

Read source
aws.amazon.com /1 month ago

Build a real-time event pipeline with Spark Real-Time Mode on AWS Glue 6.0

With AWS Glue 6.0, you can build real-time, near-real-time, and batch data pipelines on a single platform. Using a financial market-risk example, learn how to flag high-risk trades...

Read source
aws.amazon.com /3 weeks ago

Aurora PostgreSQL zero-ETL integration with Amazon SageMaker

Amazon Aurora PostgreSQL zero-ETL integration with Amazon SageMaker replicates your operational data to a lakehouse in near real time, without building custom ETL pipelines. Learn...

Read source
rubyflow.com /1 week ago

Kill SSH on ECS: get a Rails console via ECS Exec with Terraform

Still keeping port 22 open and a key pair in your launch template just so someone can run rails console in staging? This guide walks through replacing SSH with AWS SSM Session Mana...

Read source
aws.amazon.com /2 weeks ago

Cost-effective ETL with DuckDB and Amazon S3 Tables on AWS Glue

Learn how to pair DuckDB with AWS Glue 6.0 to run SQL-centric ETL on a single worker, reading Parquet from Amazon S3 and writing Apache Iceberg tables to Amazon S3 Tables. This pos...

Read source
aws.amazon.com /1 week ago

Introducing Reflection Beam: an open-weight model for agentic coding on AWS

Reflection AI's Beam is an open-weight model for agentic coding and long-horizon tasks. This post shows how to deploy Beam on AWS: start with an Amazon SageMaker JumpStart real-tim...

Read source
aws.amazon.com /1 month ago

Build declarative ETL pipelines with AWS Glue 6.0

AWS Glue 6.0 introduces Spark Declarative Pipelines. In this post, you build a single declarative AWS Glue 6.0 job that turns raw order records into validated, aggregated tables th...

Read source
aws.amazon.com /3 weeks ago

Build a multi-agent music production pipeline on Amazon Bedrock AgentCore Runtime Instances

Amazon Bedrock AgentCore Runtime Instances gives multi-agent workflows AWS managed EC2 infrastructure with GPUs, persistent volumes, and multi-day sessions. In this post, we deploy...

Read source
aws.amazon.com /1 week ago

Build cross-account streaming pipelines with the Amazon Kinesis Producer Library and Kinesis Client Library

Learn how to build an enterprise-scale cross-account streaming pipeline with Amazon Kinesis Data Streams using the Kinesis Producer Library (KPL) and Kinesis Client Library (KCL) a...

Read source

Turn fresh research into a full content calendar

Use SocialBu to discover ideas, generate post drafts, and schedule them across your social channels.

Start creating posts

Sources covering Amazon Emr On Eks

aws.amazon.com

Recent coverage from public sources
Public source

aws.amazon.com

Recent coverage from public sources
Public source

rubyflow.com

Recent coverage from public sources
Public source

Related searches

Advanced (300) Technical How-to Amazon EMR Announcements AWS Glue Amazon SageMaker HyperPod Amazon Elastic Kubernetes Service Amazon Managed Workflows for Apache Airflow (Amazon MWAA) Amazon S3 Tables Amazon SageMaker AI Expert (400) Intermediate (200)

Recent searches

Artificial Intelligence Machine Learning Data Science Data Engineering Business Intelligence Web Development Software Development Software Testing Programming JavaScript DevOps Cyber Security Product Management Project Management Design UX UX Research Graphic Design Video Production Video Editing