Logo

Job Debugger & Optimizer

Modern Spark Web UI tab that enriches your production logs, makes plans and stages readable, flags bottlenecks, and helps you fix failing or slow jobs in minutes, not hours. Free and open source.

Enterprise UI

Enriches Spark logs and turns raw metrics into a clear plan/stage view anyone can scan.

Fast deployment

Drop-in tab; no code changes to your jobs.

Secure & compliant

Production context stays in your environment - metadata stays in your cluster by default (Apache-2.0).

Install with just two lines of code

Add DataFlint's log enrichment JAR to your Spark session configuration - no code changes required.

Add these 2 configs to your pyspark session builder:
builder = pyspark.sql.SparkSession.builder
...
.config("spark.jars.packages", "io.dataflint:spark_2.12:0.5.0") \
.config("spark.plugins", "io.dataflint.spark.SparkDataflintPlugin") \
...

Everything in OSS PLUS Advanced Capabilities

Get the complete DataFlint experience: powerful open-source foundation enhanced with enterprise-grade development integration.

SaaS

DataFlint Plugin (OSS Foundation)

OSS
Real-time query & cluster status
Plan & stage breakdown with heat map
Run summary for each application
Performance alerts & suggestions (skew, spill, retries, inefficient joins)
Failure pinpointing with links to the failing stage/node
Open source Apache-2.0 license with full community access
Active Slack community for support and collaboration
View on GitHub

Enhanced Capabilities

Close the loop with your IDE - turn findings into fixes without leaving your editor.

Bridge production to development: alert → stage → code line via MCP integration.
Inline fix proposals in VS Code/Cursor/IntelliJ, grounded in the exact run.
Impact preview on each suggestion (runtime and $ delta).
Managed Spark History Server and cross-job search to reproduce, compare, and verify improvements.
Spark UI Tab (OSS)SaaS dashboardIDE Extension
Open source•Apache-2.0•No external export by default

Job Debugger & Optimizer Interface

Loading demo...

Ready to debug Spark jobs in minutes, not hours?

See how our open-source Job Debugger enriches your production logs and transforms your Spark troubleshooting workflow. Get a personalized demo of the enhanced Web UI and learn how to integrate it with your existing infrastructure.