Overage Execution Rows Protections

zmenzer

Just had a user accidentally do a join that we identified 7 hrs after the process started that was up to 72 billion rows. It was going to top out over 300 billion, but we hit cancel. However, that overflow credit usage topped out over 3,000 credits, which is not the kind of mistake you'd like your team to be making. So, similar to the new frequency locks that were discussed at DP25, I'd love for there to be some kind of overage protection on ETL executions so that we can auto-cancel ETLs that are accidentally going off the rails. Maybe it's not a bad join now, but someone updates a downstream dataset and then it kicks off your ETL and now you're spending your months supply of credits on an accidental "oops" moment. I would love for this to be an instance-wide setting.