Intermittent Databricks SQL warehouse errors Invalid OperationHandle, Query could not be scheduled INTERNAL_ERROR

Pete Valentine 0 Reputation points
2026-06-15T20:11:11.2833333+00:00

Since Jun 12, 2026, 12:00 PM NZ Time in East us2 and since Jun 13, 2026, 6:00 AM NZ Time in Australia East we have been recieving intermittent error of the following when running dbt models against a serverless SQL warehouse:

00:47:00 Database Error in model XXX (models/XXX.sql) Invalid OperationHandle: OperationHandle [opType=EXECUTE_STATEMENT, getHandleIdentifier()={GUID}] compiled code at target/run/models/XXX.sql

00:47:00 Database Error in model YYY (models/YYY.sql) Query could not be scheduled: INTERNAL_ERROR: INTERNAL_ERROR: Retry with idempotency token for 4 attempts which exceed 3 (requestId={GUID}) compiled code at target/run/YYY.sql

It is not always the same models, sometimes it is simple full table runs with a table that has 5 records, sometimes it is with larger incremental tables with millions of records.

The SQL Warehouse reports one of the following:

  • Query failed because the execution engine did not respond.
  • [INTERNAL_ERROR] Query could not be scheduled: HTTP Response code: 503. Please try again later. SQLSTATE: XX000

Is there somewhere that has more information on what the error is? or any suggestions on how to fix?

Azure Databricks
Azure Databricks

An Apache Spark-based analytics platform optimized for Azure.

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. SAI JAGADEESH KUDIPUDI 3,640 Reputation points Microsoft External Staff Moderator
    2026-07-29T15:14:02.0933333+00:00

    Hi @Pete Valentine ,Thank you for your patience while we worked with the Product Group team on this issue.

    Root Cause
    Our engineering team completed a detailed analysis of the affected Serverless SQL Warehouse, including
    JVM profiling of the warehouse driver during the incident period.

    The investigation determined that the intermittent failures were caused by driver memory pressure due to very high concurrency. During peak activity, a large number of concurrent dbt model executions created thousands of worker threads, causing the driver process to exceed its container memory limits. As a result, the driver was restarted by the operating system.

    When the driver restarted:

    • In-flight operations lost their server-side handles, resulting in "Invalid OperationHandle" errors.
    • Requests were temporarily rejected with HTTP 503, "execution engine did not respond", and "Query could not be scheduled" errors.
    • The warehouse recovered automatically within seconds, which is why retry attempts were successful and the issue appeared intermittent.

    Key Findings

    • The issue was not caused by any specific query or data volume.
    • Increasing the warehouse size did not resolve the issue because the constraint was related to driver memory consumption, not compute capacity.
    • The reduction in concurrent jobs by approximately 70% successfully mitigated the issue, confirming concurrency as the primary contributing factor.

    The engineering team has confirmed that there has been no recurrence of the issue in either workspace for approximately one week, which aligns with the reduced concurrency configuration currently in place.

    We hope this information helps. Please let us know if you have any further questions.

    If this answer was helpful, please Accept Answer so that it may assist other members of the community facing similar issues.

    Was this answer helpful?

    0 comments

  2. AI answer

    2026-06-15T20:11:23.6433333+00:00

    The errors described are internal Databricks SQL warehouse issues rather than problems with the dbt models or query syntax.

    The messages

    • Query failed because the execution engine did not respond.
    • [INTERNAL_ERROR] Query could not be scheduled: HTTP Response code: 503. Please try again later. SQLSTATE: XX000

    indicate that the Databricks backend could not start or schedule an execution environment or engine instance for the query. HTTP 503 and INTERNAL_ERROR at scheduling time are transient service-side conditions (capacity, control-plane, or engine startup issues) and are not caused by query complexity or data size, which matches the observation that both very small and large models fail intermittently.

    There is no additional public error-class entry in the provided documentation for these specific INTERNAL_ERROR / scheduling / 503 conditions; they are not mapped to a documented Databricks SQL error class like the ones listed for data sources, stateful streaming, or SQL script limits. Because of that, there is no further detail available from documentation beyond what the warehouse UI already shows.

    Given that:

    1. Treat these as transient platform errors, not model errors.
    2. Implement retry logic on the client side (dbt) if possible, so that failed models are retried when the error text includes INTERNAL_ERROR or HTTP 503.
    3. Since the failures started suddenly at specific times and affect multiple regions and arbitrary queries, open a support case with Azure Databricks/Microsoft, providing:
      • Workspace ID and region(s)
      • SQL warehouse name and type (serverless)
      • Approximate timestamps and request IDs / operation handles from the error messages
      • Example queries (even simple ones that fail)

    Only the service team can investigate the internal cause (capacity, control-plane regression, or regional incident) and apply a platform fix or advise on any required configuration changes.

    References:

    AI-generated content may be incorrect. Read our transparency notes for more information.

    Was this answer helpful?

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.