Skip to content

[FEA] Support streaming shift/diff expressions in cuDF-Polars #23593

Description

@rjzamora

Row-order-sensitive expressions such as plain shift() are supported by the GPU engine in single-partition/in-memory execution, but are not supported by streaming multi-partition execution.

This issue should cover the common row-order expression variants that need boundary state across streaming partitions:

  • shift
  • diff
  • pct_change
  • forward_fill
  • backward_fill

Example:

def test_expr_shift_streaming(engine: GPUEngine) -> None:
    """``shift`` on the full frame.

    Status
    ------
    - In-memory OK
    - Streaming NOT supported
    """
    lf = pl.LazyFrame({"x": [1, 2, 3, 4, 5]})
    q = lf.select(
        pl.col("x").shift(1).alias("lag_1"),
        pl.col("x").shift(-1).alias("lead_1"),
    )
    assert_gpu_result_equal(q, engine=engine)

Metadata

Metadata

Assignees

No one assigned

    Labels

    cudf-polarsIssues specific to cudf-polarsfeature requestNew feature or request

    Type

    No type

    Projects

    Status
    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions