Skip to content

Enable OpenVINO backend fallback to CPU backend#130

Open
zhaixuejun1993 wants to merge 25 commits into
ravi9:dev_backend_openvinofrom
zhaixuejun1993:xuejun/openvino-fallback-cpu-v1
Open

Enable OpenVINO backend fallback to CPU backend#130
zhaixuejun1993 wants to merge 25 commits into
ravi9:dev_backend_openvinofrom
zhaixuejun1993:xuejun/openvino-fallback-cpu-v1

Conversation

@zhaixuejun1993
Copy link
Copy Markdown
Collaborator

This pull request introduces enhancements to tensor tracking and backend scheduling in the GGML library, primarily to improve handling of in-place operations and debugging. The changes add a new field to the tensor structure for tracking original source tensors, propagate this information through backend scheduling and OpenVINO decoder logic, and improve debugging by modifying tensor naming.

Tensor structure and tracking improvements:

  • Added an org_src pointer to the ggml_tensor struct to track the original source tensor in in-place operations, and increased the padding size to accommodate this change. (ggml/include/ggml.h, ggml/src/ggml.c) [1] [2]
  • When copying tensors during backend graph splitting, the org_src field is set to the source tensor, preserving provenance information. (ggml/src/ggml-backend.cpp)

Backend scheduling and debugging:

  • Improved backend assignment for view operations by ensuring backend IDs are propagated from view sources when supported, and by updating view nodes' backend IDs when necessary. (ggml/src/ggml-backend.cpp) [1] [2]
  • Enhanced debugging by appending node IDs to tensor names during backend scheduling, making it easier to trace tensors in logs or debugging output. (ggml/src/ggml-backend.cpp)

OpenVINO decoder logic:

  • Updated the OpenVINO decoder to utilize the new org_src field when determining dynamic dimensions and input/output relationships, ensuring correct handling of tensors originating from in-place operations. (ggml/src/ggml-openvino/ggml-decoder.cpp, ggml/src/ggml-openvino/ggml-decoder.h) [1] [2] [3]## Overview

Additional information

Requirements

cavusmustafa and others added 25 commits April 8, 2026 17:13
…lm-params

Fix LLM parameter computation error in normal softmax
@wine99 wine99 force-pushed the dev_backend_openvino branch from 36daf24 to c6e06ee Compare April 22, 2026 09:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants