Skip to content

Enhance OpenVINO backend with view operations and new ops#157

Open
zhaixuejun1993 wants to merge 6 commits into
ravi9:dev_backend_openvinofrom
zhaixuejun1993:xuejun/enable_ops
Open

Enhance OpenVINO backend with view operations and new ops#157
zhaixuejun1993 wants to merge 6 commits into
ravi9:dev_backend_openvinofrom
zhaixuejun1993:xuejun/enable_ops

Conversation

@zhaixuejun1993
Copy link
Copy Markdown
Collaborator

This pull request adds support for the CONCAT, ARGSORT, and SOFTPLUS operations in the OpenVINO backend, and improves view op handling for several operators. It introduces new translation functions for these operations, updates operator tables and supported op lists, and refines how view ops are processed to improve compatibility and correctness.

New OpenVINO operator support:

  • Added support for GGML_OP_CONCAT (concatenation) and GGML_OP_ARGSORT (argsort) in the OpenVINO backend, including new translation functions (translate_concat and translate_argsort), operator table entries, and support checks. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]
  • Added support for the GGML_UNARY_OP_SOFTPLUS unary operation, including the translation function and operator table entry. [1] [2] [3] [4] [5]

View op handling improvements:

  • Updated translation functions for several operators (e.g., GET_ROWS, SILU, all 1-to-1 unary ops) to use the new process_view_input_new utility, ensuring correct handling of view ops as inputs. [1] [2] [3]
  • In the backend comparison logic, added a special case to skip view op checks for OpenVINO, which does not handle view ops directly.
  • Relaxed restrictions on view op inputs for unary ops in OpenVINO, now allowing them except for a small set of explicitly unsupported ops. [1] [2]

These changes expand the set of supported operations for OpenVINO and improve robustness and compatibility when working with view-based tensors.## Overview

Additional information

Requirements

…sult, VIEW node in OpenVINO no need compare, the whole graph result is correct
Comment thread ggml/src/ggml-backend.cpp
ggml_is_view_op(g1->nodes[i]->op)) {
verified = true;
continue;
}
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we avoid this? Hardcoding backend-specific checks for OpenVino into the GGML code isn't ideal

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can not avoid, the VIEW op is handled in following node, which can be fused maybe, so can not compare, it has to be isolated.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it required for a model support or unit test currently?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants