Add KHR_interactivity draft

[lexaknyazev]

Rendered version: https://github.com/KhronosGroup/glTF/blob/interactivity/extensions/2.0/Khronos/KHR_interactivity/Specification.adoc

[bhouston]

Great work! What sections are next? Also what could we release as a draft extension for SIGGRAPH 2023?

[robertdorn83]

@lexaknyazev

the term "index" for the input is not specific enough here. Which variable index? From the config array? Or the graph variable index?
Let's say whe have an config here with [3,6,9]. Would be the input names "3", "6", "9" OR "0", "1", "2" ?

[lexaknyazev]

@robertdorn83 The operation details located after the table make it quite clear that input value socket ids correspond to graph variable indices, that is 3, 6, and 9 in your example. We'll update the table for clarity.

[robertdorn83]

@lexaknyazev
Currently the readonly pointer for activeCamera contains only position and rotation. Would it make also sense to add matrix to it ? I know we can build a matrix out of position and rotation because its global space, but just for consistency with the node transforms.

Also, and we already talked about partly when i remember correctly, but it would be also more consistent to add the other gltf camera properties such as activeCamera.orthographic.xmag, activeCamera.orthographic.zfar, activeCamera.perspective.aspectRatio etc. These are all known standard gltf properties.
Also maybe adding activeCamera.isOrtho, activeCamera.viewMatrix, activeCamera.projectionMatrix.
My wishlist for christmas this year ;)

[Jmangles]

"goto to the step 11." Reads a bit odd. This step also instructs you to set isValid to false while Step 11 also instructs you to do the same

[lexaknyazev]

The step 11 checks if isValid is false; it doesn't set it.

[lexaknyazev]

The step 11 checks if isValid is false; it doesn't set it.

[lexaknyazev]

The step 11 checks if isValid is false; it doesn't set it.

[CLAassistant]

All committers have signed the CLA.

[robertdorn83]

@lexaknyazev
hey alexey, why have you added a "ref event" output on the onStart and onTick event? I have nothing in mind why we could need a reference to the event here. In Unity - and im sure in a lot of other engines too - such events don't even have a Event-Object to pass through.

[robertdorn83]

@lexaknyazev Some questions about the ref types:

• I wasn't able to find some information how to resolve a Ref (let's say a Node) to the graph node index. (can also be usefull to check if a ref belongs to the own gltf file)
  would it be with
  pointer/get (template: /node/{nodeRef}/index, type: int)
  or
  pointer/get (template: /node/{nodeRef}/, type: int) ?

• what when i want to get a mesh from a node: is /node/ []{} /mesh returning a index or ref now? Or depends it now how i setup the type in the node config? so type=Ref would give me the ref to the mesh, type=Int would give me the index? But the index would only work/is usefull, when the mesh belongs to the same gltf file as the graph.

• initial Values for Variables from type Ref or Input Sockets from type Ref: so we use here as a value just a string with /node/6/?

These pointer access cases and ref initial values seems to bemissing in the specs currently .. ?

[lexaknyazev]

how to resolve a Ref (let's say a Node) to the graph node index

That's impossible. Only the opposite direction is currently supported.

what when i want to get a mesh from a node: is /node/ []{} /mesh

All such pointers return refs, the Object Model is going to be updated.

initial Values for Variables from type Ref or Input Sockets from type Ref

Their syntax and semantics are defined in the Variables section (currently 5.3). When not set, the value is null; otherwise it must be a static (no parameters) JSON pointer string that will be resolved to a runtime reference value (if resolvable) or to null (if unresolvable) when the graph is loaded.

[robertdorn83]

I updated the Samples and Tests with the new Ref spec changes. They are currently in a separate branch: gltf Interactivity Assets
Was e bit more work than expected, so sorry for the delay :)