X3D V4 Open Meeting

From Web3D.org
Revision as of 13:31, 8 June 2016 by Walroy (Talk | contribs)

Jump to: navigation, search

X3D V4.0 Open Workshop / Meeting June 8th 2016


Topics

Note: The red question marks at the end of each question are place-holders for any short answers to emerge from the discussions.

  • What level of X3D integration into HTML5 do we want? ???
    • Do we want to be fully integrated like SVG? ???
  • Do we want/need a DOM spec? If so: ???
    • Which DOM version should it be based on? ???
    • Do we want to fully support all DOM/HTML features? ???
  • Do we want to maximize the backwards compatibility of V4.0 with V3.3? Or break away completely? ???
    • Do we want to retain SAI? ???
  • What features do we want? For example, ???
    • How is animation to be handled? The X3D way of TimeSensor and ROUTEs, or an HTML way, such as CSS3 animations, or else JavaScript? ???
    • How is user interaction to be handled? The X3D way of Sensors, or the HTML way with event handlers? ???
    • Do we need any different nodes? One example might be a mesh node? ???
    • Do we want Scripts and Prototypes in HTML5? ???
    • How do we want to handle styling? ???
  • What profile(s) do we need for HTML? ???

Attendees and contributors

E-mail contributors: Don Brutzman, John Carlson, Leonard Daly, Andreas Plesch, Philipp Slusallek

Meeting Attendees:

Apologies: Don Brutzman, Andreas Plesch


Prior e-mail contributions:

Note 1: Contributions are presented in chronological order. Note 2: Links within the preformatted sections do not appear. So any used have been added as references at the end of the contribution.

Contribution 1

I think the bigger question of what should be done with X3D. Is X3D solely going to exist within HTML or will X3D have a separate life inside and outside of HTML.

If the life is solely within HTML, then the questions below become inclusive of all X3D. If there are separate existences, then the first question is what is the cross-compatibility between X3D/HTML and X3D/other.

Contribution 2

Relevant working-group references follow.  A lot of excellent work has been accomplished already.

	X3D Version 4
	http://www.web3d.org/x3d4

	Web3D Consortium Standards Strategy
	http://www.web3d.org/strategy

	X3D Graphics Standards: Specification Relationships
	http://www.web3d.org/specifications/X3dSpecificationRelationships.png

	X3D Version 4.0 Development
	http://www.web3d.org/wiki/index.php/X3D_version_4.0_Development

A 5-10 minute quicklook discussion across these resources might help.  We are pretty far up X3D4 Mountain already!

The posted discussion-topics list is a good start for renewed activity, and an important way to keep track of everyone's many valuable ideas.  Suggestion: create some kind of topics-discussion page, probably easily linked off the preceding wiki page.

My general inputs for each of these topics are guiding questions:

a. What do the HTML5/DOM/CSS/SVG/MathML specifications actually say?

b. How is cross-language HTML page integration actually accomplished, as shown in best practices by key exemplars?

c. What is the minimal addition needed to achieve a given technical goal using current X3D capabilities?

Editorial observation: the word "want" appears 9 times in this list...  Understandable from common usage, but not a very good way to achieve consensus over a long-term effort.  Also not very useful for measuring successful resolution.

Pragmatic engineering rephrase: "what problem are you trying to fix?"

Over 20 years of successful working-group + community efforts can guide us in these endeavors - we know how to succeed together.  An effective path for building consensus is to:
- define goals that are illustrated by use cases,
- derive technical requirements,
-  perform gap analysis, and then
-  execute loosely coordinated task accomplishment according to each participant's priorities.

How to execute each specification addition: write prose, create examples, implement, evaluate. Repeat until done, topic by topic.

References:

Contribution 3

The discussion on introducing an id field seemed to point towards the need to have fuller integration in the sense that it is difficult to isolate features. It may be necessary to define a x3d dom similar to the svg dom, with the corresponding interfaces. svg is very successful on the web but it took a long time to arrive there.

x3dom has a dual graph approach. There is the x3d graph and in parallel the page dom graph which are kept in sync but are both fully populated. Johannes  Behr would know better how to explain the concept.
It looks like FHG decided that x3dom is now considered community (only?) supported. This probably means it will be out of sync as newer web browsers arrive, or webgl is updated.

I explored Aframe a bit more. It will be popular for VR. It is still in flux and evolves rapidly. The developers (mozilla) focus on its basic architecture (which is non-hierarchical, a composable component system) and expects users to use javascript to develop more advanced functionality (in the form of shareable components). So it is quite different, fun for developers, and for basic scenes easy for consumers. Since most mobile VR content at this point is basic (mostly video spheres and panos), it is a good solution for many.

(As a test I also implemented indexedfaceset as an Aframe component, and it was pretty easy - after learning some Three.js. So it would be possible to have x3d geometry nodes on top of aframe. Protos, events and routes are another matter but also may not be impossible).

There is still space for x3d as a more permanent, and optionally sophisticated 3d content format on the web.

Event system: My limited understanding is that on a web page, the browser emits events when certain things happen. Custom events can also be emitted by js code (via DOM functions) for any purpose. (All ?) events have a time stamp and can have data attached. Then, events can be listened to. There is no restriction to listening, eg. all existing events are available to any listener. A listener then invokes a handler which does something related to the event. js code can consume, cancel, or relay events as needed (via DOM functions). It is not unusual that many events are managed on a web page. events can be used to guarantee that there is a sequence of processing.

So how does the x3d event system relate ? There is a cascade, and directivity. How long does an event live ? one frame ? Until it fully cascaded through the scene graph ?

Since x3dom and cobweb are currently the only options, from a practical stand point a question to ask may be this: what is needed to make x3dom and cobweb easy to use and interact with on a web page ? Typically, the web page would provide an UI, the connection to databases or other sources of data, and the x3d scene is responsible for rendering, and interacting with the 3d content. For VR, the UI would need to be in the scene, but connections and data sources would still be handled by the web page.

Cobweb in effect allows use of the defined SAI functions. Is it possible to define a wrapper around these functions to allow a DOM like API (createElement, element.setAttribute .. element = null) ? It may be since they are similar anyways and it would go a long way. But it still would not be sufficient to let other js libraries such as D3.js or react control and modify a scene since they would expect x3d nodes to be real DOM elements.

VR: A current issue is control devices. It would be probably useful to go over the spec. and see where there is an implicit assumption that mouse or keyboard input is available. VR HMDs have different controls (head position and orientation(pose), one button) and hand held controllers (gamepads, special sticks with their own position/orientations) or the tracked hands themselves become more popular. In VR, you do want to your hands in some way.

Perhaps, it makes sense to have <Right/LeftHand/> nodes paralleling <Viewpoint/> with position/orientation fields which can be routed to transforms to manipulate objects ? How  a browser would feed the <Hand> nodes would be up to the browser. InstantReality has a generic IOSensor.

Contribution 4

I am not sure that I will be able to join the meeting, so let me present some of our ideas in this context by email already. As you might know we have done a lot of work on declarative 3D -- in the alternate universe called XML3D. The reason for this unfortunate split, lies in the fact that it has been rather difficult to bridge the gap between our ideas of a minimum and generic extensions to HTML-5 to enable declarative 3D while staying as close as possible to the current Web technology stack on the one side and the need for backward compatibility that people strived for in X3D(OM).

Now, there may be a quite interesting these two world views could be resolved with each other. The basic idea for that was already proposed and discussed between us and the X3DOM group: Merging X3DOM and XML3D by identifying the (significant and large!) common core and a layering X3D/XMl3D compatible interfaces on top of this (or eventually identifying common set of Dec3D elements). Unfortunately, this idea was not really picked up by anyone back then.

However, with newer technology like WebComponent (a version of prototypes in HTML5) this now becomes a highly interesting and much more practical option. We are actually exploring this right now. First results look very promising and a first paper on this has just been accepted for Web3D this year.

At the core of this approach we are using a clean interface to rendering engines, where we use Three.js a default option. Other engines like game engines or ray tracing etc. would be alternatives. (We are also exploring server-based real-time ray tracing based on a generic real-time synchronization layer between scene descriptions as one example.)

On top of this is a slightly refined generic data handling layer that is derived from our Xflow. Xflow has been tremendously useful for us and makes data management significantly easier than with the specialized nodes in X3D. It is the perfect building block for WebComponents. We are also integrating into this layer a programmable data processing engine that is derived from our flexible shade.js compiler for programmable shading.

Everything on top of this is essentially fair game for WebComponents (similar to a-frame, but with mny more options). The more specialized and often domain-specific nodes from X3D would be prime examples for this. We have actually started to implemented some of the basic X3D nodes already, plus the non-core XML3D nodes. Pretty much along the lines of what we discussed with X3DOM several years ago. Given the powerful underlying engine, it actually becomes rather straight forward to implement these nodes as Web Components, especially the X3DOM subset.
But we are just starting.

We are even exploring having a public repository of WebComponents that people could develop independently and that get automatically loaded when referenced in a scene (subject to some security policy, of course).
Talk about leveraging the power of the distributed web :-).

We are finalizing the paper for the final version right now but can make a preprint available as soon as this is ready. People would be more than welcome to help design and develop this further. Maybe this could also be an interesting basis for some of the work on X3D V4?

Contribution 5

As repeated a few days ago, I remain keen for us to establish some kind of X3D v4 wiki page.  We need to collaboratively show how these different categories of concepts can be defined along with their associated pros and cons.  Email and meeting discussions can then grow those sections effectively, to show that use-case goals and design requirements can be defined and met.

Wondering if the x3dom dual-graph page dom structures align with the draft work on Shadow DOM at W3C.
https://www.w3.org/TR/shadow-dom

I'm hoping that John Carlson gets a chance to look at whether his JSON prototype expander might be adaptable as part of x3dom - that would be pretty valuable.  Although Fraunhofer has outstanding prototype support in their Instant Reality engine, it has never been clear why that code can't simply be applied in x3dom as well.  Certainly a consistent approach would seem to make both codebases more coherent and maintainable for them.  If Fraunhofer refuses to adapt or release the Instant Reality prototype code then we will just have to do it ourselves... John's work seems like a big step in that direction.

Also thanks for pointing out important questions about Fraunhofer's stewardship of the X3DOM project.  It will be good to learn more about their intentions so that our community can align effectively.  There has been no handoff.

Regarding the X3D event system: a changing value of any field in any node can be sent as an input to any field in any node, as long as types strictly match (apples to apples).  Rephrased: a ROUTE passes a time-stamped value from one node to another. Internal to an X3D scene graph, that has been implemented dozens of times.  Seems extremely simple.

External to an X3D scene graph, meaning via a browser using the Scene Access Interface (SAI), mechanisms are similarly well defined.  Since the DOM is string based, and since any X3D event value can be expressed as a string, it seems like we have a straight connect-the-dots approach awaiting us.  Forgive me for using a four-letter word, but if interested individuals might actually _read_ the HTML5/DOM and X3D specifications, then the answers to most implementation & alignment questions are likely spelled out for us.

Reference:

Contribution 6

Please do not go the route (sic!) of a string-based interface for implementing X3D routes. Yes, the DOM has a generic string-based interface, which is really important in general. But not for efficiently handling big 3D data. Any DOM node can additionally provide a "binary"
JS API as well, ideally using Typed Aarrays in JS.

Converting to strings and back will cause huge overhead and will rule out any GPU-based computation and acceleration. The latter is a must in today's environments, especially on mobiles. You do not want to create this overhead for large arrays of vertices or such and have to parse all the numbers again and again. It can also cause numerical inaccuracies in the conversion that may lead to inconsistencies in the binary representation, which can cause gaps in supposedly closed geometry.

BTW, this is exactly why we have created Xflow: To efficiently be able to specify generic typed data arrays (available as GPU buffers in the engine as early as possible), flexibly composite individual buffers into sets of buffers (<data> elements that are define all the data the input for efficient draw calls), and also to process the data as necessary along the way (e.g. flexible animation, image processing, procedural shading, transitions, etc.).

Xflow is actually much more powerful than routes and it fits much better to HTML5 -- in my opinion at least. Funded by Intel we are just extending Xflow to automatically make use of e.g. SIMD instructions (via
SIMD.js) and other JS acceleration techniques. We are also looking at WebAssembly here for better performance even if not going to the GPU.

Contribution 7

Wondering if the x3dom dual-graph page dom structures align with the draft work on Shadow DOM at W3C.
https://www.w3.org/TR/shadow-dom

The shaw-dom spec is being upstreamed to the dom spec.

https://dom.spec.whatwg.org/
 
This is a good resource, since it also explains the web page DOM in detail.

I am not sure if the shaw-dom aligns with the dual graph approach because the shadow-dom is still a dom structure and not a scene graph. I think it may be still necessary to define a x3d dom and interfaces similar to https://developer.mozilla.org/en-US/docs/DOM/DOM_Reference#SVG_interfaces
I'm hoping that John Carlson gets a chance to look at whether his JSON prototype expander might be adaptable as part of x3dom - that would be pretty valuable.  Although Fraunhofer has outstanding prototype support in their Instant Reality engine, it has never been clear why that code can't simply be applied in x3dom as well.  Certainly a consistent approach would seem to make both codebases more coherent and maintainable for them.  If Fraunhofer refuses to adapt or release the Instant Reality prototype code then we will just have to do it ourselves... John's work seems like a big step in that direction
 
Well, I would not have a good idea on how to implement prototypes in x3dom. Not sure if the Instant Reality code would really help. One idea may be to see if there is an existing declarative way to register new custom elements using web components, and extend that. But I do not think there is.
https://www.w3.org/TR/custom-elements/

 
Also thanks for pointing out important questions about Fraunhofer's stewardship of the X3DOM project.  It will be good to learn more about their intentions so that our community can align effectively.  There has been no handoff.

Well, there have been no fixes or updates to the code base on github for a long time and a recent issue response suggested that FHG considers x3dom community supported.
 
Regarding the X3D event system: a changing value of any field in any node can be sent as an input to any field in any node, as long as types strictly match (apples to apples).  Rephrased: a ROUTE passes a time-stamped value from one node to another. Internal to an X3D scene graph, that has been implemented dozens of times.  Seems extremely simple. 

External to an X3D scene graph, meaning via a browser using the Scene Access Interface (SAI), mechanisms are similarly well defined.  Since the DOM is string based, and since any X3D event value can be expressed as a string, it seems like we have a straight connect-the-dots approach awaiting us.  Forgive me for using a four-letter word, but if interested individuals might actually _read_ the HTML5/DOM and X3D specifications, then the answers to most implementation & alignment questions are likely spelled out for us.

https://dom.spec.whatwg.org/#events would be HTML5 side of events.
Is the statement at https://dom.spec.whatwg.org/#action-versus-occurance compatible with http://www.web3d.org/documents/specifications/19775-1/V3.3/Part01/concepts.html#Events ?

It will be great to get our wiki organized for clarity and go-forward action on these important X3D version 4 design issues.

Thanks for letting us know about your other explanations - very interesting!  8)

Yeah, VR is not really part of this discussion (although it could be a V4 topic) .
 
Looking forward to continuing, sustainable evolution and progress together.

I think if there is way to open spec. drafting more along the lines of other www spec. efforts on github, there is a chance to attract a wider community for more man power which would be much needed.

References:

Contribution 8

Before we can answer these questions, we need to know who the V4.0 targetted audience is.  Who are V4.0’s users?  Purely web developers and users?  Geo?  Medical?  CAD? H-Anim? Which working groups are involved?

Contribution 9

    X3D needs to run in the HTML5/DOM environment. A few nodes need to be removed, but all capabilities remain.
    Preliminary proposed V4 document at: http://tools.realism.com/specification/x3d-v40

I am going to start my position with a response to a question asked by John Carlson on a different list (x3dom-users): are we adding HTML5 capabilities to X3D or 3D (X3D in particular) capabilities to HTML5?
HTML is the dominant environment world-wide. It provides text, image, 2D graphics (SVG), video, and other capabilities. The size of the HTML5 development community far exceeds the total of the entire X3D community. Forcing HTML5 into X3D is a losing game right from the start -- whether you want all of HTML5 or just a portion. So in my mind, the only choice with a future is to add 3D to HTML5. Running in the HTML5 environment means full integration with the HTML5 DOM (or later versions when they happen). BTW, there are already a number of non-Web3D Consortium efforts to do so. We are not out in front of the effort and are about to be made irrelevant. There is no more time for delays or debates.
So now that the environment is settled, it is important to identify what in current X3D (V3.3) is incompatible with HTML5. There are only three obvious features - Script node, event handling, and case sensitivity. There are other capabilities that are dependent on these capabilities -- I'll discuss those later.

Starting with the easiest one first - case sensitivity. HTML5 is case insensitive. Relaxing X3D's rules on that allows existing X3D code to run in a browser. If everything gets converted to lower-case prior to handling (except quoted strings), then there is not a problem.

There is an obvious naming incompatibility with Script -- the name. HTML5 is already using that name. Under my initial condition there cannot be an X3D Script node. That does not mean all scripting functions are given up. HTML5 provides a wonderful script interface a more flexible structure. In X3D, Scripts are meant to process events, so the function argument is always an event (except for X3D-Script internal functions). Functions in HTML5 are a lot more flexible and can include events, objects, scalars, arrays, etc. So there is no loss of functionality by giving up X3D-Script.

Event handling is different between HTML5 and X3D. In X3D events are "routed" from one node to another. They allow one part of the scene graph to "talk" to another part. In HTML5, events "bubble-up" from the originator to the event through any handler that may be attached to any parent node of the originator until the event is cancelled. In all of my design work on V4 I have not found any instance where HTML5-Scripts could not provide the same functionality as X3D-Script+ROUTE. It requires a little different mind-set, but the HTML5 mind-set is very familiar to JavaScript programmers and other front-end developers. I also believe that a graphical development interface can be built that completely simplifies the differences.

The biggest issue I have seen with event handling is scene graph updating. X3D updates the scene graph once all non-looping events in the cascade have completed. After the scene graph is updated, a new frame is rendered. This can cause a large delay between rendered frames. HTML5 renders as it goes. Rendering happens asynchronously to changes to the DOM. There is no concept of accessing the DOM before or after all events for that frame. X3D worlds that depend on that feature will probably not be able to be ported to X3D V4.

Summarizing the three incompatibilities - with the exception of some event processing, none will prevent X3D from doing what it currently can do and all can be easily migrated to the HTML5 environment.


There are a number of features that I think should not be included in X3D V4, but these are just features and not fundamental capabilities. These include all nodes that generate geometry (e.g., Extrusion, ElevationGrid, Text) with the exception of simple solids and perhaps a couple of additions. My view here is based on the availability of free modeling software (e.g., Blender) that does all of the above, and a lot more efficiently than X3D can. Also by not including these nodes, the resultant models will look better.

Lastly (for now), I believe that there is no purpose for a PROTO node. Without a X3D-Script node, PROTOs just become convenience generators. To replace that feature, I am proposing a MACRO node that takes X3D and does string substitution prior to inserting the result into the scene graph (and DOM). I have a partial implementation of this for X3DOM. 

Summarizing: The Consortium needs to get out and lead the way for 3D on the web (and this includes VR) or it will be by-passed and left with the relics of history like blinking text, and Flash. The environment must be HTML5/DOM and X3D must stay current with the web environment. There will always be someone who needs something specialized that does not use a web environment, but those will be individual cases and not worth significant volunteer efforts.

Reference:

Contribution 10

•         What level of X3D integration into HTML5 do we want?
o   Do we want to be fully integrated like SVG?

In the spirit of openness, I say may the best implementation win. That means that there should not be full integration.  However, there should be a standard.


•         Do we want/need a DOM spec? If so:
o   Which DOM version should it be based on?

HTML5???

o   Do we want to fully support all DOM/HTML features?

We should document what tags and attributes we are adding.   If not DOM, what declarative language do you want to use? CSS may be an option
•         Do we want to maximize the backwards compatibility of V4.0 with V3.3? Or break away completely?
o   Do we want to retain SAI?

Not retain SAI.   Not be backwards compatible.


•         What features do we want? For example,
o   How is animation to be handled? The X3D way of TimeSensor and ROUTEs, or an HTML way, such as CSS3 animations, or else JavaScript?

HTML way

o   How is user interaction to be handled? The X3D way of Sensors, or the HTML way with event handlers?

HTML way and user can add own event emitters.  We can also add JavaScript emitters which can be added as attributes.

o   Do we need any different nodes? One example might be a mesh node?

We need a way to provide templates or macros, which might be provided through picking an existing one.

o   Do we want Scripts and Prototypes in HTML5?

No.


o   How do we want to handle styling?

By declaratively addressing the fragment shader on objects.


•         What profile(s) do we need for HTML?

IDK
Retrieved from "https://web3d.org./wiki/index.php?title=X3D_V4_Open_Meeting&oldid=9346"