Future Work: Facial Action Coding System (FACS) for X3D HAnim models

These plans are based on work adapting models by Dr. Myeong Won Lee, as described in the README: Original FACS Models.

Lessons Learned, So Far

1. Every head will likely have different geometry, textures, colors, etc. There is no need for every head to be the same.
2. A single head model file needs to align with HAnim joints and segments, if it is going to be interoperable with other HAnim skeleton/skin models.
3. Each head model will connect to the skeleton as shown in

       Part 1:  Humanoid animation (HAnim) architecture, Section 4 Concepts, 4.9.6.3 LOA-3 hierarchy
   
       humanoid_root : sacrum
           [...]
               vc7 : c7
               | vc6 : c6
               |   vc5 : c5
               |     vc4 : c4
               |       vc3 : c3
               |         vc2 : c2
               |           vc1 : c1
               |             skullbase : skull
               |               l_eyelid_joint : l_eyelid
               |               r_eyelid_joint : r_eyelid
               |               l_eyeball_joint : l_eyeball
               |               r_eyeball_joint : r_eyeball
               |               l_eyebrow_joint : l_eyebrow
               |               r_eyebrow_joint : r_eyebrow
               |               temporomandibular : jaw
   

4. Typically a new head will connect at HAnimJoint skullbase and HAnimSegment skull, possibly including neck joints as well.
5. Typically an interoperable HAnimHumanoid head will include HAnimJoint and HAnimSegment definitions for eyelid, eyeball, eyebrow, and jaw.
6. The HAnim specification includes guidance on connecting partial body models together in 4.8.6 Partial HAnim figures.
7. No HAnimSite feature points are required in the HAnim specification.
8. We can add HAnimSite feature points to the above HAnimSegment notes, without interfering with any of the native HAnim motion animations. However adding HAnimSite locations to the other facial geometry many need further attention. Adding a useful set of HAnimSite feature points for facial animation might be useful.

Next Steps

1. We want head models that include the ability to use an FACS Action Unit (AU) enumeration that shows a new expression.
2. We currently have single expressions demonstrated in separate model files. Showing one expression at a time is the initial goal for facial expression animations.
3. Integrating the animations together efficiently, within a single X3D model file, is the path towards having interoperably animatable heads that can move eyes/jaw and also show expressions.
4. We first want single-AU responses working, and then multiple-AU responses working, in order to achieve the full range of FACS expressions that are possible.
5. For full animation compatibility, head models need to include the standard HAnim joints and segments for eyelids, eyeballs, eyebrows, and jaw. This requirement deserves careful consideration when designing mesh geometry.
6. See the FACS Visual Guidebook for video examples with corresponding descriptions that illustrate the expressions conveyed by each Action Unit (AU), either individually or in combination.

Building example models is the path towards showing repeatable capabilities. Conceivably these capabilities might become a new HAnim node for HAnimHumanoid. Successful, repeatable design patterns can then be formalized in the future HAnim Specification.

Online at https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation/FutureWork.html

Revised 28 October 2024