Difference between revisions of "X3D Medical"
From Web3D.org
(→Papers, Tutorials and Presentations) |
|||
| (59 intermediate revisions by 4 users not shown) | |||
| Line 1: | Line 1: | ||
| − | =X3D and Volume Rendering | + | === The Medical Working Group === |
| + | The International Standards Organization (ISO) standard for 3D graphics over the Internet is Extensible 3D (X3D), which is maintained and developed by the Web3D Consortium. The initiative of the Web3D Consortium’s | ||
| + | Medical Working Group (MWG) is to specify and implement MedX3D – an extension to the open and royalty-free X3D standard to support advanced medical visualization functionality and medical data exchange (for more information see [[#MedX3D: X3D and Volume Rendering|MedX3D: X3D and Volume Rendering]]). '''The MWG has specified and demonstrated cross-platform volume rendering styles (i.e., transfer functions), segmentation and ontology support, and data import/export capabilities for interactive presentation'''. | ||
| − | The [http:// | + | The Medical Working Group is an interdisciplinary effort. The different backgrounds of the members range from medical subject matter experts, over computer scientists from academia to engineers and experts from industry. Thus potential users and future providers are involved as well as experts to work on technical solutions. In November 2012, the Web3D Consortium Medical Working Group released its [http://web3d.org/wiki/images/a/a2/Portable_and_Interoperable_Views_II.pdf 2012 Opportunities Whitepaper: Portable and Interoperable Views of Medical Image Data with ISO Extensible 3D (X3D)], outlining the path forward for cross-platform, reproducible volume rendering and the health care enterprise. |
| − | + | ==X3D and Volume Rendering== | |
| + | The reproduction of volume-rendered presentations of medical image data across platforms and the healthcare enterprise presents several challenges, especially due to data and view incompatibilities and lock-in to proprietary systems. But, explicit 3D visual presentations of medical images can provide significant advantages because this type of rendering is more truly representational of the object being imaged (the human body)- it is a more intuitive and easily-read format. It is increasingly common to render a three dimensional (3D) model from a CT, MRI, PET and X-Ray scan to better interpret the size, orientation and other spatial relationships of the patient’s anatomy as necessary for diagnosis, intervention (surgery) and therapy. | ||
| − | + | Until recently, there was little hope of interoperability for interactive 3D and 4D presentations to break out of the hospital PACS and to be archived and shared across the enterprise. With the continual advancement in computing and graphical power over the last decade, specialized workstations and software capacity has become available to display this type of 3D imaging on a common laptop. It is an imminent future when the handheld tablets on the market are capable of sustained hardware-accelerated graphics performance. | |
| + | |||
| + | Our original work (Web3D.org) for '''TATRC ([http://www.web3d.org/documents/medical/2008/Web3D_TATRC_FinalReportPublic.pdf W81XWH-06-1-0096])''' developed and demonstrated the integration of expressive volume rendering with X3D over the web with several client platforms. This set of functionalities was validated by industry experts and formalized into a specification with two separate, multi-platform implementations. The new component includes an expressive range of volume rendering styles as well as means to assign separate styles to different segments, and to create isosurfaces within the volume. In 2012, this specification has ultimately become an official part of [http://www.web3d.org/realtime-3d/specification/active ISO X3D 3.3]. | ||
| + | |||
| + | Much of the required functionality is specified in the X3D 3.3 draft International Standard, including the [http://www.web3d.org/files/specifications/19775-1/V3.3/Part01/components/texture3D.html Texturing3D Component (Clause 33)] and the [http://www.web3d.org/files/specifications/19775-1/V3.3/Part01/components/volume.html Volume Rendering Component (Clause 41)] to support several compose-able styles for Volume Rendering for Medical Imaging, geology and other non-invasive sensing modalities. A Medical Interchange Profile of X3D nodes is also defined in Annex L : http://www.web3d.org/files/specifications/19775-1/V3.3/Part01/MedInterchange.html. '''The node set of the X3D 3.3. Medical Interchange Profile collects nodes for volume and polygon rendering, lighting, text and animation; it has been demonstrated to meet the requirements of several key clinical and research applications including diagnosis, surgical planning, education and training and informed consent.''' | ||
| + | |||
| + | The [http://www.web3D.org/x3d/workgroups/medical Medical Working Group] is participating in the [http://medical.nema.org/ DICOM Working Group 11] for the purpose of defining a presentation standard for reproducible Medical Imaging. | ||
| + | * See also [http://web3d.org/wiki/images/a/a2/Portable_and_Interoperable_Views_II.pdf 2012 Opportunities Whitepaper: Portable and Interoperable Views of Medical Image Data with ISO Extensible 3D (X3D)] | ||
| + | * See also [http://mhsquadrupleaimchallenge.challenge.gov/submissions/13264-archival-and-web-based-visualization-of-volumetric-scans-and-clinical-data-for-tbi-using-the-x3d-international-standard Archival and Web-based Visualization of Volumetric Scans and Clinical Data for TBI using the X3D International Standard], a research proposal to the U.S. Military Health System Challenge | ||
| − | * A presentation | + | ==X3D Volume Rendering Examples & Videos== |
| + | * A video of an X3D presentation of a [http://www.youtube.com/watch?v=ZO3jWjW9soE&list=UUoQklIQuVbdKEBqgefLbhzw&index=2 segmented MRI of a human head and brain] (from Virginia Tech) | ||
| + | * Fill out a Survey (w/ videos) about [https://survey.vt.edu/survey/entry.jsp?id=1355866408333 Multi-channel microscopy volume rendering techniques in X3D] | ||
| + | * A video of [http://www.youtube.com/watch?v=mI7zfrH6A9U&list=UUoQklIQuVbdKEBqgefLbhzw&index=1 compiled X3D volume rendering examples] from Virginia Tech (rendered w/ H3D.org) is available here | ||
| + | * [http://www.web3d.org/x3d/content/examples/Basic/index.html X3D Examples Archive: Medical Imaging / Volume Rendering]; See also additional [https://snoid.sv.vt.edu/medical/X3DVolumes/videos/ videos] and [https://snoid.sv.vt.edu/medical/X3DVolumes/images/ images] of X3D Volume RenderStyles (from Virginia Tech). | ||
| − | * | + | ==Tools== |
| + | * [https://savage.nps.edu/X3D-Edit/ X3D-Edit 3.3] supports the Texturing3D and Volume Component nodes, including validation by DTD and Schema | ||
| + | * [http://www.h3d.org Haptics 3D Toolkit] by Sensegraphics | ||
| + | * [http://www.instantreality.org/ Instant Reality] | ||
| + | * [http://www.x3dom.org MedX3DOM] and [http://volumerc.org/ VolumeRC ] by Vicomtech | ||
| + | * [[NRRD]] volume data format | ||
| − | * [http://www.web3d.org/membership/login/memberwiki/index.php/Medical Medical Working Group members only wiki] | + | ==Papers, Tutorials and Presentations== |
| + | * SIGGRAPH 2018 BOF [http://www.web3d.org/session/scaling-3d-medical-applications-people-everywhere] | ||
| + | * Web3D 2017 paper about community-driven extensions : Ander Arbelaiz, Aitor Moreno, Luis Kabongo, Nicholas Polys, and Alejandro García-Alonso. 2017. Community-driven extensions to the X3D volume rendering component. In Proceedings of the 22nd International Conference on 3D Web Technology (Web3D '17). ACM, New York, NY, USA, Article 1, 9 pages. DOI: [https://doi.org/10.1145/3055624.3075945] | ||
| + | * Web3D 2016 paper about comparing volume and surface renderings in X3D: [Nicholas F. Polys and Abhijit A. Gurjarpadhye. 2016. Tradeoffs in multi-channel microscopy volume visualization: an initial evaluation. In Proceedings of the 21st International Conference on Web3D Technology (Web3D '16). ACM, New York, NY, USA, 187-188. DOI: https://doi.org/10.1145/2945292.2945323] | ||
| + | * SIGGRAPH 2016 BOF [http://www.web3d.org/sites/default/files/presentations/Visualization%3B%20Brain%20Imaging%3B%20Surgical%20Training/Health-Medical.pdf] | ||
| + | * SIGGRAPH 2015 [http://metagrid1.sv.vt.edu/~npolys/SIGGRAPH2015/MedicalandVolumeVisualizationBOF_SIGGRAPH2015.pdf Volume Rendering and Medical Visualization] | ||
| + | * [http://www.osehra.org/ OSEHRA (OpenSource VistA)] Summit 2014: [https://vis.arc.vt.edu/~npolys/Web3D/OSEHRA/OSEHRA_NPolys.pdf X3D Scene Graph Structures for Medical Image Metadata] by Nicholas F. Polys & Abhijit A.Gurjarpadhye | ||
| + | * SIGGRAPH 2014 Birds-Of-a-Feather (BOF) session: [https://vis.arc.vt.edu/~npolys/Web3D/Siggraph2014/S2014_BOF_Medical_NPolys.pdf Volume Rendering and Medical Visualization] by Nicholas Polys | ||
| + | * [http://www.nextmed.com/ Medicine Meets Virtual Reality] (Feb. 2014) Session 'Extending Extensible 3D (X3D): from Haptic-BasedMedical Training to Clinical Applications' Thursday Feb 20th | ||
| + | ** Michael Aratow: Medical Informatics, CEP America [http://metagrid1.sv.vt.edu/~npolys/Web3D2014/MMVR2014/MMVR_Aratow.pdf A Health IT Perspective on X3D] | ||
| + | ** Timothy Coles: The Australian e-Health Research Centre & Surgical Simulation and Planning Team, CSIRO [http://metagrid1.sv.vt.edu/~npolys/Web3D2014/MMVR2014/MMVR_TColes.pdf X3D in Medical Training & Simulation] | ||
| + | ** Tommy Forsell: SenseGraphics AB [http://metagrid1.sv.vt.edu/~npolys/Web3D2014/MMVR2014/MMVR_Forsell.pdf H3D In Real Life – Medical Simulation] | ||
| + | ** Nicholas F. Polys: Advanced Research Computing, Virginia Tech [http://metagrid1.sv.vt.edu/~npolys/Web3D2014/MMVR2014/MMVR_Polys.pdf Volume Rendering and Lossless Metadata with X3D] | ||
| + | ** Felix G. Hamza- Lup: Computer Science and Information Technology, Armstrong Atlantic State University [http://metagrid1.sv.vt.edu/~npolys/Web3D2014/MMVR2014/MMVR_HamzaLup.pdf Radiation Therapy Training with X3D] | ||
| + | * [http://metagrid2.sv.vt.edu/~npolys/Web3D2013/Med_Vol_Tutorial_Web3D2013_Merged.pdf Web3D 2013 Tutorial:Medical Imaging and Volume Rendering in X3D ] | ||
| + | * [http://www.web3d.org/wiki/images/f/ff/Wivi2013_print_v2%281%29.pdf "A Fresh Look at Immersive Volume Rendering: Challenges and Capabilities"], Polys, Ullrich, Evestedt, Wood and Aratow. IEEE Virtual Reality Workshop on Immersive Volumetric Interaction, 2013. | ||
| + | * [http://iacis.org/iis/2012/19_iis_2012_40-50.pdf "New Platforms for Health Hypermedia" by Polys and Wood], Issues in Information Systems. Vol 13 (1); pp 40-50, 2012. ''This is a good overview of the X3D capabilities and potential for Health Informatics.'' | ||
| + | * Web3D 2012 Tutorial [http://web3d2012.org/program.html#tutorial3 X3D Volume Visualization and Medical Applications] Download the slides! | ||
| + | * SIGGRAPH 2012 Birds-Of-A-Feather (BOF) '[https://vis.arc.vt.edu/~npolys/Web3D/Sigraph2012/SIGGRAPH2012BOF.pdf Medical and Volume Rendering with X3D]' by Polys, Aratow and Kabongo | ||
| + | * Best Paper Award at IEEE VR 2012: [http://www.web3d.org/realtime-3d/news/best-paper-award-ieee-vr-2012 Haptic Palpation for Medical Simulation in Virtual Environments] | ||
| + | * A presentation made at the SIGGRAPH 2011 Medical BOF is available [https://snoid.sv.vt.edu/medical/X3DVolumes/nD_X3D_2011_polysSIGGRAPH.pdf here] | ||
| + | * Ullrich, S., T. Kuhlen, N. F. Polys, D. Evestedt, M. Aratow, and N. W. John, "Quantizing the Void: Extending Web3D for Space-Filling Haptic Meshes", Medicine Meets Virtual Reality (MMVR), vol. 163, Newport Beach CA, USA, IOS Press, pp. 670-676, February, 2011. | ||
| + | * Proceedings of the [http://www.hpv.cs.bangor.ac.uk/vr10-med/ IEEE VR 2010 Medical Workshop] | ||
| + | * John, N. W., M. Aratow, J. Couch, D. Evestedt, A. D. Hudson, N. Polys, R. F. Puk, A. Ray, K. Victor, and Q. Wang, "MedX3D: Standards Enabled Desktop Medical 3D", Medicine Meets VR (MMVR), 2008. | ||
| + | ---- | ||
| + | * N.W. John, "Design and Implementation of Medical Training Simulators", Virtual Real. 12, 4 (Dec. 2008), 269-279. | ||
| + | * F.P. Vidal, N.W. John, A.E.Healey, D.A. Gould, "Simulation of Ultrasound Guided Needle Puncture using Patient Specific Data with 3D Textures and Volume Haptics", Computer Animation and Virtual Worlds. Vol. 19, Issue 2, pp111-127, May 2008, Online ISSN: 1546-427X , Print ISSN: 1546-4261, | ||
| + | * N. W. John, I.S. Lim, "Cybermedicine Tools for Communication and Learning", Journal of Visual Communication in Medicine, 2007; 30(2): 4-9. | ||
| + | * Polys, N., D. Bowman, C. North, R. Laubenbacher, and K. Duca, "PathSim Visualizer: An Information-Rich Virtual Environment for Systems Biology", Web3D Symposium, Monterey, CA, ACM Press, 2006. | ||
| + | ---- | ||
| + | '''[http://www.web3d.org/membership/login/memberwiki/index.php/Medical Medical Working Group members only wiki]''' | ||
Latest revision as of 02:33, 27 September 2018
Contents
The Medical Working Group
The International Standards Organization (ISO) standard for 3D graphics over the Internet is Extensible 3D (X3D), which is maintained and developed by the Web3D Consortium. The initiative of the Web3D Consortium’s Medical Working Group (MWG) is to specify and implement MedX3D – an extension to the open and royalty-free X3D standard to support advanced medical visualization functionality and medical data exchange (for more information see MedX3D: X3D and Volume Rendering). The MWG has specified and demonstrated cross-platform volume rendering styles (i.e., transfer functions), segmentation and ontology support, and data import/export capabilities for interactive presentation.
The Medical Working Group is an interdisciplinary effort. The different backgrounds of the members range from medical subject matter experts, over computer scientists from academia to engineers and experts from industry. Thus potential users and future providers are involved as well as experts to work on technical solutions. In November 2012, the Web3D Consortium Medical Working Group released its 2012 Opportunities Whitepaper: Portable and Interoperable Views of Medical Image Data with ISO Extensible 3D (X3D), outlining the path forward for cross-platform, reproducible volume rendering and the health care enterprise.
X3D and Volume Rendering
The reproduction of volume-rendered presentations of medical image data across platforms and the healthcare enterprise presents several challenges, especially due to data and view incompatibilities and lock-in to proprietary systems. But, explicit 3D visual presentations of medical images can provide significant advantages because this type of rendering is more truly representational of the object being imaged (the human body)- it is a more intuitive and easily-read format. It is increasingly common to render a three dimensional (3D) model from a CT, MRI, PET and X-Ray scan to better interpret the size, orientation and other spatial relationships of the patient’s anatomy as necessary for diagnosis, intervention (surgery) and therapy.
Until recently, there was little hope of interoperability for interactive 3D and 4D presentations to break out of the hospital PACS and to be archived and shared across the enterprise. With the continual advancement in computing and graphical power over the last decade, specialized workstations and software capacity has become available to display this type of 3D imaging on a common laptop. It is an imminent future when the handheld tablets on the market are capable of sustained hardware-accelerated graphics performance.
Our original work (Web3D.org) for TATRC (W81XWH-06-1-0096) developed and demonstrated the integration of expressive volume rendering with X3D over the web with several client platforms. This set of functionalities was validated by industry experts and formalized into a specification with two separate, multi-platform implementations. The new component includes an expressive range of volume rendering styles as well as means to assign separate styles to different segments, and to create isosurfaces within the volume. In 2012, this specification has ultimately become an official part of ISO X3D 3.3.
Much of the required functionality is specified in the X3D 3.3 draft International Standard, including the Texturing3D Component (Clause 33) and the Volume Rendering Component (Clause 41) to support several compose-able styles for Volume Rendering for Medical Imaging, geology and other non-invasive sensing modalities. A Medical Interchange Profile of X3D nodes is also defined in Annex L : http://www.web3d.org/files/specifications/19775-1/V3.3/Part01/MedInterchange.html. The node set of the X3D 3.3. Medical Interchange Profile collects nodes for volume and polygon rendering, lighting, text and animation; it has been demonstrated to meet the requirements of several key clinical and research applications including diagnosis, surgical planning, education and training and informed consent.
The Medical Working Group is participating in the DICOM Working Group 11 for the purpose of defining a presentation standard for reproducible Medical Imaging.
- See also 2012 Opportunities Whitepaper: Portable and Interoperable Views of Medical Image Data with ISO Extensible 3D (X3D)
- See also Archival and Web-based Visualization of Volumetric Scans and Clinical Data for TBI using the X3D International Standard, a research proposal to the U.S. Military Health System Challenge
X3D Volume Rendering Examples & Videos
- A video of an X3D presentation of a segmented MRI of a human head and brain (from Virginia Tech)
- Fill out a Survey (w/ videos) about Multi-channel microscopy volume rendering techniques in X3D
- A video of compiled X3D volume rendering examples from Virginia Tech (rendered w/ H3D.org) is available here
- X3D Examples Archive: Medical Imaging / Volume Rendering; See also additional videos and images of X3D Volume RenderStyles (from Virginia Tech).
Tools
- X3D-Edit 3.3 supports the Texturing3D and Volume Component nodes, including validation by DTD and Schema
- Haptics 3D Toolkit by Sensegraphics
- Instant Reality
- MedX3DOM and VolumeRC by Vicomtech
- NRRD volume data format
Papers, Tutorials and Presentations
- SIGGRAPH 2018 BOF [1]
- Web3D 2017 paper about community-driven extensions : Ander Arbelaiz, Aitor Moreno, Luis Kabongo, Nicholas Polys, and Alejandro García-Alonso. 2017. Community-driven extensions to the X3D volume rendering component. In Proceedings of the 22nd International Conference on 3D Web Technology (Web3D '17). ACM, New York, NY, USA, Article 1, 9 pages. DOI: [2]
- Web3D 2016 paper about comparing volume and surface renderings in X3D: [Nicholas F. Polys and Abhijit A. Gurjarpadhye. 2016. Tradeoffs in multi-channel microscopy volume visualization: an initial evaluation. In Proceedings of the 21st International Conference on Web3D Technology (Web3D '16). ACM, New York, NY, USA, 187-188. DOI: https://doi.org/10.1145/2945292.2945323]
- SIGGRAPH 2016 BOF [3]
- SIGGRAPH 2015 Volume Rendering and Medical Visualization
- OSEHRA (OpenSource VistA) Summit 2014: X3D Scene Graph Structures for Medical Image Metadata by Nicholas F. Polys & Abhijit A.Gurjarpadhye
- SIGGRAPH 2014 Birds-Of-a-Feather (BOF) session: Volume Rendering and Medical Visualization by Nicholas Polys
- Medicine Meets Virtual Reality (Feb. 2014) Session 'Extending Extensible 3D (X3D): from Haptic-BasedMedical Training to Clinical Applications' Thursday Feb 20th
- Michael Aratow: Medical Informatics, CEP America A Health IT Perspective on X3D
- Timothy Coles: The Australian e-Health Research Centre & Surgical Simulation and Planning Team, CSIRO X3D in Medical Training & Simulation
- Tommy Forsell: SenseGraphics AB H3D In Real Life – Medical Simulation
- Nicholas F. Polys: Advanced Research Computing, Virginia Tech Volume Rendering and Lossless Metadata with X3D
- Felix G. Hamza- Lup: Computer Science and Information Technology, Armstrong Atlantic State University Radiation Therapy Training with X3D
- Web3D 2013 Tutorial:Medical Imaging and Volume Rendering in X3D
- "A Fresh Look at Immersive Volume Rendering: Challenges and Capabilities", Polys, Ullrich, Evestedt, Wood and Aratow. IEEE Virtual Reality Workshop on Immersive Volumetric Interaction, 2013.
- "New Platforms for Health Hypermedia" by Polys and Wood, Issues in Information Systems. Vol 13 (1); pp 40-50, 2012. This is a good overview of the X3D capabilities and potential for Health Informatics.
- Web3D 2012 Tutorial X3D Volume Visualization and Medical Applications Download the slides!
- SIGGRAPH 2012 Birds-Of-A-Feather (BOF) 'Medical and Volume Rendering with X3D' by Polys, Aratow and Kabongo
- Best Paper Award at IEEE VR 2012: Haptic Palpation for Medical Simulation in Virtual Environments
- A presentation made at the SIGGRAPH 2011 Medical BOF is available here
- Ullrich, S., T. Kuhlen, N. F. Polys, D. Evestedt, M. Aratow, and N. W. John, "Quantizing the Void: Extending Web3D for Space-Filling Haptic Meshes", Medicine Meets Virtual Reality (MMVR), vol. 163, Newport Beach CA, USA, IOS Press, pp. 670-676, February, 2011.
- Proceedings of the IEEE VR 2010 Medical Workshop
- John, N. W., M. Aratow, J. Couch, D. Evestedt, A. D. Hudson, N. Polys, R. F. Puk, A. Ray, K. Victor, and Q. Wang, "MedX3D: Standards Enabled Desktop Medical 3D", Medicine Meets VR (MMVR), 2008.
- N.W. John, "Design and Implementation of Medical Training Simulators", Virtual Real. 12, 4 (Dec. 2008), 269-279.
- F.P. Vidal, N.W. John, A.E.Healey, D.A. Gould, "Simulation of Ultrasound Guided Needle Puncture using Patient Specific Data with 3D Textures and Volume Haptics", Computer Animation and Virtual Worlds. Vol. 19, Issue 2, pp111-127, May 2008, Online ISSN: 1546-427X , Print ISSN: 1546-4261,
- N. W. John, I.S. Lim, "Cybermedicine Tools for Communication and Learning", Journal of Visual Communication in Medicine, 2007; 30(2): 4-9.
- Polys, N., D. Bowman, C. North, R. Laubenbacher, and K. Duca, "PathSim Visualizer: An Information-Rich Virtual Environment for Systems Biology", Web3D Symposium, Monterey, CA, ACM Press, 2006.
