Distributed Shared Memory for Roaming Large Volumes
|
We present a cluster-based volume rendering system for roaming very large volumes. This system allows to move a gigabyte-sized probe inside a total volume of several tens or hundreds of gigabytes in real-time. While the size of the probe is limited by the total amount of texture memory on the cluster, the size of the total data set has no theoretical limit. The cluster is used as a distributed graphics processing unit that both aggregates graphics power and graphics memory. A hardware-accelerated volume renderer runs in parallel on the cluster nodes and the final image compositing is implemented using a pipelined sort-last rendering algorithm. Meanwhile, volume bricking and volume paging allow efficient data caching. On each rendering node, a distributed hierarchical cache system implements a global software-based distributed shared memory on the cluster. In case of a cache miss, this system first checks page residency on the other cluster nodes instead of directly accessing local disks. Using two Gigabit Ethernet network interfaces per node, we accelerate data fetching by a factor of 4 compared to directly accessing local disks. The system also implements asynchronous disk access and texture loading, which makes it possible to overlap data loading, volume slicing and rendering for optimal volume roaming.
[1] 1299 AmzaC., CoxA.L., DwarkadasS., KeleherP., LuH., RajamonyR., YuW., and ZwaenepoelW., TreadMarks: Shared Memory Computing on Networks of Workstations. Computer, 29: 18–28, February 1996.
[2] BadouelD., BouatouchK., and PriolT., Distributing Data and Control for Ray Tracing in Parallel. IEEE Computer Graphics and Applications, 14 (4): 69–77, 1994.
[3] BethelW., TierneyB., LeeJ., and GunterD. adn LauS.,, Using High-Speed WANs and Network Data Caches to Enable Remote and Distributed Visualization. In Proceedings of Supercomputing Conference, 2000.
[4] BhaniramkaP. and DemangeY., OpenGL Volumizer: A Toolkit for High Quality Volume Rendering of Large Data Sets. In Proceedings of IEEE Symposium on Volume Visualization, pages 45–54, 2002.
[5] CabralB., CamN., and ForanJ., Accelerated Volume Rendering and Tomographic Reconstruction Using Texture Mapping Hardware. In Proceedings of IEEE Symposium on Volume Visualization, pages 91–98, 1994.
[6] CarterJ.B., BennettJ.K., and ZwaenepoelW., Implementation and Performance of Munin. In Proceedings of the 13th ACM Symposium on Operating Systems Principles, pages 152–164, 1991.
[7] CarterJ.B., KhandekarD., and KambL., Distributed Shared Memory: Where We Are and Where We Should Be Headed. In Workshop on Hot Topics in Operating Systems, pages 119–122, 1995.
[8] CastaniL., LvyB., and BosquetF., VolumeExplorer: Roaming Large Volumes to Couple Visualization and Data Processing for Oil and Gas Exploration. In Proceedings of IEEE Visualization Conference, 2005.
[9] CavinX., MionC., and FilboisA., COTS Cluster-Based Sort-Last Rendering: Performance Evaluation and Pipelined Implementation. In Proceedings of IEEE Visualization Conference, 2005.
[10] ChankhunthodA., DanzigP.B., NeerdaelsC., SchwartzM.F., and WorrellK.J., A Hierarchical Internet Object Cache. In Proceedings of the USENIX Annual Technical Conference, pages 153–164, 1996.
[11] CoxM. and EllsworthD., Application-Controlled Demand Paging for Out-of-Core Visualization. In Proceedings of IEEE Visualization Conference, pages 235–244, 1997.
[12] DeMarleD., GribbleC., BoulosS., and ParkerS., Memory Sharing for Interactive Ray Tracing on Clusters. Parallel Computing, 31 (2): 221–242, February 2005.
[13] DeMarleD., ParkerS., HartnerM., GribbleC., and HansenC., Distributed Interactive Ray Tracing for Large Volume Visualization. In Proceedings of IEEE Symposium on Parallel and Large-Data Visualization and Graphics, pages 87–94, 2003.
[14] EllsworthD., Accelerating Demand Paging for Local and Remote Out-of-Core Visualization. Technical Report NASA Ames Research Center, 2001.
[15] EngelK., KrausM., and ErtlT., High-Quality Pre-Integrated Volume Rendering Using Hardware Accelerated Pixel Shading. In Proceedings of Eurographics/SIGGRAPH Workshop on Graphics Hardware, pages 9–16, 2001.
[16] Force10 Networks, Inc. The High Performance Data Center: The Role of Ethernet in Consolidation and Virtualization. http://www.force10networks.com/products/ pdfwp_datacenter_convirt.pdf, 2005.
[17] GaoJ., HuangJ., JohnsonR., AtchleyS., and KohlJ.A., Distributed Data Management for Large Volume Visualization. In Proceedings of IEEE Visualization Conference, pages 183–189, 2005.
[18] GreenS. and PaddonD., Exploiting Coherence for Multiprocessor Ray Tracing. IEEE Computer Graphics and Applications, 9 (6): 12–26, November 1989.
[19] HoustonM., Designing Graphics Clusters. Parallel Rendering Workshop — IEEE Visualization Conference, 2004.
[20] LacrouteP. and LevoyM., Fast Volume Rendering Using a Shear-Warp Factorization of the Viewing Transform. In Proceedings of ACM SIGGRAPH Conference, volume 28, pages 451–457, 1994.
[21] LevoyM., Display of Surfaces from Volume Data. IEEE Computer Graphics and Applications, 8: 29–37, 1988.
[22] LiK., Shared Virtual Memory on Loosely Coupled Multiprocessors. PhD thesis, Department of Computer Science, Yale University, 1986.
[23] LumE.B., WilsonB., and MaK.L., High-Quality Lighting and Efficient Pre-Integration for Volume Rendering. In Proceedings of Eurographics/IEEE Symposium on Visualization, pages 25–34, 2004.
[24] MaK.L., PainterJ.S., HansenC.D., and KroghM.F., Parallel Volume Rendering Using Binary-Swap Image Composition. IEEE Computer Graphics and Applications, 14 (4): 59–68, July 1994.
[25] MaxN., Optical Models for Direct Volume Rendering. IEEE Transactions on Visualization and Computer Graphics, 1 (2): 99–108, 1995.
[26] MolnarS., CoxM., EllsworthD., and FuchsH., A Sorting Classification of Parallel Rendering. IEEE Computer Graphics and Applications, 14 (4): 23–32, 1994.
[27] NVIDIA Corporation. Fast Texture Downloads and Readbacks Using Pixel Buffer Objects in OpenGL. http://developer.nvidia.com/objectfast_texture_transfers.html, August 2005.
[28] ParkerS., ParkerM., LivnatY., SloanP.P., HansenC., and ShirleyP., Interactive Ray Tracing for Volume Visualization. IEEE Transactions on Visualization and Computer Graphics, 5 (3): 238–250, 1999.
[29] PlateJ., TirtasanaM., CarmonaR., and FrhlichB., Octreemizer: A Hierarchical Approach for Interactive Roaming Through Very Large Volumes. In Proceedings of Eurographics/IEEE Symposium on Visualization, 2002.
[30] Rezk-SalamaC., EngelK., HadwigerM., KnissJ., LefhonA., and WeiskopfD., Real-Time Volume Graphics. Course 28, ACM SIGGRAPH, 2004.
[31] SilberschatzA., GagneG., and GalvinP.B., Operating System Concepts, Seventh Edition. John Wiley & Sons, Inc., Hoboken, 2005. 921 p.
[32] SiSoftware Sandra 2005. http://www.sisoftware.co.uk.
[33] StompelA., MaK.L., LumE.B., AhrensJ., and PatchettJ., SLIC: Scheduled Linear Image Composition for Parallel Volume Rendering. In Proceedings of IEEE Symposium on Parallel and Large-Data Visualization and Graphics, pages 33–40, 2003.
[34] ThakurR. and GroppW., Improving the Performance of Collective Operations in MPICH. In Proceedings of the 10th European PVM/MPI Users' Group Conference (Euro PVM/MPI 2003), pages 257–267, 2003.
[35] TierneyB., LeeJ., CrowleyB., HoldingM., HyltonJ., and DrakeF., A Network-Aware Distributed Storage Cache for Data Intensive Environments. In Proceedings of IEEE High Performance Distributed Computing Conference, 1999.
[36] WaldI., DietrichA., and SlusallekP., An Interactive Out-of-Core Rendering Framework for Visualizing Massively Complex Models. In Proceedings of Eurographics Symposium on Rendering, pages 81–92, 2004.
[37] ZekauskasM.J., SawdonW.A., and BershadB.N., Software Write Detection for Distributed Shared Memory. In Proceedings of the Symposium on Operating System Design and Implementation, pages 87–100, 1994.
[38] N. I. H. U.S. National Library of Medecine. The Visible Human Project. http://www.nlm.nih.gov/research/visiblevisible_human.html.
Index Terms:
Large volumes, volume roaming, out-of-core, hierarchical caching, distributed shared memory, hardware-accelerated volume visualization, graphics hardware, parallel rendering, graphics cluster.
Citation:
Laurent Castanie, Christophe Mion, Xavier Cavin, Bruno Levy, "Distributed Shared Memory for Roaming Large Volumes," IEEE Transactions on Visualization and Computer Graphics, vol. 12, no. 5, pp. 1299-1306, Sept. 2006, doi:10.1109/TVCG.2006.135
PURCHASE ARTICLE: $19
IEEE Xplore Subscribers
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb