An advanced interconnection network called "HCC-ABL- tree (hierarchical completely-connected tree by additional bypass linking)" is proposed forconstructing large PC clusters capable of distributed inter-node communication by using Ethernet switches with a small number of I/O-ports. A basic one-level CC-ABL-tree can be constructed by using small subarrays of processing nodes (PCs) providing interconnections with the complete connection scheme, and by connecting the nodes to a tree- structured bypass network with switches with a height of two so that every node of the subarray is allocated to a different second-level switch. A two-or three- level HCC-ABL-tree can be constructed by using the trees with one less number of levels as the components by connecting them to each other with the complete- connection scheme by using the second-level switches of the bypass network as the hierarchical interconnections. The maximum number of processing nodes in a cluster can be increased exponentially by increasing the numb er of hierarchical levels. The network diameter is two for a one-level tree, three for a two-level tree, and seven for a three-level tree. The proposed network can configure the processing nodes in the cluster as a square-mesh-connected array with any aspect ratio by graph embedding, where a newly proposed distributed routing algorithm can define the paths with no duplicated use of a link. This manner of configuring can also be achieved even when busy or faulty nodes are existing in the cluster, while bypassing these nodes in a node-disjoint manner with a small congestion and dilation in the paths.