We present a geometric method for (i) matching 2D line segments from multiple oriented images, (ii) optimally reconstructing 3D line segments and (iii) grouping 3D line segments to corners. The proposed algorithm uses two developments in combining projective geometry and statistics, which are described in this article: (i) the geometric entities points, lines and planes in 2D and 3D and their uncertainty are represented in homogeneous coordinates and new entities may be constructed including their propagated uncertainty. The construction can be performed directly or as an estimation. (ii) relations such as incidence, equality, parallelity and orthogonality between points, lines and planes can be tested statistically based on a given significance level. Using these tools, the resulting algorithm is straight-forward and gives reasonable results. It is only based on geometric information and does not use any image intensities, though it can be extended to use other information. The matching of 3D lines does not need any thresholds other than a significance value for the hypotheses tests.