This paper proposes a rule based web image annotation method which improves the precision and recall of annota- tion by the use of decision tree. This decision tree learns the relationship between images and their annotations based on the proposed 17 attributes that specify the structural rela- tionship between them in HTML documents and the visual characteristics of the images. By converting and pruning this learned tree, a set of rules with high estimated accu- racy which determines whether or not a word can be the keyword of an image can be generated. Upon experimental results, the proposed method made 57 rules and the preci- sion and recall of annotation by these rules were about 88% and 95% for the various concepts, respectively. We argue the contribution of this work in two aspects. First, we sug- gest the clear criteria for precise annotation inferred by the statistical analysis of many web pages. Second, to cope with the deterioration of recall caused by the lack of measure for the visual characteristics, the visual similarity between an image and its concept combines to the attributes that used for tree learning.