Multi-rate feature is an important advance in wireless communications. It enables wireless devices to operate at high data rate when the channel condition is sufficiently good. Existing ad hoc routing protocols can not perform very well in the multi-rate ad hoc networks because they generally assume that bandwidth of different links are identical in the system. One straightforward solution to this issue is to use special routing metrics in routing algorithm, which is called routing approach. However, we believe that only localized reaction is performed in response to the local event. Based on this design principle, we propose a relay based medium access scheme to employ multi-rate capability even if existing minimum-hop based routing protocols are used. This scheme takes advantage of the node residing between neighboring nodes as a relay node and transforms a long-range transmission into two short-range transmissions. In our scheme design, we are very careful in reducing the additional overhead caused in order to exploit the high data rate transmission. Simulations show that our scheme achieve better throughput compared with 802.11 MAC scheme, a rate-adaptive MAC protocol called RBAR and the routing approach.