Smart cards are widely known for their tamper resistance, but only contain a small amount of memory. Though very small, this memory often contains highly valuable information (identification data, cryptographic key, etc). This is why it is subject to many attacks, as the other parts of the smart card, and thus requires appropriately chosen protections. The use of memories in smart cards induces security problems, but also other more particular ones. The main constraint is naturally the limited physical expansion and integration, but fault level, aging and power consumption are not to be discarded. Indeed, deducing the context of a ROM using a microscope has been proven to work. Interactions with light or eddy current on silicon can produce faults that might reveal important information, as well. This article details the role of memory in smart card industries, in current context and future perspectives of smart cards and their applications. It then gives a survey of published physical attacks targeting memory and all the existing techniques to counter them. Great efforts are undertaken by industries and academics to tackle specific memory problems introducing hardware and software countermeasures in the designs. This struggle between security and hackers permits in the one hand tremendous breakthroughs in research but in the other hand makes rather difficult for manufacturers to maintain cost effectiveness, that is one important factor for smart card.