Cadmium (Cd) pollution renders many soils across the world unsuited or unsafe for food- or feed-orientated agriculture. The main mechanism of Cd phytotoxicity is the induction of oxidative stress, amongst others through the depletion of glutathione. Oxidative stress can damage lipids, proteins, and nucleic acids, leading to growth inhibition or even cell death. The plant cell has a variety of tools to defend itself against Cd stress. First and foremost, cell walls might prevent Cd from entering and damaging the protoplast. Both the primary and secondary cell wall have an array of defensive mechanisms that can be adapted to cope with Cd. Pectin, which contains most of the negative charges within the primary cell wall, can sequester Cd very effectively. In the secondary cell wall, lignification can serve to immobilize Cd and create a tougher barrier for entry. Changes in cell wall composition are, however, dependent on nutrients and conversely might affect their uptake. Additionally, the role of ascorbate (AsA) as most important apoplastic antioxidant is of considerable interest, due to the fact that oxidative stress is a major mechanism underlying Cd toxicity, and that AsA biosynthesis shares several links with cell wall construction. In this review, modifications of the plant cell wall in response to Cd exposure are discussed. Focus lies on pectin in the primary cell wall, lignification in the secondary cell wall and the importance of AsA in the apoplast. Regarding lignification, we attempt to answer the question whether increased lignification is merely a consequence of Cd toxicity, or rather an elicited defense response. We propose a model for lignification as defense response, with a central role for hydrogen peroxide as substrate and signaling molecule.