The management and recycling of heavy metal-rich sludge have posed serious environmental challenges. High levels of potassium permanganate are often used to treat eutrophic water bodies, which produce manganese-rich drinking water treatment sludge (DWTS). Although there have been some studies on recycling sludge in cement-based materials, little attention has been paid to the multiscale behavior of concrete with manganese-rich sludge. In this study, eco-friendly concrete was produced by substituting part of the cement with calcined DWTS (CWTS), and its mechanical properties, drying shrinkage, microstructure, and environmental characteristics were systematically investigated. The results showed that the incorporation of an appropriate amount of CWTS improved the mechanical properties and volume stability of the concrete, and the best performance was achieved when the substitution level was 10%. The 90-d compressive strength of 10% CWTS-modified concrete was relatively increased by 7.91%, while the 150-d drying shrinkage was relatively reduced by 14.84% compared with plain concrete. The pozzolanic action of CWTS consumed part of Ca(OH)2 and increased the degree of reactivity of the paste at appropriate substitution levels. From the perspective of nanoscale characteristics, the incorporation of 10% CWTS reduced the content of the pore phase and clinker and increased the content of the C-S-H phase relative to plain concrete, where high-density C-S-H and low-density C-S-H increased by 12.5% and 6.8%, respectively. Moreover, incorporating an appropriate amount of CWTS also reduced the width of the interfacial transition zone (ITZ), but with the further increase of its content, the width of the ITZ increased. In addition, CWTS-modified concrete not only exhibited safe heavy metal leaching behavior but also had lower costs and carbon emissions. This study provides new insights into the management and recycling of DWTS, especially manganeserich sludge obtained from the treatment of algal blooms.