In this paper, we evaluated the Ag concentration dependence of the build-up effect of radio-photoluminescence (RPL) in Ag-doped P2O5–Al2O3–Na2O–SiO2 (PANS) glasses. After X-ray irradiation, the Ag-doped PANS glasses showed two emission peaks at around 460 and 630 nm, assigned to blue and orange RPL components, respectively. The build-up curves of the orange RPL component had two components corresponding to the formation of the Ag+2 dimer and Ag2+ ions. The intensity of the component due to the Ag-2 dimer increased with increasing Ag concentration, whereas that of the component due to Ag2+ ions decreased. It is assumed that at a high Ag concentration, the Ag+ ions have a higher probability of diffusing to Ag0 sites to form the Ag+2 dimer because the average distance between neighboring Ag atoms in the glass structure decreases with increasing Ag concentration. Therefore, the intensity of the component due to the Ag-2 dimer was dominant at a high concentration of Ag in the glass structure.