Using Adversarial Attacks for Localized Generation of Super-Resolution Artifacts

The task of image super-resolution, addressed using deep neural networks, particularly generative adversarial models, faces the problem of visual artifacts. These distortions degrade the result quality, and their automatic detection is challenging due to the lack of large-scale labeled datasets. This work aims to develop an automated method for creating such datasets to train and evaluate artifact detection models. The proposed method utilizes an adversarial attack approach to deliberately create artifacts in the output images of super-resolution models. The core of the method is a modification of the Iterative Fast Gradient Sign Method. The key innovation lies in the modified loss function, which maximizes distortions in a specified image area, defined by a binary mask, while simultaneously minimizing them in the remaining parts. This enables the generation of localized artifacts that mimic natural defects. To validate the method, a dataset containing over 2000 examples has been created. Experimental results confirmed that the proposed dataset possesses high-quality annotations. Detection methods demonstrated an IoU value exceeding 0.7 on it, which is substantially higher than results achieved on existing datasets. The developed method allows for the efficient creation of scalable and high-quality labeled datasets. A neural network method was also developed, which shows better results compared to the baseline method. This opens up opportunities for developing more robust super-resolution methods, their subsequent post-processing, and creating effective artifact detectors.

1. Wang, X., Xie, L., Dong, C., Shan, Y. (2021). Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data. International Conference on Computer Vision Workshops (ICCVW).

2. Xie, L., Wang, X., Chen, X., Li, G., Shan, Y., Zhou, J., & Dong, C. (2023). DeSRA: Detect and Delete the Artifacts of GAN-based Real-World Super-Resolution Models.

3. Zhang, L., Zhou, Y., Barnes, C., Amirghodsi, S., Lin, Z., Shechtman, E., Shi, J. (2022). Perceptual artifacts localization for inpainting. European Conference on Computer Vision, 146-164.

4. Zhang, L., Xu, Z., Barnes, C., Zhou, Y., Liu, Q., Zhang, H., Amirghodsi, S., Lin, Z., Shechtman, E., Shi, J. (2023). Perceptual Artifacts Localization for Image Synthesis Tasks. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 7579–7590.

5. Liang, J., Zeng, H., Zhang, L. (2022). Details or Artifacts: A Locally Discriminative Learning Approach to Realistic Image Super-Resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.

6. Wang, Z., Bovik, A. C., Sheikh, H. R., Simoncelli, E. P. (2004). Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612. DOI: 10.1109/TIP.2003.819861.

7. Ding, K., Ma, K., Wang, S., Simoncelli, E. P. (2022). Image Quality Assessment: Unifying Structure and Texture Similarity. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(5), 2567-2581. DOI: 10.1109/TPAMI.2020.3045810.

8. Zhang, R., Isola, P., Efros, A. A., Shechtman, E., Wang, O. (2018). The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 586-595. DOI: 10.1109/CVPR.2018.00068.

9. Goodfellow, I. J., Shlens, J., Szegedy, C. (2014). Explaining and Harnessing Adversarial Examples. CoRR, abs/1412.6572. Available at: https://arxiv.org/pdf/1412.6572, accessed 14.03.2026.

10. Hsu, C.-C., Lee, C.-M., Chou, Y.-S. (2024). DRCT: Saving Image Super-Resolution Away from Information Bottleneck. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 6133-6142.

11. Kuznetsova, A., Rom, H., Alldrin, N. et al. The Open Images Dataset V4. Int J Comput Vis 128, 1956-1981 (2020). DOI: 10.1007/s11263-020-01316-z.

12. Xu, M., Zhang, Z., Wei, F., Hu, H., Bai, X. (2023). SAN: Side Adapter Network for Open-Vocabulary Semantic Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(12), 15546-15561. DOI: 10.1109/TPAMI.2023.3311618.

13. J. Liang, J. Cao, G. Sun, K. Zhang, L. Van Gool R. Timofte, "SwinIR: Image Restoration Using Swin Transformer", 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), Montreal, BC, Canada, 2021, pp. 1833-1844, DOI: 10.1109/ICCVW54120.2021.00210.