- -

A deep multimodal system for provenance filtering with universal forgery detection and localization

RiuNet: Repositorio Institucional de la Universidad Politécnica de Valencia

Compartir/Enviar a

Citas

Estadísticas

  • Estadisticas de Uso

A deep multimodal system for provenance filtering with universal forgery detection and localization

Mostrar el registro sencillo del ítem

Ficheros en el ítem

dc.contributor.author Jabeen, Saira es_ES
dc.contributor.author Khan, Usman Ghani es_ES
dc.contributor.author Iqbal, Razi es_ES
dc.contributor.author Mukherjee, Mithun es_ES
dc.contributor.author Lloret, Jaime es_ES
dc.date.accessioned 2022-11-07T16:34:12Z
dc.date.available 2022-11-07T16:34:12Z
dc.date.issued 2021-05 es_ES
dc.identifier.issn 1380-7501 es_ES
dc.identifier.uri http://hdl.handle.net/10251/189337
dc.description.abstract [EN] Traditional multimedia forensics techniques inspect images to identify, localize forged regions and estimate forgery methods that have been applied. Provenance filtering is the research area that has been evolved recently to retrieve all the images that are involved in constructing a morphed image in order to analyze an image, completely forensically. This task can be performed in two stages: one is to detect and localize forgery in the query image, and the second integral part is to search potentially similar images from a large pool of images. We propose a multimodal system which covers both steps, forgery detection through deep neural networks(CNN) followed by part based image retrieval. Classification and localization of manipulated region are performed using a deep neural network. InceptionV3 is employed to extract key features of the entire image as well as for the manipulated region. Potential donors and nearly duplicates are retrieved by using the Nearest Neighbour Algorithm. We take the CASIA-v2, CoMoFoD and NIST 2018 datasets to evaluate the proposed system. Experimental results show that deep features outperform low-level features previously used to perform provenance filtering with achieved Recall@50 of 92.8%. es_ES
dc.language Inglés es_ES
dc.publisher Springer-Verlag es_ES
dc.relation.ispartof Multimedia Tools and Applications es_ES
dc.rights Reserva de todos los derechos es_ES
dc.subject Provenance filtering es_ES
dc.subject Convolutional neural networks es_ES
dc.subject Forgery detection and localization es_ES
dc.subject Manipulation detection es_ES
dc.title A deep multimodal system for provenance filtering with universal forgery detection and localization es_ES
dc.type Artículo es_ES
dc.identifier.doi 10.1007/s11042-020-09623-w es_ES
dc.rights.accessRights Abierto es_ES
dc.description.bibliographicCitation Jabeen, S.; Khan, UG.; Iqbal, R.; Mukherjee, M.; Lloret, J. (2021). A deep multimodal system for provenance filtering with universal forgery detection and localization. Multimedia Tools and Applications. 80(11):17025-17044. https://doi.org/10.1007/s11042-020-09623-w es_ES
dc.description.accrualMethod S es_ES
dc.relation.publisherversion https://doi.org/10.1007/s11042-020-09623-w es_ES
dc.description.upvformatpinicio 17025 es_ES
dc.description.upvformatpfin 17044 es_ES
dc.type.version info:eu-repo/semantics/publishedVersion es_ES
dc.description.volume 80 es_ES
dc.description.issue 11 es_ES
dc.relation.pasarela S\473265 es_ES
dc.description.references Murabayashi A (2018) The problem of fake photos in fake news. https://petapixel.com/2017/01/19/problem-fake-photos-fake-news/, Accessed: 2019-01-11 es_ES
dc.description.references Fridrich AJ, Soukal BD, Lukáš AJ (2003) Detection of copy-move forgery in digital images. In: Proceedings of Digital Forensic Research Workshop, Citeseer es_ES
dc.description.references Lloret J, Bosch I, Sendra S, Serrano A (2011) A wireless sensor network for vineyard monitoring that uses image processing. Sensors 11(6):6165–6196 es_ES
dc.description.references Birajdar GK, Mankar VH (2013) Digital image forgery detection using passive techniques: A survey. Elsevier Digital investigation 10(3):226–245 es_ES
dc.description.references Farid H (2009) Image forgery detection. IEEE Signal Process. Mag. 26(2):16–25 es_ES
dc.description.references Ahmed N, Natarajan T, Rao KR (1974) Discrete cosine transform. IEEE transactions on Computers 100(1):90–93 es_ES
dc.description.references Lowe DG (2004) Distinctive image features from scale-invariant keypoints. International journal of computer vision 60(2):91–110 es_ES
dc.description.references Bay H, Tuytelaars T, VanGool L (2006) Surf: Speeded up robust features. In: European conference on computer vision, Springer, pp 404–417 es_ES
dc.description.references Amerini I, Ballan L, Caldelli R, DelBimbo A, Serra G (2011) A sift-based forensic method for copy–move attack detection and transformation recovery. IEEE Trans. Informat. Forensics and Security 6(3):1099–1110 es_ES
dc.description.references Shivakumar BL, Baboo SS (2011) Detection of region duplication forgery in digital images using surf. International Journal of Computer Science Issues (IJCSI) 8(4):199 es_ES
dc.description.references Bhatti MH, Khan J, Khan M UG, Iqbal R, Aloqaily M, Jararweh Y, Gupta B (2019) Soft computing-based eeg classification by optimal feature selection and neural networks. IEEE Transactions on Industrial Informatics 15 (10):5747–5754 es_ES
dc.description.references Khan MZ, Harous S, Hassan SU, Khan M UG, Iqbal R, Mumtaz S (2019) Deep unified model for face recognition based on convolution neural network and edge computing. IEEE Access 7:72622–72633 es_ES
dc.description.references Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L (2015) ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV) 115(3):211–252. https://doi.org/10.1007/s11263-015-0816-y es_ES
dc.description.references Simonyan K, Zisserman A (2014Sept.) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 es_ES
dc.description.references Rao Y, Ni J (2016) A deep learning approach to detection of splicing and copy-move forgeries in images. In: 2016 IEEE International Workshop on Information Forensics and Security (WIFS), IEEE, pp 1–6 es_ES
dc.description.references Bayar B, Stamm MC (2016) A deep learning approach to universal image manipulation detection using a new convolutional layer. In: Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security, ACM, pp 5–10 es_ES
dc.description.references Chen J, Kang X, Liu Y, Wang ZJ (2015) Median filtering forensics based on convolutional neural networks. IEEE Signal Processing Letters 22(11):1849–1853 es_ES
dc.description.references Wu Y, Abd-Almageed W, Natarajan P (2018September) Busternet: Detecting copy-move image forgery with source/target localization. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 168–184 es_ES
dc.description.references Pinto A, Moreira D, Bharati A, Brogan J, Bowyer K, Flynn P, Scheirer W, Rocha A (2017) Provenance filtering for multimedia phylogeny. In: 2017 IEEE International Conference on Image Processing (ICIP), IEEE, pp 1502–1506 es_ES
dc.description.references Asghar K, Habib Z, Hussain M (2017) Copy-move and splicing image forgery detection and localization techniques: a review. Taylor & Francis Australian Journal of Forensic Sciences 49(3):281–307 es_ES
dc.description.references Steinbach M, Karypis G, Kumar V, et al. (2000) A comparison of document clustering techniques. In: KDD workshop on text mining, vol 400, Boston, pp 525–526 es_ES
dc.description.references Bo X, Junwen W, Guangjie L, Yuewei D (2010) Image copy-move forgery detection based on surf. In: Proc. IEEE Int. Conf. on Multimedia Information Netw. and Security, pp 889–892 es_ES
dc.description.references Qu Z, Qiu G, Huang J (2009) Detect digital image splicing with visual cues. In: Springer Int. workshop on Information hiding, pp 247–261 es_ES
dc.description.references Lin SD, Wu T (2011) An integrated technique for splicing and copy-move forgery image detection. In: Proc. IEEE 4th International Congress on Image and Signal Processing, vol 2, pp 1086–1090 es_ES
dc.description.references Lin Z, He J, Tang X, Tang C-K (2009) Fast, automatic and fine-grained tampered jpeg image detection via dct coefficient analysis. Pattern Recogn 42(11):2492–2501 es_ES
dc.description.references Cao Y, Gao T, Fan L, Yang Q (2012) A robust detection algorithm for copy-move forgery in digital images. Forensic science international 214 (1-3):33–43 es_ES
dc.description.references Mahmood T, Mehmood Z, Shah M, Saba T (2018) A robust technique for copy-move forgery detection and localization in digital images via stationary wavelet and discrete cosine transform. J Vis Commun Image Represent 53:202–214 es_ES
dc.description.references Muhammad G, Al-Hammadi MH, Hussain M, Bebis G (2014) Image forgery detection using steerable pyramid transform and local binary pattern. Mach Vis Appl 25(4):985–995 es_ES
dc.description.references Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks?. In: Prof. Advances in neural information processing systems, pp 3320–3328 es_ES
dc.description.references Wang P, Wei Z, Xiao L (2016) Pure spatial rich model features for digital image steganalysis. Multimedia Tools and Applications 75(5):2897–2912 es_ES
dc.description.references Kang X, Stamm MC, Peng A, Liu KJRay (2013) Robust median filtering forensics using an autoregressive model. IEEE Transactions on Information Forensics and Security 8(9):1456–1468 es_ES
dc.description.references Yuan H-D (2011) Blind forensics of median filtering in digital images. IEEE Transactions on Information Forensics and Security 6(4):1335–1345 es_ES
dc.description.references Cao G, Zhao Y, Ni R, Yu L, Tian H (2010) Forensic detection of median filtering in digital images. In: 2010 IEEE International Conference on Multimedia and Expo, IEEE, pp 89–94 es_ES
dc.description.references Bunk J, Bappy JH, Mohammed TM, Nataraj L, Flenner A, Manjunath BS, Chandrasekaran S, Roy-Chowdhury AK, Peterson L (2017) Detection and localization of image forgeries using resampling features and deep learning. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), IEEE, pp 1881–1889 es_ES
dc.description.references Helgason S, Helgason S (1999) The radon transform. Springer, vol 2 es_ES
dc.description.references Chum O, Philbin J, Zisserman A, et al. (2008September) Near duplicate image detection: min-hash and tf-idf weighting.. In: BMVC, vol 810, pp 812–815 es_ES
dc.description.references Li M, Wang B, Ma W-Y, Li Z (2010) Detecting duplicate images using hash code grouping, Google Patents. US Patent 7,647,331 es_ES
dc.description.references Dong W, Wang Z, Charikar M, Li K (2012) High-confidence near-duplicate image detection. In: Proceedings of the 2nd acm international conference on multimedia retrieval, ACM, p 1 es_ES
dc.description.references Agarwal R, Verma OP (2019) An efficient copy move forgery detection using deep learning feature extraction and matching algorithm. Multimedia Tools and Applications 79:7355–7376 es_ES
dc.description.references Jaiswal AK, Srivastava R (2019) Image splicing detection using deep residual network. Available at SSRN 3351072 es_ES
dc.description.references Ahmed B, Gulliver TA, et al. (2020) Image splicing detection using mask-rcnn. SIViP, pp 1–8 es_ES
dc.description.references Dias Z, Goldenstein S, Rocha A (2013) Toward image phylogeny forests: Automatically recovering semantically similar image relationships. Forensic science international 231(1-3):178–189 es_ES
dc.description.references (2016) Nist media forensics challenge. https://www.nist.gov/itl/iad/mig/media-forensics-challenge, Accessed: 2018-07-26 es_ES
dc.description.references Medibrowser. http://medifor.rankone.io/ es_ES
dc.description.references Moreira D, Bharati A, Brogan J, Pinto A, Parowski M, Bowyer KW, Flynn PJ, Rocha A, Scheirer WJ (2018) Image provenance analysis at scale. IEEE Trans Image Process 27(12):6109–6123 es_ES
dc.description.references Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826 es_ES
dc.description.references Wojna Z, Ferrari V, Guadarrama S, Silberman N, Chen L-C, Fathi A, Uijlings J (2017July) The devil is in the decoder. arXiv preprint arXiv:1707.05847 es_ES
dc.description.references Dong J, Wang W, Tan T (2013) Casia image tampering detection evaluation database. In: 2013 IEEE China Summit and International Conference on Signal and Information Processing, IEEE, pp 422–426 es_ES
dc.description.references Tralic D, Zupancic I, Grgic S, Grgic M (2013) Comofodnew database for copy-move forgery detection. In: Proceedings ELMAR-2013, IEEE, pp 49–54 es_ES
dc.description.references (2018) Nist media forensics challenge. https://www.nist.gov/itl/iad/mig/media-forensics-challenge-2018, Accessed: 2019-06-11 es_ES
dc.description.references Christlein V, Riess C, Jordan J, Riess C, Angelopoulou E (2012) An evaluation of popular copy-move forgery detection approaches. IEEE Transactions on information forensics and security 7(6):1841–1854 es_ES
dc.description.references Cozzolino D, Poggi G, Verdoliva L (2015) Efficient dense-field copy–move forgery detection. IEEE Transactions on Information Forensics and Security 10 (11):2284–2297 es_ES
dc.description.references Wu Y, Abd-Almageed W, Natarajan P (2017) Deep matching and validation network: An end-to-end solution to constrained image splicing localization and detection. In: Proceedings of the 25th ACM international conference on Multimedia, ACM, pp 1480–1502 es_ES
dc.description.references Garcia M, Canovas A, Edo M, Lloret J (2008) A QoE management system for ubiquitous IPTV devices. In: The Third International Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies (UBICOMM) es_ES


Este ítem aparece en la(s) siguiente(s) colección(ones)

Mostrar el registro sencillo del ítem