- -

Identifying the Machine Learning Family from Black-Box Models

RiuNet: Repositorio Institucional de la Universidad Politécnica de Valencia

Compartir/Enviar a

Citas

Estadísticas

  • Estadisticas de Uso

Identifying the Machine Learning Family from Black-Box Models

Mostrar el registro sencillo del ítem

Ficheros en el ítem

Thumbnail
Nombre: Fabra-Boluda;Ferr ...
Tamaño: 1.337Mb
Formato: PDF
Descripción: Versión del Autor.
Abrir
[Cerrado]
Nombre: PUBL031_CAEPIA_20 ...
Tamaño: 1.245Mb
Formato: PDF
Descripción: Versión editorial
Solicitar una copia al autor
dc.contributor.author Fabra-Boluda, Raúl es_ES
dc.contributor.author Ferri Ramírez, César es_ES
dc.contributor.author Hernández-Orallo, José es_ES
dc.contributor.author Martínez-Plumed, Fernando es_ES
dc.contributor.author Ramírez Quintana, María José es_ES
dc.date.accessioned 2020-07-09T03:31:59Z
dc.date.available 2020-07-09T03:31:59Z
dc.date.issued 2018-09-27 es_ES
dc.identifier.issn 0302-9743 es_ES
dc.identifier.uri http://hdl.handle.net/10251/147681
dc.description.abstract [EN] We address the novel question of determining which kind of machine learning model is behind the predictions when we interact with a black-box model. This may allow us to identify families of techniques whose models exhibit similar vulnerabilities and strengths. In our method, we first consider how an adversary can systematically query a given black-box model (oracle) to label an artificially-generated dataset. This labelled dataset is then used for training different surrogate models (each one trying to imitate the oracle¿s behaviour). The method has two different approaches. First, we assume that the family of the surrogate model that achieves the maximum Kappa metric against the oracle labels corresponds to the family of the oracle model. The other approach, based on machine learning, consists in learning a meta-model that is able to predict the model family of a new black-box model. We compare these two approaches experimentally, giving us insight about how explanatory and predictable our concept of family is. es_ES
dc.description.sponsorship This material is based upon work supported by the Air Force Office of Scientific Research under award number FA9550-17-1-0287, the EU (FEDER), and the Spanish MINECO under grant TIN 2015-69175-C4-1-R, the Generalitat Valenciana PROMETEOII/2015/013. F. Martinez-Plumed was also supported by INCIBE under grant INCIBEI-2015-27345 (Ayudas para la excelencia de los equipos de investigacion avanzada en ciberseguridad). J. H-Orallo also received a Salvador de Madariaga grant (PRX17/00467) from the Spanish MECD for a research stay at the CFI, Cambridge, and a BEST grant (BEST/2017/045) from the GVA for another research stay at the CFI. es_ES
dc.language Inglés es_ES
dc.publisher Springer-Verlag es_ES
dc.relation.ispartof Lecture Notes in Computer Science es_ES
dc.relation.ispartof ADVANCES IN ARTIFICIAL INTELLIGENCE, CAEPIA 2018 es_ES
dc.rights Reserva de todos los derechos es_ES
dc.subject Machine learning families es_ES
dc.subject Black-box model es_ES
dc.subject Dissimilarity measures es_ES
dc.subject Adversarial machine learning es_ES
dc.subject.classification LENGUAJES Y SISTEMAS INFORMATICOS es_ES
dc.title Identifying the Machine Learning Family from Black-Box Models es_ES
dc.type Artículo es_ES
dc.type Comunicación en congreso es_ES
dc.type Capítulo de libro es_ES
dc.identifier.doi 10.1007/978-3-030-00374-6_6 es_ES
dc.relation.projectID info:eu-repo/grantAgreement/MECD//PRX17%2F00467/ es_ES
dc.relation.projectID info:eu-repo/grantAgreement/GVA//BEST%2F2017%2F045/ es_ES
dc.relation.projectID info:eu-repo/grantAgreement/GVA//PROMETEOII%2F2015%2F013/ES/SmartLogic: Logic Technologies for Software Security and Performance/ es_ES
dc.relation.projectID info:eu-repo/grantAgreement/MINECO//TIN2015-69175-C4-1-R/ES/SOLUCIONES EFECTIVAS BASADAS EN LA LOGICA/ es_ES
dc.relation.projectID info:eu-repo/grantAgreement/AFOSR//FA9550-17-1-0287/US/Who's behind these predictions? Reconciling transparency and privacy in machine learning/ es_ES
dc.relation.projectID info:eu-repo/grantAgreement/INCIBE//INCIBEI-2015-27345/ es_ES
dc.rights.accessRights Abierto es_ES
dc.contributor.affiliation Universitat Politècnica de València. Departamento de Sistemas Informáticos y Computación - Departament de Sistemes Informàtics i Computació es_ES
dc.description.bibliographicCitation Fabra-Boluda, R.; Ferri Ramírez, C.; Hernández-Orallo, J.; Martínez-Plumed, F.; Ramírez Quintana, MJ. (2018). Identifying the Machine Learning Family from Black-Box Models. Lecture Notes in Computer Science. 11160:55-65. https://doi.org/10.1007/978-3-030-00374-6_6 es_ES
dc.description.accrualMethod S es_ES
dc.relation.conferencename XVIII Conferencia de la Asociación Española para la Inteligencia Artificial (CAEPIA'18) es_ES
dc.relation.conferencedate Octubre 23-26,2018 es_ES
dc.relation.conferenceplace Granada, España es_ES
dc.relation.publisherversion https://doi.org/10.1007/978-3-030-00374-6_6 es_ES
dc.description.upvformatpinicio 55 es_ES
dc.description.upvformatpfin 65 es_ES
dc.type.version info:eu-repo/semantics/publishedVersion es_ES
dc.description.volume 11160 es_ES
dc.relation.pasarela S\386164 es_ES
dc.contributor.funder Generalitat Valenciana es_ES
dc.contributor.funder European Regional Development Fund es_ES
dc.contributor.funder Air Force Office of Scientific Research es_ES
dc.contributor.funder Ministerio de Educación, Cultura y Deporte es_ES
dc.contributor.funder Instituto Nacional de Ciberseguridad es_ES
dc.contributor.funder Ministerio de Economía y Competitividad es_ES
dc.description.references Angluin, D.: Queries and concept learning. Mach. Learn. 2(4), 319–342 (1988) es_ES
dc.description.references Benedek, G.M., Itai, A.: Learnability with respect to fixed distributions. Theor. Comput. Sci. 86(2), 377–389 (1991) es_ES
dc.description.references Biggio, B., et al.: Security Evaluation of support vector machines in adversarial environments. In: Ma, Y., Guo, G. (eds.) Support Vector Machines Applications, pp. 105–153. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-02300-7_4 es_ES
dc.description.references Blanco-Vega, R., Hernández-Orallo, J., Ramírez-Quintana, M.J.: Analysing the trade-off between comprehensibility and accuracy in mimetic models. In: Suzuki, E., Arikawa, S. (eds.) DS 2004. LNCS (LNAI), vol. 3245, pp. 338–346. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30214-8_29 es_ES
dc.description.references Dalvi, N., Domingos, P., Sanghai, S., Verma, D., et al.: Adversarial classification. In: Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 99–108. ACM (2004) es_ES
dc.description.references Dheeru, D., Karra Taniskidou, E.: UCI machine learning repository (2017). http://archive.ics.uci.edu/ml es_ES
dc.description.references Domingos, P.: Knowledge discovery via multiple models. Intell. Data Anal. 2(3), 187–202 (1998) es_ES
dc.description.references Duin, R.P.W., Loog, M., Pȩkalska, E., Tax, D.M.J.: Feature-based dissimilarity space classification. In: Ünay, D., Çataltepe, Z., Aksoy, S. (eds.) ICPR 2010. LNCS, vol. 6388, pp. 46–55. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-17711-8_5 es_ES
dc.description.references Fernández-Delgado, M., Cernadas, E., Barro, S., Amorim, D.: Do we need hundreds of classifiers to solve real world classification problems. J. Mach. Learn. Res. 15(1), 3133–3181 (2014) es_ES
dc.description.references Ferri, C., Hernández-Orallo, J., Modroiu, R.: An experimental comparison of performance measures for classification. Pattern Recognit. Lett. 30(1), 27–38 (2009) es_ES
dc.description.references Giacinto, G., Perdisci, R., Del Rio, M., Roli, F.: Intrusion detection in computer networks by a modular ensemble of one-class classifiers. Inf. Fusion 9(1), 69–82 (2008) es_ES
dc.description.references Huang, L., Joseph, A.D., Nelson, B., Rubinstein, B.I., Tygar, J.: Adversarial machine learning. In: Proceedings of the 4th ACM Workshop on Security and Artificial Intelligence, pp. 43–58 (2011) es_ES
dc.description.references Kuncheva, L.I., Whitaker, C.J.: Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Mach. Learn. 51(2), 181–207 (2003) es_ES
dc.description.references Landis, J.R., Koch, G.G.: An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics 33, 363–374 (1977) es_ES
dc.description.references Lowd, D., Meek, C.: Adversarial learning. In: Proceedings of the 11th ACM SIGKDD International Conference on Knowledge Discovery in Data mining, pp. 641–647. ACM (2005) es_ES
dc.description.references Martınez-Plumed, F., Prudêncio, R.B., Martınez-Usó, A., Hernández-Orallo, J.: Making sense of item response theory in machine learning. In: Proceedings of 22nd European Conference on Artificial Intelligence (ECAI). Frontiers in Artificial Intelligence and Applications, vol. 285, pp. 1140–1148 (2016) es_ES
dc.description.references Papernot, N., McDaniel, P., Goodfellow, I.: Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:1605.07277 (2016) es_ES
dc.description.references Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z.B., Swami, A.: The limitations of deep learning in adversarial settings. In: 2016 IEEE European Symposium on Security and Privacy (EuroS&P), pp. 372–387. IEEE (2016) es_ES
dc.description.references Papernot, N., McDaniel, P., Wu, X., Jha, S., Swami, A.: Distillation as a defense to adversarial perturbations against deep neural networks. In: 2016 IEEE Symposium on Security and Privacy (SP), pp. 582–597. IEEE (2016) es_ES
dc.description.references Sesmero, M.P., Ledezma, A.I., Sanchis, A.: Generating ensembles of heterogeneous classifiers using stacked generalization. Wiley Interdiscip. Rev.: Data Min. Knowl. Discov. 5(1), 21–34 (2015) es_ES
dc.description.references Smith, M.R., Martinez, T., Giraud-Carrier, C.: An instance level analysis of data complexity. Mach. Learn. 95(2), 225–256 (2014) es_ES
dc.description.references Tramèr, F., Zhang, F., Juels, A., Reiter, M.K., Ristenpart, T.: Stealing machine learning models via prediction APIs. In: USENIX Security Symposium, pp. 601–618 (2016) es_ES
dc.description.references Valiant, L.G.: A theory of the learnable. Commun. ACM 27(11), 1134–1142 (1984) es_ES
dc.description.references Wallace, C.S., Boulton, D.M.: An information measure for classification. Comput. J. 11(2), 185–194 (1968) es_ES
dc.description.references Wolpert, D.H.: Stacked generalization. Neural Netw. 5(2), 241–259 (1992) es_ES


Este ítem aparece en la(s) siguiente(s) colección(ones)

Mostrar el registro sencillo del ítem