- -

Deployment of Elastic Virtual Hybrid Clusters Across Cloud Sites

RiuNet: Repositorio Institucional de la Universidad Politécnica de Valencia

Compartir/Enviar a

Citas

Estadísticas

  • Estadisticas de Uso

Deployment of Elastic Virtual Hybrid Clusters Across Cloud Sites

Mostrar el registro sencillo del ítem

Ficheros en el ítem

Thumbnail
Nombre: CaballerAntonacciSustr ...
Tamaño: 2.232Mb
Formato: PDF
Descripción: Versión del Autor.
Abrir
[Cerrado]
Nombre: Caballer2021_Arti ...
Tamaño: 1.948Mb
Formato: PDF
Descripción: Versión editorial
Solicitar una copia al autor
dc.contributor.author Caballer Fernández, Miguel es_ES
dc.contributor.author Antonacci, Marica es_ES
dc.contributor.author Sustr, Zdenek es_ES
dc.contributor.author Perniola, Michele es_ES
dc.contributor.author Moltó, Germán es_ES
dc.date.accessioned 2022-01-18T19:02:19Z
dc.date.available 2022-01-18T19:02:19Z
dc.date.issued 2021-03 es_ES
dc.identifier.issn 1570-7873 es_ES
dc.identifier.uri http://hdl.handle.net/10251/179913
dc.description.abstract [EN] Virtual clusters are widely used computing platforms than can be deployed in multiple cloud platforms. The ability to dynamically grow and shrink the number of nodes has paved the way for customised elastic computing both for High Performance Computing and High Throughput Computing workloads. However, elasticity is typically restricted to a single cloud site, thus hindering the ability to provision computational resources from multiple geographically distributed cloud sites. To this aim, this paper introduces an architecture of open-source components that coherently deploy a virtual elastic cluster across multiple cloud sites to perform large-scale computing. These hybrid virtual elastic clusters are automatically deployed and configured using an Infrastructure as Code (IaC) approach on a distributed hybrid testbed that spans different organizations, including on-premises and public clouds, supporting automated tunneling of communications across the cluster nodes with advanced VPN topologies. The results indicate that cluster-based computing of embarrassingly parallel jobs can benefit from hybrid virtual clusters that aggregate computing resources from multiple cloud back-ends and bring them together into a dedicated, albeit virtual network. es_ES
dc.description.sponsorship The work presented in this article has been partially funded by project DEEP Hybrid-DataCloud (grant agreement No 777435). GM and MC would also like to thank the Spanish "Ministerio de Economia, Industria y Competitividad" for the project "BigCLOE" with reference number TIN2016-79951-R. Computational resources at CESNET, used in the real-world use case, were supplied by the project "e-Infrastruktura CZ" (e-INFRA LM2018140) provided within the program Projects of Large Research, Development and Innovations Infrastructures es_ES
dc.language Inglés es_ES
dc.publisher Springer-Verlag es_ES
dc.relation.ispartof Journal of Grid Computing es_ES
dc.rights Reserva de todos los derechos es_ES
dc.subject Cloud Computing es_ES
dc.subject Network virtualization es_ES
dc.subject Cluster computing es_ES
dc.subject.classification CIENCIAS DE LA COMPUTACION E INTELIGENCIA ARTIFICIAL es_ES
dc.title Deployment of Elastic Virtual Hybrid Clusters Across Cloud Sites es_ES
dc.type Artículo es_ES
dc.identifier.doi 10.1007/s10723-021-09543-5 es_ES
dc.relation.projectID info:eu-repo/grantAgreement/EC/H2020/777435/EU/ es_ES
dc.relation.projectID info:eu-repo/grantAgreement/AEI//TIN2016-79951-R//COMPUTACION BIG DATA Y DE ALTAS PRESTACIONES SOBRE MULTI-CLOUDS ELASTICOS/ es_ES
dc.rights.accessRights Abierto es_ES
dc.contributor.affiliation Universitat Politècnica de València. Departamento de Sistemas Informáticos y Computación - Departament de Sistemes Informàtics i Computació es_ES
dc.contributor.affiliation Universitat Politècnica de València. Instituto de Instrumentación para Imagen Molecular - Institut d'Instrumentació per a Imatge Molecular es_ES
dc.description.bibliographicCitation Caballer Fernández, M.; Antonacci, M.; Sustr, Z.; Perniola, M.; Moltó, G. (2021). Deployment of Elastic Virtual Hybrid Clusters Across Cloud Sites. Journal of Grid Computing. 19(1):1-16. https://doi.org/10.1007/s10723-021-09543-5 es_ES
dc.description.accrualMethod S es_ES
dc.relation.publisherversion https://doi.org/10.1007/s10723-021-09543-5 es_ES
dc.description.upvformatpinicio 1 es_ES
dc.description.upvformatpfin 16 es_ES
dc.type.version info:eu-repo/semantics/publishedVersion es_ES
dc.description.volume 19 es_ES
dc.description.issue 1 es_ES
dc.relation.pasarela S\428626 es_ES
dc.contributor.funder AGENCIA ESTATAL DE INVESTIGACION es_ES
dc.contributor.funder COMISION DE LAS COMUNIDADES EUROPEA es_ES
dc.description.references Kranzlmüller, D., de Lucas, J.M., Öster, P.: The European Grid Initiative (EGI). In: Remote Instrumentation and Virtual Laboratories. https://doi.org/10.1007/978-1-4419-5597-5∖_6, pp 61–66. Springer, US (2010) es_ES
dc.description.references Altunay, M., Avery, P., Blackburn, K., Bockelman, B., Ernst, M., Fraser, D., Quick, R., Gardner, R., Goasguen, S., Levshina, T., Livny, M., McGee, J., Olson, D., Pordes, R., Potekhin, M., Rana, A., Roy, A., Sehgal, C., Sfiligoi, I., Wuerthwein, F.: A science driven production cyberinfrastructure-the open science grid. J Grid Comput. 9(2), 201–218 (2011). https://doi.org/10.1007/s10723-010-9176-6 es_ES
dc.description.references Medeiros, C.B., Katz, D.S.: EScience today and tomorrow. https://doi.org/10.1016/j.future.2015.10.016 (2016) es_ES
dc.description.references Mell, P., Grance, T.: The NIST definition of cloud computing. NIST special publication 800-145 (Final), Tech. rep. http://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-145.pdf (2011) es_ES
dc.description.references Slurm workload manager. https://slurm.schedmd.com/ es_ES
dc.description.references Thain, D., Tannenbaum, T., Livny, M.: Distributed computing in practice: The Condor experience. https://doi.org/10.1002/cpe.938 (2005) es_ES
dc.description.references Fajardo, E.M., Dost, J.M., Holzman, B., Tannenbaum, T., Letts, J., Tiradani, A., Bockelman, B., Frey, J., Mason, D.: How much higher can HTCondor fly?. In: Journal of Physics: Conference Series, vol 664, Institute of Physics Publishing. https://doi.org/10.1088/1742-6596/664/6/062014(2015) es_ES
dc.description.references De Alfonso, C., Caballer, M., Alvarruiz, F., Moltó, G.: An economic and energy-aware analysis of the viability of outsourcing cluster computing to a cloud. Futur. Gener. Comput. Syst. 29(3), 704–712 (2013). 10.1016/j.future.2012.08.014 es_ES
dc.description.references INFN, INDIGO PaaS orchestrator. https://www.indigo-datacloud.eu/paas-orchestrator es_ES
dc.description.references CESNET, INDIGO virtual router. https://github.com/indigo-dc/ansible-role-indigovr es_ES
dc.description.references U. of Zurich, ElastiCluster. https://github.com/elasticluster/elasticluster es_ES
dc.description.references MIT, StarCluster. http://web.mit.edu/stardev/cluster/ es_ES
dc.description.references Coulter, J.E., Abeysinghe, E., Pamidighantam, S., Pierce, M.: Virtual clusters in the jetstream cloud: A story of elasticized hpc. In: Proceedings of the Humans in the Loop: Enabling and Facilitating Research on Cloud Computing, HARC ’19. https://doi.org/10.1145/3355738.3355752, pp 8:1–8:6. ACM, New York (2019) es_ES
dc.description.references Yu, L., Cai, Z.: Dynamic scaling of virtual clusters with bandwidth guarantee in cloud datacenters. In: IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications, pp 1–9. https://doi.org/10.1109/INFOCOM.2016.7524355 (2016) es_ES
dc.description.references Yu, L., Shen, H., Cai, Z., Liu, L., Pu, C.: Towards bandwidth guarantee for virtual clusters under demand uncertainty in multi-tenant clouds. IEEE Trans. Parallel Distrib. Syst 29(2), 450–465 (2018). https://doi.org/10.1109/TPDS.2017.2754366 es_ES
dc.description.references Caballer, M., De Alfonso, C., Alvarruiz, F., Moltó, G.: EC3: Elastic cloud computing cluster. J. Comput. Syst. Sci. 79(8), 1341–1351 (2013). https://doi.org/10.1016/j.jcss.2013.06.005 es_ES
dc.description.references Sipos, G., La Rocca, G., Scardaci, D., Solagna, P.: The EGI applications on demand service. Futur. Gener. Comput. Syst. 98, 171–179 (2019). https://doi.org/10.1016/j.future.2019.03.009 es_ES
dc.description.references Calatrava, A., Romero, E., Moltó, G., Caballer, M., Alonso, J.M.: Self-managed cost-efficient virtual elastic clusters on hybrid Cloud infrastructures. Futur. Gener. Comput. Syst. 61, 3–25 (2016). https://doi.org/10.1016/j.future.2016.01.018 es_ES
dc.description.references Azure, Azure cyclecloud. https://azure.microsoft.com/en-us/features/azure-cyclecloud/ es_ES
dc.description.references AWS, AWS parallel cluster. https://aws.amazon.com/es/blogs/opensource/aws-parallelcluster/ es_ES
dc.description.references AWS Batch. https://aws.amazon.com/batch/ es_ES
dc.description.references ALCES flight. https://alces-flight.com/ es_ES
dc.description.references Istio. Connect, secure, control, and observe services. https://istio.io/ es_ES
dc.description.references Submariner k8s project documentation website. https://submariner.io/ es_ES
dc.description.references Open netorking environment. https://networkencyclopedia.com/open-network-environment/ es_ES
dc.description.references Open vSwitch. https://www.openvswitch.org/ es_ES
dc.description.references Cloudify. https://cloudify.co/ es_ES
dc.description.references Caballer, M., Zala, S., García, Á. L., Moltó, G., Fernández, P. O., Velten, M.: Orchestrating complex application architectures in heterogeneous clouds. J.Grid Comput. 16 (1), 3–18 (2018). https://doi.org/10.1007/s10723-017-9418-y arXiv:1711.03334 es_ES
dc.description.references Caballer, M., Donvito, G., Moltó, G., Rocha, R., Velten, M.: TOSCA-based orchestration of complex clusters at the IaaS level. J Phys. Conf. Ser. 898, 082036 (2017). https://doi.org/10.1088/1742-6596/898/8/082036http://stacks.iop.org/1742-6596/898/i=8/a=082036?key=crossref.af71f04f17660fdd1e050f7c1e00b643 es_ES
dc.description.references Barisits, M., Beermann, T., Berghaus, F., Bockelman, B., Bogado, J., Cameron, D., Christidis, D., Ciangottini, D., Dimitrov, G., Elsing, M., Garonne, V., di Girolamo, A., Goossens, L., Guan, W., Guenther, J., Javurek, T., Kuhn, D., Lassnig, M., Lopez, F., Magini, N., Molfetas, A., Nairz, A., Ould-Saada, F., Prenner, S., Serfon, C., Stewart, G., Vaandering, E., Vasileva, P., Vigne, R., Wegner, T.: Rucio: Scientific data management. Comput. Softw. Big Sci. 3(1), 11 (2019). https://doi.org/10.1007/s41781-019-0026-3 es_ES
dc.description.references Caballer, M., Blanquer, I., Moltó, G., de Alfonso, C.: Dynamic management of virtual infrastructures. J. Grid Comput. 13(1), 53–70 (2015). https://doi.org/10.1007/s10723-014-9296-5 es_ES
dc.description.references Palma, D., Rutkowski, M., Spatzier, T.: Simple-Profile-YAML-v1.1.html TOSCA Simple Profile in YAML Version 1.1, Tech. rep. http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-YAML/v1.1/TOSCA-Simple-Profile-YAML-v1.1.html (2016) es_ES
dc.description.references de Alfonso, C., Caballer, M., Alvarruiz, F., Hernández, V.: An energy management system for cluster infrastructures. Comput. Electr. Eng. 39(8), 2579–2590 (2013). http://www.sciencedirect.com/science/article/pii/S0045790613001365 es_ES
dc.description.references de Alfonso, C., Caballer, M., Calatrava, A., Moltó, G., Blanquer, I.: Multi-elastic Datacenters: Auto-scaled Virtual Clusters on Energy-Aware Physical Infrastructures. J. Grid Comput. 17(1), 191–204 (2019). https://doi.org/10.1007/s10723-018-9449-z es_ES
dc.description.references Google, Google IPv6. https://www.google.com/intl/en/ipv6/statistics.html es_ES
dc.description.references Addepalli, S.R.: OVN4NFVK8s Plugin. https://github.com/opnfv/ovn4nfv-k8s-plugin (Sood, R.) es_ES
dc.description.references Audio model. https://marketplace.deep-hybrid-datacloud.eu/modules/deep-oc-audio-classification-tf.html es_ES
dc.description.references DEEP open catalog. https://marketplace.deep-hybrid-datacloud.eu es_ES
dc.description.references AudioSet: A large-scale dataset of manually annotated audio events. https://research.google.com/audioset/ es_ES
dc.description.references Gomes, J., Bagnaschi, E., Campos, I., David, M., Alves, L., Martins, J., Pina, J., López-García, A., Orviz, P.: Enabling rootless linux containers in multi-user environments: The udocker tool. Comput. Phys. Commun. 232, 84–97 (2018). https://doi.org/10.1016/j.cpc.2018.05.021 arXiv:1711.01758 es_ES
dc.description.references Salamon, J., Jacoby, C., Bello, J.P.: A dataset and taxonomy for urban sound research. In: MM 2014 - Proceedings of the 2014 ACM Conference on Multimedia, Association for Computing Machinery, Inc, pp 1041–1044. https://doi.org/10.1145/2647868.2655045 (2014) es_ES


Este ítem aparece en la(s) siguiente(s) colección(ones)

Mostrar el registro sencillo del ítem