- -

On-The-Fly Processing of continuous high-dimensional data streams

RiuNet: Institutional repository of the Polithecnic University of Valencia

Share/Send to

Cited by

Statistics

On-The-Fly Processing of continuous high-dimensional data streams

Show simple item record

Files in this item

dc.contributor.author Vitale, Raffaele es_ES
dc.contributor.author Zhyrova, Anna es_ES
dc.contributor.author Fortuna, Joao F. es_ES
dc.contributor.author De Noord, Onno E. es_ES
dc.contributor.author Ferrer, Alberto es_ES
dc.contributor.author Martens, Harald es_ES
dc.date.accessioned 2018-07-09T06:38:42Z
dc.date.available 2018-07-09T06:38:42Z
dc.date.issued 2017 es_ES
dc.identifier.issn 0169-7439 es_ES
dc.identifier.uri http://hdl.handle.net/10251/105527
dc.description.abstract [EN] A novel method and software system for rational handling of time series of multi-channel measurements is presented. This quantitative learning tool, the On-The-Fly Processing (OTFP), develops reduced-rank bilinear subspace models that summarise massive streams of multivariate responses, capturing the evolving covariation patterns among the many input variables over time and space. Thereby, a considerable data compression can be achieved without significant loss of useful systematic information. The underlying proprietary OTFP methodology is relatively fast and simple it is linear/bilinear and does not require a lot of raw data or huge cross-correlation matrices to be kept in memory. Unlike conventional compression methods, the approach allows the high-dimensional data stream to be graphically interpreted and quantitatively utilised in its compressed state. Unlike adaptive moving-window methods, it allows all past and recent time points to be reconstructed and displayed simultaneously. This new approach is applied to four different case-studies: (i) multi-channel Vis-NIR spectroscopy of the Belousov Zhabotinsky reaction, a complex, ill understood chemical process; (ii) quality control of oranges by hyperspectral imaging; (iii) environmental monitoring by airborne hyperspectral imaging; (iv) multi-sensor process analysis in the petrochemical industry. These examples demonstrate that the OTFP can automatically develop high-fidelity subspace data models, which simplify the storage/transmission and the interpretation of more or less continuous time series of high-dimensional measurements to the extent there are covariations among the measured variables. es_ES
dc.description.sponsorship This research work was partially supported by the Spanish Ministry of Economy and Competitiveness under the project DPI2014-55276-C5-1R, Shell Global Solutions International B.V. (Amsterdam, The Netherlands), Idletechs AS (Trondheim, Norway), the Norwegian Research Council (Grant 223254) through the Centre of Autonomous Marine Operations and Systems (AMOS) at the Norwegian University of Science and Technology (Trondheim, Norway) and the Ministry of Education, Youth and Sports of the Czech Republic (CENAKVA project CZ.1.05/2.1.00/01.0024 and CENAKVA II project L01205 under the NPU I program). The authors want to acknowledge Prof. Bjorn Alsberg for providing the Vis-NIR equipment and the Laboratorio de Sistemas e Tecnologia Subaquatica of the University of Porto, the Hydrographic Institute of the Portuguese Navy and the University of the Azores for carrying out the REP15 exercise, during which the hyperspectral push broom image was collected. en_EN
dc.language Inglés es_ES
dc.publisher Elsevier es_ES
dc.relation info:eu-repo/grantAgreement/MINECO//DPI2014-55276-C5-1-R/ES/BIOLOGIA SINTETICA PARA LA MEJORA EN BIOPRODUCCION: DISEÑO, OPTIMIZACION, MONITORIZACION Y CONTROL/ es_ES
dc.relation.ispartof Chemometrics and Intelligent Laboratory Systems es_ES
dc.rights Reserva de todos los derechos es_ES
dc.subject On-the-fly processing (OTFP) es_ES
dc.subject Bilinear modelling es_ES
dc.subject High-dimensional data streams es_ES
dc.subject Generalised Taylor expasion es_ES
dc.subject Singular value decomposition (SVD) es_ES
dc.subject BIG DATA analytics es_ES
dc.subject.classification ESTADISTICA E INVESTIGACION OPERATIVA es_ES
dc.title On-The-Fly Processing of continuous high-dimensional data streams es_ES
dc.type Artículo es_ES
dc.identifier.doi 10.1016/j.chemolab.2016.11.003 es_ES
dc.rights.accessRights Abierto es_ES
dc.date.embargoEndDate 2019-06-15 es_ES
dc.contributor.affiliation Universitat Politècnica de València. Departamento de Estadística e Investigación Operativa Aplicadas y Calidad - Departament d'Estadística i Investigació Operativa Aplicades i Qualitat es_ES
dc.description.bibliographicCitation Vitale, R.; Zhyrova, A.; Fortuna, JF.; De Noord, OE.; Ferrer, A.; Martens, H. (2017). On-The-Fly Processing of continuous high-dimensional data streams. Chemometrics and Intelligent Laboratory Systems. 161:118-129. doi:10.1016/j.chemolab.2016.11.003 es_ES
dc.description.accrualMethod S es_ES
dc.relation.publisherversion https://doi.org/10.1016/j.chemolab.2016.11.003 es_ES
dc.description.upvformatpinicio 118 es_ES
dc.description.upvformatpfin 129 es_ES
dc.type.version info:eu-repo/semantics/publishedVersion es_ES
dc.description.volume 161 es_ES
dc.relation.pasarela S\336092 es_ES
dc.contributor.funder Ministerio de Economía, Industria y Competitividad es_ES
dc.contributor.funder Shell Global Solutions International B.V. es_ES


This item appears in the following Collection(s)

Show simple item record