Abstract—Recent advancements in deep learning (DL) frameworks based on deep neural networks (DNN) drastically improved accuracy in image recognition, natural language processing and other applications. The key advantage of DL is systematic approach for independent training of groups of DNN layers including unsupervised training of auto-encoders for hierarchical representation of raw input data (i.e., automatic feature selection and dimensionality reduction) and supervised re-training of several final layers in the transfer learning that compensate for data incompleteness. However, severe data limitations and/or absence of relevant problem for transfer learning can drastically reduce advantages of DNN-based DL. For example, pure data-driven auto-encoders dealing with high-dimensional input data require large amount of data for effective operation. However, hierarchical data representations can be also implemented without NN. Previously we have shown robustness of boosting-like algorithms for effective utilization of existing domain knowledge (e.g. analytical models) via discovery of compact ensembles of complementary low-complexity components. This approach can tolerate significant data incompleteness and boost accuracy of individual base models as was demonstrated in cardiac diagnostics applications. Here we argue that hybrid DL framework with auto-encoders replaced by components discovered by boosting followed by supervised NN could be more tolerant to data incompleteness compared to pure DNN-based DL. Illustrations based on cardio data from www.physionet.org are presented. The proposed framework could be utilized in many applications dealing with incomplete data including personalized medicine and rare or complex abnormalities.
Index Terms—Auto-encoders, boosting, cardiac diagnostics, complexity measures, deep learning, ECG, ensemble learning, heart rate variability, hybrid learning, neural networks, physiological time series.
V. Gavrishchaka is with the West Virginia University, Physics Department, Morgantown, WV 26506 USA (e-mail: firstname.lastname@example.org).
Z. Yang and R. Miao are with the Applied Quantitative Solutions for Complex Systems (www.aqscs.com), Falls Church, VA 22041 USA (e-mail: email@example.com, firstname.lastname@example.org).
O. Senyukova is with the Lomonosov Moscow State University, Faculty of Computational Mathematics and Cybernetics, Moscow 119991 Russian Federation (e-mail: email@example.com).
Cite: Valeriy Gavrishchaka, Zhenyi Yang, Rebecca Miao, and Olga Senyukova, "Advantages of Hybrid Deep Learning Frameworks in Applications with Limited Data," International Journal of Machine Learning and Computing vol. 8, no. 6, pp. 549-558, 2018.