Abstract

Current state-of-the-art out-of-distribution algorithm does not address the variation in dynamic and static behavior between malware variants from the same family as evidence in their poor performance against an out-of-distribution malware attack. We aims to address this limitation by: 1) exploitation of the in-dimensional embedding space between variants from the same malware family to account for all variations 2) exploitation of the inter-dimensional space between different malware family 3) building a deep learning-based model with a shallow neural network with maximum of two connected layers to overcome overfitting from the scratch 4) building a Bayesian inference based computation algorithm that intertwine with connected network and is able to create new and adjust existing data points in response to an exposure to new out-of-distribution variants of existing or new malware family which determines the extent at which model weight is adjusted which in turn triggers update on the gradient. Preliminary result of our proposed framework gave an accuracy of 81\% in the successful classification of a novel out-of-distribution malware attack, something that could not be achieve by any of the state-of-the-art algorithms on novel malware classification.