Predictive Maintenance (PdM) helps to determine the condition of in-service industrial equipment and components and their timely replacement. This can be achieved by Artificial Intelligence (AI) enabled information systems. AI has been used extensively in addressing the condition monitoring problems. Most existing Deep Neural Network (DNN) models which are capable of solving PdM problems have a large memory foot print and are functional on remote machines using cloud based infrastructure only. In order to inference them close to the process, they need to run on memory constrained devices like microcontrollers (MCUs). In this work, we propose a weighted pruning algorithm to reduce the number of trainable parameters in the DNN model for bearing fault classification to enable its execution on the MCU. In addition to the pruning, we reduce the trainable model parameters by making an extensive filter size search. The model size is reduced without compromising on the performance of the pruned models by using the magnitude based method. In case of AlexNet, LeNet and Autoencoder we could reduce the model size upto 89%, 39% and 54% respectively with the new approach in comparison to the magnitude based state of the art approach.