Automatic plant species identification is a very significant innovation that enables botanists and layman to identify the plants from their leaf images. An efficient plant recognition system will be valuable to many parts of society like students, medical field, botanical researches and plant taxonomy studies. The motivation behind the research is our country’s richness in medicinal plants which are used to treat different diseases that might require complex procedures and lots of expense. It is hard to identify these plants unless we are expert in this area. Manual species identification is carried out by answering dozens of often-ambiguous questions and require quite a lot of times which prone to human error. We have considered a dataset of 15 common medicinal plant leaf images captured using digital camera as case study for this thesis work. We have used 105 images per each class and 80% of a dataset used for training and the rest are used for validation. After data organization, image preprocessing such as resizing, histogram equalization by contrast limited adaptive histogram equalization, smoothing with bilateral filter and Otsu's thresholding for segmentation were applied to the labeled data. In this work, for feature extraction convolutional neural network is assisted by handcrafted features like gabor, gray level co-occurrence matrix, Oriented FAST and Rotated BRIEF, and histogram of oriented gradient features. CNN SoftMax, support vector machine, decision tree, k-nearest neighbors, random forest and light GBM algorithms are chosen for comparison. In this proposed system, Support Vector machine classification model accomplishes highest test accuracy of 92.1% and 100% training accuracy as compared to the above state-of-the-art algorithms. Keywords: Oriented FAST and Rotated BRIEF; Histogram of Oriented Gradient; Medicinal plants; Gray Level Co-occurrence Matrix; CNN deep features