Waste is any useless substance that disposed of by the people after primary use. It is a resource when it managed properly and placed in the proper place unless it disturbs the societies. Currently, solid wastes can categorize manually by using human labor. This types of classification method has less efficiency, low accuracy, and it takes time. Accurate and proper classification of solid wastes plays an important role to recover recyclable waste materials, protect other wastes from contamination, protecting the environment, and increasing soil fertility in the agricultural sector. There are lots of research done on solid waste identification and classification using different approaches. We develop a model, which works with small sample dataset by merging local descriptor Oriented FAST and Rotated BRIEF features, and high-level Convolutional Neural Network features for the classification of three solid wastes. We used end-to-end Convolutional Neural Network using Softmax classifier and SVM for classification of three waste classes. In this study, median and Gaussian filtering techniques are used to compare their results. In order to proposed the best local feature extractor, we performed a series of experiments on Scale Invariant Feature Transform, Scale UP Robust Features, Histogram Oriented Gradient, and Oriented FAST and Rotated BRIEF, and we achieved an accuracy of 82.05%, 73.85%, 67.69%, and 84.36% respectively. We are margining four local feature vector with the Convolutional Neural Network feature vector in order to get a more discriminative feature of solid waste, then given to SVM classifier to classify the hybrid feature vectors. We achieved an accuracy of 81.70% on end-to-end Convolutional Neural Network using softmax classifier and 98.21 % on Convolutional Neural Network hybrid with Oriented FAST and Rotated BRIEF using SVM classifier. Therefore, Oriented FAST and Rotated BRIEF local features hybrid with high-level Convolutional Neural Network features results in good performance rather than using them individually. Key words: Solid waste classification, feature extraction, CNN, ORB, and SVM