Nowadays, natural language processing became a hot research area, that is mainly focused on maximizing the capability of the computer to understand and communicate with human language or natural language. Therefore, to communicate with natural language or human language, grammatical correctness of the spoken language is important. So, developing a natural language application is important to identify the grammatical error that may occur in natural language texts. To say a sentence is grammatically correct, the arrangement of the words inside the sentence should agree in number, person, gender, tense, and other agreement rules. If the input sentence is incorrect, it may have agreement problems, such as subject-verb, object-verb, adjective-noun, incorrect word order or it may be adverb-verb agreement problems. In order to check the grammatical correctness of a sentence, several researches have been conducted for different languages with different grammar checking approaches, like rule-based, statistical-based and hybrid-based. Nowadays, deep learning becomes the most promising approach for natural language processing. The objective of this proposed work is to develop deep learning based Amharic grammar error detection. To this end, we propose a deep learning grammar checker approach. We apply two deep learning approaches such as long short-term memory recurrent neural network and bidirectional long short-term memory recurrent neural network. We have used python 3.7, Keras TensorFlow as a backend, Pyqt5 to design the interface, and HornMorpho to analyze the feature of Amharic words. The evaluation is done for two test cases. The first one is for long short-term memory and the second one is for bidirectional long short-term memory recurrent neural network. Finally, the experimental result shows that, long short-term memory performs accuracy of 88.27%, recall 88.27%, precision of 88.33%, and f1 measure of 88.5%. The bidirectional long short-term memory performs 88.89% accuracy, 88.89% of recall, 89 % precision and 89% of f1 measure. The challenge of this research work was the quality of morphologically annotated Amharic sentence especially words having more than two meanings and words that tell respect. The grammar error detector can be more effective when we have a larger morphologically annotated sentence and hence further research needs to be done to enhance the result of this study.