Ethiopia is a country of linguistic diversity. There are about 80 and more languages spoken in the country. Amharic and Kistanigna are among those languages. Moreover, English is one of technologically favored languages in which bunch of useful documents published. The necessity of automatic translation, Machine Translation, comes here to exchange knowledge, information, and ideas between languages as fast as possible as human translation tends to be slower than machines in comparison. Machine Translation is a Natural Language Processing application, that studies the use of computer software to translate a natural language into another language in the form of text or speech. This study is intended to design trilingual Amharic, Kistanigna, and English bi-directional translation. The experiment is conducted using a self-owned attention model which is transformer. 10,233 pairs of datasets of the three languages were collected to conduct our experiment, and 80% of the data (8,187) was used for training and 20% (2,046) for testing. The experiment was conducted on word-level and morpheme-level. Morfessor was used in segmenting words into morphemes. Amharic – Kistanigna, Kistanigna – English, and Amharic – English bi-directional translation in word and morpheme-base translational unit tested and morpheme-base translation unit got high BLEU score result and lesser training time scored. Morpheme based Amharic – Kistanigna, and Kistanigna – Amharic got 24.75 and 23.50 BLEU score, respectively. The Kistanigna – English scored 20.70 and the revers way scored BLEU score of 18.99. In addition, Amharic – English and English – Amharic got BLEU score of 22.89 and 19.67, respectively. Key Words: Machine Translation, Trilingual Machine Translation, Deep learning, Transformer, Amharic, Kistanigna, Kistane af