Deep Insights of Erroneous Bengali–English Code-Mixed Bilingual Language

ABSTRACTCode-switching, the combination of more than one language within a single utterance is popular in social media sites for informal communication, generating a huge amount of data, though not fully analysed for knowledge extraction due to lack of documentation. Moreover, the code-mixed data is erroneous, and to detect and correct different types of errors, the model requires large erroneous code-mixed language data, not publicly available. This paper first defines generic rules to write Bengali–English code-mixed language in English script, considering the inherent complexity of the Bengali language. Different types of typographical and cognitive errors are induced to obtain the huge erroneous data, based on human behaviour and perception as consulted with the language experts. The errors considered here are applicable to other code-mixed Indic languages and would be beneficial for the researchers. To demonstrate the applicability of the model, we have included the errors in the Hindi–English code-mixed Indic language. An attention-based two-level deep network architecture (uses LSTM as a basic unit) is used for error detection, correction, and translation of code-mixed sentences into a monolingual sentence. Results are reported in terms of accuracy, ROUGE score, and BLEU scores at word level and sentence level for both Bengali–English and Hindi–English code-mixed languages.