Natural Language Processing: Language, Phases and Applications

Hello students, welcome to Kwickprep. A three year old child understands the sentence, the bank is closed. A powerful computer can still get it wrong. Why? Today we learn why human language is hard for computers, the five phases of NLP, and where NLP helps you every single day.

First, let us place NLP on the map of AI. A domain of AI is a big area, grouped by the kind of data it works on. Data Science works on numbers and tables, like predicting a cricket score from past matches. Computer Vision works on images and videos, like face unlock on a phone. Natural Language Processing, or NLP, works on text and speech, like a voice assistant.

So what exactly is NLP? Natural language means a language humans speak, like Hindi, English, Tamil or Gujarati. NLP helps computers understand what we type or say. It also helps computers answer back in our language. It joins linguistics, the study of language, with computer science and AI.

To see the problem, compare a computer language with a human language. A computer language like Python has strict, fixed rules. Every line has exactly one meaning. If you make a small mistake, the program stops with an error. But humans understand each other even with spelling or grammar mistakes. And human language keeps growing, with new words like selfie added all the time.

Here are four reasons why human language is hard for computers. One word can have many meanings, like bank, which can be a river bank or a money bank. The same words in a different order can change the meaning. A sentence can have perfect grammar and still make no sense. And people use sarcasm, slang, and mixed languages like Hinglish.

Let us see each problem in a real sentence. He hit the bat could mean a cricket bat or the flying animal, and only the context tells us. Riya called Aman and Aman called Riya use the same words, but the caller changes. Blue samosas sing loudly has correct grammar, but no meaning at all. And great, it rained, may be said happily by a farmer or angrily by a cricket fan.

India makes NLP even more challenging. The Constitution lists twenty two scheduled languages, and hundreds more are spoken. They use many different scripts. People also mix languages in one sentence, which is called code mixing. And one Hindi word typed in English letters can be spelled in many ways, so a computer sees them as different words.

To handle all this, NLP works in five phases, one after another. Lexical analysis breaks the text into words and checks each word. Syntactic analysis checks the grammar and the order of words. Semantic analysis checks whether the sentence has a sensible meaning. Discourse integration connects the sentence with the sentences before it. Pragmatic analysis finds the real intention, using the situation. At the end, the machine understands what was meant.

Now let us see each phase with a small example. In the lexical phase, I love chai is split into three words, I, love and chai. In the syntactic phase, chai love I is marked wrong, because the order breaks English grammar. In the semantic phase, blue samosas sing is rejected, because it has no sensible meaning. In discourse integration, if Riya came late, she was tired, then she means Riya. In the pragmatic phase, can you pass the salt is understood as a request, not a question about your ability.

Let us test you with one sentence. The cricket bat flew to its cave. Every word is a real English word, so the lexical phase is happy, and the grammar is correct too. So pause and predict, which phase will say something is wrong? It is semantic analysis, because a wooden bat cannot fly to a cave, so the meaning fails.

Now the best part, where NLP is used. Machine translation changes text from one language to another, like Google Translate turning English into Marathi. Voice assistants like Alexa, Siri and Google Assistant hear your question and reply in speech. Spam filters read emails and move junk mail to the spam folder. Autocomplete on your phone keyboard guesses the next word you want to type.

Let us follow a voice assistant step by step. First, speech recognition turns your voice into text. Next, NLP reads the text and finds your intention, for example, you want the weather in Pune. Then the assistant finds the answer from the internet or an app. Text to speech turns the answer back into a voice. Finally, you hear the reply, all within a second or two.

A spam filter is a great example of NLP with machine learning. It learns from lakhs of emails that people have already marked as spam or not spam. It notices word clues, like free prize, click now, or urgent. For every new email, it calculates a spam score. If the score is high, the email goes to the spam folder, and if you mark a mistake, it learns again.

NLP powers many more tools you already use. Chatbots answer customer questions on bank and railway websites. Sentiment analysis reads product reviews and finds if people are happy or unhappy. Text summarisation turns a long news article into a few short lines. And grammar and spelling checkers underline your mistakes while you type.

Let us revise what we learned today. NLP is the domain of AI that works with human text and speech. Language is hard for computers because of many meanings, word order, sarcasm and mixed languages. The five phases are lexical, syntactic, semantic, discourse integration and pragmatic analysis. And NLP powers translation, voice assistants, spam filters and much more.