Natural Language Processing with Python-NLP Series (Part 1)- Introduction

This guide will help you navigate learning Natural Language Processing concepts from beginner to advance level. This will include the learning resources that I found effective in the past, along with supplementary explanations. I will also share my production level experience where ever required.

To keep this series digestible, I will cover everything from basic’s of NLP to production level projects in various parts and you can use this to start building your knowledge one after the other to be a NLP Hero.

At the end, I will share the python code which will help you apply what we have discussed, this will help learning by doing.

Introduction:

What is Natural Language ? For us,it is a mean to express our thoughts and ideas.But in the Digital age we are constantly interacting with machines around us through various means, such as typing instruction in form of words like interacting with chatbots, voice command like Alexa, Siri. Thus it has become essential to develop mechanisms by which human language can be well understood by computers. NLP helps us achieve this.

NLP can be defined as a field of computer science that is concerned with enabling computer algorithms to understand,analyze and generate natural language.

Let’s look at an example. You have probably interacted with Siri or Alexa at some point. Ask Alexa for weather status, and it will reply with the current condition. The technology behind this is NLP. Siri and Alexa use techniques such as Speech to Text with the help of a search engine to do this magic. As the name suggests, Speech to Text is an application of NLP in which computers are trained to understand verbally spoken words.

Difference between Text Analytics and NLP:

Text analytics is the process of extracting meaningful insights and answering questions from textual data, such as the length of sentences, length of words, word count, and finding words from the text.Lets understand it better with an example.

Ex: Suppose we want to find the name of top five countries who have contributed the most in the field of Medical science in past 5 years, steps involved are:

• collect all the medical science related news from past 5 years using google news API.
• extract names of countries in these news articles.
• create a dictionary in which keys will be the country names and their
  values will be the number of times the country name is found in the news articles.
• After we have completed searching all the news articles, we can sort the country names by the values associated with them.

In this way, we will come up with the top five countries that contributed the most to the medical science field in the last 5 years.This is a typical example of text analytics, in which we are generating insights from text without getting into the semantics of the language.

NLP helps us in understanding the semantics i.e. the underlying meaning of text, like the sentiment of a sentence whether its positive of negative, top keywords in a document i.e. the major contributor in a document using Topic modeling (we will discuss this in upcoming articles), and parts of speech for different words. NLP is not restricted to text data, we have methods to make a machine understand voice i.e. speech recognition which enable us to communicate with machine in more Natural way.Before we move ahead its important to understand two more terms NLU and NLG.

Difference between NLU and NLG : Both are subset of NLP.

NLU : Natural Language Understanding is the ability of machine to understand the text.Sometimes what is said or written may not be same, there can be flaws and mistakes, here NLU comes into picture and it make sure that it will infer correct intent and meaning even the data is written and spoken with some error.NLU helps achieving better NLP.It reads the data and convert it into structured data.

NLG : Natural Language Generation helps generating human like text based on structured data.As we discussed above NLU generate structure data but it is not necessary that the generated text is easy to understand by human, here NLG make sure that the text generated is human readable.

So, we can say when a human speaks to a machine (Speech to Text), the machine interprets the language with the help of the NLU process. By using the NLG process, the machine generates an appropriate response and shares it with the human (Text to Speech), thus making it easier for humans to understand the machine response.In short :

• NLP is what happen when computer’s read the language.
• NLG is what happen when computer’s write the language.
• NLU is what happen when computer’s understand the language.

Wrap Up

Hopefully by the end of this article, you are familiar with an introduction to NLP, how we are making machines understand and response to our questions in more natural way using the NLP,NLU and NLG concepts.

We have just touched the surface and a lot to cover in this series.In the future post we will talk about building blocks of NLP task i.e.Tokenizing,
parts-of-speech, Tagging, Stemming, Lemmatization concepts and more.

P.S. If you liked this article, it would mean a lot if you hit the recommended button or share with friends.