N-grams for text processing

N-gram is an N sequence of words. It can be a unigram (one word), bigram (sequence of two words), trigram (sequence of three words), and so on. It focuses on a sequence of words. Such a method is very useful in speech recognition and predicting input text. It helps us to predict the next words that could occur in a given sequence. Search engines also use the n-gram technique...

Read More

Word embedding techniques for text processing

 It is difficult to perform analysis on the text so we use word embedding techniques to convert the texts into numerical representation. This is also called vectorization of the words. It is a representation technique for text in which words having same meaning are given similar representation. It helps to extract features from the text.i. Bag of Words (BoW)It is one...

Read More

Easy understanding of Confusion matrix with examples

Confusion matrix is an important metric to evaluate the performance of classifier. The performance of a classifier depends on their capability predict the class correctly against new or unseen data. It is one of the easiest metrics for finding the correctness and accuracy of the model. The confusion matrix in itself is not a performance measure but all the performance metrics...

Read More

Topic Modeling: Working of LSA (Latent Semantic Analysis) in simple terms

 LSA (Latent Semantic Analysis) is another technique used for topic modeling. The main concept behind topic modeling is that the meaning behind any document is based on some latent variables so we use various topic modeling techniques to unravel those hidden variables i.e., topics so that we can make sense of the given document. LSA is mostly suitable for large sets of...

Read More

Bi-LSTM in simple words

 In traditional neural networks, inputs and outputs were independent of each other. To predict next word in a sentence, it is difficult for such model to give correct output, as previous words are required to be remembered to predict the next word. For example, to predict the ending of a movie, it depends on how much one has already watched the movie and what contexts...

Read More

Topic Modeling

 Topic modeling is an unsupervised technique as it is used to perform analysis on text data having no label attached to it. As the name suggests, it is used to discover a number of topics within the given sets of text like tweets, books, articles, and so on. Each topic consists of words where the order of the words does not matter. It performs automatic clustering of...

Read More

Challenges of Sentiment analysis

 Lack of availability of enough domain-specific datasetMulti-class classification of the dataDifficult to extract the context of the post made by the usersHandling of neutral postsAnalyzing the posts of multiple langua...

Read More

Visualizing Decision tree in Python (with codes included)

List of libraries required to be installed (if not already installed). Here installation is done through Jupyter Notebook. For terminal use only "pip install library-name".#import sys#!{sys.executable} -m pip install numpy#!{sys.executable} -m pip install pandas#!{sys.executable} -m pip install sklearn#!{sys.executable} -m pip install hvplot#!{sys.executable} -m pip install...

Read More

Web scrapping using a single line of code in python

We will scrap the data of wikipedia using a single line of code in python. No extra libraries are required. Only Pandas can do the job.Step 1: Install and import pandas libraryimport numpy as np Step 2: Read the data of web (here Wikipedia website) using pd.read_html('Website link here')[integer]df = pd.read_html('https://en.wikipedia.org/wiki/COVID-19_pandemic_by_country_and_territory')[1]Step...

Read More

Build a colorful Word Cloud in python using mask image

Word cloud is a data visualization tool in data science. It is very efficient to visualize various words in a text according to the quantum of their repetition within the text. The stopwords have been ignored while visualization. A text file called "skill.txt" has been used to visualize. Mask image of map of Nepal has been used to visualize the word cloud.The libraries required...

Read More

Covid-19 Data Visualization across the World using Choropleth map

IntroductionThis project visualizes the Covid-19 data (i.e. Total cases, Deaths and Recoveries) across various Provinces and Districts of Nepal as of 12th August, 2020. Geojson file of Nepal's states and districts have been used. Also python library i.e. Folium has been used to generate Choropleth map whose geo_data value is the geojson of Nepal.The libraries imported are:Data...

Read More