Data Wrangling using Python- Part 1

In this blog, we will show some of the commonly used data wrangling steps using Python.  We will be using pandas data frame as our data object to show all the steps. Importing Python Packages In this part of blog, we will use pandas and numpy packages available in Python. We need to import these ... Read more

Reading Text File in Python

Reading data into an analytical tool is one of the first steps before proceeding to analytics. Some of the common challenges while reading a text file are Knowing Delimiter Presence of Missing  or null values Column values holding date values In this blog, we will read text file in which values are separated by a ... Read more

Python Learning - Finding Answers

In this blog, we are sharing some of the scenario arose while working on a project and we are providing the steps to get the work done. I am sure, there would be multiple ways to achieve the outcome but I am sharing my solutions.   Q1:  How do we extract id value from html ... Read more

Visualisation using R - Commonly used functions

We will discussing some of the commonly used Base R Graphic functions.  Some of the commonly used functions are plot: Plotting Line Chart and Scatter Plot boxplot: Box Whiskers Plot for a continuous variable  or distributions by different groups hist: Histogram Scatter  Plot We will create a sample data points and then use for the scatter ... Read more

Powerful Proc Tabulate Explained

In this blog, we will use PROC TABULATE , one of the most powerful PROCS for data summarization using SAS. For creating summary table of the information (similar to Pivot in Excel), we need to define Classification Variables, also called dimensions Measurement Variables, also called Facts Structure of the Output Summary Tables PROC FREQ, PROC ... Read more

Concatenating Datasets in SAS

Author: Mrinmoy Saikia Data preparation is one of the most significant steps in Data Science, Analytics or Reporting Projects. In this blog, we focus on learning - "Combining 2 or more SAS datasets Vertically". Combining SAS datasets vertically is also referred as Concatenating Datasets. Concatenating is combining 2 or more datasets one below another. From ... Read more

Data Science: Profile Screening Model for Mid-Management Roles

Business Context: The client was an executive search firm. It has built a candidate database with over a million candidate profiles.  The client wanted to leverage the candidate database for smart candidate selection and recruitment process. For this project, the aim was to build a predictive model which will help in identifying a list of ... Read more

SAS Control Statements

In this blog, we will discuss some of the SAS control statements with examples. IF/WHERE Statement IF or WHERE statement is applied to select observations. WHERE condition is applied while reading observations from Input dataset where as IF condition is applied at Program Data Vector (PDV). Using the Where statement may improve the efficiency of ... Read more

Chi Square Test using SAS

A chi-square test is an statistical method to test association between two categorical variables (especially between nominal variables).  Type of Variables. Correlation Analysis: When both the variables are continuous, and it can be done using Pearson Correlation Coefficient.  Correlation Analysis. ANOVA: One variable is categorical and other variable is continuous. Finding how levels of categorical variable ... Read more

ANOVA using SAS and Example

Analysis of Variance (ANOVA) is used for comparing means across multiple samples. Focus here is only 1-Way ANOVA and there are a few different ways of applying similar concepts to different scenarios. If number of samples or groups is one or two, we can use T Test (T Test using SAS). Using one categorical variables, ... Read more

Retain Statement - Explained with Examples

Author: Rameshwari, done SAS for Data Analytics Training course from DnI Institute SAS programs are made of statements and each statements end with semi comma -";" . One of the important statements in SAS is RETAIN statement. Why do we need Retain Statement? Important point about SAS program is that in majority of  cases SAS ... Read more

Machine Learning for Retailers

Retail industry has been in the forefront in adopting Data Science and Machine Learning. Some of the common applications of Data Science and Machine Learning are list below. Definitely, these are not exhaustive and smart professionals are bringing new applications everyday. Some of the commonly used Machine Learning and Data Science Techniques are Logistic Regression ... Read more

Machine Learning - Steps to Build Regression Model

A number of real life business decisions are of a regression in nature. We have prepared a  detailed list of regression modelling scenarios. Business Scenario: House Price Prediction We all want to have our own house and price of a house is a imprtant link betewen our wish and owning a house. We want to build ... Read more

Scenarios: Binary Predictive Models

A long list of business decisions are of binary in nature and we will list a few of such scenarios. When a decision variable (also referred as target variable, Response variable or dependent variable) is binary (takes only two value), a long list of supervised statistical and machine learning algorithms can be used. Some of ... Read more

Logistic Regression using R: German Credit Example

Logistic Regression is one of the oldest  and widely used Statistical/Machine Learning techniques for Binary Decision Variable scenarios. In the previous blog, we have explained the overall steps to build a predictive model using Logistic Regression.   Also, if you are interested to understand Binary Model Performance Statistics, you can read a detailed blog on Model ... Read more

10 Most Commonly Used Character Functions in SAS

In this blog, we will discuss 10 most common text manipulation functions in SAS. These functions can be used for preparing data for text analytics or predictive model development. Length of String: LENGTH, LENGTHC and LENGTHN Change Case of String: LOWCASE, UPCASE and PROPCASE String Alignment: LEFT and RIGHT Remove or Trim Leading and Trailing Blanks: TRIM ... Read more

K Means Clustering Examples and Practical Applications

Pricing Segmentation: E-retailers or e-commerce companies have taken the retail industry by storm. They are offering luring offers and discounts. They aim to move from discount led to convenience or differentiation led offering over a period in time. Some of them have been forced to start the journey. One of the large retailer wanted to ... Read more

Data Science for Schools and Educational Institutes

Data is pervasive  but analytics & insights are not. Schools and educational institutions are not an exception. In this blog, we will discuss on various ideas to leverage data and analytics for Schools and Institutions. Student Analytics Schools and Institutions captures various details about the students and their learning. Some of the data captured are ... Read more