Financial Services

Consumer finance industry has gone through transformation with FinTech...

Know More

Insurance

Digital insurers are disrupting the insurance industry by harnessing the...

Know More

Retail

Retail industry is changing rapidly with the advent of online ecommerce...

Know More

Utilities

Electricity generation, transmission and distribution businesses are changing...

Know More

Recent Blogs


blog grid image

TIME SERIES ANALYSIS &...

Observations of any variable recorded over time in sequential order are considered a time series. The measurements may be taken every hour, day, week, month, or year, or at any other regular interval. The time interval over which data are collected is called periodicity. There are two common approaches to forecasting: -

1)Qualitative Forecasting method: When historical data are unavailable or not relevant to future. Forecasts generated subjectively by the forecaster. For example – a manager may use qualitative forecasts when he/she attempts to project sales for a brand-new product. Although qualitative forecasting method is attractive in certain scenarios, it’s often criticised as it’s prone to optimism and overconfidence.

2)Quantitative Forecasting method: When historical data on variables of interest are available. Methods are based on an analysis of historical data concerning the time-series of the specific variable of interest. Forecasts are generated through mathematical modelling. Quantitative forecasting methods are subdivided into two types:

1)Time Series Forecasting methods: forecast of future values based on the past and present values of the variable being forecasted. These are also known as non-casual forecasting methods, they are purely time series models and do not present any explanation of the mechanism generating the variable of interest and simply provide a method for projecting historical dat

2)Casual Forecasting methods: It attempts to find casual variables to account for changes (for the variable to be forecasted) in a time series. It forecasts the future values by examining the cause and effect relationships. Casual forecasting methods are based on a regression framework, where the variable of interest is related to a single or multiple independent variables. Here, forecasts are caused by the known values of the independent variables.

Basic assumptions of time-series forecasting are: -

  • Factors that have influenced activities in the past and present will continue to do so in more or less the same way in the future.
  • As the forecast horizon shortens, forecast accuracy increases.
  • Forecasting in the aggregate is more accurate than forecasting individual items.
  • Forecasts are seldom accurate (therefore it is wise to offer a forecast range)

In this blog, we are going to focus on Time Series Forecasting when there are no Trends in the model. The main aim of which is to identify and isolate influencing factors to make predictions. To achieve this objective, we need to explore the fluctuations using mathematical models, the most basic of which is the classical ‘multiplicative’ model.

                                                                               

Figure 1: This shows a Time Series Plot which shows the monthly sales for two companies over two years, where the vertical axis measures the variable of interest and the horizontal axis corresponds to the time periods.

Figure 2 shows the components of a time series. The pattern or behaviour of the data in a time series involves several components:

  • Trend - the long-term increase or decrease in a variable being measured over time (such as the growth of national income). Forecasters often describe an increasing trend by an upwards sloping straight line and a decreasing trend by a downward sloping straight line.
  • Cyclical - a wave like pattern within the time series that repeats itself throughout the time series and has a recurrence period of more than one year (such as prosperity, recession, depression and recovery).
  • Seasonal - a wave like pattern that is repeated throughout a time series and has a recurrence period of at most one year (such as sales of ice-cream or garden supplies)
  • Irregular - changes in time-series data that are unpredictable and cannot be associated with the other components (such as floods, strikes).

The classical multiplicative time series model states that any value in a time-series is the product of trend, cyclical, seasonal and irregular, as the multiplicative model assumes that the effect of these four components in a time series model are interdependent.

Classical multiplicative time series model for annual data: Yi = Ti * Ci * Ii

where, Ti is the value for the trend component in the year ‘i’,

Ci is the value of the cyclical component in the year ‘i’,

Ii is the value of the irregular component in the year ‘i’.

Classical multiplicative time series model includes the seasonal component where there is quarterly or monthly data available: Yi = Ti * Ci * Ii * Si

Where, Si is the value of the seasonal component in time period ‘i’.

                                                                                                             

Since in this blog we are primarily focusing on Non-Trend Models (which means after plotting the data there are no patterns that occur over time, neither an upward nor a downward trend), we use smoothing techniques to smooth series and provide an overall long term impression. When there’s no trend, we use smoothing techniques such as the method of moving averages or the method of exponential smoothing to smooth the series.

Time-series smoothing methods: If, for instance, we use annual data, a smoothing technique can be used to smooth a time series by removing unwanted cyclical and irregular variations.

Let’s take an example of Gasoline sales (in 1000s of Gallons) over a period of time:

Year

1

2

3

4

5

6

7

8

9

10

11

12

Sales (Yi)

17

21

19

23

18

16

20

18

22

20

15

22

We drew a scattered diagram using the above-mentioned data. In figure 4, our visual impression of the long-term trend in the series is obscured by the amount of variation from year to year. It becomes difficult to judge whether any long term upward or downward trend exists in the series. To get a better overall impression of the pattern of movement in the data over time, we smooth the data.

One of the ways is using the Moving Averages method: here the mean of the time series data is taken from several consecutive periods. The term moving is used because it’s continually recomputed as new data becomes available, it progresses by dropping the earliest value and adding the latest value. To calculate moving averages, we need to know the length of periods chosen to be included in the moving average. Moving Averages are represented by MA(L ) where L denotes the length of periods chosen. A Weighted Moving Average (WMA) is prepared as It helps to smooth the price curve for better trend identification. It places even greater importance on recent data.

Using the above example, we prepare a table to show the Weighted Moving Averages:

In the above figures (5 & 6), we can observe that the 5 year moving averages smooth the series more than the 3 year moving averages because the period is longer. So, as L increases, it smoothens the variations better but the number of moving averages that we can calculate becomes fewer, this is because too many moving averages will be missing at the beginning and end of the series.

A Moving Average has two main disadvantages:

It involves the loss of the first and last sets of time periods. This could be a significant loss of information if there are few observations in the time series.

The process of dropping the last observation in current set causes the moving average to forget most of the previous time series values A technique that addresses both of these problems is called Exponential Smoothing. It’s a forecasting technique in which a weighting system is used to determine the importance of previous time periods in the forecast. It’s used to weight data from previous time periods with exponentially decreasing importance in the forecast. The aim is to estimate the current level and use it as a forecast of future value.

To calculate an exponentially smoothed value in time period ‘i’, we use the following understanding: -

E1 = Y1        Ei = WYi + (1-W)Ei-1,

where,

Ei is the value of the exponentially smoothed series being calculated in the time period ‘i’

Ei-1 is the value of the exponentially smoothed series already calculated in the time period ‘i-1’

Yi is the observed value of the time series in period ‘i’

W is subjectively assigned weight or smoothing coefficient (where, 0 < W < 1)

Let us use the same example of Gasoline sales (in 1000s of Gallons) over a period of time:

(Assume W = 0.5)

From figure 7, we can observe how exponentially smoothening the series with lesser variations. Now comes the point where we take a decision to choose the smoothing coefficient. When we use a small W (such as W = 0.05) then there’s heavy smoothing, as there’s more emphasis on the previous time period (Yi-1), therefore, slow adoption to recent data. If there’s moderate smoothing (such as W = 0.2) then there’s moderate smoothing or moderate adaptation to recent data. And if we choose a high value for W (such as W = 0.8) then there’s little smoothing and quick adaptation to the recent data.

Therefore, the selection has to be somewhat subjective. So, if our goal is to only smooth a series by eliminating unwanted cyclical and irregular variations, we should select a small value for W (thus less responsive to recent changes). If our goal is forecasting, then we should choose a large value for W (in this case more weight is being put on the actual value than the forecast value as large W assigns more weights to the more recent values).

blog grid image

IMAGE RECOGNITION

Social media has transformed our way of communication and socialization in today’s world. Facebook and twitter are always on the lookout for more information about their users from their users. People eagerly share their information with the public which is used by the media agents to improve their business and services. This information comes from customers in the form of text, image or video. In the age of selfie, capturing every moment in the cell phone is a norm. Be it a private holiday, an earth quake shaking some part of the world or a cyclone blowing the roof over the head, everything is clicked and posted. These images are used as data by social media and researchers for image recognition, also known as computer vision.

Image recognition is the process of detecting and identifying an object or a feature in a digital image or video in order to add value to customers and enterprises. Billions of pictures are being uploaded daily on the internet. These images are identified and analysed to extract useful information. This technology has various applications as shown below. In this blog we will touch upon some of these applications and the techniques used therein.

Text Recognition

e will begin with the technique used to recognise a handwritten number. Machine learning technologies like deep learning can be used to do so. A brief note on AI, ML, DL and ANN before we proceed further. Artificial intelligence (AI) is human intelligence exhibited by machines by training the machines. Whereas Machine Learning (ML) is an approach to achieve artificial intelligence and deep learning is a technique for implementing machine learning. Artificial Neural Network (ANN) is based on the biological neural network. A single neuron will pass a message to another neuron across this network if the sum of weighted input signals from one or more neurons into this particular neuron exceeds a threshold. The condition when the threshold is exceeded and the message is passed along to the next neuron is called as activation1.

There are different ways to recognize images. We will use neural networks to recognize a simple handwritten text, number 8. A very critical requirement for machine learning is data, as much data as possible to train the machine well. A neural network takes numbers as input. An image is represented as a grid of numbers to the computer and these numbers represent how dark each pixel is. The handwritten text of number 8 is represented as below.

This 18x18 pixel image is treated as an array of 324 numbers. These are the 324 input nodes to the neural network as shown below.

The neural network will have two outputs. The first output will predict the likelihood that the image is an ’8’ and the second output will predict the likelihood that it is not an ’8’. The neural network is trained with different handwritten numbers to differentiate between ’8’and not an ’8’. So, when it is fed with an ’8’, it is trained to identify that the probability of it being an ’8’ is 100% and not being an ’8’ is 0%. So, now it can recognize ’8’ but only a particular pattern of 8. If there is a slight change in position or size, it may not recognise it. There are various ways to train it to identify ’8’ in any position and size. Deep neural network technique can be used to do so. To train better, we need more data and with increase in data, the network becomes bigger. This is done by stacking more layers of nodes and this is known as deep neural network. It does so by treating ’8’at the top separately from ’8’ at the bottom of a picture. This is avoided by using another technique called convolutional neural network. All these technologies are evolving rapidly with improved and refined approach to get better output.

Face Recognition

Face recognition is used to convey a person’s identity. It uniquely identifies us. Biometric face recognition technology has applications in various areas including law enforcement and non-law enforcement.

The conventional pipeline of face recognition consists of four stages4

Face detection is easier than face identification as all faces have the same features eyes, ears, nose, and mouth, almost in the same relative positions. Face identification is a lot more difficult as our face is constantly changing, unlike our fingerprints. With every smile, every expression our face gets transformed as the shape of our face contorts with our expression. Though humans can identify us even when we sport a different hairstyle, systems have to be trained to do so. Computers struggle with the problem of A-PIE or aging, pose, illumination, and expression. These are considered as sources of noise which make it difficult to distinguish between faces. A technique called deep learning helps reduce this noise and disclose the statistical features that the images of a single person have in common to uniquely identify that person.

DeepFace is a deep learning facial recognition system created by Facebook. It identifies human faces in digital images and employs a nine-layer neural net with over 120 million connection weights, and was trained on four million images uploaded by more than 4000 Facebook users5. This method reached an accuracy of 97.35%, almost approaching human-level performance.

Computer recognizes faces as collections of lighter and darker pixels. The system first clusters the pixels of a face into elements such as edges that define contours. Subsequent layers of processing combine elements into nonintuitive, statistical features that faces have in common but are different enough to discriminate them. The output of the processing layer below serves as the input to the layer above. The output of deep training the system is a representational model of a human face. The accuracy of the result depends on the amount of data, which in this case is the number of faces the system is trained on.

FBI’s Next Generation Identification (NGI)

FBI’s Criminal Justice Information Services (CJIS) Division developed and incrementally integrated a new system called the Next Generation Identification (NGI) system to replace the Integrated Automated Fingerprint Identification System (IAFIS). NGI provides the criminal justice community with the world’s largest and most efficient electronic repository of biometric and criminal history information6. The accuracy of identification using NGI is much less compared to Facebook’s DeepFace. One of the reasons is the poor quality of pictures that FBI uses. FBI normally uses the images obtained through public cameras which do not provide a face straight-on photograph. Whereas Facebook already has the information of all our friends and works with over 250 billion photos and over 4.4 million labelled faces compared to FBI’s over 50 billion photos. Thus, with more data Facebook has an edge in better identification. Facebook also has more freedom to make mistakes, since a false photo-tag carries much less weight than a mistaken police ID7. Facial recognition is of great use in automatic photo-tagging, but there is risk of false-accept rate while trying to identify a suspect and an innocent could be in trouble because of this.

Search and e-commerce

Google’s Cloud Vision API and Microsoft’s Project Oxford’s Computer Vision, face, and emotion APIs provide image-recognition solutions using deep, machine-learning algorithms to provide powerful ecommerce and retail applications that will enhance shopping experience of users and create new marketing opportunities for retailers8.

Cortexica9 uses its findSimilar™ software to provide services to retailers like Macy’s and Zalando. Cortexica does this by providing the retailer with an API. First, the images of all the items in the inventory are ingested in the software. The size and completeness of the dataset is important. Second, a Key Point Files (KPF) for each image, which is a proprietary Cortexica file, is produced. This file contains all the visual information needed to describe the image and help with future searches. Third, this system is then connected to the customer’s app or website search feature. Fourth, when the consumer sends an incoming query image, it is converted into a KPF, the visual match is computed and the consumer gets the matched results in order of visual similarity in couple of seconds.

This hot topic that is "visual search" is all driven by the alignment of consumer activity, with regards to their propensity to taking pictures, and the innovation of how retailers want their inventory to be discovered by consumers using their mobile devices. Facts like colour, texture, distinctive parts and shapes all need to be considered in designing the algorithm to meet the challenges of the broad range of retail fashion requirements.

Companies like Zugara11 use augmented reality (AR) shopping applications that allow a customer to try clothing in a virtual dressing room by overlaying an image of a dress or shirt and find what suits best. Here the app looks at the shopper via web camera and can capture the emotions of the consumer and send it to Google or Microsoft API for emotional analysis. Depending on the feedback from the API’s image analysis, the AR application can be guided to provide similar or different outfit to the customer12.

According to MarketsandMarkets, a global market research company and consulting firm, the image recognition market is estimated to grow from USD 15.95 Billion in 2016 to USD 38.92 Billion by 2021, at a CAGR of 19.5% between 2016 and 2021. The future of image recognition seems very interesting.

blog grid image

CASINOS & EVOLVING ANA...

nalytics are used by companies to be more competitive and the financial services industry has known this for decades.  In fact, many financial services analytics professionals are moving to gaming as both industries need to balance risks and returns. More and more, casinos are using analytics to make decisions in areas that have traditionally relied upon “expertise” rather than data-driven approaches to increase profits…

    Where to strategically place games on the casino floor

              

Today, modeling teams at a number of casinos use software such as SAS to predict the impact of moving games from one area of a casino floor to another1.  

To set a baseline, data is collected on how much money each game, whether table games or slots, currently brings in as well as how people move about the casino. When the gathered data is combined with the odds of a particular game paying out, the analytics team can model what the performance would look like in different locations to help determine where the game should be placed in order to achieve the optimal performance level.  This is a similar technique used by supermarket companies. Just as with a grocery store, where on the casino floor would you get the best yield?

       A holistic data-driven approach for all casino operations

           

Gaming revenue is not the largest portion of what casinos bring in. They derive much of their revenue from their resort operations. For example, a good way to encourage gambling is to give customers free nights or discounted dinners in the hotel that houses a casino. But the casino would lose money if it did so for everyone, because some people don’t gamble much. To help pinpoint such offers, savvy Casinos run customer analytics applications on data it has collected showing how often individual guests gamble, how much money they tend to spend in the casino and what kinds of games they like. This is all part of a significant shift in how casinos do business where it’s getting to the point that casinos are being run like financial services firms.

       The challenges of shifting to Big Data

       
   

At MGM Resorts’ 15 casinos across the United States, thousands of visitors are banging away at 25,000 slot machines. Those visitors rang up nearly 28 percent of the company’s $6 billion in annual domestic revenue in 2013. Using the game and customer data that MGM collects daily and the behind-the-scenes software that transforms the data into critical insights, in turn, boost the customer experience and profit margins2.

Lon O’Donnell, MGM’s first-ever director of corporate slot analytics, is challenged to show why big data is a big deal when it comes to plotting MGM’s growth. “Our goal is to make that data more digestible and easier to filter,” says O’Donnell, who estimates that Excel still handles an incredible 80 percent of the company’s workload. In the near term, that means the team is experimenting with data visualization tools (Slotfocus dashboard - right) to make slot data more crunchable. Heavy-lifting analytics are a goal down the road3.  MGM isn’t the only gaming company interested in big data - nor was it the first. That distinction goes to Gary Loveman, who left teaching at Harvard Business School for Las Vegas in the late 1990s and turned Harrah’s into gaming’s first technology-centric player.

History has caught up with the industry. For decades, Las Vegas casinos were some of the only legal gambling outfits in the country, so they could afford to be complacent. That advantage disappeared during the past two decades with the rise of legal gambling in 48 states. The switch to slicker, more sophisticated cloud apps is still on the horizon. One reason why is the regulatory nature of gaming: Casinos tend to organize data in spreadsheets to report to regulators, who review the accounting and verify that slots perform within legal specifications. But those reports are not ideal business intelligence sources.

       Using Big Data to catch cheaters

       

Casinos are at the forefront of new tools to help them make more money and reduce what they consider to be fraud. One tool is something called non-obvious relationship awareness (NORA) software that allows casinos to determine quickly if a potentially colluding player and dealer have ever shared a phone number or a room at the casino hotel, or lived at the same address4,5. “We created the software for the gaming industry,” says Jeff Jonas, founder of Systems Research & Development, which originally designed NORA. The technology has proved so effective that Homeland Security adapted it to sniff out connections between suspected terrorists. “Now it’s used as business intelligence for banks, insurance companies and retailers,” Jonas says.  The image above shows three types of cameras feed the video wall in the Mirage’s surveillance room (top-right):  Fixed-field-of-view units focus on tables; motorized pan-tilt-zoom cameras survey the floor; and 360-degree cams take in an entire area.

Big Data and attendant technologies are starting to transform businesses right before our very eyes. Old ways of doing things are beginning to fall by the wayside. When specific examples like NORA become more public, Big Data suddenly becomes less abstract to those who make decisions.

Analytics are used by companies to be more competitive and the financial services industry has known this for decades.  In fact, many financial services analytics professionals are moving to gaming as both industries need to balance risks and returns. More and more, casinos are using analytics to make decisions in areas that have traditionally relied upon “expertise” rather than data-driven approaches to increase profits…

    Where to strategically place games on the casino floor

       

Today, modeling teams at a number of casinos use software such as SAS to predict the impact of moving games from one area of a casino floor to another1.

To set a baseline, data is collected on how much money each game, whether table games or slots, currently brings in as well as how people move about the casino. When the gathered data is combined with the odds of a particular game paying out, the analytics team can model what the performance would look like in different locations to help determine where the game should be placed in order to achieve the optimal performance level.  This is a similar technique used by supermarket companies. Just as with a grocery store, where on the casino floor would you get the best yield?

       A holistic data-driven approach for all casino operations

       

Gaming revenue is not the largest portion of what casinos bring in. They derive much of their revenue from their resort operations. For example, a good way to encourage gambling is to give customers free nights or discounted dinners in the hotel that houses a casino. But the casino would lose money if it did so for everyone, because some people don’t gamble much. To help pinpoint such offers, savvy Casinos run customer analytics applications on data it has collected showing how often individual guests gamble, how much money they tend to spend in the casino and what kinds of games they like. This is all part of a significant shift in how casinos do business where it’s getting to the point that casinos are being run like financial services firms.

       The challenges of shifting to Big Data

       

At MGM Resorts’ 15 casinos across the United States, thousands of visitors are banging away at 25,000 slot machines. Those visitors rang up nearly 28 percent of the company’s $6 billion in annual domestic revenue in 2013. Using the game and customer data that MGM collects daily and the behind-the-scenes software that transforms the data into critical insights, in turn, boost the customer experience and profit margins2.      

Lon O’Donnell, MGM’s first-ever director of corporate slot analytics, is challenged to show why big data is a big deal when it comes to plotting MGM’s growth. “Our goal is to make that data more digestible and easier to filter,” says O’Donnell, who estimates that Excel still handles an incredible 80 percent of the company’s workload. In the near term, that means the team is experimenting with data visualization tools (Slotfocus dashboard - right) to make slot data more crunchable. Heavy-lifting analytics are a goal down the road3.  MGM isn’t the only gaming company interested in big data - nor was it the first. That distinction goes to Gary Loveman, who left teaching at Harvard Business School for Las Vegas in the late 1990s and turned Harrah’s into gaming’s first technology-centric player.

History has caught up with the industry. For decades, Las Vegas casinos were some of the only legal gambling outfits in the country, so they could afford to be complacent. That advantage disappeared during the past two decades with the rise of legal gambling in 48 states. The switch to slicker, more sophisticated cloud apps is still on the horizon. One reason why is the regulatory nature of gaming: Casinos tend to organize data in spreadsheets to report to regulators, who review the accounting and verify that slots perform within legal specifications. But those reports are not ideal business intelligence sources.

       Using Big Data to catch cheaters

       

Casinos are at the forefront of new tools to help them make more money and reduce what they consider to be fraud. One tool is something called non-obvious relationship awareness (NORA) software that allows casinos to determine quickly if a potentially colluding player and dealer have ever shared a phone number or a room at the casino hotel, or lived at the same address4,5. “We created the software for the gaming industry,” says Jeff Jonas, founder of Systems Research & Development, which originally designed NORA. The technology has proved so effective that Homeland Security adapted it to sniff out connections between suspected terrorists. “Now it’s used as business intelligence for banks, insurance companies and retailers,” Jonas says.  The image above shows three types of cameras feed the video wall in the Mirage’s surveillance room (top-right):  Fixed-field-of-view units focus on tables; motorized pan-tilt-zoom cameras survey the floor; and 360-degree cams take in an entire area.

Big Data and attendant technologies are starting to transform businesses right before our very eyes. Old ways of doing things are beginning to fall by the wayside. When specific examples like NORA become more public, Big Data suddenly becomes less abstract to those who make decisions.

Realize your pilot benefits in weeks

Discussion on Problem Statement (4 hrs)

A meeting with the business sponsor and other senior stakeholders to understand the problem and business objectives.

Data Analysis (2-3 weeks)

Analyse data gaps, data quality and prepare data set for machine learning.

Model Development (4-8 weeks)

Develop and train alternative models, compare results and decide best suited model for the business purpose.

Trial Run (2-3 months)

Parallel run the model in lab and compare the predicted outcomes against production data set.

Production Deployment (4-5 months)

Deploy the pilot solution into production and realise the benefit of AI in your business.

Discussion on Problem Statement (4 hrs)

Data Analysis (2-3 weeks)

Model Development (4-8 weeks)

Trial Run (2-3 months)

Production Deployment (4-5 months)