Overview

Recommendation System is a facility of predicting user responses to options. Examples:

• Offering news articles to on-line newspaper readers, based on interest(topics) prediction.
• Online retailer suggestions, based on history purchase/search.
  Read more »

GMM, HMM

1. Curse of Dimensionality: different narrations

Similarity function: Almost all pairs of points are equally far away from one another, and almost any two vectors are almost orthogonal.

Statistical Inference: This sparsity is problematic for any method that requires statistical significance. In order to obtain a statistically sound and reliable result, the amount of data needed to support the result often grows exponentially with the dimensionality. Think about the features are supposed to be IDPT in a simple regression problem.

General ML: Given finite number of data samples in a high-dimensional feature space with each feature having a number of possible values, an enormous amount of training data is required to ensure that there are several samples with each combination of values.

2.

Link1
Link2

1. Overview

Clustering algorithm types

1
2
3
4
5
6
7
8
9
10
11
12
13
14

All types
│———Prototype Based: point assignment
│   |   
|   |———Centroids: distance metric applied to center of a subgroup
|   |
|   |———Medoids: representative point of the graph  
|
└─── Hierarchical Based
│   │
│   │———Agglomeration
│   │
│   └───Division
│   
└───Density Based

Clustering by geographic data

Clustering directly by Sale performance

Initially the budget function aims to find the relation between sale data and “labor goal”, my first thought is to do linear regression without any transformation.

1. Why Sampling

A misconception I used to have is that the era of big data means the end of a need for sampling, actually, in a Big Data project, like the Bosch production line performance prediction, our models are still developed and piloted with samples. More generally speaking, to understand a statistical task, most times we have to design experiments which will inevitably use sampling.

Number of Data Points to Estimate?

Overall, it’s very well normal-shaped, with a little bit “long tail”.

1

sns.distplot(df_demand['actual_sales'].values,bins=100)

Some concepts that look similar may lead to confusion, especially when given their abbreviations. This article will try to distinguish OLS, GLS, WLS, LARS, ALS

• This work is under guidance of Professor Saravanan Kesavan in UNC Kenan-Flagler Business School.

Given a retailer’s data across 123 stores through 47 weeks, we want to apply the Newsvendor model to estimate service level for managers in each store. Further research will be, to discover what kind of factors are affecting manager service level.