Week 1: RKHS Kernel Implementation

-

     My first long term goal is to improve the existing conditional probability calculation module by making use of RKHS mapping to get rid of the curse of dimensionality problem that has been haunting the traditional method. Here is a day-wise summary of my progress throughout the week:

Monday:

My Internship officially started on this day, I had a 1 on 1 meeting with Roger Sir regarding this week’s assignment which would be to implement an RKHS kernel for the purpose of predicting a probability distribution function, given just a set of data points ( 1000 in my case) belonging to the said distribution. This would bolster my understanding of RKHS working and will come in use when implementing more complex kernels in the future.

Tuesday:

Implemented the RKHS kernel and the evaluation function. Dataset for the arbitrary function X = logistic(-2,1) if choice([0,1]) else logistic(2,1) was created using the synthDataGen.py script and used as the basis for the evaluation function. Made a simplistic plotting function just to see how it turned out. Received valuable advice from Roger Sir, specifically on how to improve the plotting function by limiting it to a shorter range and increasing the point density.

Wednesday:

Created the ideal probability density function for the distribution we based our dataset on. This serves as the standard we wish to achieve with our RKHS implementation. After improvements to the plotting function, it allowed us to properly compare the curves derived from our evaluation function vs the ideal pdf() function for different values of sigma. However the graphs are not quite aligning due to a bug.  



Thursday:

Fixed the bug in the pdf() function and finetuned the plotting function to now accommodate varying the scale parameter of the pdf() function to test results. Turns out, the sigma of our RKHS and scale of the pdf() function are quite similar, the smaller their value, the sharper the resulting curve is and vice versa. After experimenting for a while, I was able to observe a near perfect overlap at values sigma = 0.6 and scale = 1. 

Presented my results on the weekly meeting with the team. Got to observe the tests and techniques sir had applied to a similar problem of higher complexity. There is also merit in considering the CDF along with the PDF of a distribution as it allows us to determine density of points on the curve. The main takeaway from this assignment has been that, for any distribution there exists a kernel we can derive from just the dataset of the distribution and that the results will be accurate under the right parameter values. My code is visible at:

Friday:

Spent some hours analyzing and experimenting with Roger Sir’s code on a generalized RKHS function and the testing module. Gained understanding on how the tests and graphs shown on the previous day were implemented.

Upcoming week goal:

To implement RKHS for computation of conditional probabilities which is explored in a paper. ( https://geometry.stanford.edu/papers/shsf-hsecdads-08/shsf-hsecdads-08.pdf )

Comments

Post a Comment

Popular posts from this blog

Week 12: Final Week

-
  These past 12 weeks have flown by so fast! It still feels like yesterday I started working on this project. There were highs and lows but at the end of the day we overcame our obstacles to create something to be proud of. It was a great learning experience and I had fun working with the team.  Monday: Implemented the prototype of the K decrement algorithm to reduce the number of filter variables in case of shortage of data points upon filtration. Here are some results: K decrement- condP() K = 100 rf = 0.8, minmin = 20 dims, datSize, tries, K =  3 1000 1 100 Average R2: JP, UP, PS =  0.67025996194      0.726423475099       0.7434199290171855 K=100 , rf = 0.8, minmin = 5 dims, datSize, tries, K =  3 1000 5 100 Average R2: JP, UP, PS =  0.70784384556      0.87296540935         0.764876980843549 K=100 , rf = 0.8, minmin = 4 dims, datSize, tries, K =  4 1000 1 100 Average R2: JP, UP, PS =...
Read more

Week 0: An Introduction

-
Image
What is Causality?      Causality (also referred to as causation, or cause and effect) is influence by which one event, process, state or object (a cause) contributes to the production of another event, process, state or object (an effect) where the cause is partly responsible for the effect, and the effect is partly dependent on the cause. That is the definition according to wikipedia. In simpler terms though, if a variable b is dependant on another variable a , then we can say a   causes   b  or that a   is a causal factor to b . Why Study Causality?      Traditional statistics only deals with pure values of variables and attributes. It does not tell us of the causal relationships between factors in a large dataset. ML algorithms in their current state can be biased, suffer from a relative lack of explainability, and are limited in their ability to generalize the patterns they find in a training data set for multiple applications. ...
Read more

Week 5: Random Features and Multiple Conditionals

-
Image
       This week I've digested multiple papers and they are all not necessarily related but I believe it was still required to gain understanding on how to move forward with the project. Here is a day-wise summary of my progress throughout the week: Monday: Studied the Fast Nonparametric Conditional Density Estimation paper . The paper details a method to calculate the ideal bandwidths for the Kernel methods(sigma in case of the Gaussian Kernel). They have made use of Dual-tree recursion to reduce the time complexity of the operation to O(n2). The previous best method, calculating integrated squared errors(ISE) was of O(n3) so this method saves a lot of time without compromising accuracy when dealing with large datasets. However, calculating bandwidths is not a priority right now, so after discussion with Roger sir, I have shifted my attention towards extending our kernel technique to condition on multiple variables. Tuesday:      Revisited the Nonpa...
Read more