Week 4: Two Step Estimator

-

     This week I implemented the 2-Step estimator from this paper (Nonparametric Conditional Density Estimation) and compared the results with the 1-Step estimator I had implemented in the previous week from the same paper. Also found out a shortcut method to calculate E(Y|X=x) that is not only more accurate than the existing ProbSpace method, but also much faster.

Monday:

    Studied the 2-step Estimator method from the Nonparametric Conditional Density Estimation paper. Had a discussion with Roger sir regarding future goals and objectives. Apart from optimizing the RKHS method further using the Dual Tree method , we have to integrate the RKHS method into the ProbSpace module. 

Implemented an optimization metric during the RKHS calculation phase that allows us to skip calculations that have very little effect towards our answer. This allowed us to reduce the time for calculations significantly: 


  • “Or Optimization”, bound = 3*sigma

    • Average Error: 1.3968811031293926 

    • Max error: 5.431449902740937 

    • Time: 153.31915044784546



  • “And Optimization”, bound=3*sigma

    • Average Error: 1.561447953266133 

    • Max error: 5.426098406537719 

    • Time: 38.010313987731934



For reference :

  • No Optimization

    • Average Error: 1.3966743172847627 

    • Max error: 5.431449933619589 

    • Time: 719.6718242168427


Tuesday & Wednesday:

Implemented the 2 step Estimator, here is a comparison for 2-Step vs 1-Step methods. (Note, we are using the “and optimization” (i.e, (abs(r1.X[i] - x) <= r1bound and abs(r2.X[i]-y) <= r2bound) condition for filtering).

Method

Average Error

Max Error

Time taken

1 Step Estimator

1.561447953266133

5.426098406537719

27.177362203598022

2 Step Estimator

1.4623417529653346

6.610000000001165

133.49220895767212







As seen, 2-Step Estimator does produce lower average error but at the cost of time and max-error. At this point I tried to calculate the same graphs but with “or optimization” (similar to “and” but with or in the middle), but that just takes too much time incase of the 2-step estimator. So I settled on using “or optimization” for 1-Step and “and optimization” for 2-Step.

Method

Average Error

Max Error

Time taken

1-Step (OR)

1.3968811031293926

5.431449902740937

113.14020037651062

2-Step (AND)

1.4623417529653346

6.610000000001165

131.32312035560608





As seen, 1-Step with OR optimization has 2-Step beat in all regards, all the while taking lesser time to calculate. The saving grace for the 2-Step method seems to be that it has the potential to reach a higher level of accuracy with some modifications to the optimization metric, however the time taken makes it difficult to consider.


Thursday:

Due to a suggestion from Roger sir, we discovered a shortcut method (this was a part of the 2-step estimator process) to calculate E(Y|X=x) without going through the full procedure of calculating P(Y=y|X=x) like we were before. This yielded amazing results, highest accuracy we’ve seen so far plus it’s even faster than the ProbSpace method!


Method

Average Error

Max Error

Time taken

ProbSpace

10.97397258530662

74.3476459734108

7.661393880844116

RKHS Sigma =  0.24

1.2084530592994442

19.74293374241948

2.8335468769073486






Friday:

Started studying the Fast Nonparametric Conditional Density Estimation[2] paper to implement the dual tree optimization technique.

Upcoming Week Plans:

Explore the Dual-Tree approximation from the Fast Nonparametric Conditional Density Estimation paper. 

References:

[1] Hansen, Bruce E. (2004) "Nonparametric Conditional Density Estimation", University of Wisconsin Department of Economics

[2]Holmes, Michael P.; Gray, Alexander G.; Isbell, Charles Lee Jr. (2007) "Fast nonparameteric conditional density estimation", URL: https://arxiv.org/ftp/arxiv/papers/1206/1206.5278.pdf

Comments

Post a Comment

Popular posts from this blog

Week 1: RKHS Kernel Implementation

-
Image
      My first long term goal is to improve the existing conditional probability calculation module by making use of RKHS mapping to get rid of the curse of dimensionality problem that has been haunting the traditional method. Here is a day-wise summary of my progress throughout the week: Monday: My Internship officially started on this day, I had a 1 on 1 meeting with Roger Sir regarding this week’s assignment which would be to implement an RKHS kernel for the purpose of predicting a probability distribution function, given just a set of data points ( 1000 in my case) belonging to the said distribution. This would bolster my understanding of RKHS working and will come in use when implementing more complex kernels in the future. Tuesday: Implemented the RKHS kernel and the evaluation function. Dataset for the arbitrary function X = logistic(-2,1) if choice([0,1]) else logistic(2,1) was created using the synthDataGen.py script and used as the basis for the evaluation function. Made
Read more

Week 10: Implementing UPROB

-
Image
Week 10 Report This week, I developed the UPROB module (you can check out the code here ). Since this was the main focus throughout the week and I spent all my time developing, bug-fixing , testing and documenting results, I feel that a day-wise summary would not be the most effective method to show results. Instead I have opted to explain the functionalities, working and usage of this module. This will also serve as documentation for future reference. I have listed the results and conclusion towards the end of this report. UPROB Init: Initializing a UPROB Object is very similar to that of rkhsMV except for the final argument k , which denotes the % of variables to be filtered before applying JPROB methods on the remaining ones.  R1 and R2 are two separate rkhsMV instances, one for each of the UPROB methods, condE() and condP(). The idea is to store R1 and R2 such that if we are repeatedly performing operations on the same dataset, we need not initialize a new kernel each time thus sa
Read more

Week 12: Final Week

-
  These past 12 weeks have flown by so fast! It still feels like yesterday I started working on this project. There were highs and lows but at the end of the day we overcame our obstacles to create something to be proud of. It was a great learning experience and I had fun working with the team.  Monday: Implemented the prototype of the K decrement algorithm to reduce the number of filter variables in case of shortage of data points upon filtration. Here are some results: K decrement- condP() K = 100 rf = 0.8, minmin = 20 dims, datSize, tries, K =  3 1000 1 100 Average R2: JP, UP, PS =  0.67025996194      0.726423475099       0.7434199290171855 K=100 , rf = 0.8, minmin = 5 dims, datSize, tries, K =  3 1000 5 100 Average R2: JP, UP, PS =  0.70784384556      0.87296540935         0.764876980843549 K=100 , rf = 0.8, minmin = 4 dims, datSize, tries, K =  4 1000 1 100 Average R2: JP, UP, PS =  0.70162883565      0.80123074862         0.6089856810665888  K=100 in condP() function means that w
Read more