Week 11: Improving UPROB

-

 This week I improved the UPROB module, implementing a UPROB(K-100) functionality, internalizing the filtration operation to automatically choose optimal minimum and maximum points. I also overcame last weeks challenge of caching and managed to implement a working cache functionality for CondP().

Monday :

Discussed functionalities of UPROB with Roger sir and got some suggestions and feedback. Some of the things I will be implementing to improve UPROB are:

  • Calculate minPoints and maxPoints for filtering operations internally and abstract it from the user. 

  • Implement UPROB(K=100)

  • Caching the RKHS depending on the filter values rather than filtered variables to avoid re-initializing it. This solves the issue I had last week with caching.



Tuesday and Wednesday:

Implemented the caching functionality. This will especially be helpful in condP() calculation to perform operations where we are only altering the value of expected variable with constant conditionals. For ex:

The following operations can be performed with a single rkhs 

P(A = 1 | B = 2, C = 3)
   P(A = 3 | B = 2, C = 3)

P(A = 5 | B = 2, C = 3)


Implemented a variant of UPROB(K=100) not involving the usage of an RKHS. To explain the working :


In a simple example of 3 dimensions, A, B and C, When we are trying to find out say, P(A|B= 1, C= 2), after filtering the dataset, we are left with some N data-points of the form:

(a1, 1, 2) ; (a2, 1 , 2) ; (a3, 1 , 2) ......... (an, 1 , 2). for some values of a1,a2,a3...,an

 

In such a dataset I deduced that the average of all these a1,a2,a3....,an values should provide us with the expected value of A. I implemented this and sure enough, the r2 values of this method is very similar to ProbSpace. 

Here are some results for dims = 2,3,4,5,6;   datSize = 1000;   K = 100

 

dims, datSize, tries, K =  2 1000 10 100

Average R2: JP, UP, PS =  0.95948351161 0.9917388944263 0.9618719349801685

 

 

dims, datSize, tries, K =  3 1000 10 100

Average R2: JP, UP, PS =  0.9470697228 0.759582385234 0.8081795917910626

 

dims, datSize, tries, K =  4 1000 10 100

Average R2: JP, UP, PS =  0.9078011903 0.687716501589 0.6617432174774956

 

dims, datSize, tries, K =  5 1000 10 100

Average R2: JP, UP, PS =  0.8003123156 0.404339122151 0.3788512381207686

 

dims, datSize, tries, K =  6 1000 10 100

Average R2: JP, UP, PS =  0.528478955 0.037851936471 -0.0110369733116401

 

Thursday and Friday:

Implemented UPROB(K=100) but this time, using a single-dimensional RKHS. 


P(A = 1| B = 2, C = 3)


Similar to the previous method, I filter first on all of the conditional variables (B,C), I then pass the remaining data points (only the values of A from the results after filtration) into an RKHS to calculate the expected value of the A. I run this query for different values of A and the mean of these results will give me the E(A | B = 2, C = 3).


I was pleasantly surprised that the results of this method are highly accurate:


dims, datSize, tries =  2 1000 1

Average R2: JP, UP, PS =  0.842644618 0.9828869871 0.9696438228341947



dims, datSize, tries =  3 1000 1

Average R2: JP, UP, PS =  0.717250601 0.73071595754 0.7154192883214503



dims, datSize, tries =  4 1000 1

Average R2: JP, UP, PS =  0.664790677 0.72162423052 0.6011758633380084



Next I implemented minPoints and maxPoints calculations:


  • self.minPoints  = ceil(self.N**((dims-filter_len)/dims))*self.rangeFactor

  • self.maxPoints = ceil(self.N**((dims-filter_len)/dims))/self.rangeFactor


Here the goal was to find the optimal rangeFactor value so that the filtering operations would be optimal.

While I was running tests, I detected a fatal error in the filtering operation, ProbSpace.filter() we have been using so far. The way ProbSpace is configured, if Probspace cannot get at least minPoints number of data-points within a certain range of the conditional value, it increases the range and checks again. It tries this  for 8 iterations. If by chance it was unable to find minPoints number of datapoints even then, it returns 0 data-points (this generally only happens if the dataset we are using is of small size (N ~= 1000).


This is a problem because we cannot initialize our RKHS with 0 data points and this error crashes the testing program. After discussion with Roger sir, I will be implementing a failsafe in place such that if  UPROB(K=100) returns 0 data points, I try again with UPROB(K=80) or whatever value of K corresponding to all but 1 conditional variable being filtered. In other words, we progressively reduce the number of variables to be filtered until we obtain the required minimum amount of data points. 


I have implemented this new K-decrement filtering but the variables need some fine tuning to work properly as in its current state, the filtering algorithm always drops to calculating UPROB(K=0) which is just JPROB, reducing accuracy.


Next week plan:

Perfect the K decrement algorithm such that it only reduces K when the obtained number of data points from filtration is 0.


Comments

Post a Comment

Popular posts from this blog

Week 1: RKHS Kernel Implementation

-
Image
      My first long term goal is to improve the existing conditional probability calculation module by making use of RKHS mapping to get rid of the curse of dimensionality problem that has been haunting the traditional method. Here is a day-wise summary of my progress throughout the week: Monday: My Internship officially started on this day, I had a 1 on 1 meeting with Roger Sir regarding this week’s assignment which would be to implement an RKHS kernel for the purpose of predicting a probability distribution function, given just a set of data points ( 1000 in my case) belonging to the said distribution. This would bolster my understanding of RKHS working and will come in use when implementing more complex kernels in the future. Tuesday: Implemented the RKHS kernel and the evaluation function. Dataset for the arbitrary function X = logistic(-2,1) if choice([0,1]) else logistic(2,1) was created using the synthDataGen.py script and used as the basis for the evaluation function. Made
Read more

Week 10: Implementing UPROB

-
Image
Week 10 Report This week, I developed the UPROB module (you can check out the code here ). Since this was the main focus throughout the week and I spent all my time developing, bug-fixing , testing and documenting results, I feel that a day-wise summary would not be the most effective method to show results. Instead I have opted to explain the functionalities, working and usage of this module. This will also serve as documentation for future reference. I have listed the results and conclusion towards the end of this report. UPROB Init: Initializing a UPROB Object is very similar to that of rkhsMV except for the final argument k , which denotes the % of variables to be filtered before applying JPROB methods on the remaining ones.  R1 and R2 are two separate rkhsMV instances, one for each of the UPROB methods, condE() and condP(). The idea is to store R1 and R2 such that if we are repeatedly performing operations on the same dataset, we need not initialize a new kernel each time thus sa
Read more

Week 12: Final Week

-
  These past 12 weeks have flown by so fast! It still feels like yesterday I started working on this project. There were highs and lows but at the end of the day we overcame our obstacles to create something to be proud of. It was a great learning experience and I had fun working with the team.  Monday: Implemented the prototype of the K decrement algorithm to reduce the number of filter variables in case of shortage of data points upon filtration. Here are some results: K decrement- condP() K = 100 rf = 0.8, minmin = 20 dims, datSize, tries, K =  3 1000 1 100 Average R2: JP, UP, PS =  0.67025996194      0.726423475099       0.7434199290171855 K=100 , rf = 0.8, minmin = 5 dims, datSize, tries, K =  3 1000 5 100 Average R2: JP, UP, PS =  0.70784384556      0.87296540935         0.764876980843549 K=100 , rf = 0.8, minmin = 4 dims, datSize, tries, K =  4 1000 1 100 Average R2: JP, UP, PS =  0.70162883565      0.80123074862         0.6089856810665888  K=100 in condP() function means that w
Read more