java - Testing Logistic Regression. Doesn't converge using 1 or 0, but does using logit -


i have created simple logistic regression using gradient descent using linear regression code here: gradient descent linear regression in java

now i'm adapting logistic regression changing hypothesis adding logit transform: 1/(1+e^(-z)), z original theta^t * x. , not scaling population size.

when trying test results confusing behavior: set independent variables(x) random number * expected weight , dependent(y) logit of sum, y= logit(w0*1 + w1*x1 + w2*x2).

now in case converges correct answer, , can recover expected weights. should have y 0 or 1. when round or down, no longer converge.

here generating training data:

@test public void testlogisticdescentmultiple() {     //...      //initialize independent xi     //going create test data y= 10 + .5(x1) + .33(x2)      for( int x=0;x<num_examples;x++) {         independent.set(x, 0, 1); //x0 set 1 intercept         independent.set(x, 1, random.nextgaussian()); //x1         independent.set(x, 2, random.nextgaussian() ); //x2     }      //initialize dependent yi     for( int x=0;x<num_examples;x++) {         double val      = w0 +  (w1*independent.get(x,1)) + (w2*independent.get(x,2));         double logitval = logit( val );          //converges without code block         if( logitval < 0.5 ) {             logitval = 0;         }else {             logitval = 1;         }         //          dependent.set(x, logitval );     }     //... }  public static double logit( double val ) {     return( 1.0 / (1.0 + math.exp(-val))); }  //updated logistic regression public doublematrix1d logisticdescent(double         alpha,                                       doublematrix1d thetas,                                       doublematrix2d independent,                                       doublematrix1d dependent ) {     algebra algebra     = new algebra();      //hypothesis 1/( 1+ e ^ -(theta(transposed) * x))     //start theata(transposed)*x     doublematrix1d hypothesies = algebra.mult( independent, thetas );      //h = 1/(1+ e^-h)     hypothesies.assign(new doublefunction() {         @override         public double apply (double val) {             return( logit( val ) );         }     });      //hypothesis - y     //now have each xi, difference between predicted hypothesis , actual yi     hypothesies.assign(dependent, functions.minus);      //transpose examples(mxn) nxm can matrix multiply hypothesis nx1     doublematrix2d transposed = algebra.transpose(independent);      doublematrix1d deltas     = algebra.mult(transposed, hypothesies );      // thetas = thetas - (deltas*alpha)  in 1 step     thetas.assign(deltas, functions.minusmult(alpha));      return( thetas ); } 


Comments

Popular posts from this blog

ruby - Trying to change last to "x"s to 23 -

jquery - Clone last and append item to closest class -

c - Unrecognised emulation mode: elf_i386 on MinGW32 -