java - Testing Logistic Regression. Doesn't converge using 1 or 0, but does using logit -
i have created simple logistic regression using gradient descent using linear regression code here: gradient descent linear regression in java
now i'm adapting logistic regression changing hypothesis adding logit transform: 1/(1+e^(-z)), z original theta^t * x. , not scaling population size.
when trying test results confusing behavior: set independent variables(x) random number * expected weight , dependent(y) logit of sum, y= logit(w0*1 + w1*x1 + w2*x2).
now in case converges correct answer, , can recover expected weights. should have y 0 or 1. when round or down, no longer converge.
here generating training data:
@test public void testlogisticdescentmultiple() { //... //initialize independent xi //going create test data y= 10 + .5(x1) + .33(x2) for( int x=0;x<num_examples;x++) { independent.set(x, 0, 1); //x0 set 1 intercept independent.set(x, 1, random.nextgaussian()); //x1 independent.set(x, 2, random.nextgaussian() ); //x2 } //initialize dependent yi for( int x=0;x<num_examples;x++) { double val = w0 + (w1*independent.get(x,1)) + (w2*independent.get(x,2)); double logitval = logit( val ); //converges without code block if( logitval < 0.5 ) { logitval = 0; }else { logitval = 1; } // dependent.set(x, logitval ); } //... } public static double logit( double val ) { return( 1.0 / (1.0 + math.exp(-val))); } //updated logistic regression public doublematrix1d logisticdescent(double alpha, doublematrix1d thetas, doublematrix2d independent, doublematrix1d dependent ) { algebra algebra = new algebra(); //hypothesis 1/( 1+ e ^ -(theta(transposed) * x)) //start theata(transposed)*x doublematrix1d hypothesies = algebra.mult( independent, thetas ); //h = 1/(1+ e^-h) hypothesies.assign(new doublefunction() { @override public double apply (double val) { return( logit( val ) ); } }); //hypothesis - y //now have each xi, difference between predicted hypothesis , actual yi hypothesies.assign(dependent, functions.minus); //transpose examples(mxn) nxm can matrix multiply hypothesis nx1 doublematrix2d transposed = algebra.transpose(independent); doublematrix1d deltas = algebra.mult(transposed, hypothesies ); // thetas = thetas - (deltas*alpha) in 1 step thetas.assign(deltas, functions.minusmult(alpha)); return( thetas ); }
Comments
Post a Comment