machine learning - Huge number of classes with Multinominal Naive Bayes (scikit-learn) -

-

whenever start having bigger number of classes (1000 , more) multinominalnb gets super slow , takes gigabytes of ram. same true scikit learn classification algorithms support .partial_fit() (sgdclassifier, perceptron). when working convolutional neural networks 10000 classes no problem. when want train multinominalnb on same data 12gb of ram not enough , very slow. understanding of naive bayes, lot of classes, should lot faster. might problem of scikit-learn implementation (maybe of .partial_fit() function) ? how can train multinominalnb/sgdclassifier/perceptron on 10000+ classes (batchwise)?

short answer without information:

  • the multinomialnb fits independent model each of classes, thus, if have c=10000+ classes fit c=10000+ models , therefore, model parameters [n_classes x n_features], quite lot of memory if n_features large.

  • the sgdclassifier of scikits-learn uses ova (one-versus-all) strategy train multiclass model (as sgdc not inherently multiclass) , therefore, c=10000+ models need trained.

  • and perceptron, documentation of scikits-learn:

perceptron , sgdclassifier share same underlying implementation. in fact, perceptron() equivalent sgdclassifier(loss=”perceptron”, eta0=1, learning_rate=”constant”, penalty=none).

so, 3 classifiers mention don't work high number of classes, independent model needs trained each of classes. recommend try inherently support multiclass classification, such randomforestclassifier.


Comments

Post a Comment

Popular posts from this blog

c# - Can I intercept a SOAP response in .NET before a content type binding mismatch ProtocolException? -

-
i trying debug c# code wrote consume web service. i'm hitting error when receiving response because content type of response doesn't match binding. exception snippet: an exception of type 'system.servicemodel.protocolexception' occurred in mscorlib.dll not handled in user code additional information: content type text/xml of response message not match content type of binding (application/soap+xml; charset=utf-8) the web service external can't change on end. know there lot of troubleshooting steps can take besides (and have tried many, it's not simple changing binding), there way view response message, or more info response, before hit exception? have custom message inspector written afterreceivereply method in it, exception occurs before method hit.
Read more

python - Terminate a gnome-terminal opened with subprocess -

-
using subprocess , command ' gnome-terminal -e bash ' can open gnome-terminal desired (and have stick around). done either p=subprocess.popen(['gnome-terminal', '-e', 'bash']) or p=subprocess.popen(['gnome-terminal -e bash'], shell=true) but cannot close terminal using p.terminate() or p.kill() . understand, little trickier when using shell=true did not expect run problems otherwise. to terminate terminal , children (in same process group): #!/usr/bin/env python import os import signal import subprocess p = subprocess.popen(['gnome-terminal', '--disable-factory', '-e', 'bash'], preexec_fn=os.setpgrp) # here... os.killpg(p.pid, signal.sigint) --disable-factory used avoid re-using active terminal can kill newly created terminal via subprocess handle os.setpgrp puts gnome-terminal in own process group os.killpg() used send signal group
Read more

c - Unrecognised emulation mode: elf_i386 on MinGW32 -

-
i'm trying make kernel, , cannot link c output assembly. ld . i'm getting error: unrecognized emulation mode: elf_i386 i'm using windows 10 professional mingw32 , msys. code using: link.ld /* * link.ld */ output_format(elf32-i386) entry(start) sections { . = 0x100000; .text : { *(.text) } .data : { *(.data) } .bss : { *(.bss) } } kernel.c /* * kernel.c */ void kmain(void) { const char *str = "my first kernel"; char *vidptr = (char*)0xb8000; //video mem begins here. unsigned int = 0; unsigned int j = 0; /* loops clears screen * there 25 lines each of 80 columns; each element takes 2 bytes */ while(j < 80 * 25 * 2) { /* blank character */ vidptr[j] = ' '; /* attribute-byte - light grey on black screen */ vidptr[j+1] = 0x07; j = j + 2; } j = 0; /* loop writes string video memory */ while(str[j] != '\0') { /...
Read more