java - find the max temperature of each month in a year using map reduce program -


my code seems correct me on cmd not giving correct output, please me find problem in code. runs output wrong:

package test;  import java.io.ioexception; import org.apache.hadoop.conf.configuration; import org.apache.hadoop.fs.path; import org.apache.hadoop.io.intwritable; import org.apache.hadoop.io.text; //import org.apache.hadoop.mapred.jobconf; import org.apache.hadoop.mapreduce.job; import org.apache.hadoop.mapreduce.mapper; import org.apache.hadoop.mapreduce.reducer; import org.apache.hadoop.mapreduce.lib.input.fileinputformat; import org.apache.hadoop.mapreduce.lib.output.fileoutputformat;   public class temp {     public static class mymapper extends mapper<object, text, intwritable,text> {          public void map(object key, text value,context context) throws ioexception, interruptedexception{              int month=integer.parseint(value.tostring().substring(16, 18));             intwritable mon=new intwritable(month);             string temp=value.tostring().substring(26,30);             string t=null;             for(int i=0;i<temp.length();i++){                 if(temp.charat(i)==',')                         break;                 else                     t=t+temp.charat(i);             }             text data=new text(value.tostring().substring(21, 25)+t);             context.write(mon, data);         }     }      public static class myreducer extends  reducer<intwritable,text,intwritable,intwritable> {          public void reduce(intwritable key,iterable<text> values,context context) throws ioexception, interruptedexception{             string temp="";             int max=0;             for(text t:values)             {                 temp=t.tostring();                 if(temp.substring(0, 4)=="tmax"){     if(integer.parseint(temp.substring(4,temp.length()))>max){                         max=integer.parseint(temp.substring(4,temp.length()));                     }                 }             }              context.write(key,new intwritable(max));         }          }      public static void main(string[] args) throws exception {         configuration conf = new configuration();         job job = job.getinstance(conf, "temp");         job.setjarbyclass(temp.class);         job.setmapperclass(mymapper.class);          job.setreducerclass(myreducer.class);         job.setmapoutputkeyclass(intwritable.class);         job.setmapoutputvalueclass(text.class);         job.setoutputkeyclass(intwritable.class);         job.setoutputvalueclass(intwritable.class);          fileinputformat.addinputpath(job, new path(args[0]));         fileoutputformat.setoutputpath(job, new path(args[1]));         job.waitforcompletion(true);          } } 

input file:

usc00300379,19000121,tmax,-78,,,6,
usc00300379,19000131,tmax,-133,,,6,
usc00300379,19000111,tmax,127,,,6,

output code is:

12 0
13 0
11 0

it looks have shifted index 1 (should substring(15, 17) instead of substring(16, 18)), can't figure out why based on input file snipped (maybe missed here?)

due shifted index receive months 12, 13, 11 instead of 01, 01, 01. , receive 0 max temperature due shifted index "tmax" isn't presented @ beginning of string.

suggestions:

  1. you have comma separated csv file, don't use substring. can use value.tostring().split(",") receive parts of record.
  2. see parts need (date , temperature).
  3. fetch month date part using substring method.
  4. you don't need tmax part in reducer. can produce intwitable, intwritable (month, temperature) mapper output. 5.get rid of tmax checks in reducer (use temperature intwritable values).

suggestions: don't create text , intwritable instances in mapper , reducer every time. can make instance fields , use set methods (it optimizes memory)


Comments

Popular posts from this blog

ruby - Trying to change last to "x"s to 23 -

jquery - Clone last and append item to closest class -

c - Unrecognised emulation mode: elf_i386 on MinGW32 -