Rebalancing Cassandra 1.2 cluster when a seed node (from total of 2) is died using virtual nodes -


i have cassandra 1.2 cluster in local machine modified hosts:

#cassandra nodes m.y.i.p               m.y.i.p   (node 1)<---- seed 1 m.y.i.p               127.0.0.3 (node 2) m.y.i.p               127.0.0.4 (node 3)<---- seed 2 m.y.i.p               127.0.0.5 (node 4) m.y.i.p               127.0.0.6 (node 5) 

... cluster has total of 5 nodes. i'm using virtual nodes, don't havo set initial_token anywhere.

the steps i've followed:

  1. start seed nodes (1 , 3)
  2. start nodes 2 , 4.
  3. turn off node 3.
  4. try join node 5.

the problem in 4th step, when try join node 5 cassandra throws exception:

   java.lang.illegalstateexception: unable find sufficient sources streaming range (-7204018021044318047,-6965052533844316617]         @ org.apache.cassandra.dht.rangestreamer.getrangefetchmap(rangestreamer.java:205)         @ org.apache.cassandra.dht.rangestreamer.addranges(rangestreamer.java:129)         @ org.apache.cassandra.dht.bootstrapper.bootstrap(bootstrapper.java:81)         @ org.apache.cassandra.service.storageservice.bootstrap(storageservice.java:890)         @ org.apache.cassandra.service.storageservice.jointokenring(storageservice.java:659)         @ org.apache.cassandra.service.storageservice.initserver(storageservice.java:514)         @ org.apache.cassandra.service.storageservice.initserver(storageservice.java:411)         @ org.apache.cassandra.service.cassandradaemon.setup(cassandradaemon.java:278)         @ org.apache.cassandra.service.cassandradaemon.activate(cassandradaemon.java:366)         @ org.apache.cassandra.service.cassandradaemon.main(cassandradaemon.java:409) java.lang.illegalstateexception: unable find sufficient sources streaming range (-7204018021044318047,-6965052533844316617]         @ org.apache.cassandra.dht.rangestreamer.getrangefetchmap(rangestreamer.java:205)         @ org.apache.cassandra.dht.rangestreamer.addranges(rangestreamer.java:129)         @ org.apache.cassandra.dht.bootstrapper.bootstrap(bootstrapper.java:81)         @ org.apache.cassandra.service.storageservice.bootstrap(storageservice.java:890)         @ org.apache.cassandra.service.storageservice.jointokenring(storageservice.java:659)         @ org.apache.cassandra.service.storageservice.initserver(storageservice.java:514)         @ org.apache.cassandra.service.storageservice.initserver(storageservice.java:411)         @ org.apache.cassandra.service.cassandradaemon.setup(cassandradaemon.java:278)         @ org.apache.cassandra.service.cassandradaemon.activate(cassandradaemon.java:366)         @ org.apache.cassandra.service.cassandradaemon.main(cassandradaemon.java:409) exception encountered during startup: unable find sufficient sources streaming range (-7204018021044318047,-6965052533844316617] error 16:55:47,634 exception in thread thread[storageserviceshutdownhook,5,main] java.lang.nullpointerexception         @ org.apache.cassandra.service.storageservice.stoprpcserver(storageservice.java:321)         @ org.apache.cassandra.service.storageservice$1.runmaythrow(storageservice.java:479)         @ org.apache.cassandra.utils.wrappedrunnable.run(wrappedrunnable.java:28)         @ java.lang.thread.run(thread.java:722) 

the range (-7204018021044318047,-6965052533844316617] belongs died node (node 3). due i'm using virtual nodes, think ring must rebalance:

rebalancing cluster no longer necessary when adding or removing nodes. when node joins cluster, assumes responsibility portion of data other nodes in cluster. if node fails, load spread evenly across other nodes in cluster.

but i'm wrong, :).

what can join new node in existing cluster when previous node down? please, note have 256 tokens per node, nodetool move isn't option (i think).

thank in advance, info welcome!

the solution remove node 3 (which turned off) token manually.


Comments

Popular posts from this blog

ruby - Trying to change last to "x"s to 23 -

jquery - Clone last and append item to closest class -

c - Unrecognised emulation mode: elf_i386 on MinGW32 -