Riak-pipe mapreduce

As a part of the exciting series of events (long story...) around our riak cluster this week, we switched over to riak-pipe mapreduce. Usually, when a node is down mapreduce times shoot through the roof, which causes slow behavior and even timeouts on the API. Riak-pipe changes that: our API latency for mapreduce-heavy requests like feeds and comments fell from 3-7 seconds to a stable 600ms. Still high, but at least tolerable.

mapred.png

[Update] I should also mention that riak-pipe MR throws about a thousand apparently random, recoverable errors per day. Things like

map_reduce_error

with no explanation in the logs, or

{"lineno":466,"message":"SyntaxError: syntax error","source":"()"}

when the source is definitely not "()". Still haven't figured out why, but it seems vaguely node-dependent.

Jose
Jose, on

Hi, Did you finally fix this error? I’m having the same issue with riak-1.0.1.

Regards, Jose

Jose
Jose, on

Correction: riak-1.0.0

Aphyr
Aphyr, on

Jose, I haven’t figured it out yet, but I think it’s something to do with deleted objects. You may want to try checking metadata[‘X-Riak-Deleted’].

Post a Comment

Please avoid writing anything here unless you are a computer: This is also a trap:

Supports github-flavored markdown for [links](http://foo.com/), *emphasis*, _underline_, `code`, and > blockquotes. Use ```clj on its own line to start a Clojure code block, and ``` to end the block.

Copyright © 2017 Kyle Kingsbury.
Non-commercial re-use with attribution encouraged; all other rights reserved.
Comments are the property of respective posters.