@Thomas The n1 lost its leadership as soon as there was a partition because there are only 2 nodes on its side (n1 and n2). The requirement is that there should be (n/2)+1 available in the cluster so in this case it would be 3.
Colin Scott, on
For what it’s worth, the lein invocation should now be: lein run redis-sentinel
Andre, on
“N5 wins the race, and proceeds to accept writes. If we heal the partition with salticid jepsen.heal, and wait a few seconds, the nodes will detect the fully connected cluster and the new primary will step down, to allow n1 to resume its place.”
This sound to me like n1 will be the primary again. But the drawing indicates that n5 remains the primary (what makes sense to me).
Who is the primary after “salticid jepsen.heal” ?
Ivan, on
Well, now I see you use e.g.
(merge craft initial-space-center)
not sure if this would be possible using records (fresh Clojure noob here). So probably you prefer maps for flexibility? Is there a way to combine this with records or protocols to make it safer, or would you stick with this in a serious / bigger app?
Ivan, on
Very interesting tutorial, thanks!
Just a question, do you have anything against records and protocols? As far I understand this is be suitable for model data, as presented here. It would be safer than using only maps. Maybe, you consider this approach is too OO? Or you just want to keep it simple?
Yaroslav Nosa, on
Nice, what is it, Oregon,Colorado?
Colin Scott, on
For what it’s worth the pg app needs to be invoked from the top-level jepsen directory (the cd salticid step above is incorrect)
Amy Jenkins, on
One of the most useful things for a beginner to do is to muck around with the command line. Thanks for this!
Marc, on
Hi Kyle
Thank you for these very nice series of article on Clojure. Could you give us “the” solution — or the more idiomatic one — to the problems you list at the end of this article? I started solving them, but I’m kind of stuck on the third :/ https://gist.github.com/falzm/9543903
Just to counterbalance the opening slightly, and perhaps this is just an encouraging view of the way things are where I live, but certainly at my software company we’ve interviewed female applicants whose CVs weren’t good enough, but we wanted to hire more women. One colleague hired this way didn’t know the difference between megabytes and gigabytes, but the point is it certainly wasn’t an environment that discouraged female hires.
When I was at school and university white guys such as myself weren’t encouraged socially to keep doing tech/geeky studies; in fact one chief idea is that women don’t like guys who do that sort of thing. I think facing opposition is something we all do if we want to do something that isn’t socially normal, and it’s hard (but necessary) to try and not confuse that opposition that almost everyone faces with some genuine victimisation that one has felt in the past, or lump all opposition together as having the same cause. E.g. if you have two guys who don’t like you, one of them might not because of your gender or sexuality or whatever (i.e. unfair victimisation) and the other might not because he doesn’t feel you work hard enough. If that happens, don’t think that both guys don’t like your for the victimisation reason, easy though it is to do.
I’m a Brit working in South Africa, and Afrikaaners sometimes give me a hard time. I keep smiling and chatting nicely, and eventually they stop thinking of me as British and just think of me as a person. I haven’t yet written a technical blog post that starts by encouraging Brits to work in South Africa :)
fco, on
bah… can’t see what you’ve put in your “heal” and few other scripts, only browsed rapidly through that unreadable clojure thing (sorry, never liked lisp-like languages). So I quite don’t understand what’s behind your “survivors” messages.
Additionnally, with PG and other R-DMBS, 2PC is not used for single-branch transactions, so I have serious doubts against the whole article. And the behaviour you describe seems normal and expected (I have already seen it with Oracle+JDBC).
FWIW with real 2PC you would have ended with stale transactions ready to commit, which would require manual intervention to definitely commit or rollback.
fco, on
You are basically showing off that a Redis cluster doesn’t work when it’s not used for what it’s designed… (too long for detailed explanation here)
Nobody was doubting it.
I’m the first to say that Redis Sentinel doesn’t work without careful understanding and configuration, especially since it’s based on a bad “quorum” definition.
The problem here, is it should dismiss n1 as soon as it would detect it lost the “true” quorum of 3 (see below). But hopelessly this isn’t the coded algorithm, and the quorum is only applied to slave->master transition :(
However it could be made to work correctly only if you enforced a sane quorum definition, such as mainly with “min-slaves-to-write 2”.
Lauri, on
What a great post! There are a few bugs in the code if you follow along with the snippets:
the makes-orbit test should use (flight trajectory) rather than trajectory when analyzing the crash time etc.
ascent should be defined as [0 300] and circularization as [400 1000] for the tests to pass
tiensonqin, on
This series is so helpful, very great work!
pron, on
I think more care should be taken with the definition of “consistent”. You define consistency in the strictest of senses: all nodes see all acknowledged operation in the same order. For many (most?) systems, a more useful definition of consistency is the integrity of data constraints specific to the application (e.g. the sum of all accounts in the system must remain constant). That is the meaning of consistency in ACID. Ordering guarantees a stronger consistency, which is a strict subset of ACID consistency, but is not required by many systems. In the asynchronous replication model, if you only allow reads from the master, then while you may lose data upon failover, the data may still be consistent in the ACID sense; what you lose is ACID’s “D”: durability.
Seunghoon, on
Awesome post! Thanks for sharing!
Aphyr, on
Wilson, Flo, actually, the value of the var does change. Symbols are immutable; vars are mutable. Subsequent calls to def don’t create a new var–instead they modify the var’s value. That’s how you can redefine functions and values in Clojure programs piecewise. See the chapter on state. :)
user=> (let [x (def a "foo") y (def a "bar")] (identical? x y))
true
Thomas Beck, on
But what happened to the two leaders once the partition was resolved? Did n1 drop his lead responsibility?
Matt, on
Noah is right, your mass function should be adding in the upper stage for the fuel and dry mass. If you modify the function to add that, then the craft can no longer make it into orbit either!
eGerula, on
Back again to (source class) when you can see included something like: {:added “1.0”
:static true}. This is metadata and looks like a map. In this example it has 2 keys, :added and :static. In some functions declarations you can find the :type key also in metadata. You can find an example here: http://clojuredocs.org/clojure_core/clojure.core/type
eGerula, on
class - “Returns the Class of x”
type - “Returns the :type metadata of x, or its Class if none”
Therefore, type does what class does when metadata of the x doesn’t contain a :type.
Also, (source type) shows you the body of the type as:
(or (get (meta x) :type) (class x)))
Adrian, on
Just a heads up …basic woodworking tells us that tops should be attached with notches so that as the top expands it does not crack where the screws were used to attach it to its base. I would hate to see your efforts ruined. …I realise you’re table is inside but humidity changes from summer to winter can be enough to cause damage. Think of old windows or doors that stick in summer…Remove the corner 90 brackets and notch them. Then put a washer between the top and the bracket so the screw can slide
good luck
Aphyr, on
Thanks for the kind words, all. Goal has always been to produce a full-length book; hopefully some time by the end of the year.
Noah Easterly, on
Shouldn’t the atlas-v use its next-stage’s mass to determine its dry-mass?
Great series, thanks!
anonymous, on
This is a nice intro to functional techniques, but there are hundreds of these around the web. What’s really missing is more and better examples of dealing with resource management and side effects in FP.
Honza, on
There is a tiny typo in the definition of trajectory. The dt argument is ignored and 1 is hard-coded.
Donald Parish, on
Great addition. One of my first programs was a TRS-80 simulation of a space ship orbiting a planet, I hope to interest my son in using it for his physics class.
jeff, on
I’m in the throes of deciding on a finish for my black walnut slab dining table. I am seriously considering Rubio, but, am worried that it won’t stand up, nor look as good, as a wipe on poly finish. Of course, it takes 1/10th the time and is 1/50th less stressful too.
Any updates on durability for yours?
Aphyr, on
Thank you! :)
Jeremy, on
Cool, thanks. Really enjoying this series!
Aphyr, on
Think I finally convinced myself to fix the comment formatting code, haha. :)
@Thomas The n1 lost its leadership as soon as there was a partition because there are only 2 nodes on its side (n1 and n2). The requirement is that there should be (n/2)+1 available in the cluster so in this case it would be 3.
For what it’s worth, the lein invocation should now be:
lein run redis-sentinel“N5 wins the race, and proceeds to accept writes. If we heal the partition with salticid jepsen.heal, and wait a few seconds, the nodes will detect the fully connected cluster and the new primary will step down, to allow n1 to resume its place.”
This sound to me like n1 will be the primary again. But the drawing indicates that n5 remains the primary (what makes sense to me).
Who is the primary after “salticid jepsen.heal” ?
Well, now I see you use e.g.
(merge craft initial-space-center)
not sure if this would be possible using records (fresh Clojure noob here). So probably you prefer maps for flexibility? Is there a way to combine this with records or protocols to make it safer, or would you stick with this in a serious / bigger app?
Very interesting tutorial, thanks!
Just a question, do you have anything against records and protocols? As far I understand this is be suitable for model data, as presented here. It would be safer than using only maps. Maybe, you consider this approach is too OO? Or you just want to keep it simple?
Nice, what is it, Oregon,Colorado?
For what it’s worth the
pgapp needs to be invoked from the top-level jepsen directory (thecd salticidstep above is incorrect)One of the most useful things for a beginner to do is to muck around with the command line. Thanks for this!
Hi Kyle
Thank you for these very nice series of article on Clojure. Could you give us “the” solution — or the more idiomatic one — to the problems you list at the end of this article? I started solving them, but I’m kind of stuck on the third :/ https://gist.github.com/falzm/9543903
Cheers,
m.
What’s more idiomatic:
(fn pair (second pair))
or
#(apply * %)
?
Just to counterbalance the opening slightly, and perhaps this is just an encouraging view of the way things are where I live, but certainly at my software company we’ve interviewed female applicants whose CVs weren’t good enough, but we wanted to hire more women. One colleague hired this way didn’t know the difference between megabytes and gigabytes, but the point is it certainly wasn’t an environment that discouraged female hires.
When I was at school and university white guys such as myself weren’t encouraged socially to keep doing tech/geeky studies; in fact one chief idea is that women don’t like guys who do that sort of thing. I think facing opposition is something we all do if we want to do something that isn’t socially normal, and it’s hard (but necessary) to try and not confuse that opposition that almost everyone faces with some genuine victimisation that one has felt in the past, or lump all opposition together as having the same cause. E.g. if you have two guys who don’t like you, one of them might not because of your gender or sexuality or whatever (i.e. unfair victimisation) and the other might not because he doesn’t feel you work hard enough. If that happens, don’t think that both guys don’t like your for the victimisation reason, easy though it is to do.
I’m a Brit working in South Africa, and Afrikaaners sometimes give me a hard time. I keep smiling and chatting nicely, and eventually they stop thinking of me as British and just think of me as a person. I haven’t yet written a technical blog post that starts by encouraging Brits to work in South Africa :)
bah… can’t see what you’ve put in your “heal” and few other scripts, only browsed rapidly through that unreadable clojure thing (sorry, never liked lisp-like languages). So I quite don’t understand what’s behind your “survivors” messages.
Additionnally, with PG and other R-DMBS, 2PC is not used for single-branch transactions, so I have serious doubts against the whole article. And the behaviour you describe seems normal and expected (I have already seen it with Oracle+JDBC).
FWIW with real 2PC you would have ended with stale transactions ready to commit, which would require manual intervention to definitely commit or rollback.
You are basically showing off that a Redis cluster doesn’t work when it’s not used for what it’s designed… (too long for detailed explanation here) Nobody was doubting it.
I’m the first to say that Redis Sentinel doesn’t work without careful understanding and configuration, especially since it’s based on a bad “quorum” definition. The problem here, is it should dismiss n1 as soon as it would detect it lost the “true” quorum of 3 (see below). But hopelessly this isn’t the coded algorithm, and the quorum is only applied to slave->master transition :(
However it could be made to work correctly only if you enforced a sane quorum definition, such as mainly with “min-slaves-to-write 2”.
What a great post! There are a few bugs in the code if you follow along with the snippets:
the
makes-orbittest should use(flight trajectory)rather thantrajectorywhen analyzing the crash time etc.ascentshould be defined as[0 300]andcircularizationas[400 1000]for the tests to passThis series is so helpful, very great work!
I think more care should be taken with the definition of “consistent”. You define consistency in the strictest of senses: all nodes see all acknowledged operation in the same order. For many (most?) systems, a more useful definition of consistency is the integrity of data constraints specific to the application (e.g. the sum of all accounts in the system must remain constant). That is the meaning of consistency in ACID. Ordering guarantees a stronger consistency, which is a strict subset of ACID consistency, but is not required by many systems. In the asynchronous replication model, if you only allow reads from the master, then while you may lose data upon failover, the data may still be consistent in the ACID sense; what you lose is ACID’s “D”: durability.
Awesome post! Thanks for sharing!
Wilson, Flo, actually, the value of the var does change. Symbols are immutable; vars are mutable. Subsequent calls to
defdon’t create a new var–instead they modify the var’s value. That’s how you can redefine functions and values in Clojure programs piecewise. See the chapter on state. :)user=> (let [x (def a "foo") y (def a "bar")] (identical? x y)) trueBut what happened to the two leaders once the partition was resolved? Did n1 drop his lead responsibility?
Noah is right, your mass function should be adding in the upper stage for the fuel and dry mass. If you modify the function to add that, then the craft can no longer make it into orbit either!
Back again to (source class) when you can see included something like: {:added “1.0” :static true}. This is metadata and looks like a map. In this example it has 2 keys, :added and :static. In some functions declarations you can find the :type key also in metadata. You can find an example here: http://clojuredocs.org/clojure_core/clojure.core/type
class - “Returns the Class of x” type - “Returns the :type metadata of x, or its Class if none” Therefore, type does what class does when metadata of the x doesn’t contain a :type. Also, (source type) shows you the body of the type as: (or (get (meta x) :type) (class x)))
Just a heads up …basic woodworking tells us that tops should be attached with notches so that as the top expands it does not crack where the screws were used to attach it to its base. I would hate to see your efforts ruined. …I realise you’re table is inside but humidity changes from summer to winter can be enough to cause damage. Think of old windows or doors that stick in summer…Remove the corner 90 brackets and notch them. Then put a washer between the top and the bracket so the screw can slide
good luck
Thanks for the kind words, all. Goal has always been to produce a full-length book; hopefully some time by the end of the year.
Shouldn’t the
atlas-vuse itsnext-stage’s mass to determine itsdry-mass?Great series, thanks!
This is a nice intro to functional techniques, but there are hundreds of these around the web. What’s really missing is more and better examples of dealing with resource management and side effects in FP.
There is a tiny typo in the definition of
trajectory. Thedtargument is ignored and1is hard-coded.Great addition. One of my first programs was a TRS-80 simulation of a space ship orbiting a planet, I hope to interest my son in using it for his physics class.
I’m in the throes of deciding on a finish for my black walnut slab dining table. I am seriously considering Rubio, but, am worried that it won’t stand up, nor look as good, as a wipe on poly finish. Of course, it takes 1/10th the time and is 1/50th less stressful too.
Any updates on durability for yours?
Thank you! :)
Cool, thanks. Really enjoying this series!
Think I finally convinced myself to fix the comment formatting code, haha. :)