Thanks for this. You write well and the technical pace is perfect and easy to follow.
Peter, on
There is no link here to the next chapter!
aanrechtblad, on
This continually is amazing to me how bloggers such as yourself can find the time and also the dedication to keep on creating fantastic blog posts. This is wonderful and one of my have to read on the web. I simply want to say thanks
Peter, on
I especially like the effort to explain how the language works rather than just getting readers hacking away. This makes your book really stand out from common language tutorials. I am excited for more chapters!
Kevin, on
I agree with Rob. The beginning of this blog post is a bit too much and needs some counterbalancing. Honestly, I think your post would be much better without it. Perhaps you could just add a link to your rant? And put a disclaimer saying, “This is for everyone and here is why .” In my opinion rants like this are why people think of others so differently. As a child I never really noticed/thought about someone else being different. I always just saw them as a person without paying any attention to these differences I had yet to learn about. As I grew older I read rants like this one, and listened in history classes, etc. Eventually you are taught to think people are different. It is not that teachers or people around you meant to teach you to think that way, but just the mere discussion of differences implants those thoughts at least subconsciously into the brain. If I had never had anyone tell me the word racism or try to explain differences, I would not know what those concepts were in the first place. I am not saying that history is unimportant, but that if conversations like this were less… Perhaps children would not repeat the same mistakes. Perhaps children would not see these differences that were taught to me. Perhaps children will be able to read the actual content of a blog post instead of a long rant about equality followed by the content they visited the blog post for in the first place.
Peter Mogensen, on
The new PostgreSQL 9.4 bidirectional replication offers something they call “Global Sequences”.
They are supposed to be globally unique among several masters. They are not incremented in consensus, but different masters are allocated a subset of the sequence range in consensus.
At least - so the documentation says:
Although I have not read any post yet I finished saving them to my drive. I say thanks for your open intro and support for the ladies. Great job man and continue the good fight.
Boris Adryan, on
If you’re going to get this into actual book form, I’m happy to go through the explanations and exercises again and tell you were I was stuck and relied on other help.
Superb work. I love that Mitchell
incorporates Jepsen into the Consul build tests. I ask data store vendors how they do and use their response as a gauge.
Due props. Fabulous work thanks for the intelligent hard work.
Zhangjing, on
Really wonderful tutorial !!!
Bruno, on
This tutorial is bookmarked in my browser for a long time now. Please, just… keep it up!
Cheers!
Alvo, on
Great article, thanks. Which version of ElasticSearch were you using during these tests?
Vladimir Voznesensky, on
You cannot cure false failures in a system that does not guarantee A. If your client lose connection, it could not get ack and returns error to your application.
So, the optimal way seems to check if the data to be written again is already in the database.
Yu Shen, on
Thank you very much for providing very grounded tutorial for Clojure. It’s the best for beginner who has little experience with JVM/Java. The others often assumed background with Java. With your tutorial, I now feel that I’m more comfortable to use Clojure for real world problem, not just toy ones.
I’ve finished reading all up to this debugging post, and eagerly am waiting for more.
From your example in this section of picture frame material estimation, while I appreciate your effort in creating a somehow fuzzy example to embed some bugs. it occurred to me that some of such bugs might be better avoided with more explicit design modeling, especially visualized modeling of the concepts involved, such as rectangle, mat, etc.
It’s my observation, that Clojure/Lisp community has heavy focus on elegant syntactic expression of concepts, but somehow avoided visual modeling of the concepts. Maybe, it’s consider too trivial, and too much “object oriented”, such as UML.
With some experience with UML, I feel that its graphic modeling language really helps to clarify conceptual modeling, and help to grasp more concept relationship with certain degree of intuition.
Although object oriented programming as program construction has the fatal drawback of state dependency, lack of reference transparency, but it’s conceptual modeling techniques helps for programming at large for higher level reasoning of complex system design and construction.
For functional programming (FP), I feel that it may also need some visual design language to visualize in two dimensions the function relationship to make FP better handle higher level of abstraction, required for programming at large. For instance, in your blog section on modeling with Clojure, I found it very hard to follow the model and intuition of rocket launch model. I feel some diagram would help me to grasp the reasoning, and your insights.
So it’s my wish, if you could teach us on (visual) modeling design with Clojure (FP) paradigm.
Again thanks a lot for your great contribution to Clojure community to make Clojure further grounded.
Yu
Peter, on
Was going to use this resource until I saw the psychotic rants of the author. No way I would support someone with such hatred for other people. get over it - most coders are male, most people are straight. No reason to become a hate-monger.
Brendan, on
These tutorials are amazing. Thank you for all the time you’re clearly putting into them.
nowherekai, on
these articles about clojure are awesome, most clojure book is hard to study for beginners of clojure, help me a lot.
Nickola, on
+1 for confirmation that problem still exists in 2.6 release.
insider, on
RSS not working. Any alternative?
Diatribe, on
People asking to test Solr are missing the point - it’s not Solr vs. Elasticsearch, it’s ZenDisco vs. Zookeeper. Solr was smart enough to support the cluster manager everyone else in distributed computing is using (ZK) while ES made something that was easier to use. Unfortunately in their rush to make things easy, they forgot to make sure they didn’t lose data. Data loss is a really bad thing in indexing because you don’t know what you are missing. So when you search for things in Big Data apps (like bad actors, fraud cases, exceptions, unhappy customers, etc.) you don’t find them and you don’t know that you didn’t find them. You can solve this by reindexing constantly, but that doesn’t really work well in TB-sized data sets.
ulzha, on
Powerful read. So amazed.
Michael Young, on
Great article. I am also very interested in a similar test with Solr.
m, on
This is my favorite intro to recursion example I’ve seen so far!
Richard Raseley, on
Very interesting article - it really challenged some of my understanding around RabbitMQ.
Were you able to test how the addition of publisher confirms into the mix affects the number of lost messages?
My thought is that this would help mitigate the number of lost messages, as it helps ensure the consistency of a message in a particular queue across all nodes participating in that queue.
I’m new to Clojure. I was considering implementing data structures like BST etc in this and thought having a namespace clojure.tree might be a good idea. But others felt it was a horrible idea as there were other data structures that does all this work. Could you explain more about those?
Thanks!
Robert de Lorimier, on
It would be great to do a similar test with solr, as it is the nearest technology to elastic search.
mathiasx, on
I think there’s a small mistake in the highlighting for the spares function. It looks like it parsed the square brackets for fn arguments as a markdown link in the anonymous function inside the mapcat.
Daniel Compton, on
Another extremely useful library I use for debugging is clojure.tools.trace. Among other things it lets you trace (print) the values passed in and returned from a function, either your own or a library that you are using. It often makes debugging very quick because you can see the context of function calls that led to an exception being thrown.
Thanks for this. You write well and the technical pace is perfect and easy to follow.
There is no link here to the next chapter!
This continually is amazing to me how bloggers such as yourself can find the time and also the dedication to keep on creating fantastic blog posts. This is wonderful and one of my have to read on the web. I simply want to say thanks
I especially like the effort to explain how the language works rather than just getting readers hacking away. This makes your book really stand out from common language tutorials. I am excited for more chapters!
I agree with Rob. The beginning of this blog post is a bit too much and needs some counterbalancing. Honestly, I think your post would be much better without it. Perhaps you could just add a link to your rant? And put a disclaimer saying, “This is for everyone and here is why .” In my opinion rants like this are why people think of others so differently. As a child I never really noticed/thought about someone else being different. I always just saw them as a person without paying any attention to these differences I had yet to learn about. As I grew older I read rants like this one, and listened in history classes, etc. Eventually you are taught to think people are different. It is not that teachers or people around you meant to teach you to think that way, but just the mere discussion of differences implants those thoughts at least subconsciously into the brain. If I had never had anyone tell me the word racism or try to explain differences, I would not know what those concepts were in the first place. I am not saying that history is unimportant, but that if conversations like this were less… Perhaps children would not repeat the same mistakes. Perhaps children would not see these differences that were taught to me. Perhaps children will be able to read the actual content of a blog post instead of a long rant about equality followed by the content they visited the blog post for in the first place.
The new PostgreSQL 9.4 bidirectional replication offers something they call “Global Sequences”.
They are supposed to be globally unique among several masters. They are not incremented in consensus, but different masters are allocated a subset of the sequence range in consensus. At least - so the documentation says:
“BDR uses a voting/election system to assign blocks from global sequences. The details are in the source code.” https://wiki.postgresql.org/wiki/BDR_Global_Sequences#How_global_sequences_are_managed_internally
Although I have not read any post yet I finished saving them to my drive. I say thanks for your open intro and support for the ladies. Great job man and continue the good fight.
If you’re going to get this into actual book form, I’m happy to go through the explanations and exercises again and tell you were I was stuck and relied on other help.
The hat not cap paper in the post doesn’t have the consistency hierarchy diagram. Maybe you meant: http://www.bailis.org/papers/hat-vldb2014.pdf
Superb work. I love that Mitchell incorporates Jepsen into the Consul build tests. I ask data store vendors how they do and use their response as a gauge.
Due props. Fabulous work thanks for the intelligent hard work.
Really wonderful tutorial !!!
This tutorial is bookmarked in my browser for a long time now. Please, just… keep it up!
Cheers!
Great article, thanks. Which version of ElasticSearch were you using during these tests?
You cannot cure false failures in a system that does not guarantee A. If your client lose connection, it could not get ack and returns error to your application.
So, the optimal way seems to check if the data to be written again is already in the database.
Thank you very much for providing very grounded tutorial for Clojure. It’s the best for beginner who has little experience with JVM/Java. The others often assumed background with Java. With your tutorial, I now feel that I’m more comfortable to use Clojure for real world problem, not just toy ones.
I’ve finished reading all up to this debugging post, and eagerly am waiting for more.
From your example in this section of picture frame material estimation, while I appreciate your effort in creating a somehow fuzzy example to embed some bugs. it occurred to me that some of such bugs might be better avoided with more explicit design modeling, especially visualized modeling of the concepts involved, such as rectangle, mat, etc.
It’s my observation, that Clojure/Lisp community has heavy focus on elegant syntactic expression of concepts, but somehow avoided visual modeling of the concepts. Maybe, it’s consider too trivial, and too much “object oriented”, such as UML.
With some experience with UML, I feel that its graphic modeling language really helps to clarify conceptual modeling, and help to grasp more concept relationship with certain degree of intuition.
Although object oriented programming as program construction has the fatal drawback of state dependency, lack of reference transparency, but it’s conceptual modeling techniques helps for programming at large for higher level reasoning of complex system design and construction.
For functional programming (FP), I feel that it may also need some visual design language to visualize in two dimensions the function relationship to make FP better handle higher level of abstraction, required for programming at large. For instance, in your blog section on modeling with Clojure, I found it very hard to follow the model and intuition of rocket launch model. I feel some diagram would help me to grasp the reasoning, and your insights.
So it’s my wish, if you could teach us on (visual) modeling design with Clojure (FP) paradigm.
Again thanks a lot for your great contribution to Clojure community to make Clojure further grounded.
Yu
Was going to use this resource until I saw the psychotic rants of the author. No way I would support someone with such hatred for other people. get over it - most coders are male, most people are straight. No reason to become a hate-monger.
These tutorials are amazing. Thank you for all the time you’re clearly putting into them.
these articles about clojure are awesome, most clojure book is hard to study for beginners of clojure, help me a lot.
+1 for confirmation that problem still exists in 2.6 release.
RSS not working. Any alternative?
People asking to test Solr are missing the point - it’s not Solr vs. Elasticsearch, it’s ZenDisco vs. Zookeeper. Solr was smart enough to support the cluster manager everyone else in distributed computing is using (ZK) while ES made something that was easier to use. Unfortunately in their rush to make things easy, they forgot to make sure they didn’t lose data. Data loss is a really bad thing in indexing because you don’t know what you are missing. So when you search for things in Big Data apps (like bad actors, fraud cases, exceptions, unhappy customers, etc.) you don’t find them and you don’t know that you didn’t find them. You can solve this by reindexing constantly, but that doesn’t really work well in TB-sized data sets.
Powerful read. So amazed.
Great article. I am also very interested in a similar test with Solr.
This is my favorite intro to recursion example I’ve seen so far!
Very interesting article - it really challenged some of my understanding around RabbitMQ.
Were you able to test how the addition of publisher confirms into the mix affects the number of lost messages?
My thought is that this would help mitigate the number of lost messages, as it helps ensure the consistency of a message in a particular queue across all nodes participating in that queue.
For those wanting to view linked code, use “…/jepsen/tree/old/…” instead of “…/jepsen/blob/master/…”. For example, https://github.com/aphyr/jepsen/blob/master/src/jepsen/cassandra.clj#L116-L149 should be https://github.com/aphyr/jepsen/tree/old/src/jepsen/cassandra.clj#L116-L149
good example
Hi,
I’m new to Clojure. I was considering implementing data structures like BST etc in this and thought having a namespace clojure.tree might be a good idea. But others felt it was a horrible idea as there were other data structures that does all this work. Could you explain more about those?
Thanks!
It would be great to do a similar test with solr, as it is the nearest technology to elastic search.
I think there’s a small mistake in the highlighting for the
sparesfunction. It looks like it parsed the square brackets for fn arguments as a markdown link in the anonymous function inside themapcat.Another extremely useful library I use for debugging is clojure.tools.trace. Among other things it lets you trace (print) the values passed in and returned from a function, either your own or a library that you are using. It often makes debugging very quick because you can see the context of function calls that led to an exception being thrown.
Jepsen Knossos was recently subpoenaed to appear in court as an expert witness of the trail of Paxos Agorithm. Apparently he is a really snappy dresser and a hard as nails private detective. You can read extracts from the court transcriptions over at http://simbo1905.wordpress.com/2014/10/05/in-defence-of-the-paxos-consensus-algorithm/