Aaron, you’re looking for the “Crime in the United States” reports–the index is on that UCR page. I didn’t link the individual reports because a.) there’s a dozen of them, and b.) I had to trawl through old PDFs to get them. The FBI’s published data only goes back to 1992, sadly, but I did submit a request for their full crime statistics, including scanned PDFs back to 1930. That arrived this week; when I get a chance I’ll dig into it and make an expanded chart.

“…I’m surprised by the fact that nobody remarks on the place of the US here : it appears to be in between developed countries and developing ones. For the superpower whose GDP per capita is among the highest, whose military is the strongest on earth, isn’t that a bit disquieting?”

Yeah, I’m unsettled too. I think most people should. Income inequality (by most metrics I’m aware of) has been rising for the past fifty years or so.

There’s an argument that increasingly industrialized markets will tend to have higher levels of income inequality as smaller pools of more skilled individuals take on the workloads that used to occupy many. Since technology amplifies the abilities of individuals, natural variation in our activities due to hard work, family background, dumb luck, etc, are amplified as well, resulting in more and more disparate incomes.

I’m not sure I buy this argument–my naive, unconsidered objection might start by noting that many heavily industrialized societies like Japan and Germany have much lower income and wealth inequalities than ours.

Regardless of whether there’s a causal link between advancing technology and income inequality, the combination of the two suggests an obvious policy. We could decide that as our collective powers increase, we can afford as a society to provide increasing levels of basic needs to every person: free housing, transport, food, healthcare, and education, financed by the generosity of the rich and by a large-scale welfare state. Even highly progressive tax curves retain some financial incentive for work: look at the US from 1914 to 1984.

“I’m curious about inequality as a predictor. GINI isn’t great. The folks at the Equality Trust in the UK might have an interesting approach to analyzing these data.”

Quite right. Both income and wealth distributions are measured differently in every country, and even if they were, the Laffer curve in both cases isn’t preserved by the Gini function. We should also note that hypothetical societies with completely even income distribution can show unequal wealth distribution due to savings. At best the Gini coefficient is only a rough approximation of relative access to capital, goods, and services. We could also be measuring a systematic bias: “countries with income reporting schemes which bias towards equality because of social or policy factors, also tend to have lower rates of gun crime,” that sort of thing.

You’re right, David–there are several variables at play here. Obviously “Guns per capita” is a poor predictor of “deaths caused by a firearm”–the graph tells us that much. “Percentage of the population which owns guns” might be a better predictor, but since the fraction of the population which kill people with firearms is so small compared to the larger pool of gun owners, and criminals are less likely to report their gun use in surveys, that metric could have its difficulties too.

I should also be clear that this comparison tell us only about the relationship of gun prevalence in each country, not the policymaking question “How would the US behave if we changed the prevalence of guns?”

There’s a very simple and politically untenable solution to that question, which is simply to determine some observables like “amount of gun-related violence” and “amount of non-gun-related violence”, sign and implement a strict gun-control law–say, outlawing possession of handguns nationally–and watch what happens for five years. The more assertive the law is, the easier it’ll be to determine its effects.

Local bans won’t be sufficient: plenty of municipalities have imposed strict firearms ownership restrictions but without controlled borders these laws have little impact on firearm density. Surveys of felons who used firearms in an offense indicate cost is an important factor in weapon selection. A national buyback might raise the price of grey-market weapons used in crime, and cause substitution to other, less deadly weapons or crime.

This is certainly consistent with the NFA study, and perhaps with Britain’s firearms ban, but there’s a distinct lack of solid data. Since the US has so many guns, a unique mix of cultural factors, and is so large, I suspect controlled experiment could be the best way to determine the effects of policy measures.

Yeah, folks do tend to best understand the languages they speak. I find Chinese pretty hard to follow, too!

That said, this is nontrivial Clojure code. It uses several parts of the stdlib including the STM and futures, has no comments at all, uses the same functions to do multiple things at once, and the variable names are extremely short. Why? Because this gist is a transcript of my REPL session–I wrote this code live and never intended it to be read. Production Clojure tends to look a little different. ;-)

Maybe some narration will help.

https://gist.github.com/3200862

The code lives in a namespace, which I’ve called “messagepassing.core”. It’s going to use the standard java library’s LinkedTransferQueue. We also set a constant ’m': the number of messages to exchange.

The main function can do two things: test the use of queues, or the use of atoms. It just looks at the first command-line argument to decide what to do.

queue-test defines a function “bounce”, which takes a number from one queue, increments it, and puts it back into the other queue. It keeps doing this until the number is bigger than m. In the node.js code this function is also called bounce, but it’s not recursive because Node is calling the function as a part of its IO loop. In fact you can’t express any other type of flow control in Node.

Then queue-test creates a pair of queues (q1 and q2) and a pair of workers (w1 and w2). The workers are just independent threads (futures) which bounce a number between queues. @w2 expands to (deref w2), and dereferencing a future means “wait for the future’s return value”. So in the main thread, @w2 waits until the test is complete. Then we clean up the other worker thread.

Atom-test uses the STM, so there’s no real node analogy here. It defines a function called bump, which increments an atomic reference ‘a’ until it reaches m. swap! is a clojure core function which changes the value of an atomic reference using a function of its current value. Since swap! also returns the value it set, we can use (when) to determine whether the threshold is reached–without race conditions.

The test creates a single atomic reference x to the number one, then spawns a pair of worker threads w1 and w2 which bump x until it reaches the threshold m. We resolve both futures, which blocks until each has determined that the atom’s value is now equal to m.

Hope that helps. :)

Great point, Joe. Storing timestamped transaction logs does not provide CP semantics; I was thinking purely of the transactional serializability. Readers can still miss an arbitrary amount of history, so the operational semantics are not CP.

One of the reasons I wrote this post, in addition to your talk, is that I’ve tried to build systems based on last-write-wins, and advocated using external coordinators for stronger consistency to others on the mailing list! I hadn’t considered that since Riak doesn’t yet provide negative guarantees around failures, you can’t control when your writes succeed and destroy other information! I’m excited to see your work on 2PC develop, even with the possibility of unattainable quorum, there are some practical partition cases where this will not arise and serializable semantics continue operating. And ultimately, safety is preserved–the most important property for CP structures. :)

I’ve been kinda nuked by health issues this year, only got halfway to finishing aphyr.com. :-( Lots of folks have asked for a feed; it’s a high priority for me.

Bascule, I think this is the best possible outcome. It’ll be really nice for folks who have invested time in learning Node to be able to take advantage of parallelism and keep using the same patterns and libraries. I imagine the work that’s being done on isolates will come into play as well.

TJ, it’s great that folks are working on high-performance socket message passing–but I suspect you may not have read the post.

Regarding node isolates, I think that it would probably be less difficult to port node.js to a better VM than to modify the VM to support real or faked parallelism. Node does a few things right: smart packaging, a rich library of single-purpose tools at hand, a simple-to-understand concurrency model, etc. It also does some important things spectacularly wrong: being single-threaded, inheriting Javascript’s laughable type system, offering syntactically heavy closures as the end-all-be-all of nonlinear programming, etc. Since JS and Dart are single-threaded by specification, I have serious doubts about the semantics of parallel extensions to the core language.

Re: TLB shootdowns, thanks! This is honestly out of my depth, and I’ve read conflicting explanations as to what exactly triggers them. If I understand correctly, modern Linux kernels only need to reach an interprocessor barrier for TLB flushes when memory maps change. I assume that v8 makes relatively few allocations, and watching /proc/interrupts appears to confirm this: there are basically no TLB shootdowns for any of my message-passing examples above.

I agree that stateless servers are boring, but this is a.) good, because boring things are easy to understand, write, and analyze; and b.) many things can be accomplished with stateless components. Because shared state is complicated and expensive, I try to isolate the stateful part of my systems from the stateless ones. The stateless parts may well be good candidates for Node.js. That said, you’re correct to observe that stateless servers invariably offload their state-sharing to another system: a database, queue, internal services, third party APIs, and so on.