Comments on: Popularity vs Performance vs Productivity

By: Isaac Gouy

Isaac Gouy — Sun, 17 Apr 2011 17:35:24 +0000

Kyle Hailey :I have no data on Java interpreted vs just in time.

Yes you do! You show 2 charts from the benchmarks game - the first compares Ruby 1.8.7 with JIT'd Java; the second compares Ruby 1.9 with interpreted Java not JIT'd Java.

Kyle Hailey :It makes me wonder about the Linux tests. I assume these tests are with a running JVM and already compiled into machine code. What about something that has to start up a JVM and has not yet compiled the machine code?

Your assumption is not correct! The Help page tells you how the programs were measured. The Help page also provides a comparison between starting up JVM and a running JVM.

By: Kyle Hailey

Kyle Hailey — Tue, 15 Mar 2011 17:19:53 +0000

I have no data on Java interpreted vs just in time usage in the field. Just in time I’d expect would be about the only way people run Java in a server, which makes sense on server side code. I’ve have never been closely involved on server side code, other than Oracle connection session pooling which I wrote in my own Python process that the server communicated with. For server side code, java has the pros of the ability to do most anything one can do these days as well as the speed of execution. On the down side the code bloat and over architecting and lower productivity might be enough to cancel out gains. Gains from algorithmic speeds are going to be easily lost in badly architected code. Code that is smaller, easier to read, easier to write, easier to change will lend itself to faster changes and corrections in architecture.
It comes down to how well the path to be implemented is known. If the path is well known the Java might just be the best choice – slower coding but a solid fast stable finished product, but how many projects know exactly what they are going to do? For projects that are going to change, which is especially true of any consumer space, the flexibility of the architecture and functionality or at least the speed at which the team can re-implement code is going to be the ultimate criteria.
I agree that the data on coding productivity is weak, I wish there was more, but the simple code example of java vs python is powerful. If it takes me 5 days to write something in Java and 1 in Python, then hands down Python is the winner for me.
And as far as execution speed goes, java speed was my first and still biggest issue. Most of my experience is with client side Java, and maybe Java is just not meant for client side apps, but again I see things done in 10M memory spaces running 10x faster than their java equivalents.
It makes me wonder about the Linux tests. I assume these tests are with a running JVM and already compiled into machine code. What about something that has to start up a JVM and has not yet compiled the machine code? Isn’t that the way most client side apps go?
Wondering if Ruby or Python will ever have machine code compiled versions. A quick google search and there seem to be a number of projects working on python compilation to machine code and/or a JVM with JIT.

By: Isaac Gouy

Isaac Gouy — Tue, 15 Mar 2011 15:52:15 +0000

@Kyle Hailey

When people talk about the execution time of Java programs do you think they are talking about JIT’d Java or Java without JIT?

When you type java –version does that show mixed mode or interpreted mode?

By: Kyle Hailey

Kyle Hailey — Mon, 14 Mar 2011 21:24:53 +0000

python 3 looks to be much less than 10x slower compared to Java 6 -Xint: http://dboptimizer.com/wp-content/uploads/2011/03/python_vs_java_xint.png but gets fairly trounced vs Java 6 -Server: http://dboptimizer.com/wp-content/uploads/2011/03/python_vs_java_Xserver.png

By: Isaac Gouy

Isaac Gouy — Mon, 14 Mar 2011 19:44:49 +0000

And given that Dion Hinchcliffe was trying to make a point about “Web development” let’s look at a recent published study of web development, and note the authors remark:

- “First, the most complete solution is a Java solution. Many people might find this surprising, because scripting languages (such as Perl or PHP) tend to have higher productivity at least for small tasks.” page 3

http://page.mi.fu-berlin.de/prechelt/Biblio/platforms07-cacm-2010.pdf

By: Isaac Gouy

Isaac Gouy — Mon, 14 Mar 2011 19:13:25 +0000

Incidentally, the “study of productivity in C, Java, Perl, Python” only *measured* development time for the Java, C++ and C programs. Those programs were written under controlled conditions.

In constrast, the programs written in other languages were contributed by a self selected group of program authors who responded to an online request. The development time recorded for those programs was whatever the program authors claimed :-)

By: Isaac Gouy

Isaac Gouy — Mon, 14 Mar 2011 19:08:00 +0000

The text still says “Python is roughly 10x times slower in execution time” when from that graphic Python averages 20x slower, and the benchmarks game shows a bunch of measurements >30x slower than Java http://shootout.alioth.debian.org/u32/python.php

Of course, the problem with Dion’s graphic is that although it’s titled “Hard Metrics” only the program times were measurements – the “expected development efficiency gains” is plucked out of thin air ;-)

By: Kyle Hailey

Kyle Hailey — Mon, 14 Mar 2011 18:45:43 +0000

absolutely right. I changed the text to reflect this.
What I wanted to see in a single graphic is the performance verses productivity and had thought that Dion had pointed this out with the text on the graphic showing “Range of expected development efficiency gains from productivity oriented approaches.”
I will look into making a graphic showing productivity vs performance.

By: Isaac Gouy

Isaac Gouy — Mon, 14 Mar 2011 14:19:47 +0000

> “Now the above graphic might be hard to read (wish it was larger) but what it is indicating is that Python is roughly 10x faster to program in than Java.”

No, that is not what weblanguagecomparison2.png indicates.

The graphic indicates the run time performance of programs written in different programming languages.

The graphic indicates the Python programs ran on average 20x slower than the Java programs.

The graphic has no information about how it took the Python programmers to write the Python programs or how long it took the Java programmers to write the Java programmers.

The graphic is simply (an out of date) summary from the benchmarks game website, here it is – http://shootout.alioth.debian.org/gp4/which-language-is-best.php

Here’s an up-to-date summary http://shootout.alioth.debian.org/u32/which-programming-languages-are-fastest.php