Go to content Go to navigation Go to search

Scala vs. Groovy: static typing is key to performance
Thursday September 6, 2007 by Derek Young

I was curious about the performance of two of the JVM based languages, Scala and Groovy, relative to Java itself. I suspected that by having more type information available at compile time Scala’s bytecode would look more like the output of javac and therefore have performance closer to that of straight Java.

I started with the Java version of the (somewhat notorious) Ray tracer language comparison benchmark that has been used to compare the performance of various languages. The benchmark generates a ray traced image of recursively positioned spheres. I took the minimal optimization version and ported it into both Scala and Groovy.

I kept the code as close to the Java version as possible. Neither Scala nor Groovy supports Java/C style for loops, so I substituted the recommended loops in each language. Groovy doesn’t support inner classes so I had to pull some code out to the top level. Groovy uses BigDecimal for floating point literals, so I suffixed them all with “d” to make them doubles. I left all type declarations in the Groovy version, although Groovy programs are typically not written this way.

Results

ray.java 12.89s
ray.scala 11.224s
ray.groovy 2h 31m 42s

I was expecting the Groovy code to run longer than the Scala code but was shocked at the actual difference. All three versions of the code produce identical images: (fullsize here)

Because Scala knows the types of all variables, it produces floating point operations for floats and integer operations for ints, like Java. The compiled output of Groovy treats everything dynamically, determining at runtime which functions to use for simple operations and using objects like java.lang.Double instead of double. The more the bytecode looks like Java code the better it will perform.

Unlike most software, a ray tracer executes almost entirely in its own code, not in any external libraries or waiting for I/O. This highlights the difference between the three compilers, but most software will see less of a difference.

The Scala timings demonstrate that algorithms can be coded in Scala without much loss in performance, even for critical areas of code.

I should point out it’s possible I made a mistake in the translation from Java to Groovy, but I tried to make it as fair as I could. I would be interested in hearing if there is something I should have done differently that explains the difference.

Timing Notes
All times recorded with time from the command line, which does include startup overhead. With over a 2 hour difference this doesn’t matter.

OS: Ubuntu Feisty
javac -version: build 1.6.0-b105
Groovy execution: time groovy ray.groovy 8 512
Scala compilation: scalac ray.scala
Scala execution: scala scalaRay 8 512
Java compilation: javac ray.java
Java execution: java ray 8 512

  1. >>>groovy ray.groovy 8 512
    Why you dont use groovyc unlike javac & scalac???


    — Vadim Voituk    Sep 6, 10:57 AM    #

  2. The only reason I didn’t use groovyc was because the difference was so great, and the compilation overhead at the beginning of the run only takes a couple seconds. I decided it wasn’t worth waiting another two and a half hours to time the compiled output. Running with groovy first compiles the code just like groovyc does, then executes that code. It doesn't interpret the source code or run any differently. I’ll make another timing tonight after work when my machine’s free and post the results though.


    — Derek Young    Sep 6, 01:29 PM    #

  3. Wouldn’t matter if he used groovyc. Cuts off 7 sec at the most, and that’s if it’s a Pentium II 200mhz. The real issue is that using Groovy to do ray tracing is like wearing high heel shoes to a marathon. Groovy was never meant for heavy calculations driven apps like this.

    The real groovy solution would be to write the renderer in Java or Scala, and call it from Groovy. Ray tracing isn’t groovy, it’s gnarly, and requires a gnarly language to do efficiently.


    — Danno Ferrin    Sep 6, 01:30 PM    #

  4. Just a little stylistic note on your Scala code. You can write 1 until n instead of 1 to (n-1).


    — psykotic    Sep 6, 01:37 PM    #

  5. It’s not clear whether you’re measuring -server or -client

    Scala while loops will probably be faster than for comprehensions over ints


    — Isaac Gouy    Sep 6, 01:54 PM    #

  6. There is no reason for a dynamically typed language to be slower than a statically typed language. It’s a matter of implementation. Perhaps performance is not a priority for Groovy, but I wish Groovy fans would just admit this upfront, instead of claiming that dynamic typing is inherently slow, because this has been disproven time and time again.


    — Slava Pestov    Sep 6, 03:36 PM    #

  7. I’ve compiled the groovy script using groovyc and ran the example. Killed the run after 4 minutes (on a 2.16ghz dualcore) nowhere near the 12 second mark.


    — p3t0r    Sep 6, 04:11 PM    #

  8. the biggest mistake people make is using explicit types all other. That makes Groovy slower in the current implementation, and maybe later too, because the runtime has to check and transform the types. The next thing is that many calculations in Groovy are way slower than in Java because Groovy is doing them in BigDecimal. I plan to add optimzed double and integer modes that will speed up the code very much. But that will take some more months, because before this the MetaClass system will be changed. I am not sure Groovy will ever be as fast as Java. The method calls are doing so much more in Groovy.. but I expect Groovy to become much faster. Compared to Java I think a factor below 10 is reachable.


    — Jochen "blackdrag" Theodorou    Sep 6, 04:20 PM    #

  9. Yes, you can speed up your Groovy version by removing static types which slow it down, using traditional for loops (Groovy 1.1 only) and not using BigDecimal (currently does calculations in BigDecimal and then converts to Double). It will currently still be much slower – but mostly nothing to do with dynamic vs static typing.


    — Paul King    Sep 7, 09:30 PM    #

  10. You might also use -server for about 2 times speedup and consider operator overloading for code style so that instead of:

    Vec o = Vec.add(ray.orig, Vec.add(Vec.scale(i.lambda, ray.dir), Vec.scale(delta, i.normal)))

    you would have:

    def o = ray.orig + (i.lambda * ray.dir + delta * i.normal)


    — Paul King    Sep 7, 09:36 PM    #

  11. Great to get some numbers even tho they are kinda special case!
    Any plans to try out Jython or/and JRuby to?


    — krs    Sep 9, 06:30 AM    #

  12. What is the groovy version? I get the following exception when running against groovy 1.0:

    Caught: java.lang.ArrayIndexOutOfBoundsException: 1

    at ray.run(ray.groovy:144) at ray.main(ray.groovy)
    — Marcos Silva Pereira    Sep 9, 10:57 PM    #

  13. Marcos: I used groovy V1.0. You need to pass in two arguments on the command line or it will crash like that. The first argument is the recursion level that determines how complex the scene is. The second is the resolution (the images are always square). The tests I ran were with the arguments 8 512.


    — Derek Young    Sep 10, 09:06 AM    #

  14. What were your runtimes for the C/C++/etc versions on your machine?


    — Warren Henning    Oct 2, 01:54 PM    #

  15. These are my results. I’m using a Macbook 2.0 GHz Intel Core Duo with 2 GB 667 MHz DDR2 SDRAM running MacOSX Tiger 10.4.11

    $ uname -a
    Darwin goku2s-computer.local 8.11.1 Darwin Kernel Version 8.11.1: Wed Oct 10 18:23:28 PDT 2007; root:xnu-792.25.20~1/RELEASE_I386 i386 i386

    JAVA
    $ java -version
    java version “1.5.0_13”
    Java™ 2 Runtime Environment, Standard Edition (build 1.5.0_13-b05-241)
    Java HotSpot™ Client VM (build 1.5.0_13-121, mixed mode, sharing)

    $ time javac ray.java
    Note: ray.java uses unchecked or unsafe operations.
    Note: Recompile with -Xlint:unchecked for details.

    real 0m0.465s
    user 0m0.359s
    sys 0m0.063s

    $ time java ray 8 512

    real 0m37.126s
    user 0m35.219s
    sys 0m1.638s

    SCALA

    $ scala -version
    Scala code runner version 2.6.1-final — © 2002-2007 LAMP/EPFL

    $ time scalac ray.scala

    real 0m6.055s
    user 0m4.190s
    sys 0m0.317s

    $ time scala scalaRay 8 512

    real 0m40.592s
    user 0m39.019s
    sys 0m1.406s

    GROOVY

    $ groovy -version
    Groovy Version: 1.5.4 JVM: 1.5.0_13-121

    $ time groovyc ray.groovy

    real 0m1.238s
    user 0m1.096s
    sys 0m0.123s

    $ time groovy ray.groovy 8 512

    real 194m57.459s
    user 192m56.086s
    sys 1m0.091s

    Compiled version
    Obs: A changed the lines:

    (new rayMain()).run(Integer.parseInt(args1), Integer.parseInt(args0), 4);
    to

    public class Tester { public static void main(args) { (new rayMain()).run(Integer.parseInt(args1), Integer.parseInt(args0), 4); }
    }

    To be able to call the main method os the class Tester.

    $time groovyc ray.groovy

    real 0m1.290s
    user 0m1.114s
    sys 0m0.127s

    $ time java -cp /Applications/groovy-1.5.4/embeddable/groovy-all-1.5.4.jar:. Tester 8 512

    real 201m29.815s
    user 198m4.698s
    sys 1m10.802s

    We have to note that scala and groovy versions are direct translations from the java source. But that’s not a excuse to have such a diferent performance in this case. Aprox. 300 times slower than scala and java version is a huge diference.


    — Goku2    Feb 9, 02:17 PM    #
  Textile Help