Codon Compiler For Python Is Fast - but With Some Caveats

For 16 years, Rik Farrow has been an editor for the long-running nonprofit Usenix. He's also been a consultant for 43 years (according to his biography at Usenix.org) - and even wrote the 1988 book Unix System Security: How to Protect Your Data and Prevent Intruders. Today Farrow stopped by Slashdot to share his thoughts on Codon. rikfarrow writes:Researchers at MIT decided to build a compiler focused on speeding up genomics processing... Recently, they have posted their code on GitHub, and I gave it a test drive. "Managed" languages produce code for a specific runtime (like JavaScript). Now Farrow's article at Usenix.org argues that Codon produces code "much faster than other managed languages, and in some cases faster than C/C++." Codon-compiled code is faster because "it's compiled, variables are typed at compile time, and it supports parallel execution." But there's some important caveats:The "version of Python" part is actually an important point: the builders of Codon have built a compiler that accepts a large portion of Python, including all of the most commonly used parts - but not all... Duck typing means that the Codon compiler uses hints found in the source or attempts to deduce them to determine the correct type, and assigns that as a static type. If you wanted to process data where the type is unknown before execution, this may not work for you, although Codon does support a union type that is a possible workaround. In most cases of processing large data sets, the types are known in advance so this is not an issue... Codon is not the same as Python, in that the developers have not yet implemented all the features you would find in Python 3.10, and this, along with duck typing, will likely cause problems if you just try and compile existing scripts. I quickly ran into problems, as I uncovered unsupported bits of Python, and, by looking at the Issues section of their Github pages, so have other people. Codon supports a JIT feature, so that instead of attempting to compile complete scripts, you can just add a @codon.jit decorator to functions that you think would benefit from being compiled or executed in parallel, becoming much faster to execute... Whether your projects will benefit from experimenting with Codon will mean taking the time to read the documentation. Codon is not exactly like Python. For example, there's support for Nvidia GPUs included as well and I ran into a limitation when using a dictionary. I suspect that some potential users will appreciate that Codon takes Python as input and produces executables, making the distribution of code simpler while avoiding disclosure of the source. Codon, with its LLVM backend, also seems like a great solution for people wanting to use Python for embedded projects. My uses of Python are much simpler: I can process millions of lines of nginx logs in seconds, so a reduction in execution time means little to me. I do think there will be others who can take full advantage of Codon. Farrow's article also points out that Codon "must be licensed for commercial use, but versions older than three years convert to an Apache license. Non-commercial users are welcome to experiment with Codon."

Read more of this story at Slashdot.