Monday, November 23, 2009

Extending Soy Templates

One of the more interesting features of Soy templates is that you can extend the templating system. While working with Soy from JavaScript, I found that I needed to do exactly that.

In order to avoid some confusion as to how you can extend the template system in Scala, even though you will be using the templates from JavaScript, I'll quickly explain how Soy templates work when you're using JavaScript. Essentially, you precompile your .soy file into a .js file using a template compiler (which is written in Java). It's the compiler that you're extending and it's output is what you use in your web app. So you don't directly use the template from JavaScript.

What I wanted to do was something like this:

{switch typeof($msgparm)}
{case 'number'}
<i4>{$msgparm}</i4>
{case 'string'}
<string>{$msgparm}</string>
{/switch}

Unfortunatly, there is no typeof function "out of the box" so I needed to write my own in a plugin. As I was only interested in using this function in templates that I'm going to use in JavaScript, I needed to write a class that implements a single interface - SoyJsSrcFunction. If I wanted to use this function from Java, I'd need to implement SoyToFuFunction instead. There are only 3 methods to implement, 2 of which are one-liners in Scala. In this instance, all three are one liners, but computeForJsSrc could be more complicated, depending on what you're doing.


@Singleton
class TypeOfFunction @Inject() extends SoyJsSrcFunction {
def getName = "typeof"

def getValidArgsSizes: java.util.Set[Integer] = ImmutableSet.of[Integer](1);

def computeForJsSrc(args: java.util.List[JsExpr]): JsExpr = new JsExpr("typeof(" + args.get(0).getText() + ")", Integer.MAX_VALUE)

}


Scala Source Code

As you can probably tell, all this does is use JavaScript's typeof operator to implement the functionality I was looking for.

You might notice the @Singleton and @Inject annotations - Soy uses Google Guice for dependency injection. This means that we're not done yet - we also need to write a subclass of Guice's AbstractModule so that our extension can be injected into Soy. Fortunately it's trivial and XML-free:

class ExtendedFunctionsModule extends AbstractModule {
def configure() = {
val soyFunctionsSetBinder = Multibinder.newSetBinder(binder(), classOf[SoyFunction])
soyFunctionsSetBinder.addBinding().to(classOf[TypeOfFunction])
}

}

Scala Source Code

To use the extension all I need to do is add my Module class to the template compiler's pluginModules option:


java -cp typeof.jar:scala-library.jar:SoyToJsSrcCompiler.jar com.google.template.soy.SoyToJsSrcCompiler --pluginModules com.maethorechannen.widgets.ExtendedFunctionsModule --outputPathFormat rpc.js rpc.soy

Consice Java Maps in Scala

This post is a follow up to my previous post on using Closure Templates in Scala.

One of the things that bugged me in the Scala example was how verbose it was compared to the Groovy version. So I had an idea - use YAML and the snakeyaml library to generate a Java Map with my data. While it's still a couple lines longer than the Groovy version, it's a lot simpler than the Scala original.

The biggest change is this:

val y = """widget: {
name: Widget Name,
display: {
width: 240,
height: 320
}
}"""

val yaml = new Yaml()
val map = yaml.load(y).asInstanceOf[java.util.Map[String, _]]


Scala Source Code

Using Google's Closure Templates in Scala and Groovy

Google recently released it's Closure Tools - which is made up of a JavaScript compiler, a JavaScript framework and a templating system which can be used from both Java and JavaScript. I'm guessing from the package names that Closure Templates are actually called Soy templates internally at Google, and that's what I'm going to call them for the rest of this post.

I'm not going to go into the syntax of Soy templates - if you've ever used a templating system like FreeMarker or Django Templates then you shouldn't have any trouble using Soy templates (though you should be aware that Soy eats white space, so your output might not be what you expect if you don't use special character commands to put whitespace into the output).

Here's an example template (that I saved as "widget.soy"):

{namespace com.maethorechannen.widgets}

/** Widget template
* @param widget Widget Data
*/
{template .widget}
<?xml version="1.0" encoding="utf-8"?>{\n}
<widget>{\n}
{\t}<widgetname>{$widget.name}</widgetname>{\n}
{\t}<width>{$widget.display.width}</width>{\n}
{\t}<height>{$widget.display.height}</height>{\n}
</widget>
{/template}


Using the templates from Scala:

At a minimum, you'll need the following imports:

import com.google.template.soy.SoyFileSet
import com.google.template.soy.data.SoyMapData
import com.google.template.soy.tofu.SoyTofu
import java.io.File


To pass your data to the ToFu rendering engine, you'll need to have your data in either a SoyMapData object or in a plain old Java Map object. As I wanted to use Scala's sugar for quickly building up a Map, I needed to write up an implicit def from Scala's Map to a SoyMapData (quick aside - the sort of thing that stops me from loving Scala is having to do something like this - the impedence mismatch between Java collections and Scala collections is still too big). This function isn't complete, but it will convert the values I was using:


implicit def mapToSoy(m: Map[String, _]): SoyMapData = {
val sm = new SoyMapData()
m.keys.foreach {k =>
val v: Any = m(k)
v match {
case s: String => sm.put(k,s)
case mm: Map[String, _] => sm.put(k, mapToSoy(mm))
case i: Int => sm.put(k, i)
}
}
return sm
}

Here's the Map I'm passing in:

val map = Map[String, Any](
"widget" -> Map[String, Any](
"name" -> "Scala Test",
"display" -> Map[String, Any](
"width"-> 240,
"height" -> 320)))

To actually use the template, I need to load the template into a SoyFileSet, and then compile the template to a rendering object


val sfs = (new SoyFileSet.Builder()).add(new File("widget.soy")).build();
val tofu = sfs.compileToJavaObj()

Finally, I render the template with my data

println(tofu.render("com.maethorechannen.widgets.widget", map, null));

Which renders the following:

<?xml version="1.0" encoding="utf-8"?>
<widget>
<widgetname>Scala Test</widgetname>
<width>240</width>
<height>320</height>
</widget>


Scala Source Code

Using the templates from Groovy:

Using Soy from Groovy is almost identical to using Soy from Scala, but without the need for the implicit def (as Groovy Map sugar returns a Java Map).

import com.google.template.soy.SoyFileSet
import com.google.template.soy.data.SoyMapData
import com.google.template.soy.tofu.SoyTofu

def map = [widget: [
name: "Widget Name",
display: [
width: 240,
height: 320
]
]]

def sfs = (new SoyFileSet.Builder()).add(new File("widget.soy")).build();
def tofu = sfs.compileToJavaObj()
println(tofu.render("com.maethorechannen.widgets.widget", map, null));


Groovy Source Code

Wednesday, July 15, 2009

Musical Fun With YQL

Vernian Process have released a large amount of their back catalog available for free download from last.fm (63 mp3 files worth)1. As someone who never says no to free dark ambient, I thought I would quickly screen scrape the free download urls from the 6 month chart page and feed them into wget instead of manually downloading them2.

For the screen scraping, I turned to Yahoo's YQL service. In effect, YQL makes the web "programmable" with a SQL-like language. One of YQL's features is that it can use any webpage as a data source (it also supports various web APIs, like flickr's, but all I need for this is it's html ability). Here's the YQL query:


select * from html
where url="http://www.last.fm/music/Vernian+Process/+charts?rangetype=6month&subtype=tracks"
and xpath="//a[starts-with(@href, 'http://freedownloads.last.fm')]"


That will return all the free download anchors contained on the page.

It was then a single command to download all the tracks (thanks to curl and wget)

curl "http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20html%20where%20url%3D%22http%3A%2F%2Fwww.last.fm%2Fmusic%2FVernian%2BProcess%2F%2Bcharts%3Frangetype%3D6month%26subtype%3Dtracks%22%20and%20xpath%3D%22%2F%2Fa%5Bstarts-with(%40href%2C%20'http%3A%2F%2Ffreedownloads.last.fm')%5D%22&format=xml" | wget --force-html -i -


1. They've done it because they've improved changed styles and have gone more Industrial.

2. In case you're wondering, I used the Chart and not the Free Tracks page as the Free Tracks page only had 4 of the freely downloadable tracks.

Monday, June 29, 2009

Sieve of Eratosthenes

This is my response to the Scala slide in the "JavaOne Talk: Performance Comparisons of Dynamic Languages on the Java Virtual Machine" presentation. In particular, the Scala example seemed quite verbose and "unscalalike", so I thought I would see what I could quickly knock up. It's the Sieve of Eratosthenes (sort of - it's recursive and it doesn't bother with Step 5 of the algorithm on the Wikipedia page, so it does more work than is needed).

It's quick and dirty and could stand to be improved - my main motivation was to show that it could be done quickly and in a lot fewer lines in Scala than was shown on the presentation slide so it's not the greatest example of code ever written. I've put it up here because blogger's commenting system nukes source code layout.


def primes(n: Int): List[Int] = {
def nomults(s: Int, xs: Seq[Int]): List[Int] = (for (x <- xs if x % s != 0) yield x).toList
def sieve(xs: List[Int]): List[Int] = {
if (xs.isEmpty) xs
else {
val p = xs.first
val nxs = nomults(p, xs drop 1)
p :: sieve(nxs)
}
}

val odds = nomults(2, 2 to n)
2 :: sieve(odds)
}

println( primes(100) )

Monday, April 20, 2009

Oracle To Buy Sun

Article at the NY Times

Sun's Press Release

My first impression is that this should be good for Java, less good for MySQL.

Wednesday, April 8, 2009

Scala on the Google AppEngine

Good News - It was fairly trivial to use Scala with the new Java support in AppEngine.


I used a text editor and ant instead of mucking about with Eclipse. It was pretty much a case of following the docs for Java, just using Scala instead of Java for writing the servlet. The main difference between using Java and Scala is that you need to merge some Scala specific items into the build.xml file. Here's my build.xml




<project>
<property name="scala.home" location="../scala-2.7.3.final" />
<property name="sdk.dir" location="../appengine-java-sdk-1.2.0" />
<property name="sources.dir" value="${base.dir}/src" />
<property name="build.dir" value="${base.dir}/build"/>

<import file="${sdk.dir}/config/user/ant-macros.xml" />

<path id="project.classpath">
<pathelement path="war/WEB-INF/classes" />
<fileset dir="war/WEB-INF/lib">
<include name="**/*.jar" />
</fileset>
<fileset dir="${sdk.dir}/lib">
<include name="shared/**/*.jar" />
</fileset>
</path>

<target name="init">
<property
name="scala-library.jar"
value="${scala.home}/lib/scala-library.jar"
/>
<path id="build.classpath">
<pathelement location="${scala-library.jar}" />

<pathelement location="${build.dir}" />
</path>
<taskdef resource="scala/tools/ant/antlib.xml">
<classpath>
<pathelement location="${scala.home}/lib/scala-compiler.jar" />
<pathelement location="${scala-library.jar}" />
</classpath>
</taskdef>
</target>


<target name="copyscala"
description="Copies the App Engine JARs to the WAR.">
<copy
todir="war/WEB-INF/lib"
flatten="true">
<fileset dir="${scala.home}/lib">
<include name="**/scala-library.jar" />
</fileset>
</copy>
</target>

<target name="copyjars"
description="Copies the Scala JARs to the WAR.">
<copy
todir="war/WEB-INF/lib"
flatten="true">
<fileset dir="${sdk.dir}/lib/user">
<include name="**/*.jar" />
</fileset>
</copy>
</target>

<target name="compile" depends="copyscala, copyjars, init"
description="Compiles Scala source and copies other source files to the WAR.">
<mkdir dir="war/WEB-INF/classes" />
<copy todir="war/WEB-INF/classes">
<fileset dir="src">
<exclude name="**/*.scala" />
</fileset>
</copy>
<scalac
srcdir="src"
destdir="war/WEB-INF/classes"
classpathref="project.classpath"
/>
</target>


<target name="runserver" depends="compile"
description="Starts the development server.">
<dev_appserver war="war" />
</target>

</project>