Unit testing potential thread leakage

So you've bewildered yourself into the dark world of pain threads, and you wanna make sure your app doesn't leak threads.

I found I had to calm my nerves by having tonnes of these Spock tests covering my Thread spawning.

I took my actual code, pasted it into this blog, stripped and rewrote it to cut straight to the point.

This silly service only illustrates the two methods for starting and stopping threads. This sample only spawns 1 thread but the point here are tests below.


To be sure your startJobs actually creates your thread(s), you should have a test confirming that first.


And here is the test looking for leakage:


The whole idea is finding the threads by your given thread name. And yes, i know Thread.sleep isn't ideal, and I may not need it, but it just feels reasonable to make sure the thread gets a little time to get going.

Side note: I learned that you have to cancel both the Timer and the TimerTask. If you only cancel the Timer, you'll have to wait for the GC to clean up the thread, and your test will fail.

grails run-app and tomcat jmx

Want the standard Tomcat JMX exposed during development (grails run-app) ?

Add this to your Config.groovy:
grails.tomcat.jvmArgs=["-Dcom.sun.management.jmxremote", "-Dcom.sun.management.jmxremote.port=8099", "-Dcom.sun.management.jmxremote.ssl=false", "-Dcom.sun.management.jmxremote.authenticate=false"]

Fire up jconsole and hit localhost:8099.

</note to future self blog entry>

Unit test your logging

I have a few log statements which are really important for me in production. Without them, I would be screwed when tracking whats actually going on. So I decided that this important code shouldn't be left out in the cold.

Here is a trivial example. Lets say the "Replacing address" log statement in the following code is really important when stuff hits the fan.



void setAddress(Address adr) {
  log.info "Replacing address ${this.address} with ${adr} on ${this}"
  this.address = adr
  log.debug "Updated{person} with new address"
}

setup:

  def log = []

  def collectLog = {String message -> log << message}

  def finn = new Person(name:'finn', new Address(streetname:'homeless'))

  finn.metaClass.log = [debug:collectLog, info:collectLog, error: collectLog]

when:

  finn.address = new Address(streetname:'Slottsplassen 1')

then:

  assert log.find {String msg -> msg.contains "Replacing address homeless with Slottsplassen 1 on Finn"}

The code turned out simpler than I feared when I initially decided I need to test my logging. Joy!

When I say Jump, Jenkins asks how high

... if you want even faster feedback than waiting for Jenkins to poll your stuff

- In project config within Jenkins. Check: "Trigger builds remotely", add a token, like "foobar" into the textbox.

- SSH to your SVN-server and find the project home within svnroot.


- Here is my post-commit hook:
finn@subversionserver:/var/svnroot/myproject/hooks$ cat post-commit

#!/bin/sh
/usr/bin/wget -O /dev/null --auth-no-challenge --http-user=finn --http-password=JENKINS_PASSWORD http://jenkinsserver:8080/view/Finn/job/myproject/build?token=foobar

If you don't have authentication on Jenkins, remove --http-user and --http-password

uhm, there is a ~/.ssh/config file?

How did I miss this for all these years I've used ssh?

~/.ssh/config lets you type 

#ssh home
instead of
#finn@84.215.xxx.xxx -p 10001

Here is the ~/.ssh/config
Host home
HostName 84.215.xxx.xxx
Port 10001
User finn

Combine this passwordless ssh with
ssh-copy-id -i ~/.ssh/id_rsa.pub finn@84.215.xxx.xxx
and phoning home is pretty effortless.

... another one of those note-to-future-Finn (tm) posts which I'll copy and paste stuff from.

HOWTO: Grails / GORM + Oracle Schema

By simply adding schema to your domain classes, you won't be able to get it working in h2 afterwards. Here is how you fix that.

1. Update all your domain classes

grails-app/domain/Person.groovy

class Person {

        String name

static mapping = {

table name: 'PERSON', schema:"MYSCHEMA"

}

}

2. Update all your datasource

grails-app/conf/DataSource.groovy:

environments {

    development {

        dataSource {

            dbCreate = "create-drop"

            url = "jdbc:h2:mem:devDb;MVCC=TRUE;LOCK_TIMEOUT=10000;INIT=CREATE SCHEMA IF NOT EXISTS MYSCHEMA"

        }

repeat that for the test environment also.

Putting this in writing because I wish I had this blog post a while ago.

Cheat sheet: ssh + screen + irssi

Cheat Sheet: SSH + IRSSI + SCREEN 

1. ssh user@myserver

2. screen -r  resumes your screen. screen starts a new one, then type irssi

3. /server irc.freenode.org to connect to the server. /join #channelName to join a channel

4. /wc leaves a channel.

5. <ctrl+a d> detaches screen, type exit to leave your ssh session

JavaScript Continous Testing Smackdown

I'm pretty new to serious JavaScript coding. By serious, I mean writing lots of pure JavaScript, as a contrary to putting some JavaScript functions into your <script> block to fizz things up.

I immediately sought out a way to get a smooth TDD development cycle running. I already had a theoretical idea on how to do this - from some of my colleagues who are JavaScript elitists in my book.

As far as I'm aware of, the technique described in this post is THE way to do test driven JavaScript development these days.

So, what will I get from this stuff?

You'll get a terminal which automatically runs your JavaScript tests immediately as you change code. Like Spork in Ruby On Rails, or Infinitest in Eclipse.

Consider the alternative. Having to refresh your browser on every change of JavaScript code. Sound familiar? It's no good; for your quality of life or your aging process (gray hair). Little less your productivity.

Under the hood, these guys passes your JavaScript code to "browser slaves" and shoves the test-results back into your terminal window. I've tried both Buster.JS and JsTestDriver. So, which is the king of TDD JavaScript (in my opinion ofcourse)?

Buster.JS

jsTestDriver using watchr

All the code from this blog entry can be found on github. Under a busterjs and jstd branch. https://github.com/finnjohnsen/jstesting

"But hey..? There are other ways to test JavaScript code"

I've noticed, and none I've found are attractive at all.

• You can refresh your browser and see if your stuff works. You'll have to do this anyway in addition to your unit testing. But you certainly don't want this to be the only way to test your code. It often involved clicking and typing to reach the code you know you want to test. Horrible and unfortunately probably the most common way to do JavaScript development in the world today (I made that up, it's just my guts talking).
• I've seen set-ups which involves refreshing or triggering the tests from your browser. For example at the Jasmine -site, this is what they document. This is a lot better than the previous bullet point. But we can do better.
• The Selenium approach: Automate and play back clicks and input-text into your browser. These kind of behavior tests are tightly coupled to html, slow and messy. But they have value as they ensure your site actually works! But it's not what I want for unit testing.
• Pure Node.js testing. I've tossed this idea, admittedly without really digging into it. Because browsers are too important. Different browsers have different JavaScript implementations, so isolating my tests to the Node.js v8 runtime feels fundamentally wrong.

If you know of other ways, please comment.

Buster.JS or JsTestDriver: My Conclusion

Both frameworks gives you a speedy feedback when working test driven development in JavaScript. However, either will help you all the way - as you may have wished.

• jsTestDriver requires more patching, hacking and scripting, but works well once you've got it running. You'll probably polish and tune your scripts for quite some time after you've got it running, but you have something working pretty easily. JSTD has wide adaptation and you'll almost certainly find a solution to your problems just by searching around a little. You can have a look at this wrapper, which is suppose to ease the scripting from JsTestDriver. Anyway, check out my sample set up.

• Buster.JS is a bigger creature. It seems to aim to provide you with some of the pieces missing in JsTestDriver. It provides contrinous/auto testing and an built-in assertion library. These two critical components are what made the installation and set-up of jsTestDriver take a while longer than buster. So check out my sample set up.

I spent about 45 minutes setting up this JSTD project, and 15 minutes on Buster.js. However I've done this before. I suspect I spent at least 2 or 3 times as much the first time.

Buster.JS states on it's website it is under heavy development. It even and uses the word 'unstable' to describe its current state. Leaving JSTD the choice if you can't live with that. Personally I like getting as much set up for me as possible, and I found nothing mentionable when setting up Buster.JS. So I will use Buster.JS on my next green field. I can however be considered biased, one of the authors of Buster.JS work in the same company as I.

Android Sources in Eclipse

I've blogged about getting Eclipse to show Android sources before, but these techniques no longer work. So I'll toss a working how-to out there again.


I'm still baffled that Google hasn't ensured this working from day #1 with this eclipse plugin. But enough crying :)


I should also mention there is an eclipse plugin made for this. I've never tried it. I don't like bloating eclipse with tons of plugins, unless I really have to. And this boils down to 8 pretty easy steps, so an eclipse plugin isn't something Id'e consider. (But I may change my mind later)

Download the sources from github, and attach them:

• Head over to the android source at github
• Click the android SDK you're working on. (*1) I'll choose Android 2.2 SDK release 2 
• Click the tiny link which says Tree

• Click the Snapshot link which just appeared by the Tree -link

You'll start downloading a file which is about 130mb. This is the android source file. It's filename starts with base, for me it's called base-08d9d9a.tar.gz

• move the base -file to the SDK platform directory, for me that's /opt/android-sdk-linux_x86/platforms/android-8
• unpack it (tar xvfz base-08d9d9a.tar.gz).
• head into eclipse, and press F3 over some android source you're missing, like Activity. And get this pesky window, which we all hate:

• Click Attach Source, choose External Folder, and choose the core/java in the base folder. For me that's /opt/android-sdk-linux_x86/platforms/android-8/base-08d9d9a/core/java

There you go. Please post a comment and tell me how this turns out for you :)


.finn


(*1) There are loads of versions there, and it's a bit confusing. I've concluded the stuff we're looking for should end with SDK. However I'm not 100% certain about what version to download to match the Android version we see in the Android SDK download tool - this is just me guessing.

Getting that Skeleton test up

When creating a new unit test file, getting the import statics right with Hamcrest is 30 seconds of hassle, every time.

Here is the typical unit test start-up I prefer:

@Test
public void skeletonTest() {
assertThat(true, is(equalTo(true)));
}

Here is a eclipse template, which may solve this. But may also not, as I tend to forget little tricks like these. Time will tell if I've been able to incorporate this my reflexes.

So, get the template in, make the new junit test file, type skeltest and press ctrl+space.

The template:

${imp:import(org.junit.Test)}
${imps:importStatic(
org.hamcrest.Matchers.equalTo,
org.hamcrest.Matchers.is,
org.junit.Assert.assertThat)}
@Test
public void skeletonTest() {
assertThat(true, is(equalTo(true)));
}

Tired of killing browsers?

Launching the browser a lot from your IDE while working on / developing <some> web app?

Getting zillions of tabs or new browser instances?

Killing them all by hand every now and then?

Small trick which tells Firefox to put all URL requests in the same tab, reusing one tab.

in the URL type: about:config

find the property browser.link.open_newwindow

change the value to 1

Follow up tip: Keep some Firefox fork/clone (or whatever) installation which you use just for development, and apply this setting to it. This way you don't mess up your Firefox for normal surfing. I keep swiftweasel in /opt/swiftweasel

watch your code do actual work - in runtime

Short version: Use AOP to log your method invocations. Store method input parameters and result object, serialized to XML. Put it all into context with a Graphviz file generated as a side-effect.

Visual beats plain old text for a lot of purposes. So why not generate something visual from your running code, showing you what's happening using boxes and arrows. I've had tons of fun (and not too much sleep) lately digging into this using Graphviz.

The value of this is to track that the flow is as expected, and if you need to you can investigate the input and output data going into your ex. repositories or "logical services". The limitation of this is that all needs to run within the same VM and thread.

Here is what the proof of concept app produced from a dummy java stack in a unit test.


















The text on the arrows are clickable links and gives the user this if clicking the "0.Arguments":

Okok, the arguments and results are just boring serialized XML data. They could probably be graphs too, but I haven't come around to that and that isn't really what I'm after.

I also made a simple Servlet, which lists all generated graphs and makes you able to click and view any of these.

the code
The rest of this blog will be slightly in-depth and describing the knots and bolts of this.

The main idea is to accumulate a graphviz dot -file a cross a call-stack. At the points you want as opposed to everything (using reflection and looking at your entire call stack).

Aspect point cuts (Spring and AspectJ) is perfect for the job. I use ThreadLocal to keep a graph context/handler and simply append to it every time the aspect is invoked. The context saves a file and is released when the call-stack is done.

The Servlet in the proof of concept code is just a nice-to-have and requires graphviz to be installed on the "server". It invokes the 'dot' executable and pushes the output svg (or pdf / png / whatever you want) to the user.

You probably don't want this logging done all the time, due to performance and extensive logging to your temp. So I've stuck a check for an environment variable in the aspect - so you can enable and disable when you need this.

That's it really. You can check out the proof of concept from github and run the tests to see the graph-files get generated. Use the code as you please. Check out the README and please comment if you see any weaknesses or have any opinions.

/Finn

+rw clipboard

I saw some guy do a cat on some file (directly into his clipboard) and then paste/ctrl-v the text into his editor. My brain immediataly went "gimme! gimme! gimme!"

After a little googling around I found that his tools, pbcopy/pbpaste, was exclusively for OS X. A few minutes more with google showed me that the tool for the job in Ubuntu was xsel. 


#apt-cache search xsel
xsel - command-line tool to access X clipboard and selection buffers


However xsel covers both read and write and uses parameter to distinct them. My brain isn't up for the job remembering parameters so I decided Steve Jobs'es command was my way. Alias will fix this right up and I have pbcopy and pbpaste and feel happy and warm inside.

get xsel

#sudo apt-get install xsel

Here are the commands:

#cat myfile.txt | xsel -ib

#xsel -o > mycopy.txt


put these two lines into ~/.bashrc if you want this the OS X -way.

alias pbcopy='xsel -ib'
alias pbpaste='xsel -ob'

Do it like apple does:
#cat myfiles.txt | pbcopy
#pbpaste > mycopy.txt

/Finn


edit.(15.04.2011) For some reason the xsel with Ubuntu 10.10. needs xsel -ib (not just xsel -i). No idea why, but blog post is updated accordingly.

Spotify links in Ubuntu & Chrome

Spotify doesn't work in Linux, natively. (edit: it does now) So it's done using wine following this excellent howto.

This means however, that Chrome (and your Ubuntu) is unaware of spotify running "under the radar" as a wine/spotify.exe process. So when clicking a link, like this one: http://open.spotify.com/track/61K2lUXbNSvhEIOkhcozK4 spotify doesn't react.

I've Copy&Paste this dead-on recipe from another blog I found, just so I don't loose it in the future. I can confirm it's working on my Ubuntu 10.04 x64 desktop machine.

echo '#!/bin/sh' > ~/.browser2spotify

echo 'wine "$HOME/.wine/drive_c/Program Files/Spotify/spotify.exe" /uri "$@"' >> ~/.browser2spotify

chmod 755 ~/.browser2spotify

gconftool-2 -t string -s /desktop/gnome/url-handlers/spotify/command "/home/${USER}/.browser2spotify %s"

gconftool-2 -s /desktop/gnome/url-handlers/spotify/needs_terminal false -t bool

gconftool-2 -s /desktop/gnome/url-handlers/spotify/enabled true -t bool

The clue with this is to make gnome pass URLs starting with spotify: to the script which passes the URL to spotify.exe

This will do the job until Spotify makes a proper native linux client. Id'e think that wouldn't be too hard as it's an Adobe Air application, which is flash, which is pretty platform independent and very capable of running properly on Linux.

/Finn

sshfs - brilliantly simple

Got another linux/unix -box sitting somewhere in your home? I do, and I want _everything_ available to me from where I sit at my desktop.

I've used nfs quite a bit in the past, while quite easy to set-up it's a hassle compared to the simpleness of sshfs. And setting up samba is like getting skinned alive on this scale.

(ubuntu)

#apt-get install sshfs

(ubuntu, the other machine / slave)

#apt-get install openssh-server

put this in /etc/fstab, and you can on-demand mount/unmount the other computer when you see fit:

sshfs#finn@10.0.0.53:/ /home/finn/theslavecomputer fuse noauto,defaults,users,idmap=user 0 0

Now you have the root of the filesystem of the other machine sitting in your home directory. Sweet?

#mount theslavecomputer

... any time you need it. I prefer not using auto mount as I only need access to the other machine like 1 in 20 times I boot. A good rule on fstab in my experience is to not put a lot of stuff on auto-mount. It will get you in trouble when booting eventually which makes you so miserable you'll not click 'like' on any comments you'll read Facebook that day.

Tip: make this password less by uploading the public certificate. Google it and you'll get a step-by-step in a sec.

/Finn