colored log(back) for unit tests in two steps




The goal

Make my own stuff yellow, and keep the other stuff grey.

1. src/test/resources/logback.groovy
(modify the namespace you wanna highlight as your own)




2. Add this to your build.gradle (in addition to logback):

testCompile "org.fusesource.jansi:jansi:1.12"

Done.

Windows 10 - United States International AltGr No Dead Keys

I use US keyboard. However I loose my native æøå letters if I use standard US layout. AltGR No Dead Keys fixes this problem.
AltGR + z makes the "æ"
AltGr + l makes the "ø"
AltGr + w makes the "å"

Holding shift makes them uppercased.

I couldn't figure out how to use the .klc file I've used in previous Windows versions. So I ended up having to make a .msi installer in "Microsoft Keyboard Layout Creator". Tedious stuff, so I decided to put it in my blog where I (and maybe you) can get it from in the future.








USIntNDK_amd64.msi


PS. You shouldn't trust executables from random blogs, like here. So honoring that best practice I recommend you use the .klc keyboard layout file and create the .msi using Microsoft Keyboard Layout Creator yourself.

Minecraft server in Docker

Minecraft server, in a docker container

 

Feel free to use the docker image uploaded to Docker Hub during this christmas holiday.

Modify the /home/finn/minecraft_data -path to fit your server, and run this command:




docker run -d -v /home/finn/minecraft_data:/data -p 25565:25565 -p 25575:25575 --name minecraft -i finnjohnsen/minecraft-srv:1.8.1

The Dockerfile is on github

Have fun gaming

Digging into EHCache (and TTL)

Using a caching frameworks (like Grails Cache and Grails EhCache) instead of writing the caching mechanism yourself is comfortable. You find yourself smacking @Cacheable on listing operations and get methods - and CacheEvict on the CRUD operations. Right?

But this is extremely distant from the actual implementation - so I think it's a good idea to have a close look and confirm it works as you think.

So let's investigate, using 2 techniques achieving 3 goals.

Goals are:
1. Confirming Configuration run-time
2. Watching the Cache run-time
3. Confirming TTL timers in run-time
4. Unit testing

Techniques described:
1. JMX
2. Grails Console Plugin

First you need to verify that EhCache is JMX exposed. I have this in Bootstrap.groovy.

Then, fire up jConsole

Confirming configuration

EhCache works "great" without any configuration at all - and then runs purely on defaults - which can trick you to believe your system works as you configured.

So let's see that the configuration you intended are correct. We do this by browsing in jConsole to "net.sf.ehcache" -> "CacheConfiguration" and click the cache name.

I've now verified my cache TTL has the exact value 1800 seconds (aka 30 minutes), matches my config file. Meaning my config file works.

Watching the Cache run-time

Head over to CacheStatistics in jConsole, and watch for CacheHits and CacheMiss

CacheHit means the cache did its job and returned cached results instead of invoking the annotated method. CacheMiss means the method was invoked. A high number on CacheHits is probably a sign of a healthy set up.

Confirming TTL timers
This may seem uneccesary, but I wanted to look at how this worked, so I dug into it. I couldn't find the TTL timers exposed in JMX, so I dug into the runtime using Grails Console Plugin.

So grab the CacheManager, find your cache, and ask for ExpirationTime and Creation Time.

This code on Gist

As you see, there is a perfect 30 minutes between Expiration and Creation time. Perfect.

Now sit back and wait until the ExpirationTime passes - and watch a new CacheMiss increasse in jConsole! If it does, everything is working as you intented.


Unit testing
This kind of configurations doesn't feel like a good fit for unit testing in my book. But maybe I'm wrong.

Cheers
.finn

Easier Killing

Who are you?

"I take the long road" -guy (this has been me for years and years)

1. ps -ax | grep <something describing my process>
2. look for the <pid>
3. type kill -s 9 <pid>

"I made a shortcut" -guy

Make a clever script which uses ps | grep output, cut, awk or whatever to get the pid for the kill.

"I use the existing shortcut" -guy

pkill -9 -f  <something describing my process>

or pgrep first if I'm insecure about the regex

Mocking grailsApplication

You don't have to mock grailsApplication, unless you've put yourself  in the same boat as me and like writing pure jUnit. Keep running grails test-app all the time is just annoying and slow. Note I havent tested Grails 2.3 yet, which is supposed to be faster than the predecessors.

Anyway. This "pure" approach has one downside: your service/controller aren't getting its dependencies injected for free. You have to do everything yourself.

Injecting services is easy, as you just @grails.test.mixin.Mock them in during setUp(). But there is that def grailsApplication sitting at the top of your file. How to best mock it so you don't get NullPointerException in your test? I (mostly) use grailsApplication for configuration, so there lies my need for mocking.

There are several ways of getting this done and I have two ways which I use. I can't decide on a favorite. Here they are, I'll let you decide what you like:

nested maps

@Before
public void setUp() {
  service = MyService() //service is a global variable for the entire test.
  service.grailsApplication = [config:[myApp:[jms:[firstQueue:"firstQueue"]]]]

This works best if you have really few properties. Maybe just one. But it's a one-liner, which I like.

ConfigObject

@Before
public void setUp() {
  service = MyService()
  def mockConfig = new ConfigObject()
  mockConfig.myApp.jms.firstQueue="firstQueue"
  service.grailsApplication = [config: mockConfig]

I know deep down that ConfigObject is the proper way of doing this - but it's more stuff to write.

Which do you like best?

/Finn

Groovy / Grails / GORM - File Name Too Long

Yet another "File Name Too Long" workaround :-(

Normal day - was just gonna add one more check to my nice .where{} closure. But when I did, everything blew up in my face - and I'm sitting here with no eyebrows.

The file will no longer compiles!

#grails test-app
/home/finn/src/stratos/stratos/target/classes/no/nsb/stratos/MyService$_updateAllWithChanges_closure5_closure7_closure8_closure9_closure10_closure12_closure14_closure16.class (File name too long)

When I commented back in the validTo everything went melt-down.

List gsmList = DistributionText.where {
  status.enabled==true &&
  status.ok==false &&
  status.validFrom <= new Date() && 
//status.validTo >= new Date() &&
  status.station != null &&
  channelCode=="FOO" &&
  language == "eng"
}.list()

I can't figure out why this happens! The filename isn't over the Linux file length limit. 

I've seen this before when JSON marshalling really deep objects. (the workaroud then, was to create a map from intended object, and serialize the map instead)

So, what to do when you really don't understand the error in front of you? Escape! So I that's what I did - I rewrote to rewrote to Hibernate Criterion.

List<DistributionText> gsmList = DistributionText.createCriteria().list {
status {
  eq("enabled", true)
  eq("ok", false)
  le("validFrom", new Date())
  ge("validTo", new Date())
  isNotNull("station") 
}
eq("channelCode", "FOO")
eq("language", "eng")

} 

I kinda like better the Groovy DSL/where to this Hibernate syntax, so if you find out how to get around the File Name Too Long, please leave a comment.

Unit testing potential thread leakage

So you've bewildered yourself into the dark world of pain threads, and you wanna make sure your app doesn't leak threads.

I found I had to calm my nerves by having tonnes of these Spock tests covering my Thread spawning.

I took my actual code, pasted it into this blog, stripped and rewrote it to cut straight to the point.

This silly service only illustrates the two methods for starting and stopping threads. This sample only spawns 1 thread but the point here are tests below.


To be sure your startJobs actually creates your thread(s), you should have a test confirming that first.


And here is the test looking for leakage:


The whole idea is finding the threads by your given thread name. And yes, i know Thread.sleep isn't ideal, and I may not need it, but it just feels reasonable to make sure the thread gets a little time to get going.

Side note: I learned that you have to cancel both the Timer and the TimerTask. If you only cancel the Timer, you'll have to wait for the GC to clean up the thread, and your test will fail.

grails run-app and tomcat jmx

Want the standard Tomcat JMX exposed during development (grails run-app) ?

Add this to your Config.groovy:
grails.tomcat.jvmArgs=["-Dcom.sun.management.jmxremote", "-Dcom.sun.management.jmxremote.port=8099", "-Dcom.sun.management.jmxremote.ssl=false", "-Dcom.sun.management.jmxremote.authenticate=false"]

Fire up jconsole and hit localhost:8099.

</note to future self blog entry>

Unit test your logging

I have a few log statements which are really important for me in production. Without them, I would be screwed when tracking whats actually going on. So I decided that this important code shouldn't be left out in the cold.

Here is a trivial example. Lets say the "Replacing address" log statement in the following code is really important when stuff hits the fan.



void setAddress(Address adr) {
  log.info "Replacing address ${this.address} with ${adr} on ${this}"
  this.address = adr
  log.debug "Updated{person} with new address"
}

setup:

  def log = []

  def collectLog = {String message -> log << message}

  def finn = new Person(name:'finn', new Address(streetname:'homeless'))

  finn.metaClass.log = [debug:collectLog, info:collectLog, error: collectLog]

when:

  finn.address = new Address(streetname:'Slottsplassen 1')

then:

  assert log.find {String msg -> msg.contains "Replacing address homeless with Slottsplassen 1 on Finn"}

The code turned out simpler than I feared when I initially decided I need to test my logging. Joy!

When I say Jump, Jenkins asks how high

... if you want even faster feedback than waiting for Jenkins to poll your stuff

- In project config within Jenkins. Check: "Trigger builds remotely", add a token, like "foobar" into the textbox.

- SSH to your SVN-server and find the project home within svnroot.


- Here is my post-commit hook:
finn@subversionserver:/var/svnroot/myproject/hooks$ cat post-commit

#!/bin/sh
/usr/bin/wget -O /dev/null --auth-no-challenge --http-user=finn --http-password=JENKINS_PASSWORD http://jenkinsserver:8080/view/Finn/job/myproject/build?token=foobar

If you don't have authentication on Jenkins, remove --http-user and --http-password

uhm, there is a ~/.ssh/config file?

How did I miss this for all these years I've used ssh?

~/.ssh/config lets you type 

#ssh home
instead of
#finn@84.215.xxx.xxx -p 10001

Here is the ~/.ssh/config
Host home
HostName 84.215.xxx.xxx
Port 10001
User finn

Combine this passwordless ssh with
ssh-copy-id -i ~/.ssh/id_rsa.pub finn@84.215.xxx.xxx
and phoning home is pretty effortless.

... another one of those note-to-future-Finn (tm) posts which I'll copy and paste stuff from.

HOWTO: Grails / GORM + Oracle Schema

By simply adding schema to your domain classes, you won't be able to get it working in h2 afterwards. Here is how you fix that.

1. Update all your domain classes

grails-app/domain/Person.groovy

class Person {

        String name

static mapping = {

table name: 'PERSON', schema:"MYSCHEMA"

}

}

2. Update all your datasource

grails-app/conf/DataSource.groovy:

environments {

    development {

        dataSource {

            dbCreate = "create-drop"

            url = "jdbc:h2:mem:devDb;MVCC=TRUE;LOCK_TIMEOUT=10000;INIT=CREATE SCHEMA IF NOT EXISTS MYSCHEMA"

        }

repeat that for the test environment also.

Putting this in writing because I wish I had this blog post a while ago.

Cheat sheet: ssh + screen + irssi

Cheat Sheet: SSH + IRSSI + SCREEN 

1. ssh user@myserver

2. screen -r  resumes your screen. screen starts a new one, then type irssi

3. /server irc.freenode.org to connect to the server. /join #channelName to join a channel

4. /wc leaves a channel.

5. <ctrl+a d> detaches screen, type exit to leave your ssh session

JavaScript Continous Testing Smackdown

I'm pretty new to serious JavaScript coding. By serious, I mean writing lots of pure JavaScript, as a contrary to putting some JavaScript functions into your <script> block to fizz things up.

I immediately sought out a way to get a smooth TDD development cycle running. I already had a theoretical idea on how to do this - from some of my colleagues who are JavaScript elitists in my book.

As far as I'm aware of, the technique described in this post is THE way to do test driven JavaScript development these days.

So, what will I get from this stuff?

You'll get a terminal which automatically runs your JavaScript tests immediately as you change code. Like Spork in Ruby On Rails, or Infinitest in Eclipse.

Consider the alternative. Having to refresh your browser on every change of JavaScript code. Sound familiar? It's no good; for your quality of life or your aging process (gray hair). Little less your productivity.

Under the hood, these guys passes your JavaScript code to "browser slaves" and shoves the test-results back into your terminal window. I've tried both Buster.JS and JsTestDriver. So, which is the king of TDD JavaScript (in my opinion ofcourse)?

Buster.JS

jsTestDriver using watchr

All the code from this blog entry can be found on github. Under a busterjs and jstd branch. https://github.com/finnjohnsen/jstesting

"But hey..? There are other ways to test JavaScript code"

I've noticed, and none I've found are attractive at all.

• You can refresh your browser and see if your stuff works. You'll have to do this anyway in addition to your unit testing. But you certainly don't want this to be the only way to test your code. It often involved clicking and typing to reach the code you know you want to test. Horrible and unfortunately probably the most common way to do JavaScript development in the world today (I made that up, it's just my guts talking).
• I've seen set-ups which involves refreshing or triggering the tests from your browser. For example at the Jasmine -site, this is what they document. This is a lot better than the previous bullet point. But we can do better.
• The Selenium approach: Automate and play back clicks and input-text into your browser. These kind of behavior tests are tightly coupled to html, slow and messy. But they have value as they ensure your site actually works! But it's not what I want for unit testing.
• Pure Node.js testing. I've tossed this idea, admittedly without really digging into it. Because browsers are too important. Different browsers have different JavaScript implementations, so isolating my tests to the Node.js v8 runtime feels fundamentally wrong.

If you know of other ways, please comment.

Buster.JS or JsTestDriver: My Conclusion

Both frameworks gives you a speedy feedback when working test driven development in JavaScript. However, either will help you all the way - as you may have wished.

• jsTestDriver requires more patching, hacking and scripting, but works well once you've got it running. You'll probably polish and tune your scripts for quite some time after you've got it running, but you have something working pretty easily. JSTD has wide adaptation and you'll almost certainly find a solution to your problems just by searching around a little. You can have a look at this wrapper, which is suppose to ease the scripting from JsTestDriver. Anyway, check out my sample set up.

• Buster.JS is a bigger creature. It seems to aim to provide you with some of the pieces missing in JsTestDriver. It provides contrinous/auto testing and an built-in assertion library. These two critical components are what made the installation and set-up of jsTestDriver take a while longer than buster. So check out my sample set up.

I spent about 45 minutes setting up this JSTD project, and 15 minutes on Buster.js. However I've done this before. I suspect I spent at least 2 or 3 times as much the first time.

Buster.JS states on it's website it is under heavy development. It even and uses the word 'unstable' to describe its current state. Leaving JSTD the choice if you can't live with that. Personally I like getting as much set up for me as possible, and I found nothing mentionable when setting up Buster.JS. So I will use Buster.JS on my next green field. I can however be considered biased, one of the authors of Buster.JS work in the same company as I.