21 November 2013

Digging into EHCache (and TTL)

Using a caching frameworks (like Grails Cache and Grails EhCache) instead of writing the caching mechanism yourself is comfortable. You find yourself smacking @Cacheable on listing operations and get methods - and CacheEvict on the CRUD operations. Right?

But this is extremely distant from the actual implementation - so I think it's a good idea to have a close look and confirm it works as you think.

So let's investigate, using 2 techniques achieving 3 goals.

Goals are:
1. Confirming Configuration run-time
2. Watching the Cache run-time
3. Confirming TTL timers in run-time
4. Unit testing

Techniques described:
1. JMX
2. Grails Console Plugin

First you need to verify that EhCache is JMX exposed. I have this in Bootstrap.groovy.


Then, fire up jConsole

Confirming configuration

EhCache works "great" without any configuration at all - and then runs purely on defaults - which can trick you to believe your system works as you configured.

So let's see that the configuration you intended are correct. We do this by browsing in jConsole to "net.sf.ehcache" -> "CacheConfiguration" and click the cache name.





I've now verified my cache TTL has the exact value 1800 seconds (aka 30 minutes), matches my config file. Meaning my config file works.

Watching the Cache run-time

Head over to CacheStatistics in jConsole, and watch for CacheHits and CacheMiss


CacheHit means the cache did its job and returned cached results instead of invoking the annotated method. CacheMiss means the method was invoked. A high number on CacheHits is probably a sign of a healthy set up.

Confirming TTL timers
This may seem uneccesary, but I wanted to look at how this worked, so I dug into it. I couldn't find the TTL timers exposed in JMX, so I dug into the runtime using Grails Console Plugin.

So grab the CacheManager, find your cache, and ask for ExpirationTime and Creation Time.



















This code on Gist

As you see, there is a perfect 30 minutes between Expiration and Creation time. Perfect.

Now sit back and wait until the ExpirationTime passes - and watch a new CacheMiss increasse in jConsole! If it does, everything is working as you intented.


Unit testing
This kind of configurations doesn't feel like a good fit for unit testing in my book. But maybe I'm wrong.

Cheers
.finn



29 October 2013

Easier Killing

Who are you?


"I take the long road" -guy (this has been me for years and years)


  1. ps -ax | grep <something describing my process>
  2. look for the <pid>
  3. type kill -s 9 <pid>


"I made a shortcut" -guy

Make a clever script which uses ps | grep output, cut, awk or whatever to get the pid for the kill.

"I use the existing shortcut" -guy

pkill -9 -f  <something describing my process>

or pgrep first if I'm insecure about the regex

16 October 2013

Mocking grailsApplication


You don't have to mock grailsApplication, unless you've put yourself  in the same boat as me and like writing pure jUnit. Keep running grails test-app all the time is just annoying and slow. Note I havent tested Grails 2.3 yet, which is supposed to be faster than the predecessors.

Anyway. This "pure" approach has one downside: your service/controller aren't getting its dependencies injected for free. You have to do everything yourself.

Injecting services is easy, as you just @grails.test.mixin.Mock them in during setUp(). But there is that def grailsApplication sitting at the top of your file. How to best mock it so you don't get NullPointerException in your test? I (mostly) use grailsApplication for configuration, so there lies my need for mocking.

There are several ways of getting this done and I have two ways which I use. I can't decide on a favorite. Here they are, I'll let you decide what you like:

nested maps

@Before
public void setUp() {
  service = MyService() //service is a global variable for the entire test.
  service.grailsApplication = [config:[myApp:[jms:[firstQueue:"firstQueue"]]]]

This works best if you have really few properties. Maybe just one. But it's a one-liner, which I like.

ConfigObject

@Before
public void setUp() {
  service = MyService()
  def mockConfig = new ConfigObject()
  mockConfig.myApp.jms.firstQueue="firstQueue"
  service.grailsApplication = [config: mockConfig]

I know deep down that ConfigObject is the proper way of doing this - but it's more stuff to write.

Which do you like best?
/Finn


14 October 2013

Groovy / Grails / GORM - File Name Too Long

Yet another "File Name Too Long" workaround :-(

Normal day - was just gonna add one more check to my nice .where{} closure. But when I did, everything blew up in my face - and I'm sitting here with no eyebrows.

The file will no longer compiles!

#grails test-app
/home/finn/src/stratos/stratos/target/classes/no/nsb/stratos/MyService$_updateAllWithChanges_closure5_closure7_closure8_closure9_closure10_closure12_closure14_closure16.class (File name too long)

When I commented back in the validTo everything went melt-down.

List gsmList = DistributionText.where {
  status.enabled==true &&
  status.ok==false &&
  status.validFrom <= new Date() && 

//status.validTo >= new Date() &&
  status.station != null &&
  channelCode=="FOO" &&
  language == "eng"
}.list()


I can't figure out why this happens! The filename isn't over the Linux file length limit. 


I've seen this before when JSON marshalling really deep objects. (the workaroud then, was to create a map from intended object, and serialize the map instead)

So, what to do when you really don't understand the error in front of you? Escape! So I that's what I did - I rewrote to rewrote to Hibernate Criterion.

List<DistributionText> gsmList = DistributionText.createCriteria().list {
status {
  eq("enabled", true)
  eq("ok", false)
  le("validFrom", new Date())
  ge("validTo", new Date())
  isNotNull("station") 
}
eq("channelCode", "FOO")
eq("language", "eng")



I kinda like better the Groovy DSL/where to this Hibernate syntax, so if you find out how to get around the File Name Too Long, please leave a comment.

16 April 2013

Unit testing potential thread leakage

So you've bewildered yourself into the dark world of pain threads, and you wanna make sure your app doesn't leak threads.

I found I had to calm my nerves by having tonnes of these Spock tests covering my Thread spawning.

I took my actual code, pasted it into this blog, stripped and rewrote it to cut straight to the point.

This silly service only illustrates the two methods for starting and stopping threads. This sample only spawns 1 thread but the point here are tests below.


To be sure your startJobs actually creates your thread(s), you should have a test confirming that first.


And here is the test looking for leakage:


The whole idea is finding the threads by your given thread name. And yes, i know Thread.sleep isn't ideal, and I may not need it, but it just feels reasonable to make sure the thread gets a little time to get going.

Side note: I learned that you have to cancel both the Timer and the TimerTask. If you only cancel the Timer, you'll have to wait for the GC to clean up the thread, and your test will fail.

26 March 2013

grails run-app and tomcat jmx

Want the standard Tomcat JMX exposed during development (grails run-app) ?

Add this to your Config.groovy:
grails.tomcat.jvmArgs=["-Dcom.sun.management.jmxremote", "-Dcom.sun.management.jmxremote.port=8099", "-Dcom.sun.management.jmxremote.ssl=false", "-Dcom.sun.management.jmxremote.authenticate=false"]

Fire up jconsole and hit localhost:8099.

</note to future self blog entry>

25 March 2013

Unit test your logging

I have a few log statements which are really important for me in production. Without them, I would be screwed when tracking whats actually going on. So I decided that this important code shouldn't be left out in the cold.

Here is a trivial example. Lets say the "Replacing address" log statement in the following code is really important when stuff hits the fan.



void setAddress(Address adr) {
  log.info "Replacing address ${this.address} with ${adr} on ${this}"
  this.address = adr
  log.debug "Updated{person} with new address"
}



setup:
def log = []
def collectLog = {String message -> log << message}
def finn = new Person(name:'finn', new Address(streetname:'homeless'))
finn.metaClass.log = [debug:collectLog, info:collectLog, error: collectLog]
when:
finn.address = new Address(streetname:'Slottsplassen 1')
then:
assert log.find {String msg -> msg.contains "Replacing address homeless with Slottsplassen 1 on Finn"}


The code turned out simpler than I feared when I initially decided I need to test my logging. Joy!

25 February 2013

When I say Jump, Jenkins asks how high

... if you want even faster feedback than waiting for Jenkins to poll your stuff


- In project config within Jenkins. Check: "Trigger builds remotely", add a token, like "foobar" into the textbox.

- SSH to your SVN-server and find the project home within svnroot.


- Here is my post-commit hook:
finn@subversionserver:/var/svnroot/myproject/hooks$ cat post-commit
#!/bin/sh
/usr/bin/wget -O /dev/null --auth-no-challenge --http-user=finn --http-password=JENKINS_PASSWORD http://jenkinsserver:8080/view/Finn/job/myproject/build?token=foobar

If you don't have authentication on Jenkins, remove --http-user and --http-password

12 February 2013

uhm, there is a ~/.ssh/config file?

How did I miss this for all these years I've used ssh?


~/.ssh/config lets you type 
#ssh home
instead of
#finn@84.215.xxx.xxx -p 10001

Here is the ~/.ssh/config
Host home
HostName 84.215.xxx.xxx
Port 10001
User finn


Combine this passwordless ssh with
ssh-copy-id -i ~/.ssh/id_rsa.pub finn@84.215.xxx.xxx
and phoning home is pretty effortless.

... another one of those note-to-future-Finn (tm) posts which I'll copy and paste stuff from.

29 January 2013

HOWTO: Grails / GORM + Oracle Schema

By simply adding schema to your domain classes, you won't be able to get it working in h2 afterwards. Here is how you fix that.

1. Update all your domain classes

grails-app/domain/Person.groovy
class Person {
        String name
static mapping = {
table name: 'PERSON', schema:"MYSCHEMA"
}
}

2. Update all your datasource

grails-app/conf/DataSource.groovy:
environments {
    development {
        dataSource {
            dbCreate = "create-drop"
            url = "jdbc:h2:mem:devDb;MVCC=TRUE;LOCK_TIMEOUT=10000;INIT=CREATE SCHEMA IF NOT EXISTS MYSCHEMA"
        }


repeat that for the test environment also.

Putting this in writing because I wish I had this blog post a while ago.

18 January 2013

Cheat sheet: ssh + screen + irssi

Cheat Sheet: SSH + IRSSI + SCREEN 

1. ssh user@myserver
2. screen -r  resumes your screen. screen starts a new one, then type irssi
3. /server irc.freenode.org to connect to the server. /join #channelName to join a channel
4. /wc leaves a channel.
5. <ctrl+a d> detaches screen, type exit to leave your ssh session

28 November 2012

JavaScript Continous Testing Smackdown

I'm pretty new to serious JavaScript coding. By serious, I mean writing lots of pure JavaScript, as a contrary to putting some JavaScript functions into your <script> block to fizz things up.

I immediately sought out a way to get a smooth TDD development cycle running. I already had a theoretical idea on how to do this - from some of my colleagues who are JavaScript elitists in my book.

As far as I'm aware of, the technique described in this post is THE way to do test driven JavaScript development these days.

So, what will I get from this stuff?

You'll get a terminal which automatically runs your JavaScript tests immediately as you change code. Like Spork in Ruby On Rails, or Infinitest in Eclipse.

Consider the alternative. Having to refresh your browser on every change of JavaScript code. Sound familiar? It's no good; for your quality of life or your aging process (gray hair). Little less your productivity.

Under the hood, these guys passes your JavaScript code to "browser slaves" and shoves the test-results back into your terminal window. I've tried both Buster.JS and JsTestDriver. So, which is the king of TDD JavaScript (in my opinion ofcourse)?


Buster.JS
jsTestDriver using watchr































All the code from this blog entry can be found on github. Under a busterjs and jstd branch. https://github.com/finnjohnsen/jstesting


"But hey..? There are other ways to test JavaScript code"

I've noticed, and none I've found are attractive at all.

  • You can refresh your browser and see if your stuff works. You'll have to do this anyway in addition to your unit testing. But you certainly don't want this to be the only way to test your code. It often involved clicking and typing to reach the code you know you want to test. Horrible and unfortunately probably the most common way to do JavaScript development in the world today (I made that up, it's just my guts talking).
  • I've seen set-ups which involves refreshing or triggering the tests from your browser. For example at the Jasmine -site, this is what they document. This is a lot better than the previous bullet point. But we can do better.
  • The Selenium approach: Automate and play back clicks and input-text into your browser. These kind of behavior tests are tightly coupled to html, slow and messy. But they have value as they ensure your site actually works! But it's not what I want for unit testing.
  • Pure Node.js testing. I've tossed this idea, admittedly without really digging into it. Because browsers are too important. Different browsers have different JavaScript implementations, so isolating my tests to the Node.js v8 runtime feels fundamentally wrong.
If you know of other ways, please comment.



Buster.JS or JsTestDriver: My Conclusion


Both frameworks gives you a speedy feedback when working test driven development in JavaScript. However, either will help you all the way - as you may have wished.

  • jsTestDriver requires more patching, hacking and scripting, but works well once you've got it running. You'll probably polish and tune your scripts for quite some time after you've got it running, but you have something working pretty easily. JSTD has wide adaptation and you'll almost certainly find a solution to your problems just by searching around a little. You can have a look at this wrapper, which is suppose to ease the scripting from JsTestDriver. Anyway, check out my sample set up.
  • Buster.JS is a bigger creature. It seems to aim to provide you with some of the pieces missing in JsTestDriver. It provides contrinous/auto testing and an built-in assertion library. These two critical components are what made the installation and set-up of jsTestDriver take a while longer than buster. So check out my sample set up.
I spent about 45 minutes setting up this JSTD project, and 15 minutes on Buster.js. However I've done this before. I suspect I spent at least 2 or 3 times as much the first time.



Buster.JS states on it's website it is under heavy development. It even and uses the word 'unstable' to describe its current state. Leaving JSTD the choice if you can't live with that. Personally I like getting as much set up for me as possible, and I found nothing mentionable when setting up Buster.JS. So I will use Buster.JS on my next green field. I can however be considered biased, one of the authors of Buster.JS work in the same company as I.



16 April 2011

Android Sources in Eclipse

I've blogged about getting Eclipse to show Android sources before, but these techniques no longer work. So I'll toss a working how-to out there again.


I'm still baffled that Google hasn't ensured this working from day #1 with this eclipse plugin. But enough crying :)


I should also mention there is an eclipse plugin made for this. I've never tried it. I don't like bloating eclipse with tons of plugins, unless I really have to. And this boils down to 8 pretty easy steps, so an eclipse plugin isn't something Id'e consider. (But I may change my mind later)

Download the sources from github, and attach them:
  • Click the Snapshot link which just appeared by the Tree -link

You'll start downloading a file which is about 130mb. This is the android source file. It's filename starts with base, for me it's called base-08d9d9a.tar.gz
  • move the base -file to the SDK platform directory, for me that's /opt/android-sdk-linux_x86/platforms/android-8
  • unpack it (tar xvfz base-08d9d9a.tar.gz).
  • head into eclipse, and press F3 over some android source you're missing, like Activity. And get this pesky window, which we all hate:

  • Click Attach Source, choose External Folder, and choose the core/java in the base folder. For me that's /opt/android-sdk-linux_x86/platforms/android-8/base-08d9d9a/core/java
There you go. Please post a comment and tell me how this turns out for you :)


.finn


(*1) There are loads of versions there, and it's a bit confusing. I've concluded the stuff we're looking for should end with SDK. However I'm not 100% certain about what version to download to match the Android version we see in the Android SDK download tool - this is just me guessing.

14 April 2011

Getting that Skeleton test up

When creating a new unit test file, getting the import statics right with Hamcrest is 30 seconds of hassle, every time.

Here is the typical unit test start-up I prefer:
@Test
public void skeletonTest() {
assertThat(true, is(equalTo(true)));
}
Here is a eclipse template, which may solve this. But may also not, as I tend to forget little tricks like these. Time will tell if I've been able to incorporate this my reflexes.

So, get the template in, make the new junit test file, type skeltest and press ctrl+space.
























The template:
${imp:import(org.junit.Test)}
${imps:importStatic(
org.hamcrest.Matchers.equalTo,
org.hamcrest.Matchers.is,
org.junit.Assert.assertThat)}
@Test
public void skeletonTest() {
assertThat(true, is(equalTo(true)));
}

19 July 2010

Tired of killing browsers?

Launching the browser a lot from your IDE while working on / developing <some> web app?

Getting zillions of tabs or new browser instances?

Killing them all by hand every now and then?

Small trick which tells Firefox to put all URL requests in the same tab, reusing one tab.

in the URL type: about:config
find the property browser.link.open_newwindow
change the value to 1


Follow up tip: Keep some Firefox fork/clone (or whatever) installation which you use just for development, and apply this setting to it. This way you don't mess up your Firefox for normal surfing. I keep swiftweasel in /opt/swiftweasel