Prevent spontaneous cache creation

UPDATE (2017-02-15): Spring is correctly handling the null from getMissingCache. So you won’t have to overload it. I was misled by a framework that was incorrectly layering the JCacheCacheManager. I stand corrected and the article as well. Thanks to Stéphane Nicoll for his vigilance.

UPDATE (2017-02-16): Spring has a JCacheCacheConfiguration that configure a JCache CacheManager using a list of cache names and a default cache configuration. By default, it uses new MutableConfiguration<>() so the article was updated accordingly.

UPDATE (2017-04-07): My Hibernate PR has been accepted. So you now have an elegant solution with the latest Hibernate versions. The article was updated accordingly.


I played a lot with JCache connectors lately. To plug Ehcache 3 to different things.

I noticed one really dangerous thing. Frameworks tend to spontaneously create caches that were not explicitly defined. I think it is coming from the JCache spirit that there should be a default cache configuration. It is nonetheless a bad idea.

Let’s look at two examples

Spring

Caching in Spring is implemented by spring-cache. To plug JCache to spring-cache you use the JCacheCacheManager. By default when a cache isn’t available in the CacheManager, Spring calls JCacheCacheManager.getMissingCache. So far so good.

The default implementation for this method returns null when a cache doesn’t exist. This null will then be handled at higher levels to throw a nice exception.

If you want to explicitly support spontaneous cache creation, getMissingCache is where you should put your creation code.

However, watch out if you do that. You might lost track of all the existing caches. And please, never do the following.

@Override
protected Cache getMissingCache(String name) {
    Cache cache = super.getMissingCache(name);
    if(cache == null) {
        return new JCacheCache(cacheManager.createCache(name, new MutableConfiguration<>()));
    }
    return cache;
}

It returns a cache configured using default. It is never what you want.

Then, as usual, Spring tries to be nice with us. So if you enable caching (@EnableCaching), that the JSR-107 API is in
the classpath and that you do not expose any CacheManager, Spring will create one for you.

The JCacheCacheConfiguration will get a default JSR-107 CacheManager and add a list of caches taken from the cache property spring.cache.cache-names. These caches are by default created using a new MutableConfiguration<>(). As we said above, this is not a correctly configured cache.

The solution is to expose the wanted cache configuration in a bean.

@Bean
public javax.cache.configuration.Configuration<Object, Object> cacheConfiguration() {
    CacheConfiguration<Object, Object> cacheConfiguration = CacheConfigurationBuilder
        .newCacheConfigurationBuilder(Object.class, Object.class, ResourcePoolsBuilder
            .newResourcePoolsBuilder().heap(100))
        .build();
    javax.cache.configuration.Configuration<Object, Object> configuration =
        Eh107Configuration.fromEhcacheCacheConfiguration(cacheConfiguration);
    return configuration;
}

This bean will be magically used as cache default. You should always do this and never let ``new MutableConfiguration<>()` be used.

Hibernate

To use JCache with Hibernate we need to use the JCacheRegionFactory. The problem with JCacheRegionFactory is that by default if a cache is not found, it will spontaneously create a cache by passing new MutableConfiguration(). This means that instead of using a properly configured cache, you end up with some random default configuration (infinite on heap for Ehcache).

This is really bad because it is pretty hard to detect.

What I recommend in this case is, again, to override the default. In the latest Hibernate versions (5.2.8+) (thanks to https://github.com/hibernate/hibernate-orm/pull/1783[HHH-1783], we can do the following

@Override
protected Cache<Object, Object> createCache(String regionName, Properties properties, CacheDataDescription metadata) {
    throw new IllegalArgumentException("Unknown hibernate cache: " + regionName);
}

In older versions, sadly, there is no straightforward cache creation method to override. The best we have is newDefaultConfig which provides the default configuration. One sad thing is that you don’t have the actual cache name here. You will need to debug to know it.

@Override
protected Configuration<Object, Object> newDefaultConfig(Properties properties, CacheDataDescription metadata) {
    throw new IllegalArgumentException("Unknown hibernate cache: " + metadata);
}

Again, an alternative solution would be to provide a meaningful default cache configuration in this method.

Conclusion

I do understand that frameworks do not like to fail with exceptions. This helps the feeling that they are working out of the box.

But I still think silently not caching or providing random default configuration is dangerous. Using my two workarounds should prevent you a lot of headaches.

Future behaviour through time

Yesterday, I was playing with interruption. At some point I ended up with this code.

public <T> T uninterruptibleGet(Future<T> future) throws ExecutionException {
  while(true) {
    try {
      return future.get();
    } catch (InterruptedException e) {
      Thread.currentThread().interrupt();
    }
  }
}

This is bad code. Don’t do that. It should be an infinite loop. For those not playing with interruption daily here’s why:

  1. Waiting on get
  2. Be interrupted
  3. Catch the exception and set the interruption state back (Thread.currentThread().interrupt())
  4. Go back to get
  5. Notice we are interrupted (the interruption state is on)
  6. Throw an InterruptedException
  7. Go to 4

The thing is that when running the code, there was no infinite loop.

So I wanted to know why.

The answer is in FutureTask.get().

public V get() throws InterruptedException, ExecutionException {
    int s = state;
    if (s <= COMPLETING) // here
        s = awaitDone(false, 0L);
    return report(s);
}

The get method first look at the state, if done, it just returns. If not, then it interrupts. Here we go, job done.

Then it goes to the Continuous Integration. And goes in an infinite loop. Whaaat???

At first I thought it was some race condition that was happening with other tests (they are run in parallel).

But no. The answer is much simpler than that: JDK 6

The FutureTask code above is from JDK 8. The JDK 6 code is different. It first starts to check the interrupt. So infinite loop it is.

Now, was is the right idiom for an uninterruptibleGet?

To get the long answer, I highly suggest that you go read Java Concurrency in Practice (chapter 7) right now. It is a must read for any Java programmer.

The short answer is this:

public <T> T uninterruptibleGet(Future<T> future) throws ExecutionException {
  boolean interrupted = false;
  try {
    while (true) {
      try {
        return future.get();
      } catch (InterruptedException e) {
        interrupted = true;
      }
    }
  } finally {
    if (interrupted)
      Thread.currentThread().interrupt();
  }
}

EasyMock generic typing

These days, I’m a bit annoyed about Java generics. Because it seems that when you want to be clean, you pretty much always get in troubles.

I’ll give you an example. The method to create a mock with EasyMock is currently typed like this:

public static <T> T mock(Class<T> toMock)

This seems quite straightforward.

But it is also quite annoying to use. For example, let’s try to mock a generic type:

List<String> list = mock(List.class);

It looks like something we would like to easily do but you get this nice warning:

Warning:(51, 33) java: unchecked conversion
  required: java.util.List<java.lang.String>
  found:    java.util.List

And there’s no way to get around it. Here are some attempts:

List<String> list = mock(List<String>.class); // no allowed (won't compile)
List<String> list = EasyMock.<String>mock(List.class); // won't compile either because the parameter type doesn't match)
List<String> list = (List<String>) mock(List.class); // still get a warning

So what’s the way out?

To be less accurate.

Yes. I can’t see any other way out. (Do you?)

So my plan is to change EasyMock typing with this:

public static <T> T mock(Class<?> toMock)

No relation anymore between the parameter and the returned type. WAT!!! Are you crazy?!?

So yes, this will compile without complaints :-( (but I can check the type coherence at runtime)

String list = mock(Integer.class);

But this will now work without any warning :-)

List<String> list = mock(List.class);

You can’t have your cake and eat it it seems. But as usual, I will be happy to be proved wrong.

Gatling Java DSL

My favorite stress tool has been Gatling for many years now.

Scenarios are

  • Easy to construct, It’s just code
  • Easy to version. It’s just code
  • Easy to run. It’s a simple command line that you launch in remote if needed
  • Having great result graphs
  • Easy to datamine. There is a simulation.log file that let’s you do some more digging when needed

My only complaints are

  • You can’t change the running scenario
  • It’s in Scala

The first one might be addressed by Gatling Frontline. I haven’t tried.

The second one is a bit more complicated to solve :-)

In fact, I don’t care that’s in Scala. But I would love to be able to core my scenario in Java.

So just for fun, I did a thought experiment. I’ve coded the Gatling API in Java and then coded the BasicScenario example using this API.

It’s not working for real

The API is just an emply shell. To know how different it would be from the Scala DSL.

The answer is: Not much.

My expectations were that I would need to rely on plenty of Java 8 features to get something close to the Scala DSl. But in fact, not even.

The main differences are:

  • Variables are typed
  • I’ve added a build method. But it’s just for style. I can get rid of it
  • A bunch of semi-colons
  • No syntactic sugar to create a map
  • I’m using Duration is used to time units
  • I need a run method where the setUp method is

That’s quite joyful and probably can be improved.

I’m now wondering if I could have used the Scala classes directly. I’m not good at making Java and Scala interacting.

So if someone is and want to try, please do so.

Manage emails in git

If you are like me, you tend to work on open source using your personal email and at YP using your enterprise email. But git doesn’t handle that well so you need to never forget to do

git config user.email henri.tremblay@somewhere.com

after each git init or git clone.

I finally had the time for work on an almost satisfying solution.

EasyMock 3.4 is out!

This version has finally removed the old (and ugly) partial mocking methods. You are now required to use the partialMockBuilder.

In exchange, you get really nice short methods to create your mocks: mock, niceMock and strictMock.

You also get a better stability since cglib and ASM are now embedded to remove a possible version mismatch with your own dependencies. Note that Objenesis will stay as an explicit dependency.

Change log

Out of SourceForge, all in Bintray

Now that Objenesis is in Bintray. The goal was to also have EasyMock binaries at the same place. They were hosted on SourceForge. And as you might have read, SourceForge is now on the dark side so going out of there was becoming a priority.

This time it was easier. I basically made a list of versions and release dates.

And then I made a loop around the lines to the

  • wget to get the binary and store it keeping the SourceForge structure
  • curl to create the version in Bintray
  • curl to upload the binary
I've manually added the release notes in GitHub but wasn't able to figure out how to push them to Bintray. If someone knows, please tell me.

To be helpful, here is the loop that I used to create the versions.

API_KEY=xxx
PASSPHRASE=xxx

# This is because I want the file upside down to create the older versions first
tail -r $1 > tmp.tsv

while read p; do
version=$(echo "$p" | cut -d' ' -f1)
date=$(echo "$p" | cut -d' ' -f2)

version=${version// /-}

echo $version $date

content="{ \"name\": \"$version\", \"desc\": \"$version\", \"released\": \"${date}T00:00:00.000Z\", \"github_use_tag_release_notes\": false, \"vcs_tag\": \"$version\" }"
echo "$content"

curl -v -XPOST -H "Content-Type: application/json" -H "X-GPG-PASSPHRASE: ${PASSPHRASE}" -uhenri-tremblay:${API_KEY} \
-d "$content" \
https://api.bintray.com/packages/easymock/distributions/easymock/versions

done <tmp.tsv

Migrating EasyMock Jira to GitHub

Following my previous article, I will now describe the Jira migration of EasyMock to GitHub issue tracker.

EasyMock Jira was on Codehaus. So I had to migrate it somewhere else. Instead of migrating to another Jira instance I’ve decided that EasyMock doesn’t need all that complexity anymore and that going to Github would enough.

The migration was made in two steps:

  1. Save everything on my disk
  2. Import everything in GitHub

Virtual goods are still mine: Open letter to The Economist

I'm inaugurating a new section that will be more life oriented i.e not talking about software engineering. It might be in English or French depending on my mood and the public expected to read the entry. So this first blog post is more or less an open letter to The Economist.

When I'm not doing software engineering, I'm highly interested in finance, economy and politics. Until recently, I was even a diligent subscriber of The Economist. I've decided two months ago to not renew my subscription. Not because I don't like the magazine anymore but because it was too time consuming to read. I want this year to have the time to read books.

I was receiving the paper version which allows me to have access to the online version. Of course, I'm not keeping the paper versions in my basement since I was having access to them online. I was really surprised to realize that as soon as my subscription expired, I've lost the access to the entire The Economist website content. Event the editions THAT I PAID FOR!

I found that to be quite a cheap decision from a journal as renowned as The Economist.

But I also think it shouldn't be legal. It's been one of my thoughts for a while now.

Virtual properties should have the same "rights" as physical ones. When I buy a physical book, I can sale it, lend it, give it.

If an buy a e-book, I can't do all this. I'm locked in whatever rules the seller wants to apply. And I don't feel good about it. I'm pretty sure I'm not the only one and I'm also pretty sure it creates adoption issues.

Of course you can buy virtual gears. Sellers love to sell you stuff. Virtual or not. But when it's virtual, they currently make sure you will never really be the owner of it.

Yes, I do understand the concept of service. For instance, The Economist offers the website as a service to their customer. They pay the hosting, the development so it makes sense that if I stop paying, the service stops. But it doesn't make sense that I can't download the PDF of all the issues I bought. Of course it means that I would then be allowed to illegally diffuse them. But I can allow photocopy my paper issue. Or scan it.

Meanwhile, I'm left with a bitter taste in my mouth. Until these's a law protecting my virtual properties, I'm not really sure I will trust The Economist anymore.

Finish migrating Objenesis to GitHub

With Codehaus and Google Code closing, I’m happily required to migrate Objenesis and EasyMock.

Both projects have an hybrid hosting using different platforms. I’ve decided to start with Objenesis which is easier to migrate. I’ll describe the process in this post. Hopefully, it will be helpful to someone but, in fact, I’m also looking forward to your feedback to see if I can improve my final setup.

So, Objenesis source code was originally on Google Code. I’ve moved it to GitHub some years ago mainly because pull requests are awesome. The website is also a GitHub page. Part of the documentation is on Google Code wiki, the binaries are on Google Code as well as the issue tracker.

Google being nice folks, they used to provide a nice way (https://code.google.com/export-to-github, now down) to migrate everything to GitHub. But I couldn’t use that because

  • I don’t want the project to end up in henri-tremblay/objenesis. I want it where it is now, in the EasyMock organisation
  • I only want to migrate the wiki and the issues since the sources are already there

So here’s what I did instead.

Issue tracker

The issue tracker was migrated using the IssueExporterTool. It worked perfectly (but is a bit slow as advertised).

Wiki pages

At first, I tried to export the entire Objenesis project to Github to be able to retrieve the wiki pages and then move them to the real source code. The result was quite bad because the tables are not rendered correctly. So I ended up manually migrating each page to markdown. There was only 3 pages so it wasn’t too bad.

Binaries

This was more complicated. Maven binaries are already deployed through the Sonatype Nexus. But I needed to migrate the standalone bundles. I’ve looked into three options:

  • GitHub releases
  • Bintray
  • Both (GitHub releases exported to Bintray)

GitHub releases are created automatically when you tag in git. But you also seem to be able to add some release notes and binaries over that. I didn’t want to dig too much into it. I knew I wanted to be in Bintray in the end. And I wanted easy automation. Bintray was easy to use so I went for Bintray only. Be aware, the way I did the migration is low-tech (but works).

  1. Make a list of all the binaries to migrate
    • Get them with wget https://objenesis.googlecode.com/files/objenesis-xxx.zip
  2. Create an organisation, a distribution repository and an [objenesis package])(https://bintray.com/easymock/distributions/objenesis) in Bintray
  3. Add my GPG key to my Bintray account
  4. Upload everything using REST (curl -T objenesis-xxx.zip -H "X-GPG-PASSPHRASE: ${PASSPHRASE}" -uhenri-tremblay:${API_KEY} https://api.bintray.com/content/easymock/distributions/objenesis/xxx/objenesis-xxx-bin.zip?publish=1)

It works, the only drawback is that no release notes are provided. They’ve always been on the website

Project moved

Finally, I’ve set the “Project Moved” flag to target GitHub. This quite aggressively redirects you to GitHub if you try to access https://code.google.com/p/objenesis.