Friday, April 22, 2011

Checked Exceptions-2

So after the big rant over checked exceptions I've been bitten by the problems that come with the them, not for the first time but it seemed all the more acute since I just wrote about them. By modifying the signature of a method to convert ints to enum types I started a change that meant modifying forty files from the the depths of the application to it's highest spire, and overall creating three different exceptions to encapsulate the added exception and IOExceptions that could already occur. Take into account that some of these files were unit tests that would need to change to throw the exception I fit occurred but did not need ant more modification to pass.

The problem with specifying a checked exception here would be the same if Integer.valueOf(String) threw a checked version of NumberFormatException, try-catches and throws statements appeared everywhere. This case is pretty much the worst case for this change as the method is used in more places than I expected, this is an API meant for eventual consumption by people who I may never meet. Who may not have the code there to view the documentation. I left work tonight without committing that work into SVN to think whether this change was really worth all the upheaval it had caused.

To clarify, the method already threw an IllegalArgumentException that was duly documented. The reason I created a custom checked exception was because the unthinkable happened, it actually threw the runtime exception and although it was caught it didn't give me any comfort. Luckily the the method executing the logic throwing the exception handled and logged it. While adding the tries and throws I realised that it was only caught appropriately in about 20% of the cases that it was used. I think this is a good estimate at how many runtime exceptions are caught appropriately in most software, luckily things mostly go our way ;)

After some serious thought I realised that had the method threw the exception when it was created the same amount of error handling code would exist and I would be happy because now the software is more robust. The exception will never be thrown out of core application code i.e. libraries such as Swing with unforeseen results.

The lesson I learned from this isn't that checked exceptions are hard, it's that they are hard to change after the code is in use. This is in exactly the same way that changing method parameters is hard once the code is in use. A couple of things to look out for next time:

Next time the method is going to throw runtime exception perhaps a custom checked exception should be thrown. The runtime exception will most likely not be caught, should the software be allowed to crash if this happens?

Exception hierarchy is your friend. By throwing subclasses of MyCompanyException (which is common but pointless on its own) catchers can easily cope with and log this but many custom versions can be created for more fine grained strategies. Also this may already be caught so no modification required in those cases.

One more conclusion that surprised me is that I actually found some existing errors lurking in code, two stream.close() cases that weren't contained in a finally block. If an exception was thrown then they would not be closed. Not the worst problem in the world but it shows how when I was in error handling mode how I saw code differently.

Friday, April 15, 2011

Generated methods

Overview

The Java compiler generates extra methods which appear in stack traces. These can be confusing at first. What are they and why do they exist?

The access model in the JVM

The access model in the JVM has not changed significantly since version 1.0. However, Java 5.0 added features such as inner/nested classes and convariant return types which were not in the original design. So how are they supported?

The issue with nested classes

Nested classes can access private methods and fields of an outer class. However the JVM does not support this so the compiler generates methods as a workaround.

Private Methods


In this example, the inner class calls a private method init() of the outer class.

public class PrivateMethod { // line 4
    private void init() {
        throw new UnsupportedOperationException(); // line 6
    };

    class Inner {
        Inner() {
            init();
        }
    }

     public static void main(String... args) {
        PrivateMethod pm = new PrivateMethod();
        Inner inner = pm.new Inner();
    }
}

So what does the stack trace look like when we run this?

Exception in thread "main" java.lang.UnsupportedOperationException
 at com.google.code.java.core.javac.generated.PrivateMethod.init(PrivateMethod.java:6)
 at com.google.code.java.core.javac.generated.PrivateMethod.access$000(PrivateMethod.java:4)
 at com.google.code.java.core.javac.generated.PrivateMethod$Inner.(PrivateMethod.java:11)
 at com.google.code.java.core.javac.generated.PrivateMethod.main(PrivateMethod.java:17)

In the stack trace there is a method called access$000 on line 4 which is the first line of the class. Is this a real method? What do we see in the byte code?

$ javap -c -private -classpath . com.google.code.java.core.javac.generated.PrivateMethod

static void access$000(com.google.code.java.core.javac.generated.PrivateMethod);
  Code:
   0:   aload_0
   1:   invokespecial   #1; //Method init:()V
   4:   return

$ javap -c -private -classpath . com.google.code.java.core.javac.generated.PrivateMethod\$Inner
Compiled from "PrivateMethod.java"
public class com.google.code.java.core.javac.generated.PrivateMethod$Inner extends java.lang.Object{
final com.google.code.java.core.javac.generated.PrivateMethod this$0;

com.google.code.java.core.javac.generated.PrivateMethod$Inner(com.google.code.java.core.javac.generated.PrivateMethod);
  Code:
   0:   aload_0
   1:   aload_1
   2:   putfield        #1; //Field this$0:Lcom/google/code/java/core/javac/generated/PrivateMethod;
   5:   aload_0
   6:   invokespecial   #2; //Method java/lang/Object."":()V
   9:   aload_1
   10:  invokestatic    #3; //Method com/google/code/java/core/javac/generated/PrivateMethod.
                        access$000:(Lcom/google/code/java/core/javac/generated/PrivateMeth
   13:  return

}
The compiler has added a static method which is not private so the Inner class can access the private method of the outer class indirectly.

Private Fields

Private fields are accessed via a getter and setter, but the compiler can generate methods for assignment operations such as ++ and *=
public class PrivateField {
    private int num = 0;

    public class Inner {
        public void set(int n) {
            num = n;
        }

        public int get() {
            return num;
        }

        public void increment() {
            num++;
        }

        public void multiply(int n) {
            num *= n;
        }
    }
}
Compiles with four generated methods, one for each of get, set, ++ and *=
$ javap -c -private -classpath . com.google.code.java.core.javac.generated.PrivateField
setter
static int access$002(com.google.code.java.core.javac.generated.PrivateField, int);
  Code:
   0:   aload_0
   1:   iload_1
   2:   dup_x1
   3:   putfield        #1; //Field num:I
   6:   ireturn

getter
static int access$000(com.google.code.java.core.javac.generated.PrivateField);
  Code:
   0:   aload_0
   1:   getfield        #1; //Field num:I
   4:   ireturn

increment ++
static int access$008(com.google.code.java.core.javac.generated.PrivateField);
  Code:
   0:   aload_0
   1:   dup
   2:   getfield        #1; //Field num:I
   5:   dup_x1
   6:   iconst_1
   7:   iadd
   8:   putfield        #1; //Field num:I
   11:  ireturn

multiplier *=
static int access$028(com.google.code.java.core.javac.generated.PrivateField, int);
  Code:
   0:   aload_0
   1:   dup
   2:   getfield        #1; //Field num:I
   5:   iload_1
   6:   imul
   7:   dup_x1
   8:   putfield        #1; //Field num:I
   11:  ireturn

Covariant return types

Covariant return types is a feature which was added to Java 5.0 but isn't supported in the JVM. For this reason, the compiler generates a method which overrides all the super methods and their return types.
public class CovariantReturnTypeA {
    public Object method() { return 1L; }
}

public class CovariantReturnTypeB extends CovariantReturnTypeA {
    public Number method() { return 2.0; }
}

public class CovariantReturnTypeC extends CovariantReturnTypeB {
    public Integer method() { return 3; }
}

The last method() overrides the method which returns Number and the method which returns Object. To implement this, the compile generates a method with the same signature which calls the method which returns Integer
$ javap -c -private -classpath . com.google.code.java.core.javac.generated.CovariantReturnTypeC

public java.lang.Integer method();
  Code:
   0:   iconst_3
   1:   invokestatic    #2; //Method java/lang/Integer.valueOf:(I)Ljava/lang/Integer;
   4:   areturn

public java.lang.Number method();
  Code:
   0:   aload_0
   1:   invokevirtual   #3; //Method method:()Ljava/lang/Integer;
   4:   areturn

public java.lang.Object method();
  Code:
   0:   aload_0
   1:   invokevirtual   #3; //Method method:()Ljava/lang/Integer;
   4:   areturn
There are three implementations of method() in the byte code.

Performance implications

The performance implications for Java Standard Edition are small as these methods can be inlined and effectively have no impact. In Java Mobile Edition, this might not be the case.

They can cause some confusion and if this is a problem you can make the fields/methods package-local instead and the generated methods will go away.

Thursday, April 14, 2011

Checked Exceptions-1

I've always felt that exceptions were never used to their full potential in my code. It is so common in software to do a try-catch and log the exception that perhaps there should be a keyword built into the language for this :)

Every time the system logs or returns null because something couldn't happen then maybe a checked exception should be thrown.

Before using the eclipse refactor “surround with try-catch” we have to think is this really the best and most flexible way to handle the potential problem? For example say you have a method with a signature of void foo(obj) and something goes wrong, the client may be able to handle the problem but at least needs to be aware that something went wrong, indicating that the task didn't complete as expected.

The are a couple of problems with this
Thrown exceptions become part of the API so don't be too implementation specific.
Custom exceptions are not so easy to create, a new class needs to be created, more code more maintenance.
More responsibility on the API consumer and code duplication, catching exceptions and handling them in the same way many times throughout the code.

The argument that checked exceptions pollute code within implementation specific details is probably the biggest argument against them, and rightly so. For example if a component used a file to store data it might throw an IOException. Later on you might scale that up to a database, now you want to throw an SQLException but you can't change it because backwards compatibility would be broken. Not too bad if you own all of the code but an impossibility if you do. The simple work around is to create an exception for this situation and put the exception as the cause in the Exception(Throwable) constructor.

It is a shame that types in Java are not easily created, it would be great to do a typedef where the result is a completely new type. It would be especially good if generics could be included in that way like templates can in c++. This idea came about from reading about Erlang atoms which are a value represent only themselves and can be easily created so custom exceptions are cheap to create.

I saw some excellent c++ code not so long ago, types with templates were typedef'd into a type with a legible name. This was great although I'm not sure using it globally a lot is great idea, locally within a class would be excellent.

Lastly putting the onus on the consumer of the code to handle the exception correctly could be potentially dangerous. There are a lot of ways to badly handle exceptions, wrap in a RuntimeException and throw being the worst in most situations. This could happen because the code is in a Runnable or other interface which does not allow a checked exception to be thrown but doesn't make me feel any better about it.

Even with all of this, checked exceptions are the only maintainable way of using exceptions in your code as an error handling strategy. Can you imagine looking for which exceptions should be caught even a few calls deep? This is almost impossible and a maintenance nightmare.

There are some good counter arguments to all of these points and more below. Although I do agree that there are problems, especially in the google testing article the predictability of checked exceptions is awsome.

http://googletesting.blogspot.com/2009/09/checked-exceptions-i-love-you-but-you.html
and...
http://www.artima.com/intv/handcuffs.html

Wednesday, April 13, 2011

Working with Money in Java

Overview

A long time ago I wrote an article on using double for money. However, it is still a common fear for many developers when the solution is fairly simple.

Why not use BigDecimal

@myfear (Markus) asks a good question, doesn't BigDecimal do rounding already, (and with more options)

IMHO, There are two reason you may want to avoid BigDecimal

  1. Clarity.  This may be a matter of option but I find x * y / z clearer than x.multiply(y).divide(z, 10, BigDecimal.ROUND_HALF_UP)
  2. Performance. BigDecimal is often 100x slower.

Took an average of 6 ns for rounding using cast
Took an average of 17 ns for rounding using Math.round
Took an average of 932 ns for rounding using BigDecimal.setScale

You might like to test the difference on your machine.

The code

RoundingPerformanceMain.java

When to use BigDecimal


The better question is why would you ever use BigDecimal?

  1. Arbitary precision. If you want more than 15 decimal places of precision, use BigDecimal.
  2. Precise rounding. If you need to have full control over rounding options, it will be simpler using BigDecimal
  3. The project standard is to use BigDecimal. Human factors can be more important than technical arguments.

The problem with using double for money

double has two types of errors. It have representation error. i.e. it cannot represent all possible decimal values exactly. Even 0.1 is not exactly this value. It also has rounding error from calculations. i.e. as you perform calculations, the error increases.

double[] ds = {
        0.1,
        0.2,
        -0.3,
        0.1 + 0.2 - 0.3};
for (double d : ds) {
    System.out.println(d + " => " + new BigDecimal(d));
}
prints
0.1 => 0.1000000000000000055511151231257827021181583404541015625
0.2 => 0.200000000000000011102230246251565404236316680908203125
-0.3 => -0.299999999999999988897769753748434595763683319091796875
5.551115123125783E-17 => 5.5511151231257827021181583404541015625E-17

You can see that the representation for 0.1 and 0.2 is slightly higher than those values, and -0.3 is also slightly higher. When you print them, you get the nicer 0.1 instead of the actual value represented 0.1000000000000000055511151231257827021181583404541015625

However, when you add these values together, you get a value which is slightly higher than 0.

The important thing to remember is that these errors are not random errors. They are manageable and bounded.

Correcting for rounding error

Like many data types, such as date, you have an internal representation for a value and how you represent this as a string.

This is true for double. You need to control how the value is represented as a string. This can can be surprise as Java does a small amount of rounding for representation error is not obvious, however once you have rounding error for operations as well, it can some as a shock.

A common reaction is to assume, there is nothing you can do about it, the error is uncontrollable, unknowable and dangerous. Abandon double and use BigDecimal

However, the error is limited in the IEE-754 standards and accumulate slowly.

Round the result


And just like the need to use a TimeZone and Local for dates, you need to determine the precision of the result before converting to a String.

To resolve this issue, you need to provide appropriate rounding. With money this is easy as you know how many decimal places are appropriate and unless you have $70 trillion you won't get a rounding error large enough you cannot correct it.

// uses round half up, or bankers' rounding
public static double roundToTwoPlaces(double d) {
    return Math.round(d * 100) / 100.0;
}
// OR
public static double roundToTwoPlaces(double d) {
    return ((long) (d < 0 ? d * 100 - 0.5 : d * 100 + 0.5)) / 100.0;
}
If you add this into the result, there is still a small representation error, however it is not large enough that the Double.toString(d) cannot correct for it.
double[] ds = {
        0.1,
        0.2,
        -0.3,
        0.1 + 0.2 - 0.3};
for (double d : ds) {
    System.out.println(d + " to two places " + roundToTwoPlaces(d) + " => " + new BigDecimal(roundToTwoPlaces(d)));
}
prints
0.1 to two places 0.1 => 0.1000000000000000055511151231257827021181583404541015625
0.2 to two places 0.2 => 0.200000000000000011102230246251565404236316680908203125
-0.3 to two places -0.3 => -0.299999999999999988897769753748434595763683319091796875
5.551115123125783E-17 to two places 0.0 => 0

Conclusion

If you have a project standard which says you should use BigDecimal or double, that is what you should follow. However, there is not a good technical reason to fear using double for money.

Related Links

JodaMoney

My First Feeling towards her!

em naku matrame ila enduku anipinchali,
thanatho matladalani,
Thana gurinche vintu undalani,
thanatho smsing cheyalani,
thana swaram vinalani..!!!

Thana daggari nunchi vachina message chusthe entha santhashomo ... !! Ika missed call chusthe ... ha ..  anthe..!!
thanathone matladalani untundi eppudu..!! midnight onti ganta ki phone chesi just hello ani thanu chepthe ... entho happy ga feel ayi phone pettesina sandrbhalu .. kokollalu ... !!!

Thanani kanisam chudan kuda chudaledu ... !! just phone lo parichayam.. anthe.. mari nakendukila jaruguthundi .. !!

April 13th, early morning, inka daily news paper printing poorthi kaledu, taka taka ani okate sabdam .. samayam sumaru 12:35 am.. ... thellarithe 3 rasthra la elections andulo ma tamilanadu okati, andaru hai nidra pothunnaru... kani naku matram nidra pattadam ledu ... em cheyalo exact ga thelidu..!! chivaraki oka decission ki vachesa ... emaina elagaiana thana tho matladali ... thana voice vinali ..... edo manasulo theliyani alajadi, bhayam, bhadha, andholana... IKa labham ledani.. mobile theesi thana number notated kavadantho dail chesa ..... kani .. makiddari madya okay bond undi .... oka month lo kevalam okasare phone cheyali.. !!!!! kani chivaraki edaithe adi ayindani dail chesa .... atu vaipu nunchi sannaga, entho madhuramaina swaramtho "Hello"..!!! hmm ..... dani tharuvatha enno kathinamaina words ... "etime lo phone enti ... !!enduku chesav.. .!! Cheppa kada oke oka sari month ki ani .. !!! "

kani na manasu matram entho thelikaindi, entho anandanga undi, International level lo award gelichinappudu kuda antha anandam kalagamledu, and I was very very happy..!!!

oka vela thanu kuda inthe feel avuthunda .... !!! ;)

-- cheers,
Varra

Monday, April 11, 2011

Checking RAM size in linux

There are multiple ways to check the RAM size in a linux based system.

1) cat /proc/meminfo
2) free -m
3) dmidecode -t memory

If you want TOTAL PHYSICAL RAM, use this...

free, free -m or top doesn't seem to give you total physical RAM, the value is minus kernel size and some other stuff. The only way I've found to get total physical RAM so far is with this:

4) dmesg | grep ^Memory:


The second value is the physical RAM size. It'll also tell you what other junk is subtracted from that to give you what you get with "free"...

Sunday, April 10, 2011

log4j config

Which app server are you using? Each one puts its logging config in a different place, though most nowadays use Commons-Logging as a wrapper around either Log4J or java.util.logging.
Using Tomcat as an example, this document explains your options for configuring logging using either option. In either case you need to find or create a config file that defines the log level for each package and each place the logging system will output log info (typically console, file, or db).
In the case of log4j this would be the log4j.properties file, and if you follow the directions in the link above your file will start out looking like:

#------------------------------------------------------------------------------
#
#  The following properties set the logging levels and log appender.  The
#  log4j.rootCategory variable defines the default log level and one or more
#  appenders.  For the console, use 'S'.  For the daily rolling file, use 'R'.
#  For an HTML formatted log, use 'H'.
#
#  To override the default (rootCategory) log level, define a property of the
#  form (see below for available values):
#
#        log4j.logger. =
#
#    Available logger names:
#      TODO
#
#    Possible Log Levels:
#      FATAL, ERROR, WARN, INFO, DEBUG, TRACE
#
#------------------------------------------------------------------------------

log4j.rootCategory=DEBUG, S, ALL, ERROR

log4j.logger.com.dappit.Dapper.parser=ERROR
log4j.logger.org.w3c.tidy=FATAL

#------------------------------------------------------------------------------
#
#  The following properties configure the console (stdout) appender.
#  See http://logging.apache.org/log4j/docs/api/index.html for details.
#
#------------------------------------------------------------------------------
log4j.appender.S = org.apache.log4j.ConsoleAppender
log4j.appender.S.layout = org.apache.log4j.PatternLayout
log4j.appender.S.layout.ConversionPattern = [%d{dd-MM-yyyy HH:mm:ss.SSS}] [%p] [%c]: %m%n

#------------------------------------------------------------------------------
#
#  The following properties configure the Daily Rolling File appender which includes all the log level messages.
#  See http://logging.apache.org/log4j/docs/api/index.html for details.
#
#------------------------------------------------------------------------------
log4j.appender.ALL = org.apache.log4j.DailyRollingFileAppender
log4j.appender.ALL.File = {ur_app_log_dir}/{fileName}.log
log4j.appender.ALL.Append = true
log4j.appender.ALL.DatePattern = '.'yyyy-MM-dd
log4j.appender.ALL.layout = org.apache.log4j.PatternLayout
log4j.appender.ALL.layout.ConversionPattern = [%d{dd-MM-yyyy HH:mm:ss.SSS}] [%p] [%c{1}]: %m%n

#------------------------------------------------------------------------------
#
#  The following properties configure the Daily Rolling File appender which includes all the log level messages.
#  See http://logging.apache.org/log4j/docs/api/index.html for details.
#
#------------------------------------------------------------------------------
log4j.appender.ERROR = org.apache.log4j.DailyRollingFileAppender
log4j.appender.ERROR.File = {ur_app_log_dir}/error.log
log4j.appender.ERROR.Append = true
log4j.appender.ERROR.Threshold = ERROR
log4j.appender.ERROR.DatePattern = '.'yyyy-MM-dd
log4j.appender.ERROR.layout = org.apache.log4j.PatternLayout
log4j.appender.ERROR.layout.ConversionPattern = [%d{dd-MM-yyyy HH:mm:ss.SSS}] [%p] [%c{1}]: %m%n

#------------------------------------------------------------------------------
#
#  The following properties configure the package level log level filtering. you can restrict the useless log messages of ThirdParty API's.
#  Eg: SpringFramework's trace, debug, info messages are not much useful for us, so can be restricted like this.
#  log4j.logger.org.springframework=WARN
#
#  See http://logging.apache.org/log4j/docs/api/index.html for details.
#
#------------------------------------------------------------------------------
log4j.logger.{desired_package}={Level}
 
--cheers,
Varra 

Exception Handling

One of the problems with exception handling is knowing when and how to use it. In this article, I will cover some of the best practices for exception handling. I will also summarize the recent debate about the use of checked exceptions.
We as programmers want to write quality code that solves problems. Unfortunately, exceptions come as side effects of our code. No one likes side effects, so we soon find our own ways to get around them. I have seen some smart programmers deal with exceptions the following way:
public void consumeAndForgetAllExceptions(){
    try {
        ...some code that throws exceptions
    } catch (Exception ex){
        ex.printStacktrace();
    }
}
What is wrong with the code above?
Once an exception is thrown, normal program execution is suspended and control is transferred to the catch block. The catch block catches the exception and just suppresses it. Execution of the program continues after the catch block, as if nothing had happened.
How about the following?
public void someMethod() throws Exception{
}
This method is a blank one; it does not have any code in it. How can a blank method throw exceptions? Java does not stop you from doing this. Recently, I came across similar code where the method was declared to throw exceptions, but there was no code that actually generated that exception. When I asked the programmer, he replied "I know, it is corrupting the API, but I am used to doing it and it works."

It took the C++ community several years to decide on how to use exceptions. This debate has just started in the Java community. I have seen several Java programmers struggle with the use of exceptions. If not used correctly, exceptions can slow down your program, as it takes memory and CPU power to create, throw, and catch exceptions. If overused, they make the code difficult to read and frustrating for the programmers using the API. We all know frustrations lead to hacks and code smells. The client code may circumvent the issue by just ignoring exceptions or throwing them, as in the previous two examples.

The Nature of Exceptions

Broadly speaking, there are three different situations that cause exceptions to be thrown:
  • Exceptions due to programming errors: In this category, exceptions are generated due to programming errors (e.g., NullPointerException and IllegalArgumentException). The client code usually cannot do anything about programming errors.
  • Exceptions due to client code errors: Client code attempts something not allowed by the API, and thereby violates its contract. The client can take some alternative course of action, if there is useful information provided in the exception. For example: an exception is thrown while parsing an XML document that is not well-formed. The exception contains useful information about the location in the XML document that causes the problem. The client can use this information to take recovery steps.
  • Exceptions due to resource failures: Exceptions that get generated when resources fail. For example: the system runs out of memory or a network connection fails. The client's response to resource failures is context-driven. The client can retry the operation after some time or just log the resource failure and bring the application to a halt.

Types of Exceptions in Java

Java defines two kinds of exceptions:
  • Checked exceptions: Exceptions that inherit from the Exception class are checked exceptions. Client code has to handle the checked exceptions thrown by the API, either in a catch clause or by forwarding it outward with the throws clause.
  • Unchecked exceptions: RuntimeException also extends from Exception. However, all of the exceptions that inherit from RuntimeException get special treatment. There is no requirement for the client code to deal with them, and hence they are called unchecked exceptions.
By way of example, Figure 1 shows the hierarchy for NullPointerException.
Figure 1
Figure 1. Sample exception hierarchy
In this diagram, NullPointerException extends from RuntimeException and hence is an unchecked exception.
I have seen heavy use of checked exceptions and minimal use of unchecked exceptions. Recently, there has been a hot debate in the Java community regarding checked exceptions and their true value. The debate stems from fact that Java seems to be the first mainstream OO language with checked exceptions. C++ and C# do not have checked exceptions at all; all exceptions in these languages are unchecked.
A checked exception thrown by a lower layer is a forced contract on the invoking layer to catch or throw it. The checked exception contract between the API and its client soon changes into an unwanted burden if the client code is unable to deal with the exception effectively. Programmers of the client code may start taking shortcuts by suppressing the exception in an empty catch block or just throwing it and, in effect, placing the burden on the client's invoker.
Checked exceptions are also accused of breaking encapsulation. Consider the following:
public List getAllAccounts() throws
    FileNotFoundException, SQLException{
    ...
}
The method getAllAccounts() throws two checked exceptions. The client of this method has to explicitly deal with the implementation-specific exceptions, even if it has no idea what file or database call has failed within getAllAccounts(), or has no business providing filesystem or database logic. Thus, the exception handling forces an inappropriately tight coupling between the method and its callers.

Best Practices for Designing the API

Having said all of this, let us now talk about how to design an API that throws exceptions properly.

1. When deciding on checked exceptions vs. unchecked exceptions, ask yourself, "What action can the client code take when the exception occurs?"

If the client can take some alternate action to recover from the exception, make it a checked exception. If the client cannot do anything useful, then make the exception unchecked. By useful, I mean taking steps to recover from the exception and not just logging the exception. To summarize:
Client's reaction when exception happens Exception type
Client code cannot do anything Make it an unchecked exception
Client code will take some useful recovery action based on information in exception Make it a checked exception
Moreover, prefer unchecked exceptions for all programming errors: unchecked exceptions have the benefit of not forcing the client API to explicitly deal with them. They propagate to where you want to catch them, or they go all the way out and get reported. The Java API has many unchecked exceptions, such as NullPointerException, IllegalArgumentException, and IllegalStateException. I prefer working with standard exceptions provided in Java rather than creating my own. They make my code easy to understand and avoid increasing the memory footprint of code.

2. Preserve encapsulation.

Never let implementation-specific checked exceptions escalate to the higher layers. For example, do not propagate SQLException from data access code to the business objects layer. Business objects layer do not need to know about SQLException. You have two options:
  • Convert SQLException into another checked exception, if the client code is expected to recuperate from the exception.
  • Convert SQLException into an unchecked exception, if the client code cannot do anything about it.
Most of the time, client code cannot do anything about SQLExceptions. Do not hesitate to convert them into unchecked exceptions. Consider the following piece of code:
public void dataAccessCode(){
    try{
        ..some code that throws SQLException
    }catch(SQLException ex){
        ex.printStacktrace();
    }
}
This catch block just suppresses the exception and does nothing. The justification is that there is nothing my client could do about an SQLException. How about dealing with it in the following manner?
public void dataAccessCode(){
    try{
        ..some code that throws SQLException
    }catch(SQLException ex){
        throw new RuntimeException(ex);
    }
}
This converts SQLException to RuntimeException. If SQLException occurs, the catch clause throws a new RuntimeException. The execution thread is suspended and the exception gets reported. However, I am not corrupting my business object layer with unnecessary exception handling, especially since it cannot do anything about an SQLException. If my catch needs the root exception cause, I can make use of the getCause() method available in all exception classes as of JDK1.4.
If you are confident that the business layer can take some recovery action when SQLException occurs, you can convert it into a more meaningful checked exception. But I have found that just throwing RuntimeException suffices most of the time.

3. Try not to create new custom exceptions if they do not have useful information for client code.

What is wrong with following code?
public class DuplicateUsernameException
    extends Exception {}
It is not giving any useful information to the client code, other than an indicative exception name. Do not forget that Java Exception classes are like other classes, wherein you can add methods that you think the client code will invoke to get more information.
We could add useful methods to DuplicateUsernameException, such as:
public class DuplicateUsernameException
    extends Exception {
    public DuplicateUsernameException 
        (String username){....}
    public String requestedUsername(){...}
    public String[] availableNames(){...}
}
The new version provides two useful methods: requestedUsername(), which returns the requested name, and availableNames(), which returns an array of available usernames similar to the one requested. The client could use these methods to inform that the requested username is not available and that other usernames are available. But if you are not going to add extra information, then just throw a standard exception:
throw new Exception("Username already taken");
Even better, if you think the client code is not going to take any action other than logging if the username is already taken, throw a unchecked exception:
throw new RuntimeException("Username already taken");
Alternatively, you can even provide a method that checks if the username is already taken.
It is worth repeating that checked exceptions are to be used in situations where the client API can take some productive action based on the information in the exception. Prefer unchecked exceptions for all programmatic errors. They make your code more readable.

4. Document exceptions.

You can use Javadoc's @throws tag to document both checked and unchecked exceptions that your API throws. However, I prefer to write unit tests to document exceptions. Tests allow me to see the exceptions in action and hence serve as documentation that can be executed. Whatever you do, have some way by which the client code can learn of the exceptions that your API throws. Here is a sample unit test that tests for IndexOutOfBoundsException:
public void testIndexOutOfBoundsException() {
    ArrayList blankList = new ArrayList();
    try {
        blankList.get(10);
        fail("Should raise an IndexOutOfBoundsException");
    } catch (IndexOutOfBoundsException success) {}
}
The code above should throw an IndexOutOfBoundsException when blankList.get(10) is invoked. If it does not, the fail("Should raise an IndexOutOfBoundsException") statement explicitly fails the test. By writing unit tests for exceptions, you not only document how the exceptions work, but also make your code robust by testing for exceptional scenarios.

Best Practices for Using Exceptions

The next set of best practices show how the client code should deal with an API that throws checked exceptions.

1. Always clean up after yourself

If you are using resources like database connections or network connections, make sure you clean them up. If the API you are invoking uses only unchecked exceptions, you should still clean up resources after use, with try - finally blocks.
public void dataAccessCode(){
    Connection conn = null;
    try{
        conn = getConnection();
        ..some code that throws SQLException
    }catch(SQLException ex){
        ex.printStacktrace();
    } finally{
        DBUtil.closeConnection(conn);
    }
}

class DBUtil{
    public static void closeConnection
        (Connection conn){
        try{
            conn.close();
        } catch(SQLException ex){
            logger.error("Cannot close connection");
            throw new RuntimeException(ex);
        }
    }
}
DBUtil is a utility class that closes the Connection. The important point is the use of finally block, which executes whether or not an exception is caught. In this example, the finally closes the connection and throws a RuntimeException if there is problem with closing the connection.

2. Never use exceptions for flow control

Generating stack traces is expensive and the value of a stack trace is in debugging. In a flow-control situation, the stack trace would be ignored, since the client just wants to know how to proceed.
In the code below, a custom exception, MaximumCountReachedException, is used to control the flow.
public void useExceptionsForFlowControl() {
    try {
        while (true) {
            increaseCount();
        }
    } catch (MaximumCountReachedException ex) {
    }
    //Continue execution
}

public void increaseCount()
    throws MaximumCountReachedException {
    if (count >= 5000)
        throw new MaximumCountReachedException();
}
The useExceptionsForFlowControl() uses an infinite loop to increase the count until the exception is thrown. This not only makes the code difficult to read, but also makes it slower. Use exception handling only in exceptional situations.

3. Do not suppress or ignore exceptions

When a method from an API throws a checked exception, it is trying to tell you that you should take some counter action. If the checked exception does not make sense to you, do not hesitate to convert it into an unchecked exception and throw it again, but do not ignore it by catching it with {} and then continue as if nothing had happened.

4. Do not catch top-level exceptions

Unchecked exceptions inherit from the RuntimeException class, which in turn inherits from Exception. By catching the Exception class, you are also catching RuntimeException as in the following code:
try{
..
}catch(Exception ex){
}
The code above ignores unchecked exceptions, as well.

5. Log exceptions just once

Logging the same exception stack trace more than once can confuse the programmer examining the stack trace about the original source of exception. So just log it once.

6. Do not use e.printStackTrace

e.printStackTrace prints to the console. You will only see this messages, if you have defined a console appender. If you use Tomcat or other application server with a service wrapper and define a console appender, you will blow up your wrapper.log.
try {
     ......... snip .......
 } catch ( SomeException e) {
     e.printStackTrace();
 }
You can use log.error(e,e). The second parameter passed an exception and will print the stack trace into the logfile.
try {
     ......... snip .......
 } catch (SomeException e) {
     log.error("Exception Message", e);
   // display error message to customer
 }

7. Don't log and throw again

try {

     ......... snip .......
 } catch ( SomeException e) {
     log.error("Exception Message", e);
   throw e;
 }
Do not catch an exception, log the stacktrace and then continue to throw it. If higher levels log a message as well, you will end up with a stacktrace printed 2 or more times into the log files.

8. Don't kill the stacktrace

try{
... some code
    }catch(SQLException e){
        throw new RuntimeException(?DB excpetion? +e.getMessage());
    }
This code will erase the stacktrace from the SQLException. This is not recommended, because you will loose important information about the exception. Better do the following.
try{
... some code
    }catch(SQLException e){
        throw new RuntimeException("My Exception name", e);
    }
 

Summary

These are some suggestions for exception-handling best practices. I have no intention of staring a religious war on checked exceptions vs. unchecked exceptions. You will have to customize the design and usage according to your requirements. I am confident that over time, we will find better ways to code with exceptions.

Saturday, April 9, 2011

Jaganism first anniversary

"జగనిజం" ప్రారంభించబడి ఒక సవంత్షరం పూర్తి చేసుకున్న సందర్భంగా సభ్యులు, మిత్రులు, శ్రేయాభిలాషులు, అభిమానులు, జగన్ అన్న, YSR కాంగ్రెస్ పార్టీ  అభిమానులు ఆందరూ కలిసి అభినంధన సభ ఏర్పాటు చేసుకొని కేక్ కట్ చేసి శుభాకాంక్షలు తెలుపుకున్నారు.

కేక్ కట్టింగ్ అనంతరం చెన్నై "జగనిజం" అధ్యక్షుడు వర్రా రాజాకృష్ణ రెడ్డి మాట్లాడుతూ రైతు పక్షపాతి, అపర భగీరథుడు దివంగత ముఖ్యమంత్రి శ్రీ Dr. YS రాజశేఖర రెడ్డి ఆశయ సాధనే లక్ష్యంగా, ప్రజలకిచ్చిన మాట కోసం తన పదవిని త్రునప్రాయాంగా వదిలేసి , ప్రజల చెంతకు వెళ్తూ, మీకోసం నేనున్నానంటూ వారికీ భరోసా కల్పిస్తూ తన తండ్రి అడుగు జాడల్లో నడుస్తున్న మన ప్రియతమ యువనేత శ్రీ YS జగన్మోహన రెడ్డి గారికి మా వంతుగా తోడ్పడాలనే సదుద్దేశ్యంతో, అభిమానం లోనే కాదు, ఆదర్శం లోను ముందు ఉంటాం .. అనే నినాదంతో "జగనిజం" పురుడు పోసుకుంది. 
తన తండ్రి కోసం ప్రాణాలర్పించిన వారి ఇళ్ళకి వెళ్లి ఆ ఆత్మీయులని కలుస్తూ, పేరు పేరున పిలిచి ఓదారుస్తూ, వారి జీవితాల్లో ఒక నుతనోత్తేజాన్ని నింపుతూ, వారి ముఖాల్లో పూర్వపు వెలుగులు వేదజల్లడానికి ఒక మహాన్నోత ఆశయంతో ప్రారంభించబడిన "ఓదార్పు యాత్ర" రోజునే "జగనిజం" కూడా ప్రారంభించబడినది.
అప్పటి నుంచి ఎన్నో బృహత్తర కార్యక్రమాలని చేపడుతు, అన్న చేసే ప్రతి కార్యక్రమంలో మా వంతు పాలుపంచుకుంటూ, ఎంతో మంది NRI ల ప్రోత్సాహంతో పేదల పక్షపాతి, రైతు కి చిన్న మేలు జరుగుతుందంటే చాలు అది ఎంత పెద్ద పనైనా, ఎవరు అడ్డొచ్చిన లెక్క చేయకుండా ఆ పని జరిగేటట్లు చూసి, ఆ రైతు ముఖంలో ఆనందాన్ని పంచి, ఎముక లేని చేయి అని పిలిపించుకున్న YSR  సువర్ణ పాలన స్థాపనే ధ్యేయంగా అన్ని ప్రముఖ నగరాలలో ఎన్నో వేలమంది సభ్యులతో, సూరజ్ కుమార్, జూపూడి జేసింత్ పాల్ అధ్వార్యంలో "జగనిజం" తన కార్యకలపాలని కొనసాగిస్తుంది.
ఈరోజు YSR జిల్లా పులివెందులలో "జగనిజం" రాష్ట్రా అధ్యక్షుడు సూరజ్ కుమార్, ముఖ్య కార్యదర్శి జూపూడి జేసింత్, స్టీఫెన్ (చిత్తూర్), చైతన్య (బెంగుళూరు), బజీద్ (ప్రకాశం), గంగాధర్ రెడ్డి (కడప),  మహేష్ నాయుడు (రంగారెడ్డి) మరియు అభిమానులతో జలయజ్ఞ ప్రదాత, అపర భగీరథుని సమాధి వద్ద వార్శికోత్శవ సభ కి వెళ్ళారని అందువల్ల ఇక్కడ పాల్గొనలేక పోయారని, మరియు ఆంధ్రప్రదేశ్ లోని వివిధ ప్రాంతాలలో వేర్వేరుగా ఈ వార్శికోత్శవ సభలు జరుగుతున్నాయని "జగనిజం" చెన్నై అధ్యక్షుడు వర్రా రాజాకృష్ణ రెడ్డి ఒక ప్రశ్నకి సమాధానంగా చెప్పారు.
చెన్నై విభాగపు ముఖ్య కార్యదర్శి కార్తికేయన్ రెడ్డి మాట్లాడుతూ జగన్ అన్నని ముఖ్యమంత్రిగా చూడటమే "జగనిజం" లక్ష్యమని, దీనికి అడ్వైసరి కమిటి, వొమెన్ సెల్, ఆన్లైనే వింగ్ విడివిడిగా ఏర్పాటు చేయబడి, వివిధ రూపాలలో తమ సేవలని అందిస్తున్నాయని, రీసెంట్ గా YSR పై ఒక CD ni విడుదల చేసామని, జగన్ అన్న పై అభిమానం ఉండి, సేవ చేయాలన్న ఆలోచనతో ఉన్న వారందరూ ఆహ్వానీతులే అని దీనిలో చేరగోరే వారు +91 8144 255 375 కి ఫోన్ చేయొచ్చని తెలిపారు.
ఈ కార్యక్రమంలో కార్తికేయన్ రెడ్డి, సుభ్రమణ్య శర్మ, రవి నాయుడు, పీర్ షహేబ్, మస్తాన్ వలి, యేసు దాసు, కళ్యాణ్ చౌదరి, రీనా శర్మ, మరియు తదితరులు పాల్గున్నారు.

Friday, April 8, 2011

How to avoid Garbage Collection

Overview

The unpredictability of Garbage Collection can be a serious issue for some applications. You can play with the tuning parameters, but the simplest approach is to create less garbage.

Low garbage application

For low latency trading system, minimising GC time is important. The extreme case is avoid GCs all together.

If you reduce the garbage produced and increase your Eden size you can reduce your GCs to once per day (as an over night task) In the case of this application the Full GC triggered at 5 am every morning takes 1-2 seconds (during the overnight maintenance window)

It should be possible to extend the period between GCs to once a week.

Thursday, April 7, 2011

Logging Tips

1) Keep Logging where necessary and mandatory.

  • Some places we may want the user to know what is the status of application, so let him know it by providing the sufficient info.
  • Please dont log the any secret info or un encrypted data which gets rendered in plain text format.
  • Please avoid having logging statements in loops. It consumes more cpu usage. If it is mandate then have a single log statement which conveys full info.
  • You can avoid thelogging in loops by displaying the details after/before the loops.
        Eg:  for (int i=0;i< list.size(); i++)
       {
               ------
               logger.info("i: "+i+", value: "+list.get(i));
        }
  • In the above example we haev a logger which logs all the list entries where it is unnecessary, time consuming process, and in realtime who cares about the list entries and what anyone can do that data.. !!? -- no answer .. ;)
  • Instead of that you can log something like below.
  • Eg:
  • logger.info("We have received "+list.size()+" events from the server and going to process them");
  • Eg:
        final Timer timer = new Timer(); // Starts the timer.
        int publishedCount = 0; // To count the published events
        for (int i=0;i< list.size(); i++)
       {
             ------
             publishedCount++;
       }
        logger.info(publishedCount+" events processed and published successfully, time taken: "+timer.getElapsedTime()+" millis");
  • It is very easy, most informative and gives the best performance.
  • So user can see the number of events received count and can see the processed/published count and time taken.


 2) Try to have a complete full meaning content in a single line, so that your application will be greppable.
 3) Better to inject the key words in your logging content, so even a novice can grep your log.
 4) Avoid using the classname and method names in the logging.
             eg: myLogger.info(CLASS_NAME + METHOD_NAME + " Number of Ci Ids  :" + ciDetails.size());           ;-)
5)

Wednesday, April 6, 2011

Low GC in Java: Using primitives

Overview

In a recent article I examined how using primitives and collections which support primitives natively instead of Wrappers and standard collections can reduce memory usage and improve performance.

Different way to have a Map of int/Integer

There are a number of ways you can use int/Integer and a number of collections you store them in. Depending on which approach you use can have a big difference on the performance and the amount of garbage produced.
TestPerformance RangeMemory used
Use Integer wrappers and HashMap71 - 134 (ns) 53 MB/sec
Use int primitives and HashMap45 - 76 (ns)36 MB/sec
Use int primitives and FastMap58 - 93 (ns)28 MB/sec
Use int primitives and TIntIntHashMap18 - 28 (ns) nonimal
Use int primitives and simple hash map6 - 9 (ns) nonimal
The performance range was the typical (50%tile) and one of the higher (98%tile) timings. The garbage was the result of 900,000 loops per second.


These tests were run on my new system.


The Code

Loop tested
int runs = 300 * 300;
for (int i = 0; i < runs; i++) {
    int x = i % 300;
    int y = i / 300;
    int times = x * y;
    Integer count = counters.get(times);
    if (count == null)
        counters.put(times, 1);
    else
        counters.put(times, count + 1);
}

Link to all the code is here

Directory of performance examples

Sunday, April 3, 2011

వావ్ ... !!! వాట్ ఎ గ్రేట్ థ్రిల్లింగ్ మోవ్మేంట్ ఇట్ వాస్... !!!

వావ్ ... !!! వాట్ ఎ గ్రేట్ థ్రిల్లింగ్ మోవ్మేంట్  ఇట్ వాస్ ... !!!  

యూవీ ఆల్రౌండ్ ప్రతిభ ,
జహీర్ లైన్ అండ్ లెంగ్త్  బౌలింగ్,
గంభీర్ కన్సిస్తేన్సి ,
సీనియర్స్  వెల్ ఎక్షెపిరియన్సె ,
ధోని నాయకత్వ లక్షణాలు ....    వెరసి ...... 

ఓ వరల్డ్ కప్ ..!
నెరవేరిన సచిన్ చిరకాల వాంఛ .!!
ఇండియా క్రికెట్ చరిత్ర లో ఓ సువర్ణాధ్యాయం ... !!!
భారతవాని ముద్దు బిడ్డలకి ఒక తీపి జ్ఞాపిక ... !!!!


హేట్స్  ఆఫ్ ఇండియా ....  అండ్ థాంక్స్  ఇండియా టీం ... !!!!!!