Debugging a large, complex system(Java)

So for my main contract, I am working on a fairly large and a complex system i.e. it has many moving parts. So in addition to the large code-base that I have to maintain, the project also makes use of several libraries i.e. jar files written by various teams of developers at a different location to where I work. However even though completely undocumented,  I do have access to the source code for most of those jar files. It is a Java based system which uses the Spring framework and Maven, amongst other things.

Problem

As I mentioned this system has many moving parts and often when I have to add a new component to it, I rely on a few things from the jar files written by a team of developers far away. Every now and then, the changes made to those dependant jar files by the team of developers do not work or have some "sneaky" hard to find problems within them! so for code that I did not write, I need to find out

1. why isn't it working?
2. is it possible for me to find a solution and get it to work, so I can finish my adding the new component, that I need to add?

The above points make this a debugging problem for me. The challenge here is debugging source code, that I have very little knowledge of, so yes I feel like an adventurer on a quest to find a certain stolen artefact from an unknown location. Indiana Jones anyone? Ok, so think of it this way, the unknown location is the source code that I have very little knowledge of and the stolen artefact is the solution to make the jar work, so I can continue on with my work. Hmmm, to think of it, I am Indy of modern times. Ok imagine this to be a parallel universe where Indy is a programmer who tracks down and fixes bugs in the source code, cool! that makes me Indy right? it totally does! I can't wait to see an Indiana Jones movie, where we watch an actor sitting in front of a computer screen for 2 hours looking at source code. That would make great cinema, truly exciting stuff (maybe I should put a smiley somewhere, it can be hard to identify sarcasm from written text?)! Ok enough procrastination, let's move on with the content of this post.

Solution

So there are a range of problems, starting from something very easy to something fairly complex i.e. the sneaky problems that I mentioned earlier. So let's start with easier ones,

Problem 1: Missing annotations

Spring has annotations, discussing them in detail is beyond the scope of this document, so if you would like to know more about them, I recommend following a tutorial for it. Ok how do these annotations pose a problem for me? well, in the project I work on, adding a new feature often involves autowire ...ing certain fields and to successfully autowire fields the right type annotations are needed for a class which are often missing.  So if a class in one of the jar files is missing an annotation this will cause a problem with the jar which means the jar will not run and the system too will fail to run. The solution to this problem is often just a case of adding @Component annotation to a class in the jar and then rebuilding the jar from it's source and re-deploying it. Have a read of this tutorial to know more about Spring and what I am talking about.

Problem 2: Configuration issues

So checking out the latest source code for the dependant jar causing the problem, fixing it and then rebuilding it can take too long to fix for a simple problem such as a change in port no. Wait what port no? Ok, so these dependant libs(jars) used by the system that I am working on fetch data from certain end points i.e. certain URLs and from a specific port numbers, which are defined in a certain xml file. Let's call this xml file nameDaoContext.xml i.e. Spring XML configuration file and the jar file problemLib.jar so to solve this problem, I would first move this jar to a folder called jarFixFolder copy problemLib.jar to the folder unzip the jar file using this 

jar -xf problemLib.jar

Once unzipped, delete the jarFixFolder/problemLib.jar. Unzipping the jar file gives me access to the nameDaoConext.xml, then I would find and replace the incorrect port no in the jar file with the correct port no and rebuild the jar using this command,

jar -cf problemLib.jar *.*

Important note

Ok the aforementioned approach is an absolute last resort. It is actually considered bad practice, but rest assured if I am doing something "hacky" like this, it is completely justified in solving my problem. 

Now, please remember there are multiple teams working on this project and that too at different locations. Hence in the interest of saving time, my objective is to get a working solution even if it means "hacking" a jar file which only ever happens for configuration issues. If I "hack" a jar it won't make it to production anyway, because I will be raising an issue with clear instructions of what's wrong in the jar configuration. The issue is then assigned to the team responsible for maintaining the jar so they can implement a proper fix for the jar and my instructions in the issue serves as a good starting point and saves some time when implementing a fix.

Problem 3: Naming errors

Ok, at one instance I encountered a naming error which was very difficult to pinpoint. Not sure how I can explain it, but will give it a try none the less, 

• one maven project builds two jar files i.e. jarOne and jarTwo
• both jarOne and jarTwo perform a series of CRUD operations
• both the jars have retrieve all their DB queries from a sqlQueries.properties file
• jarOne was always being used by my project whereas jarTwo was new to my project
• however when I included jarTwo in the build path for my project I would get build errors

To solve this problem, I rebuilt jarTwo from it's source and added a number of simple print statements. After some debugging I found that jarTwo could run 80% of it's queries in it's sqlQueries.properties file and fail for the remaining 20% of them. Ok this was rather strange? After some more debugging I found that sqlQueries.properties file of jarOne had the 80% of the queries of jarTwo and the sqlQueries.properties files in both jarOne and jarTwo had the same name. So there was some config issue whereby jarTwo was trying to get all it's queries from the jarOne and hence jarTwo could only run 80% of it's queries at run-time and fail for the other 20%. The solution in this case was simple and that was just to make sure the sqlQueries.properties files have different names in jarOne and jarTwo and rebuild the jar files from source.

Summary

The problems that I have mentioned before were very specific to the project I am working on. Now I will try to summarise the debugging tips from the above solutions to the aforementioned problems to something more genera, that can benefit others,

1. Never shy away from using simple print statements to print: why? well sometimes certain libs have their own logging in place and using print statements means you can knock up a relatively simple logging solution that is customisable. Whether you log to console or write to a file, it's up to you (logging to console works most of the time for me).
2. Always look out for any naming issues: If you jar is accessing a certain file, ensure it is referring to the right file by printing out the contents of the file and double checking it with the file contents.
3. Missing annotations: This really is a very specific problem to a Spring based system, but yeah watch out for problems caused by missing annotations.