Speeding Up Technology: Choosing the Right IDE

People who were born with 'Eclipse' in their hand may not be able to appreciate the woes of a programmer trying to be productive with 'Edit Plus' or 'vim'. Many managers grossly under-estimate the productivity gains that can be had by using the right IDE. Not only that, development productivity for many current technologies can be improved significantly if their IDEs see significant improvement.

If you do not mind spending, go ahead and buy 'Idea'. If you are broke, 'Net Beans' would make a very good choice. I know a lot of people would be screaming 'Eclipse' by this point of time. 'Eclipse' is, no doubt, the IDE with greatest potential. But if you are developing a web application, want to get up-to-speed asap and integrated with most of the needed Java and J2EE technologies for the project(Maven, Subversion, Database, Servers, Messaging etc.) in no time, I would recommend 'Net Beans'. If you value super customization over speed, 'Eclipse' is your baby. Eclipse can also serve as a fabulous starting point for anybody who wants to develop an IDE.

The tricky bit will come into play if you wish to develop a rich internet application. This would certainly need either JavaScript or Flex. Personally, I feel that there is no IDE that effectively addresses this problem of building a super-efficient dashboard on top of a Java based backend. In this case I would recommend using Eclipse because the Flex IDE is built on top of Eclipse. You may even use Net Beans and Eclipse in conjunction while you take time to configure Eclipse to do the complete bidding.

Speeding Up Technology: Reinventing The Wheel

Re-usability is the holy-grail of any large-scale production industry. IT is no different. This re-usability primarily manifests itself on the technical front in form of programming paradigms, reusable components and frameworks.

Programming Paradigms

Programming paradigms include computing concepts like algorithms, data structures, object oriented programming, aspect oriented programming, design pattern et al. These are fundamentals that enable programmers to 'reuse' experience. It is surprising how many developers are unaware about these things. If the team has any member who completely lacks this knowledge, training or replacement is in order as these are probably the basic pre-requisites for hiring.

Reusable Components

Reusable components can be taken from open-source communities, licensed from third party or developed in-house. In open source, Google and Apache Foundation are perhaps the biggest and most reliable contributors. There are also a lot of third party development houses that may have developed the needed components and can save the company a lot of time. It is always a good idea to not only be aware about these initiatives but to actively track them. Joining organizations like Association for Computing Machinery and attending technology seminars can help in keeping you aware and agile. Buying a library, using an open-source library or building on a tried and tested library are acceptable, and even desirable, ways of speeding up development.

Developing reusable components in-house, however, is a more tricky business. Each company, knowingly or unknowingly, does try to build upon what it knows. It is a good start but can never be enough. What is really needed is an active initiative to consciously try to build components, if not frameworks, at each step. A forum to share best practices and components across the company can multiply the benefits and speed up development significantly. In fact, bigger the company, more the benefits. The only point to be kept in mind is that this initiative should neither detract nor confuse the main objective of building the product. The danger of "initiative" turning into another bureaucratic process is an equally scary scenario.

Frameworks

The final component is using frameworks. We have seen Ruby-on-Rails, Struts, Spring et al assuming significant importance. Most of them are practical implementation of programming paradigms and demonstrate the effectiveness (and weaknesses) of the underlying paradigms. Hence, frameworks are at one higher level of abstraction than either the programming paradigms or the components. This means that efficiency gains can be significantly higher if the framework is chosen correctly. In my experience, Spring makes an excellent choice for complex, multi-tiered web applications.

In our next article we talk about IDEs.

Speeding Up Technology Projects

Speeding up technology projects can be done in various ways. I feel that there can be two fundamental stumbling blocks: technological and managerial.

Technologically, I assume that the team has hired reasonably bright developers with a decent, if not perfect, understanding of programming basics. After that,the potential bottle-necks can be:

• reinventing the wheel: programmers spend way too much time doing stuff that is already done by somebody
• IDE: a good IDE can significantly speed up the speed of development
• poor testing

From managerial point of view, the main stumbling blocks can be:

• poor choice of technology
• poor choice of development methodology
• lack of focus on long-term vision with complete focus on speed

These may be self-evident to most of you who have had some experience in IT. Nevertheless, each point is much more complicated than it appears and merits some discussion.

These issues can come to haunt a team at any point due to the extremely dynamic nature of information technology. To be ever-vigilant is the only answer. What makes it especially challenging is the fact that many problems can persist themselves without making themselves evident for a long time; and by that time the rot has set in with a vengeance and changes are pretty hard to implement. In a series of articles I shall have a closer look at these issues in perspective of my experience and observations and chronicle the same. Please feel free to give your feedback and views wrt same.

Specifying SSL Certificate and Keystore Location & Password

A small reminder about how to configure the location and password for your keystore with Tomcat. By default Tomcat will look for your keystore (with the file name .keystore) in the home directory with the default password. It is possible to change the filename, password, and the location that Tomcat looks for the keystore by configuring the SSL Connector in "server.xml" (in conf folder in tomcat). "keystoreFile" is the parameter specifying key location and "keyPass" would specify the corresponding password. the finished config may look something like:

<Connector port="8443" 
    protocol="HTTP/1.1" SSLEnabled="true"
    maxThreads="150" scheme="https" secure="true"
    clientAuth="false" sslProtocol="TLS" 
    keystoreFile="/home/user_name/my_domain.key" 
    keypass="my_keystore_password"/>

This, however, will not help you if you are using an SSL socket. This is just for making an https connection. If you want to control only keystore and truststore name and password for your socket, you need to give the system properties the values by either specifying the same in vm parameters or setting it in the code:

Using VM parameters:

-Djavax.net.ssl.keyStore=keystore.jks 
-Djavax.net.ssl.keyStorePassword=abc123 
-Djavax.net.ssl.trustStore=truststore.jks 
-Djavax.net.ssl.trustStorePassword=abc123

In Code:

Properties properties = System.getProperties();
properties.put("javax.net.ssl.keyStore", keyStoreName);
properties.put("javax.net.ssl.keyStorePassword", keyStorePassword);
properties.put("javax.net.ssl.trustStore", trustStoreName);
properties.put("javax.net.ssl.trustStorePassword", trustStorePassword);

For a better understanding of SSL Sockets, please refer to the JSSE Reference Guide. Nevertheless, this is not the end of your woes if you want to control the location as well. This would make sense if you wished to package your application with some default stores so as it can potentially work right out of the box. Here is a piece of code that can help you to do exactly that: Managing store locations

Brainstorming Basics

A question asked is if this idea is unique. Yes and no is the answer. No, as a information systems are being built in one form or another. Yes, as there is no comprehensive information management and analysis system. For example, Joomla is unique as it gives a completely different perspective on building and managing website. Similarly, the project that I am proposing is unique in its vision and application.

Is it even possible to make information systems generalist when people use them so differently? Indeed, this problem is challenging. Nevertheless, in my experience, it is not only possible but is also desirable.

Am I not worried that somebody might steal my idea? Not really. In fact, if somebody can build such a system I may use it to make more applications. On the other hand, since this application that I intend to build may end up as open-source, it may have some appeal eventually.

The first step is to start defining the key modules. Let us brainstorm the objects that we can see in the system

Data Handler: A data handler that reads data in a standardized format and then stores, updates or deletes it as need be

Information: The central class that would act as a controller or central point of directing a particular information set

Webpage: A simple html page that we wish to use display various data interpretation elements

Report: A static report

Graph: Tools to generate static graphs on data set

Tables: Tools to generate smart tables on data set

Filters: Filters for graph or table data

Dashboard: Tools to generate a group of one or more dynamic graphs, smart tables and filters

Rules: Conditional treatment of incoming data

Alert: A notification broadcasted based on some rules defined on data

Logger: Logger that takes care of information logging

Support: A UI to dynamically manage the information system and see the system state visually

Each word in this list is inevitably linked to many more. As I work on it (time permitting) one definition a time, let us see where we reach. To reach the skeleton stage, we will define minimal (and final) goals for each list as we reach them.

A Need For Generic Information Systems

Information systems are critical to any business. But what do I mean by information systems? I broadly define them as software entities that read data and perform some useful function on the same. These useful functions can include:

- Data visualization (reports & graphs)
- Aid manual data analysis and model making (with rich data dashboards)
- Generate event based alerts for stakeholders (email, sms, automated call, fax)
- Rule-based data analysis

Further, these systems can be either at real-time or passive.

The umpteen number of permutations mean infinite possibilities. Nevertheless, it is fair to assume that such systems should have some common governing principles as they perform a fairly similar job. The current market has a plethora of information system software that are sector specific and address a small subset according to needs of its users. Yet I have not seen an information engine that addresses all these concerns in a way that is generic and powerful enough. There are a lot of pieces that address some part of the puzzle, but what we are missing is something that takes into account the whole picture.

Such a project will be complex in scope and diverse in application. A modular approach that does not force user to take what he does not need is warranted. It should save the development cost for such systems for a wide array of sectors.

I would propose (naturally) a java based solution using mix of spring, maven, Apache libraries, Flex, OpenMQ, Quartz and PostGres. Making it work efficiently for real-time systems and the scale of the project are two immediate challenges that I see. Making it open-source and charging for elite training/support is the way to go; unless somebody has a better idea :D. In coming days, I will try to put down the basic pegs to work towards. Ta till then!

Writing DOM to XML File on Disk

Just wrote a small program to write a DOM to an XML on the disk using Xerces. The sample class can be seen here. If you are using maven, don't forget to add xerces dependencies:

        <dependency>
            <groupId>xerces</groupId>
            <artifactId>xmlParserAPIs</artifactId>
            <version>2.6.2</version>
        </dependency>
        <dependency>
            <groupId>apache-xerces</groupId>
            <artifactId>xercesImpl</artifactId>
            <version>2.9.1</version>
        </dependency>

For more details, refer Processing XML with Java by Elliotte Rusty Harold