Monday, February 3, 2014

Tools for development process

Collection of tools that have been used a my workplace - which are good to keep for reference


Source code control:
- Mecurial (take sometime to get used to with it, then I really like it).
- GIT, CVS, SVN, clearcase
- Perfoce (not so good for my taste).

Code review tool:
- Rhocode (Good collaboration code review tool for small teams, which can be well integrated with Mecurial).

Static Analysis: 
- Findbugs (http://findbugs.sourceforge.net/)
- Sonaqube (http://www.sonarqube.org/)

Build/Test automation:
- jenkins

Tuesday, March 26, 2013

Virtualbox: How to resize or grow a VDI disk image in one minute

The complicated steps - which you may learn other useful stuffs after going through them:

https://forums.virtualbox.org/viewtopic.php?p=29276#29276

The easiest and simplest method, which you can gain the result in just a minute or less:

1. Open a terminal window (well...here I am using a Macbook Pro).
2. Go to your VM directory (Mine is /Users/lpbinh/VirtualBox VMs/WIndows_7)
3. Issue the command "VBoxManage modifyhd <your VDI name> --resize <size_in_MB>
4. Since I am resizing a Windows VM, I have to boot up the Windows VM and perform the following additional steps:
a. Right click on My Computer, choose Manage.
b. Click on Storage, then Disk Management
c. Right click on the current partition, then select Extend Volumn. Follow the wizard to complete.

Well...take more than a minute, probably two :).

Simplest WebSocket Client

To be written.

Friday, June 15, 2012

Simplest Jetty HttpClient to work in HTTPS

Took me a while to find out how to make HttpClient in Jetty works over Https - The problem is that Jetty HttpClient does not maintain the session nor the cookies for you, and you have to set it manually after being authenticated at the first place.

Here is a simple example to illustrate it:



package Testing;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

import org.eclipse.jetty.client.ContentExchange;
import org.eclipse.jetty.client.HttpClient;
import org.eclipse.jetty.client.HttpExchange;
import org.eclipse.jetty.io.Buffer;



public class JettyHttpClientCookieTest {
    public final static void main(String[] args) throws Exception {
        String loginUri = new String("https://10.176.40.10:8443/admin?op=login&username=<username>&password=<password>");
        String setGroupUri = new String("https://10.176.40.10:8443/admin?op=setGroup&groupId=1");

        if (httpClient == null) {
            httpClient = new HttpClient(); 
        }
     httpClient.setConnectorType(HttpClient.CONNECTOR_SELECT_CHANNEL);  
     try {
         System.out.println("Start the Http client...");
         httpClient.start();    
         
         System.out.println("Login to CMS ...");
         ContentExchange exchange = new ContentExchange(true) {
             protected void onResponseHeader(Buffer name, Buffer value) throws IOException {
                 System.out.println("Buffer name: " + name.toString() + "; Buffer value: " + value.toString());
                 if (name.toString().equals("Set-Cookie")) {
                     cookies.add(value.toString());
                 }
            }
         };
         exchange.setMethod("POST");
         exchange.setURL(loginUri);
      
         httpClient.send(exchange);
         if (exchange.waitForDone() == ContentExchange.STATUS_COMPLETED) {
             System.out.println("ContentExchange content: " + exchange.getResponseContent());
         }
            
         System.out.println("Set the group");
         ContentExchange exchange2 = new ContentExchange(true);
         exchange2.setMethod("POST");
         exchange2.setURL(setGroupUri);
         for (String cookie: cookies) {
             exchange2.setRequestHeader("Cookie", cookie);
         }
         httpClient.send(exchange2);
         if (exchange2.waitForDone() == ContentExchange.STATUS_COMPLETED) {
             System.out.println("ContentExchange content: " + exchange2.getResponseContent());
         }
        
         System.out.println("Fetch Shefl...");
         ContentExchange exchange3 = new ContentExchange(true);
         String fetchShelfUri = new String("https://10.176.40.10:8443/fetch?shelf");
         exchange3.setMethod("GET");
         exchange3.setURL(fetchShelfUri);
         for (String cookie: cookies) {
             exchange3.setRequestHeader("Cookie", cookie);
         }
         httpClient.send(exchange3);
         if (exchange3.waitForDone() == ContentExchange.STATUS_COMPLETED) {
             System.out.println("ContentExchange content: " + exchange3.getResponseContent());
         }
         
         System.out.println("Done");
     }
     catch (Exception e) {
         return;
     }
    }
    
    public static class JettyHttpExchangeExt extends HttpExchange {
        // Examine the response header
        protected void onResponseHeader(Buffer name, Buffer value) throws IOException {
             System.out.println("Buffer name: " + name.toString() + "; Buffer value: " + value.toString());
        }
    }

    public static List<String> cookies = new ArrayList<String>();
    public static HttpClient httpClient = null;
}

Simplest Apache HttpClient to test HTTPS connection


package Testing;

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;

import org.apache.http.Header;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.conn.scheme.Scheme;
import org.apache.http.conn.ssl.SSLSocketFactory;
import org.apache.http.conn.ssl.TrustStrategy;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.util.EntityUtils;

/**
 * This example demonstrates how to create secure connections with a custom SSL
 * context.
 */
public class ClientCustomSSL {

    public final static void main(String[] args) throws Exception {
        DefaultHttpClient httpclient = new DefaultHttpClient();
        try {
            /* Uncomment the below line and proceed with the cert if you have one.
            KeyStore trustStore  = KeyStore.getInstance(KeyStore.getDefaultType());
            FileInputStream instream = new FileInputStream(new File("<my_keystore>"));
            try {
                trustStore.load(instream, "<mypassword to decode my_keystore>".toCharArray());
            } finally {
                try { instream.close(); } catch (Exception ignore) {}
            }
  
            SSLSocketFactory socketFactory = new SSLSocketFactory(trustStore);
             */
            SSLSocketFactory socketFactory = new SSLSocketFactory( new TrustStrategy() {
                // This function below make the session always "trusted" without using the above trustStore.
                public boolean isTrusted(final X509Certificate[] chain, String authType) throws CertificateException {
                    return true;
                }
            }, org.apache.http.conn.ssl.SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);

            Scheme sch = new Scheme("https", 8443, socketFactory);
            httpclient.getConnectionManager().getSchemeRegistry().register(sch);
            HttpPost httppost = new HttpPost("https://10.176.40.10:8443/admin?op=login&username=admin&password=AdminAdmin");
            
            Header headers[] = httppost.getAllHeaders();      
            HttpResponse response = httpclient.execute(httppost);
            HttpEntity entity = response.getEntity();
            headers = response.getAllHeaders();

            System.out.println("----------------------------------------");
            
            System.out.println(response.getStatusLine());
            if (entity != null) {
                System.out.println("Response content length: " + entity.getContentLength());
                BufferedReader br = new BufferedReader(
                        new InputStreamReader((response.getEntity().getContent())));

                String output;
                System.out.println("Output from Server .... \n");
                while ((output = br.readLine()) != null) {
                    System.out.println(output);
                }
                for(Header h:headers){
                    System.out.println(h.getName() + ": " + h.getValue());
                }
              
                // Now do another post to see that the authenticated session persists - no cookie need to send along.
                httppost = new HttpPost("https://10.176.40.10:8443/admin?op=setRbacGroup&groupId=1");
                response = httpclient.execute(httppost);
                entity = response.getEntity();
                headers = response.getAllHeaders();
                System.out.println("Output from Server .... \n");
                br = new BufferedReader(
                        new InputStreamReader((response.getEntity().getContent())));
                while ((output = br.readLine()) != null) {
                    System.out.println(output);
                }             
                // Now do a get to see that the authenticated session persists - no cookie need to send along.
                HttpGet httpget = new HttpGet("https://10.176.40.10:8443/fetch?shelf");
                response = httpclient.execute(httpget);
                entity = response.getEntity();
                headers = response.getAllHeaders();
                System.out.println("Output from Server .... \n");
                br = new BufferedReader(
                        new InputStreamReader((response.getEntity().getContent())));
                while ((output = br.readLine()) != null) {
                    System.out.println(output);
                }
            } 
            EntityUtils.consume(entity);

        } finally {
            // When HttpClient instance is no longer needed,
            // shut down the connection manager to ensure
            // immediate deallocation of all system resources
            httpclient.getConnectionManager().shutdown();
        }
    }
    

}

Friday, April 20, 2012

Java: Redirecting apache log4j to syslog

1. My log4j.properties file for logging all the "INFO" level messages to both the localhost and a local file (which is  /local/log/snmp/snmp.log4j.log.info).  Six months from now, I will wonder to myself about the reason of writing to the local sylog and also a local file -  scroll down to the end for the answer 

/local/release/ESM-1.7.0-T2/snmp # cat log4j.properties
log4j.rootLogger=INFO, SYSLOG, I
#log4j.rootLogger=DEBUG, D

# configure Syslog facility LOCAL appender
log4j.appender.SYSLOG=org.apache.log4j.net.SyslogAppender
log4j.appender.SYSLOG.Threshold=INFO
log4j.appender.SYSLOG.SyslogHost=localhost
#log4j.appender.SYSLOG.Facility=LOCAL1
log4j.appender.SYSLOG.FacilityPrinting=true
log4j.appender.SYSLOG.layout=org.apache.log4j.PatternLayout
log4j.appender.SYSLOG.layout.conversionPattern=[%p] %c:%L - %m%n

# File
log4j.appender.I = org.apache.log4j.RollingFileAppender
log4j.appender.I.File = /local/log/snmp/snmp.log4j.log.info
log4j.appender.D = org.apache.log4j.RollingFileAppender                     
log4j.appender.D.File = /local/log/snmp/snmp.log4j.log.debug
                                                                             
# Control the maximum log file size
log4j.appender.I.MaxFileSize = 1MB 
log4j.appender.D.MaxFileSize = 1MB
# Archive log files (one backup file here)
log4j.appender.I.MaxBackupIndex = 10      
log4j.appender.D.MaxBackupIndex = 10
log4j.appender.I.layout = org.apache.log4j.PatternLayout
log4j.appender.D.layout = org.apache.log4j.PatternLayout
log4j.appender.I.layout.ConversionPattern = [%d{ISO8601}]%5p%6.6r[%t]%x - %C.%M(%F:%L) - %m%n
log4j.appender.D.layout.ConversionPattern = [%d{ISO8601}]%5p%6.6r[%t]%x - %C.%M(%F:%L) - %m%n

2.  Do the following for syslog configuration on the syslog server side:
- Add the corresponding configuration for where the logs will go for the "LOCAL1" facility:

local1.* /var/log/local1.log

- Since Log4j SyslogAppender is using SyslogAppender as the underline class, and SyslogWriter is using DatagramPacket which writes to syslog remotely, the designated syslog daemon needs to enable remote access to it. Make it short - the syslog daemon needs to enable option "-r" in order to receive the logs from Log4j.

- Don't forget to restart syslog service / syslogd daemon after all.

Note that this does not work with the syslog service in busybox. It's a know bug and have not been fixed as of Busybox 1.9.0. In order to have it works in busybox, a different syslog service such as socklog needs to be ported over.


3. Reason why I was writing to both the localhost syslog and a local file: 
- With the current architecture I am working on, syslog service is managed via runit . By default, syslogs are logged into the local host. If user configures a syslog server for the device, those logs will be forwarded to the actual destination.
- so in the corresponding runit script, syslogd/socklog daemon will need to be instantiated with both "-r" and "-R" option.
People are aging, so am I. So I'd like to take this is a a place to dump my work notes over rather messing up my laptop. Thinking of keeping the first page as the "Table of Content", where I can go back and update accordingly for later reference...

I'm glad if someone finds that the notes are helpful - much gladder if you can give your comments/suggestions to make it better.