String parsing with StringTokenizer

StringTokenizer class has been defined in java.util package, which allows us to break a string into tokens. i.e.; this class will be very handy, if you would like to split a string with different values separated by a delimiter.

StringTokenizer implements the Enumeration and it comes with 3 constructors

• StringTokenizer(String str) - parses the string str and creates tokens with default delimiter white space (\s or \whitespace)
• StringTokenizer(String str, String delim) - parses the string str and creates tokens with delimiters delim
• StringTokenizer(String str, String delims, boolean returnDelims) - this is the same form as the second constructor apart from returning the delimiters also as a tokens if returnDelims is true

Since, StringTokenizer implements Enumeration interface, we can use hasMoreElements() and hasNext() methods to get the individual string tokens.

import java.util.StringTokenizer;

/**
 * A StringTokenizer demo class
 * @author SANTHOSH REDDY MANDADI
 * @version 1.0
 * @since 23-October-2012
 */
public class StringTokenizerDemo
{
  public static void main(String[] args)
  {
    StringTokenizer tokenizer1 = new StringTokenizer("Hello, I'm Santhosh Reddy");
    StringTokenizer tokenizer2 = new StringTokenizer("Hello, I'm Santhosh Reddy", ",");
    StringTokenizer tokenizer3 = new StringTokenizer("Hello, I'm Santhosh Reddy", ",'", true);
    System.out.println("Displaying the tokens with tokenizer1:");
    System.out.println("Total tokens: "+tokenizer1.countTokens());
    while(tokenizer1.hasMoreTokens()) {
      System.out.println(tokenizer1.nextToken());
    }
    System.out.println("**********************************************");
    System.out.println("Displaying the tokens with tokenizer2:");
    System.out.println("Total tokens: "+tokenizer2.countTokens());
    while(tokenizer2.hasMoreTokens()) {
      System.out.println(tokenizer2.nextToken());
    }
    System.out.println("**********************************************");
    System.out.println("Displaying the tokens with tokenizer3:");
    System.out.println("Total tokens: "+tokenizer3.countTokens());
    while(tokenizer3.hasMoreTokens()) {
      System.out.println(tokenizer3.nextToken());
    }
  }
}

C:\Santhosh>java StringTokenizerDemo
Displaying the tokens with tokenizer1:
Total tokens: 4
Hello,
I'm
Santhosh
Reddy
**********************************************
Displaying the tokens with tokenizer2:
Total tokens: 2
Hello
 I'm Santhosh Reddy
**********************************************
Displaying the tokens with tokenizer3:
Total tokens: 5
Hello
,
 I
'
m Santhosh Reddy