String Tokenizer in Java

I ran into the need to make a String tokenizer with java a few days ago. So here are a few snippets that can be used.

If you would like to include the delimiters use this function:
import java.util.regex.*;
public static String[] tokenize(String str, String pattern)
{
    return Pattern.compile("(?=("+pattern+"))|(?<=("+pattern+"))").split(str);
}
 Example:

tokenize("4+3+43/5-ab", "[\\+\\-/]+");

will return an array like this [4,+,3,+,43,/,5,-,ab]


If you would like to return the array without the delimiters included use this:
import java.util.regex.*;
public static String[] tokenizeSplit(String str, String pattern)
{
    return Pattern.compile(pattern).split(str);
}
 Example: 

tokenizeSplit("4+3+43/5-ab", "[\\+\\-/]+");

will return an array like this [4,3,43,5,ab]