public class FileTokenizer extends ProjectComponent implements Tokenizer
description, location, project| Constructor and Description | 
|---|
| FileTokenizer() | 
| Modifier and Type | Method and Description | 
|---|---|
| java.lang.String | getPostToken()Return the intra-token string | 
| java.lang.String | getToken(java.io.Reader in)Get the complete input as a string | 
clone, getDescription, getLocation, getProject, log, log, setDescription, setLocation, setProjectpublic java.lang.String getToken(java.io.Reader in)
                          throws java.io.IOException
public java.lang.String getPostToken()
getPostToken in interface Tokenizer