I tested with web.mit.edu again this morning, and strangely, the exception that appeared yesterday is gone. I didn't have time to test it fully yesterday, so I'm sorry that I don't know what it was that is causing the exception.
I fixed some other exceptions that is caused by improving the fsm but not handling all possible cases, such as: I will ignore all < &#...; etc. special characters. There are cases that these characters are trailing a normal keyword. I didn't think of this before hand, and it caused exception to be thrown. Though I fixed this problem, I am sure there are many more out there that could be potentially problematic. |