I'm trying to split English sentences correctly, and I came up with the unholy regex below:
(?<!\d|([A-Z]\.)|(\.[a-z]\.)|(\.\.\.)|etc\.|[Pp]rof\.|[Dd]r\.|[Mm]rs\.|[Mm]s\.|[Mm]z\.|[Mm]me\.)(?<=([\.!?])|(?<=([\.!?][\'\"])))[\s]+?(?=[\S])'
The problem is, Python keeps raising the following error:
Traceback (most recent call last):
File "", line 1, in
File "sp.py", line 55, in analyze
self.sentences = re.split(god_awful_regex, self.inputstr.strip())
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/re.py", line 165, in split
return _compile(pattern, 0).split(string, maxsplit)
File "/System/Library/Frameworks/Python.framework/Versions/2.6/li开发者_如何学Pythonb/python2.6/re.py", line 243, in _compile
raise error, v # invalid expression
sre_constants.error: look-behind requires fixed-width pattern
Why is this not a valid, fixed-width regex? I'm not using any repeat characters (* or +), just |.
EDIT @Anomie solved the problem - thanks a ton! Unfortunately, I cannot make the new expression balance:
(?<!(\d))(?<![A-Z]\.)(?<!\.[a-z]\.)(?<!(\.\.\.))(?<!etc\.)(?<![Pp]rof\.)(?<![Dd]r\.)(?<![Mm]rs\.)(?<![Mm]s\.)(?<![Mm]z\.)(?<![Mm]me\.)(?:(?<=[\.!?])|(?<=[\.!?][\'\"\]))[\s]+?(?=[\S])
is what I have now. The number of ('s matches the number of ('s, though:
>>> god_awful_regex = r'''(?<!(\d))(?<![A-Z]\.)(?<!\.[a-z]\.)(?<!(\.\.\.))(?<!etc\.)(?<![Pp]rof\.)(?<![Dd]r\.)(?<![Mm]rs\.)(?<![Mm]s\.)(?<![Mm]z\.)(?<![Mm]me\.)(?:(?<=[\.!?])|(?<=[\.!?][\'\"\]))[\s]+?(?=[\S])'''
>>> god_awful_regex.count('(')
17
>>> god_awful_regex.count(')')
17
>>> god_awful_regex.count('[')
13
>>> god_awful_regex.count(']')
13
Any more ideas?
Consider this subexpression:
(?<=([\.!?])|(?<=([\.!?][\'\"])))
The left side of the | is one character, while the right size is zero. You have the same issue in your larger negative look-behind too, it could be 1, 2, 3, 4, or 5 characters.
Logically, a negative look-behind of (?<!A|B|C)
should be equivalent to a series of look-behinds (?<!A)(?<!B)(?<!C)
. A positive look-behind of (?<=A|B|C)
should be equivalent to (?:(?<=A)|(?<=B)|(?<=C))
.
This doesn't answer your question. However, if you want to split a text into sentences, you might want to take a look at nltk, which include beside many other things a PunktSentenceTokenizer. Here is some example tokenizer:
""" PunktSentenceTokenizer
A sentence tokenizer which uses an unsupervised algorithm to build a model
for abbreviation words, collocations, and words that start sentences; and then
uses that model to find sentence boundaries. This approach has been shown to
work well for many European languages. """
from nltk.tokenize.punkt import PunktSentenceTokenizer
tokenizer = PunktSentenceTokenizer()
print tokenizer.tokenize(__doc__)
# [' PunktSentenceTokenizer\n\nA sentence tokenizer which uses an unsupervised
# algorithm to build a model\nfor abbreviation words, collocations, and words
# that start sentences; and then\nuses that model to find sentence boundaries.',
# 'This approach has been shown to\nwork well for many European languages. ']
It looks like you might be using the repeat chacters near the end:
[\s]+?
Unless I'm reading that wrong.
UPDATE
Or vertical bar as nightcracker mentioned, and the first answer to this question seems to confirm: determine if regular expression only matches fixed-length strings
精彩评论