Using regex to extract information from a string

This is a follow-up and complication to this question: Extracting contents of a string within parentheses.

In that question I had the following string --

"Will Farrell (Nick Hasley), Rebecca Hall (Samantha)"

And I wanted to get a list of tuples in the form of (actor, character) --

[('Will Farrell', 'Nick Hasley'), ('Rebecca Hall', 'Samantha')]

To generalize matters, I have a slightly more complicated string, and I need to extract the same information. The string I have is --

"Will Ferrell (Nick Halsey), Rebecca Hall (Samantha), Glenn Howerton (Gary), 
with Stephen Root and Laura Dern (Delilah)"

I need to format this as follows:

[('Will Farrell', 'Nick Hasley'), ('Rebecca Hall', 'Samantha'), ('Glenn Howerton', 'Gary'),
('Stephen Root',''), ('Lauren Dern', 'Delilah')]

I know I can replace the filler words (with, and, &, etc.), but can't quite figure out how to add a blank entry -- '' -- if there is no character name for the actor (in this case Stephen Root). What would be the best way to go about doing this?

Finally, I need to take into account if an actor has multiple roles, and build a tuple for each role the actor has. The final string I hav开发者_Go百科e is:

"Will Ferrell (Nick Halsey), Rebecca Hall (Samantha), Glenn Howerton (Gary, Brad), with
 Stephen Root and Laura Dern (Delilah, Stacy)"

And I need to build a list of tuples as follows:

[('Will Farrell', 'Nick Hasley'), ('Rebecca Hall', 'Samantha'), ('Glenn Howerton', 'Gary'),    
 ('Glenn Howerton', 'Brad'), ('Stephen Root',''), ('Lauren Dern', 'Delilah'), ('Lauren Dern', 'Stacy')]

Thank you.

---

import re
credits = """Will Ferrell (Nick Halsey), Rebecca Hall (Samantha), Glenn Howerton (Gary, Brad), with
 Stephen Root and Laura Dern (Delilah, Stacy)"""

# split on commas (only if outside of parentheses), "with" or "and"
splitre = re.compile(r"\s*(?:,(?![^()]*\))|\bwith\b|\band\b)\s*")

# match the part before the parentheses (1) and what's inside the parens (2)
# (only if parentheses are present)
matchre = re.compile(r"([^(]*)(?:\(([^)]*)\))?")

# split the parts inside the parentheses on commas
splitparts = re.compile(r"\s*,\s*")

characters = splitre.split(credits)
pairs = []
for character in characters:
    if character:
        match = matchre.match(character)
        if match:
            actor = match.group(1).strip()
            if match.group(2):
                parts = splitparts.split(match.group(2))
                for part in parts:
                    pairs.append((actor, part))
            else:
                pairs.append((actor, ""))

print(pairs)

Output:

[('Will Ferrell', 'Nick Halsey'), ('Rebecca Hall', 'Samantha'), 
 ('Glenn Howerton', 'Gary'), ('Glenn Howerton', 'Brad'), ('Stephen Root', ''), 
 ('Laura Dern', 'Delilah'), ('Laura Dern', 'Stacy')]

---

Tim Pietzcker's solution can be simplified to (note that patterns are modified too):

import re
credits = """   Will Ferrell (Nick Halsey), Rebecca Hall (Samantha), Glenn Howerton (Gary, Brad), with
 Stephen Root and Laura Dern (Delilah, Stacy)"""

# split on commas (only if outside of parentheses), "with" or "and"
splitre = re.compile(r"(?:,(?![^()]*\))(?:\s*with)*|\bwith\b|\band\b)\s*")

# match the part before the parentheses (1) and what's inside the parens (2)
# (only if parentheses are present)
matchre = re.compile(r"\s*([^(]*)(?<! )\s*(?:\(([^)]*)\))?")

# split the parts inside the parentheses on commas
splitparts = re.compile(r"\s*,\s*")

pairs = []
for character in splitre.split(credits):
    gr = matchre.match(character).groups('')
    for part in splitparts.split(gr[1]):
        pairs.append((gr[0], part))

print(pairs)

Then:

import re
credits = """   Will Ferrell (Nick Halsey), Rebecca Hall (Samantha), Glenn Howerton (Gary, Brad), with
 Stephen Root and Laura Dern (Delilah, Stacy)"""

# split on commas (only if outside of parentheses), "with" or "and"
splitre = re.compile(r"(?:,(?![^()]*\))(?:\s*with)*|\bwith\b|\band\b)\s*")

# match the part before the parentheses (1) and what's inside the parens (2)
# (only if parentheses are present)
matchre = re.compile(r"\s*([^(]*)(?<! )\s*(?:\(([^)]*)\))?")

# split the parts inside the parentheses on commas
splitparts = re.compile(r"\s*,\s*")

gen = (matchre.match(character).groups('') for character in splitre.split(credits))

pp = [ (gr[0], part) for gr in gen for part in splitparts.split(gr[1])]

print pp

The trick is to use groups('') with an argument ''

---

What you want is identify sequences of words starting with a capital letter, plus some complications (IMHO you cannot assume each name is made of Name Surname, but also Name Surname Jr., or Name M. Surname, or other localized variation, Jean-Claude van Damme, Louis da Silva, etc.).

Now, this is likely to be overkill for the sample input you posted, but as I wrote above I assume things will soon get messy, so I would tackle this using nltk.

Here's a very crude and not very well tested snippet, but it should do the job:

import nltk
from nltk.chunk.regexp import RegexpParser

_patterns = [
    (r'^[A-Z][a-zA-Z]*[A-Z]?[a-zA-Z]+.?$', 'NNP'),  # proper nouns
    (r'^[(]$', 'O'),
    (r'[,]', 'COMMA'),
    (r'^[)]$', 'C'),
    (r'.+', 'NN')                                   # nouns (default)
]

_grammar = """
        NAME: {<NNP> <COMMA> <NNP>}
        NAME: {<NNP>+}
        ROLE: {<O> <NAME>+ <C>}
        """    
text = "Will Ferrell (Nick Halsey), Rebecca Hall (Samantha), Glenn Howerton (Gary, Brad), with Stephen Root and Laura Dern (Delilah, Stacy)"
tagger = nltk.RegexpTagger(_patterns)    
chunker = RegexpParser(_grammar)
text = text.replace('(', '( ').replace(')', ' )').replace(',', ' , ')
tokens = text.split()
tagged_text = tagger.tag(tokens)
tree = chunker.parse(tagged_text)

for n in tree:
    if isinstance(n, nltk.tree.Tree) and n.node in ['ROLE', 'NAME']: 
        print n

# output is:
# (NAME Will/NNP Ferrell/NNP)
# (ROLE (/O (NAME Nick/NNP Halsey/NNP) )/C)
# (NAME Rebecca/NNP Hall/NNP)
# (ROLE (/O (NAME Samantha/NNP) )/C)
# (NAME Glenn/NNP Howerton/NNP)
# (ROLE (/O (NAME Gary/NNP ,/COMMA Brad/NNP) )/C)
# (NAME Stephen/NNP Root/NNP)
# (NAME Laura/NNP Dern/NNP)
# (ROLE (/O (NAME Delilah/NNP ,/COMMA Stacy/NNP) )/C)

You must then process the tagged output and put names and roles in a list instead of printing, but you get the picture.

What we do here is do a first pass where we tag each token according to the regex in _patterns, and then do a second pass to build more complex chunks according to your simple grammar. You can complicate the grammar and the patterns as you want, ie. catch variations of names, messy inputs, abbreviations, and so on.

I think doing this with a single regex pass is going to be a pain for non-trivial inputs.

Otherwise, Tim's solution is solving the issue nicely for the input you posted, and without the nltk dependency.

---

In case you want a non-regex solution ... (Assumes no nested parenthesis.)

in_string = "Will Ferrell (Nick Halsey), Rebecca Hall (Samantha), Glenn Howerton (Gary, Brad), with Stephen Root and Laura Dern (Delilah, Stacy)"    

in_list = []
is_in_paren = False
item = {}
next_string = ''

index = 0
while index < len(in_string):
    char = in_string[index]  

    if in_string[index:].startswith(' and') and not is_in_paren:
        actor = next_string
        if actor.startswith(' with '):
            actor = actor[6:]
        item['actor'] = actor
        in_list.append(item)
        item = {}
        next_string = ''
        index += 4    
    elif char == '(':
        is_in_paren = True
        item['actor'] = next_string
        next_string = ''    
    elif char == ')':
        is_in_paren = False
        item['part'] = next_string
        in_list.append(item)
        item = {}                 
        next_string = ''
    elif char == ',':
        if is_in_paren:
            item['part'] = next_string
            next_string = ''
            in_list.append(item)
            item = item.copy()
            item.pop('part')                
    else:
        next_string = "%s%s" % (next_string, char)

    index += 1


out_list = []
for dict in in_list:
    actor = dict.get('actor')
    part = dict.get('part')

    if part is None:
        part = ''

    out_list.append((actor.strip(), part.strip()))

print out_list

Output: [('Will Ferrell', 'Nick Halsey'), ('Rebecca Hall', 'Samantha'), ('Glenn Howerton', 'Gary'), ('Glenn Howerton', 'Brad'), ('Stephen Root', ''), ('Laura Dern', 'Delilah'), ('Laura Dern', 'Stacy')]