开发者

How do I create something like a negated character class with a string instead of characters?

开发者 https://www.devze.com 2022-12-23 12:30 出处:网络
I am trying to write a tokenizer for Mustache in Perl.I can easily handle most of the tokens like this:

I am trying to write a tokenizer for Mustache in Perl. I can easily handle most of the tokens like this:

#!/usr/bin/perl

use strict;
use warnings;

my $comment  = qr/ \G \{\{ !  (?<comment>  .+? ) }}              /xs; 
my $variable = qr/ \G \{\{    (?<variable> .+? ) }}              /xs; 
my $text     = qr/ \G         (?<text>     .+? ) (?= \{\{ | \z ) /xs; 
my $tokens   = qr/ $comment | $variable | $text /x;

my $s = do { local $/; <DATA> };

while ($s =~ /$tokens/g) {
    my ($type)    = keys %+;
    (my $contents = $+{$type}) =~ s/\n/\\n/;

    print "type [$type] contents [$contents]\n";
}

__DATA__
{{!this is a comment}}
Hi {{name}}, I like {{thing}}.

But I am running into trouble with the Set Delimiters directive:

#!/usr/bin/perl

use strict;
use warnings;

my $delimiters = qr/ \G \{\{    (?<start> .+? ) = [ ] = (?<end> .+?) }} /xs; 
my $comment    = qr/ \G \{\{ !  (?<comment>  .+? ) }}                   /xs; 
my $variable   = qr/ \G \{\{    (?<variable> .+? ) }}                   /xs; 
my $text       = qr/ \G         (?<text>     .+? ) (?= \{\{ | \z )      /xs; 
my $tokens     = qr/ $comment | $delimiters | $variable | $text /x;

my $s = do { local $/; <DATA> };

while ($s =~ /$tokens/g) {
    for my $type (keys %+) {
        (my $contents = $+{$type}) =~ s/\n/\\n/;

        print "type [$type] contents [$contents]\n";
    }
}

__DATA__
{{!this is a comment}}
Hi {{name}}, I like {{thing}}.
{{(= =)}}

If I change it to

my $delimiters = qr/ \G \{\{ (?<start> [^{]+? ) = [ ] = (?<end> .+?) }} /xs;

It works fine, but the point of the Set Delimiters directive is to change the delimiters, so the code will wind up looking like

my $variable = qr/ \G $start (?<variable> .+? ) $end /xs;

And it is perfectly valid to say {{{== ==}}} (i.e. change the delimiters to {= and =}). What I want, but maybe not what I need, is the ability to say something like (?:not starting string)+?. I figure I am just going to have to give up being clean about it and drop code into the regex to force it to match only what I want. I am trying to avoid that for four reasons:

  1. I don't think it is very clean.
  2. It is marked as experimental.
  3. I am not very familier with it (I think it comes down to (?{CODE}) and returning special values.
  4. I am hoping someone knows some other exotic feature that I am not familiar with that fits the situation better (e.g. (?(condition)yes-pattern|no-pattern)).

Just to make things clear (I hope), I am trying to match a constant length starting delimiter followed by the shortest string that allows a match and does not contain the starting delimiter followed 开发者_如何学Goby a space followed by an equals sign followed by the shortest string that allows a match that ends with the ending delimiter.


Use a negative look-ahead assertion. Something like this:

my $variable = qr/ \G $start (?<variable> (.(?!$end))+ ) $end /xs;


For those who are curious, what follows is the full tokenizer for Mustache written in Perl 5.10 style. Now I just need to write the parser and the renderer.

#!/usr/bin/perl

use 5.010_000;
use strict;
use warnings;

sub gen_tokenizer {
    my ($s, $e) = @_;
    my ($start, $end) = map { quotemeta } $s, $e;

    my $unescaped = "$s $e" eq "{{ }}" ?
        qr/ \G \{{3}    (?<unescaped> .+?)  }{3} /xs :
        qr{ \G $start & (?<unescaped> .+? ) $end }xs;

    return qr{
        $unescaped                                                 |
        \G $start (?:
            !  (?<comment>    .+? )                                |
            >  (?<partial>    .+? )                                |
            \# (?<enum_start> .+? )                                |
            /  (?<enum_stop>  .+? )                                |
            (?<start> (?: . (?! $end ) )+? ) = [ ] = (?<end> .+? ) |
            (?<variable>      .+? )
        ) $end                                                     |
        (?<text> .+? ) (?= $start | \z )
    }xs; 
}

my $template  = do { local $/; <DATA> };
my $tokenizer = gen_tokenizer "{{", "}}";

while ($template =~ /$tokenizer/g) {
    my @types = keys %+;

    if (@types == 1) {
        my  $type     = $types[0];
        (my $contents = $+{$type}) =~ s/\n/\\n/g;

        say "$type: [$contents]";
    } else {
        $tokenizer = gen_tokenizer $+{start}, $+{end};

        say "set_delim: [$+{start} $+{end}]";
    }
}

__DATA__
{{!this is a comment}}
{{{html header}}}
Hi {{name}}, I like {{thing}}.
{{(= =)}}
(#optional)
This will only print if optional is set
(/optional)
(&html footer)
0

精彩评论

暂无评论...
验证码 换一张
取 消