How to filter out unwanted tokens with ParseKit?
I am using PK to try and spoof the .gift file format. I am trying to do the following:
For a line like this: @ "= 2 + 2"
I'm trying to return "2 + 2" without missing out on the problem of determining if the token that goes on that line matches a character and then determining how the output line should be. What I'm trying to do is say that if [PKToken.stringValue isEqualToString: @ "="] pull that value out of the PKTokenizer and then return the rest of the formatted string is still in tact.
Let me know if this was clear enough ...
-. Skylar
source to share
The ParseKit developer is here. Here's my ParseKit marker answer .
First, I must say that simple filtering of individual characters is probably better done using regular expressions than ParseKit.
However, if you are trying to do similar things with a ParqueKit marker , here's how:
NSString *s = @"= 2 + 2";
PKTokenizer *t = [PKTokenizer tokenizerWithString:s];
t.whitespaceState.reportsWhitespaceTokens = YES;
PKToken *eof = [PKToken EOFToken];
PKToken *tok = nil;
NSMutableArray *toks = [NSMutableArray array];
while ((tok = [t nextToken]) != eof) {
if (![tok.stringValue isEqualToString:@"="]) {
[toks addObject:tok];
}
}
NSString *result = [toks componentsJoinedByString:@""];
NSLog(@"%@", result);
source to share
The ParseKit developer is here. Here's my answer, Grammar ParseKit .
Again, I have to say that simple filtering of individual characters is probably better done using regular expressions than ParseKit.
However, if you are trying to do things like this with the ParseKit grammar , here's one way:
My grammar:
@reportsWhitespaceTokens = YES; @start = (reject! | passThru)+; reject = '='; passThru = ~reject;
This !
means to revoke this token . ~
Logical not .
Define this assembler callback:
- (void)parser:(PKParser *)p didMatchPassThru:(PKAssembly *)a {
NSLog(@"%s %@", __PRETTY_FUNCTION__, a);
PKToken *tok = [a pop];
if (!a.target) {
a.target = [NSMutableArray array];
}
[a.target addObject:tok];
}
You can see we are just accumulating passThru tokens in an array stored as a build target.
My driver code:
NSString *g = // fetch grammar above
PKParser *p = [[PKParserFactory factory] parserFromGrammar:g assembler:self];
NSString *s = @"= 2 + 2";
NSArray *toks = [p parse:s];
NSString *result = [toks componentsJoinedByString:@""];
NSLog(@"res '%@'", result);
NSAssert([result isEqualToString:@"2 + 2"], @"");
At the end, we just extract the accumulated tokens (build target) and concatenate them with a string.
source to share