C::Tokenize Reduce a C file to a series of tokens
Download

C::Tokenize Ranking & Summary

C::Tokenize Tags

Perl module perl tokens

C::Tokenize Description

C::Tokenize is a Perl module to reduce a C file to a series of tokens.REGULAR EXPRESSIONSThe regular expressions can be imported using, for example, use C::Tokenize '$cpp_re'to import $cpp_re.None of the regular expressions does any capturing. If you want to capture, add your own parentheses around the regular expression.$trad_comment_re Match /* */ comments.$cxx_comment_re Match // comments.$comment_re Match both /* */ and // comments.$cpp_re Match a C preprocessor instruction.$char_const_re Match a character constant, such as 'a' or '\-'.$operator_re Match an operator such as + or --.$number_re Match a number, either integer, floating point, or hexadecimal. Does not do octal yet.$word_re Match a word, such as a function or variable name or a keyword of the language.$grammar_re Match other syntactic characters such as { or [.$single_string_re Match a single C string constant such as "this".$string_re Match a full-blown C string constant, including compound strings "like" "this".decomment my $out = decomment ('/* comment */'); # $out = " comment ";Remove the comments from a string.tokenize my $tokens = tokenize ($file);Convert $file into a series of tokens.Each token containsleading Leading whitespacename$name The value of the type, e.g. $token-{comment}> if $token-{name}> equals 'comment'.Product's homepage

C::Tokenize Related Software