Finding Comments in Source Code Using Regular Expressions

Many text editors have advanced find (and replace) features. When I’m programming, I like to use an editor with regular expression search and replace. This feature is allows one to find text based on complex patterns rather than based just on literals. Upon occasion I want to examine each of the comments in my source code and either edit them or remove them. I found that it was difficult to write a regular expression that would find C style comments (the comments that start with /* and end with */) because my text editor does not implement the “non-greedy matching” feature of regular expressions.

First Try

When first attempting this problem, most people consider the regular expression:

/\*.*\*/

This seems the natural way to do it. /\* finds the start of the comment (note that the literal * needs to be escaped because * has a special meaning in regular expressions), .* finds any number of any character, and \*/ finds the end of the expression.

The first problem with this approach is that .* does not match new lines.

/* First comment
 first comment—line two*/
/* Second comment */

Second Try

This can be overcome easily by replacing the . with [^] (in some regular expression packages) or more generally with (.|[\r\n]):

/\*(.|[\r\n])*\*/

This reveals a second, more serious, problem—the expression matches too much. Regular expressions are greedy, they take in as much as they can. Consider the case in which your file has two comments. This regular expression will match them both along with anything in between:

start_code();
/* First comment */
more_code();
/* Second comment */
end_code();

Third Try

To fix this, the regular expression must accept less. We cannot accept just any character with a ., we need to limit the types of characters that can be in our expressions:

/\*([^*]|[\r\n])*\*/

This simplistic approach doesn’t accept any comments with a * in them.

/*
 * Common multi-line comment style.
 */
/* Second comment */

Fourth Try

This is where it gets tricky. How do we accept a * without accepting the * that is part of the end comment? The solution is to still accept any character that is not *, but also accept a * and anything that follows it provided that it isn’t followed by a /:

/\*([^*]|[\r\n]|(\*([^/]|[\r\n])))*\*/

This works better but again accepts too much in some cases. It will accept any even number of *. It might even accept the * that is supposed to end the comment.

start_code();
/****
 * Common multi-line comment style.
 ****/
more_code();
/*
 * Another common multi-line comment style.
 */
end_code();

Fifth Try

What we tried before will work if we accept any number of * followed by anything other than a * or a /:

/\*([^*]|[\r\n]|(\*+([^*/]|[\r\n])))*\*/

Now the regular expression does not accept enough again. Its working better than ever, but it still leaves one case. It does not accept comments that end in multiple *.

/****
 * Common multi-line comment style.
 ****/
/****
 * Another common multi-line comment style.
 */

Solution

Now we just need to modify the comment end to allow any number of *:

/\*([^*]|[\r\n]|(\*+([^*/]|[\r\n])))*\*+/

We now have a regular expression that we can paste into text editors that support regular expressions. Finding our comments is a matter of pressing the find button. You might be able to simplify this expression somewhat for your particular editor. For example, in some regular expression implementations, [^] assumes the [\r\n] and all the [\r\n] can be removed from the expression.

This is easy to augment so that it will also find // style comments:

(/\*([^*]|[\r\n]|(\*+([^*/]|[\r\n])))*\*+/)|(//.*)

Tool Expression and Usage Notes
nedit (/\*([^*]|[\r\n]|(\*+([^*/]|[\r\n])))*\*+/)|(//.*)
Ctrl+F to find, put in expression, check the Regular Expression check box.
[^] does not include new line
grep (/\*([^*]|(\*+[^*/]))*\*+/)|(//.*)
grep -E “(/\*([^*]|(\*+[^*/]))*\*+/)|(//.*)” <files>
Does not support multi-line comments, will print out each line that completely contains a comment.
perl /((?:\/\*(?:[^*]|(?:\*+[^*\/]))*\*+\/)|(?:\/\/.*))/
perl -e “$/=undef;print<>=~/((?:\/\*(?:[^*]|(?:\*+[^*\/]))*\*+\/)|(?:\/\/.*))/g;” < <file>
Prints out all the comments run together. The (?: notation must be used for non-capturing parenthesis. Each / must be escaped because it delimits the expression. $/=undef; is used so that the file is not matched line by line like grep.
Java "(?:/\\*(?:[^*]|(?:\\*+[^*/]))*\\*+/)|(?://.*)"
System.out.println(sourcecode.replaceAll(“(?:/\\*(?:[^*]|(?:\\*+[^*/]))*\\*+/)|(?://.*)”,””));
Prints out the contents of the string sourcecode with the comments removed. The (?: notation must be used for non-capturing parenthesis. Each \ must be escaped in a Java String.

An Easier Method

Non-greedy Matching

Most regular expression packages support non-greedy matching. This
means that the pattern will only be matched if there is no other choice.
We can modify our second try to use the non-greedy matcher *? instead of the greedy matcher *. With this new tool, the middle of our comment will only match if it doesn’t match the end:

/\*(.|[\r\n])*?\*/

Tool Expression and Usage Notes
nedit /\*(.|[\r\n])*?\*/
Ctrl+F to find, put in expression, check the Regular Expression check box.
[^] does not include new line
grep /\*.*?\*/
grep -E ‘/\*.*?\*/’ <file>
Does not support multi-line comments, will print out each line that completely contains a comment.
perl /\*(?:.|[\r\n])*?\*/
perl -0777ne ‘print m!/\*(?:.|[\r\n])*?\*/!g;’ <file>
Prints out all the comments run together. The (?: notation must be used for non-capturing parenthesis.
/ does not have to be escaped because ! delimits the expression.
-0777 is used to enable slurp mode and -n enables automatic reading.
Java "/\\*(?:.|[\\n\\r])*?\\*/"
System.out.println(sourcecode.replaceAll(“/\\*(?:.|[\\n\\r])*?\\*/”,””));
Prints out the contents of the string sourcecode with the comments removed. The (?: notation must be used for non-capturing parenthesis. Each \ must be escaped in a Java String.

Caveats

Comments Inside Other Elements

Although our regular expression describes c-style comments very well, there are still problems when something
appears to be a comment but is actually part of a larger element.

someString = "An example comment: /* example */";

// The comment around this code has been commented out.
// /*
some_code();
// */

The solution to this is to write regular expressions that describe each of the possible larger elements, find these as well, decide what type of element each is, and discard the ones that are not comments. There are tools called lexers or tokenizers that can help with this task. A lexer accepts regular expressions as input, scans a stream, picks out tokens that match the regular expressions, and classifies the token based on which expression it matched. The greedy property of regular expressions is used to ensure the longest match. Although writing a full lexer for C is beyond the scope of this document, those interested should look at lexer generators such as Flex and JFlex.

时间: 2024-11-02 23:26:23

Finding Comments in Source Code Using Regular Expressions的相关文章

Using Regular Expressions in Python

1. 反斜杠的困扰(The Backslash) 有时候需要匹配的文本带有'\',如'\python',因为正则表达式有些特殊字符有特殊意义,所以需要前面加上'\'来消除特殊意义,这里匹配的正则表达式是'\\python',这时候如果要编译这个正则表达式需要re.compile('\\\\python'),因为在传递字符串的时候python本身就需要用'\\'来表示'\',也就造成了反斜杠的泛滥. 使用前缀'r'可以解决这个问题:’r'之后的'\'只是这个字符本身而没有特殊意义,比如r'\n'表

PCRE Perl Compatible Regular Expressions Learning

catalog 1. PCRE Introduction 2. pcre2api 3. pcre2jit 4. PCRE Programing 1. PCRE Introduction The PCRE library is a set of functions that implement regular expression pattern matching using the same syntax and semantics as Perl 5. PCRE has its own nat

Eloquent JavaScript #09# Regular Expressions

索引 Notes js创建正则表达式的两种方式 js正则匹配方式(1) 字符集合 重复匹配 分组(子表达式) js正则匹配方式(2) The Date class 匹配整个字符串 Choice patterns 正则匹配的机制 回溯Backtracking Replace 贪婪匹配Greed 动态构建正则表达式 Search The lastIndex property 遍历匹配项 解析INI文件 国际字符 Excercise Regexp golf Quoting style Numbers

Python re module (regular expressions)

regular expressions (RE) 简介 re模块是python中处理正在表达式的一个模块 1 r"""Support for regular expressions (RE). 2 3 This module provides regular expression matching operations similar to 4 those found in Perl. It supports both 8-bit and Unicode strings; b

Thinking in Java 4th Edition Source Code

Thinking in Java 4th Edition Source Code Instructions for downloading, installing and testing the source code Download the source code zip file from this link. Create a directory in which to install the code. For these instructions, we'll refer to th

8 Regular Expressions You Should Know

Regular expressions are a language of their own. When you learn a new programming language, they're this little sub-language that makes no sense at first glance. Many times you have to read another tutorial, article, or book just to understand the "s

Tree - AdaBoost with sklearn source code

In the previous post we addressed some issue of decision tree, including instability, lack of smoothness, sensitivity to data, and etc. One solution is Boosting Method. In simple words Boosting combines multiple weak learners to get a powerful predic

CRC32 Source Code

/* The Quest Operating System * Copyright (C) 2005-2010 Richard West, Boston University * * This program is free software: you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by * the Free Softwar

【开源】海看源代码统计工具 Haikan Source Code Counter

Haikan Source Code Counter 海看源代码统计工具 BY 杭州海看网络科技有限公司 ------------------- github上的地址: https://github.com/haikanwhf/HaikanSourceCodeCounter ------------------ 海看源代码统计工具V1.7.rar