Home > Code Samples, Essentials > How to parse source code using the DXCore integrated code parsers

How to parse source code using the DXCore integrated code parsers

April 11th, 2011

There are times when you need to parse specific source files or blocks of code. Obviously, the DXCore Framework has many built-in parsers for various programming languages. They can be used inside the Visual Studio environment, or outside an IDE in any other application type, such as a Console App, for example. Later, this kind of app (a Console App) can be used in the project building process for code validation, code clean-up, automatic refactoring and any other task.

Let’s see how can we parse the code inside a DXCore plug-in when it is loaded into the Visual Studio IDE. Here, we can use the Language and the Source Model DXCore services with several APIs specific for the parsing of the code:

  • Parsing of the files outside Visual Studio:

CodeRush.Language.Parse(“FileNameGoesHere”) – parses the specified source file on disk and returns the LanguageElement that specifies it (most likely, a SourceFile instance).

  • Parsing of the opened text documents (opened source files):

ParseActiveDocument, Parse(TextDocument) or ParseDocument(TextDocument) (from the Language DXCore service) – parses the specified text document (an opened source file) and returns the LanguageElement that specifies it (a SourceFile instance).

ParseIfTextChanged (from the SourceModel DXCore service) – parses the active or the specified document if the text has been changed (but not necessarily committed) since the last parse. Plug-in authors can call this method to ensure that the DXCore structural image is in sync with the file. For example, you might call this immediately after some text edits.

  • Specific parsing methods from the Language service:

ParseExpression – parses an expression from the given string.

ParseString – parses the specified string and returns the LanguageElement that specifies it.

ParseTypeReferenceExpression – parses a type reference expression from the given string.

  • Other Parse method overloads:

These take additional parameters, such as ParserContext, RegionDirective, CompilerDirective, SourceRange, TextStringCollection, etc. – these methods parse the specified source range of the given text document with the given context, and returns the LanguageElement that specifies it. Nodes parsed in the specified range will be appended to the end of the context’s nodes. Does not trigger the BeforeParse or AfterParse events, nor does this method call BindToCode – the calling client code must do that (this allows the calling code to bind only the nodes within the parse range, and also append any trailing nodes to the end of the newly-parsed nodes).

For example, let’s imagine we’re going to release an open source project, but it is required to clean-up the source files from the comments. Here’s a sample on how to achieve this for a specific file:

string fileName = "FileNameGoesHere";
SourceFile parsedFile = CodeRush.Language.Parse(fileName) as SourceFile;
if (parsedFile != null)
{
  CleanUpFromComments(parsedFile);
}

// ...

/// <summary>
/// Removes all comments from the given file. Uses DXCore element enumerations methods and the DXCore File service to change the file.
/// </summary>
/// <param name="parsedFile">The file to clean-up.</param>
private void CleanUpFromComments(SourceFile parsedFile)
{
  LanguageElementType[] commentTypes = new LanguageElementType[] { LanguageElementType.Comment, LanguageElementType.XmlDocComment };
  ElementEnumerable elementEnumerable = new ElementEnumerable(parsedFile, commentTypes, true);
  FileChangeCollection fileChanges = new FileChangeCollection();
  foreach (LanguageElement comment in elementEnumerable)
    fileChanges.Add(new FileChange(parsedFile.FilePath, comment.Range, String.Empty));
  CodeRush.File.ApplyChanges(fileChanges);
}

Show Visual Basic code… »

Dim fileName As String = "FileNameGoesHere"
Dim parsedFile As SourceFile = CType(CodeRush.Language.Parse(fileName), SourceFile)
If parsedFile IsNot Nothing Then
  CleanUpFromComments(parsedFile)
End If

' ...

''' <summary>
''' Removes all comments from the given file. Uses DXCore element enumerations methods and the DXCore File service to change the file.
''' </summary>
''' <param name="parsedFile">The file to clean-up.</param>
Private Sub CleanUpFromComments(ByVal parsedFile As SourceFile)
  Dim commentTypes as LanguageElementType() = new LanguageElementType() { LanguageElementType.Comment, LanguageElementType.XmlDocComment }
  Dim elementEnumerable As ElementEnumerable = New ElementEnumerable(parsedFile, LanguageElementType.Comment, True)
  Dim fileChanges As New FileChangeCollection()
  For Each comment As LanguageElement In elementEnumerable
    fileChanges.Add(New FileChange(parsedFile.FilePath, comment.Range, String.Empty))
  Next
  CodeRush.File.ApplyChanges(fileChanges)
End Sub

Don’t forget, that you can also use the DXCore standalone parser assembly (DevExpress.DXCore.Parser.dll), to parse the source code outside Visual Studio. Here’s a sample above, corrected for using the DXCore parsers in any type of application:

string fileName = "FileNameGoesHere";
string extension = Path.GetExtension(fileName);
ParserBase parser = ParserFactory.CreateParserForFileExtension(extension);
if (parser != null)
{
  SourceFile parsedFile = parser.ParseFile(fileName) as SourceFile;
  if (parsedFile != null)
  {
    CleanUpFromComments(parsedFile);
  }
}

Show Visual Basic code… »

Dim fileName As String = "FileNameGoesHere"
Dim extension As String = Path.GetExtension(fileName)
Dim parser As ParserBase = ParserFactory.CreateParserForFileExtension(extension)
If parser IsNot Nothing Then
  Dim parsedFile As SourceFile = CType(parser.ParseFile(fileName), SourceFile)
  If parsedFile IsNot Nothing Then
    CleanUpFromComments(parsedFile)
  End If
End If

However, we can’t use DXCore services outside Visual Studio. So, in the sample above, the File service won’t be available for modification of files. We will use another technique in this case – for example, directly change the text of the file.

There are two DXCore plug-ins for the CSharp and Visual Basic languages, and two Windows Forms Applications for the same languages are attached as a sample (47,780 bytes). The Windows Forms Application projects uses a different mechanism for changing files, you are welcome to use any algorithm by your preference.

—–
Products: DXCore
Versions: 10.2 and up
VS IDEs: any
Updated: Apr/12/2011
ID: D074

Similar Posts:

  1. Le
    April 13th, 2011 at 01:28 | #1

    When I build this I’m getting this error:

    “Unable to copy file “obj\Debug\ParseSourceCodeSampleCS.dll” to “E:\Addins\2011.1\bin\PlugIns\ParseSourceCodeSampleCS.dll”. Could not find a part of the path ‘E:\Addins\2011.1\bin\PlugIns’.”

    Checked the build output path and the post build events for all projects and can’t seem to find where this is…

  2. Le
    April 13th, 2011 at 01:54 | #2

    Solved,

    Go to:

    Project>Properties>Build>Output>Output path:

    Click browse and remap to the exact same path that is already there. For some reason even though the path is relative here it is stored as absolute in the project file and you just have to remap it so that it will store the correct absolute path for your machine.

  3. April 13th, 2011 at 10:32 | #3

    @ Le
    Thanks for sharing it!