Validation during Continuous Integration and Delivery

Use the same validations as in the IDE in your CI/CD setup.

Qodana to run the IntelliJ AsciiDoc plugin in CD/CD

JetBrains published Qodana that wraps the smart features of IntelliJ into a container, so it can be used on the command line as a Docker container.

For AsciiDoc files, it will highlight all the problems that are highlighted in your IDE as well, including broken links as well as grammar errors.

qodana concepts.dio
Figure 1. Overview Qodana process

The process is as follows:

  1. Read all or parts of your project sources. This can be source code in any programming language or documentation like AsciiDoc.

  2. Process the files like with all the inspections known from IntelliJ. You can add plugins from the Marketplace like in your IDE to support the languages you need.

  3. Generate a report as HTML to be used in a browser, as well as a SARIF report that can be processed by GitHub Code Scanning or other subsequent steps in your CI/CD pipeline.

Example output

HTML report

See below for an HTML report that Qodana generates. With the GitHub Action setup below, it will be attached as a ZIP file to workflow run instance.

qodana html report
Figure 2. Qodana report with AsciiDoc inspection warnings

Pull request annotation

See below for a screenshot of an annotated GitHub pull request. Using the SARIF integration, GitHub can compare the pull request with the base branch and will show only new problems introduced by the pull request.

These examples show AsciiDoc syntax errors as examples. This setup shows Grammar in the same way.

qodana pullrequest status
Figure 3. Status as shown in the pull request

Once the user clicks on the details of the code scanning, they see a list of all new violations.

qodana codescanning results
Figure 4. Details of GitHub Code Scanning

Once the user clicks on one of the annotations, GitHub shows the code.

qodana pullrequest annotation
Figure 5. Annotation in the source code

Example setup for GitHub Actions

This describes the setup used for this plugin. With the benefit of being a real-world scenario, some steps might be more complicated than a simple setup.

While being production ready, Qodana is still in an early stage. Today some workaround have been applied to make it work with GitHub Actions and GitHub Code Scanning, future versions of Qodana will provide them out-of-the box.

  1. Setup Qodana configuration file. It is placed in the root folder where it will run.

    doc/qodana.yaml
    profile:
      path: asciidoc-inspection.xml
  2. Setup file with all inspections to run. It is placed in the root folder where it will run. For this setup all inspections are specified one-by-one. Alternative setups could configure only deviations from a standard configuration.

    doc/asciidoc-inspection.yaml
    <component name="InspectionProjectProfileManager">
      <profile version="1.0" is_locked="true">
        <!-- GrazieInspection normally has TYPO preference -->
        <inspection_tool class="GrazieInspection" enabled="true" level="WARNING" enabled_by_default="true" />
        <inspection_tool class="AsciiDocAnchorWithoutId" enabled="true" level="WARNING" enabled_by_default="true" />
        <inspection_tool class="AsciiDocAttributeContinuation" enabled="true" level="WARNING" enabled_by_default="true" />
        <inspection_tool class="AsciiDocBlockMacroShouldBeInlineMacro" enabled="true" level="WARNING" enabled_by_default="true" />
        <inspection_tool class="AsciiDocDescriptionExists" enabled="true" level="WARNING" enabled_by_default="true" />
        <inspection_tool class="AsciiDocDescriptionLength" enabled="true" level="WARNING" enabled_by_default="true" />
        <inspection_tool class="AsciiDocHeadingStyle" enabled="true" level="WARNING" enabled_by_default="true" />
        <inspection_tool class="AsciiDocHorizontalRule" enabled="true" level="WARNING" enabled_by_default="true" />
        <inspection_tool class="AsciiDocInlineMacroShouldBeBlockOrPreprocessorMacro" enabled="true" level="WARNING" enabled_by_default="true" />
        <inspection_tool class="AsciiDocLinkResolve" enabled="true" level="ERROR" enabled_by_default="true" />
        <inspection_tool class="AsciiDocListingStyle" enabled="true" level="WARNING" enabled_by_default="true" />
        <inspection_tool class="AsciiDocPageBreak" enabled="true" level="WARNING" enabled_by_default="true" />
        <inspection_tool class="AsciiDocReferencePattern" enabled="true" level="ERROR" enabled_by_default="true" />
        <inspection_tool class="AsciiDocXrefWithFileExtension" enabled="true" level="WARNING" enabled_by_default="true" />
        <inspection_tool class="AsciiDocXrefWithNaturalCrossReference" enabled="true" level="WARNING" enabled_by_default="true" />
        <inspection_tool class="AsciiDocAttributeShouldBeDefined" enabled="true" level="WARNING" enabled_by_default="true" />
      </profile>
    </component>
  3. Add a workflow the GitHub repository by adding the following file (see docs.yml in the GitHub repo):

    .github/workflows/docs.yml
    name: Qodana Documentation validation
    on:
      push:
        branches-ignore:
          - 'dependabot/**'
        paths:
          - 'doc/**'
          - '.github/workflows/docs.yml'
          - 'qodana.yaml'
      pull_request:
        paths:
          - 'doc/**'
          - '.github/workflows/docs.yml'
          - 'qodana.yaml'
    jobs:
      # Verify built plugin using IntelliJ Plugin Verifier tool
      # Requires build job to be passed
      qodana-docs:
        name: Qodana Docs
        runs-on: ubuntu-latest
        permissions:
          # necessary for the runs of push to store security events in the repo
          # GitHub code scanning will treat any grammar error like a any security event.
          security-events: write
        steps:
    
          - name: Fetch Sources
            # Check out current repository
            uses: actions/checkout@v2
    
          - name: Setup Qodana cache
            # to speed up future runs of Qodana
            uses: actions/cache@v2
            with:
              path: /home/runner/work/_temp/_github_home/qodana-cache
              key: ${{ runner.os }}-qodana-${{ github.ref }}-${{ hashFiles('.github/workflows/docs.yml', '**/*.gradle', '**/*.gradle.kts', 'gradle.properties') }}
              restore-keys: |
                ${{ runner.os }}-qodana-${{ github.ref }}-${{ hashFiles('.github/workflows/docs.yml', '**/*.gradle', '**/*.gradle.kts', 'gradle.properties') }}
                ${{ runner.os }}-qodana-${{ github.ref }}
                ${{ runner.os }}-qodana-
    
          - name: Download AsciiDoc plugin for AsciiDoc checks
            run: |
              curl -L -o asciidoctor-intellij-plugin.zip https://github.com/asciidoctor/asciidoctor-intellij-plugin/releases/download/0.36.4/asciidoctor-intellij-plugin-0.36.4.zip
              unzip asciidoctor-intellij-plugin.zip
    
          - name: Download Grazie plugin for grammar checks
            run: |
              curl -L -o grazie.zip 'https://plugins.jetbrains.com/plugin/download?rel=true&updateId=135090'
              unzip grazie.zip
    
          - name: Run Qodana code inspection (docs subfolder only)
            # alternative: use "-d doc", this would avoid patching the SARIF file afterward.
            # downside of alternative: gradle will run and load all gradle and build dependencies
            run: >
              docker run
              -v ${{ github.workspace }}/doc:/data/project
              -v ${{ github.workspace }}/qodana:/data/results
              -v ${{ github.workspace }}/asciidoctor-intellij-plugin:/opt/idea/plugins/asciidoctor-intellij-plugin
              -v ${{ github.workspace }}/grazie:/opt/idea/plugins/grazie
              -v /home/runner/work/_temp/_github_home/qodana-cache:/data/cache
              jetbrains/qodana-jvm-community:2021.2
              --save-report
    
          - name: Clean gradle daemon folder
            # as it ran as root in the container / only needed if running on project's root and project is a gradle project
            run: |
              sudo rm -rf /home/runner/work/_temp/_github_home/qodana-cache/gradle/daemon
    
          - name: Copy content to folder with write permissions
            # as qodana ran as root
            run: |
              cp ${{ github.workspace }}/qodana/report/results/qodana.sarif.json ${{ github.workspace }}/qodana.sarif.json
    
          - name: Add "doc" to paths as qodana ran from "doc" folder
            # Otherwise, GitHub won't be able to match the reported problems to file in the repository
            run: |
              sed -i 's|"uri": "|"uri": "doc/|g' ${{ github.workspace }}/qodana.sarif.json
    
          - name: Fix automation ID in SARIF report
            # necessary as qodana will insert a date with "/" - this is not hierarchical,
            # and won't have an ID that can be matched across dates and branches by GitHub Code Scanning
            # old: "project - 11/14/21, 10:04 AM"
            # new: "project/qodana/2021-11-14"
            # see: https://youtrack.jetbrains.com/issue/QD-1796
            run: |
              sed -i 's|"id": "project - \(..\)/\(..\)/\(..\).*"|"id": "project/qodana/20\3-\1-\2"|g' ${{ github.workspace }}/qodana.sarif.json
    
          - name: Upload Qodana report as an artifact of the pipeline
            # users with access can then download the report as a ZIP file
            uses: actions/upload-artifact@v2
            with:
              name: qodana-report
              path: ${{ github.workspace }}/qodana/report/
    
          # https://docs.github.com/en/code-security/code-scanning/integrating-with-code-scanning/uploading-a-sarif-file-to-github#example-workflow-that-runs-the-eslint-analysis-tool
          - name: Upload SARIF report to GitHub
            # so that it is present on all pull requests and GitHub shows the comparison results
            uses: github/codeql-action/upload-sarif@v1
            with:
              # Path to SARIF file relative to the root of the repository
              sarif_file: ${{ github.workspace }}/qodana.sarif.json

Open issues with Qodana

The following issues in the Qodana issue tracker are open. Please consider voting for them to make this integration simpler.

  • QD-1289 automationDetails ID not filled according to specs, leading to problems in GitHub Code Scanning

  • QD-1596 Broken description when using qodana.sarif.json in CodeQL

  • QD-1291 Qodana easy plugin installation

  • QD-1291 Add soft wrap to code preview