CI Design

CI Design

Let’s not derail during engineering stages

One of the key areas that needs focus at all times is quality. Without good QA the defect rate rises due to uncertainty inferred from changes to the systems capability. It’s not just ensuring the solution builds – although that might be the first test – but performing analysis for optimisation and security.

Lot’s of benefit to be gained from a little effort up front. I’m a big fan of QA and DevOps as this will create a level playing field for anyone that wants to contribute.

Evolution is the process by which testing new capabilities either proves successful, or kills it. Is this capability operable and does it have value? And I don’t mean “…well it might have value if some company does ‘X’ in ten years…”.

Static Code Analysis

You can’t get all the problems via static code analysis, but you can potentially identify traits which may give clues about flaws in your code when it’s executing. I’m going to come back to the security analysis and focus on the coding standards analysis, expecting that two tool-sets will be needed which are focused on each particular job.

To get both in one and have both capabilities working well in one tool would be a win-win.

However Ringo Dingo throws another curve-ball in that I know that .NET Core, Python, HTML, ISO/IEC 9075:2003 (SQL:2003), SQL-99 and Javascript are all involved in the platform.

Code Quality Analysis

I’ve typically lent towards something like Fortify (which covers security and code-quality) or Checkmarx in enterprise-grade scenarios, but I think this calls for use of open-source approaches. VS Code already has substantial live code quality analysis tools with the C# plugins, so I don’t want to do too much in this category.

ToolLanguageNotes
SonarSource Community.NETI’ve listed SonarLint in the
security analysis section below,
but it could be that the same
tool-set covers quality too
nDepend.NETLike the technical debt estimation
aspects in here… if money was
going to be forked out this would
be near the top of the list
TBC.NETJust remembered I’m going to
need something like dotTrace for
performance analysis and
diagnosis
GitLabMultiOk so I searched for Fortify-esque
tools and found a GitLab page
describing how it’s Auto DevOps
covers this already.
Query Plan AnalysisSQLNot so much a tool but use of
RDBMS platform query &
analysis. I’m not keen on building
a toolset for GitLab CI from
scratch, as I did for MS SQL in
past years.
There’s some notation in the
SonarLint documentation, but
that’s IDE-side.

What’s clear here is that whilst I’ll need to look for a dotTrace equivalent for VS Code and Linux, the default CI can be configured in GitLab to do a large portion of the work.

Ok so I need to figure out a couple of things, but this looks very promising

The other core part of CI is running the unit tests successfully – something that should be done before code is pushed back into the working branch, but often skipped or forgotten.

For .NET it’s fairly simple right up until API call tests (perhaps that should fall into the integration testing rather than automated unit testing), and I need to look into Python and Javascript unit testing.

Security Analysis

I used the OWASP list of code analysis tools as a basis, focusing on .NET. Really this tool needs to be runnable as part of a solution build in VS Code, and somehow runnable as a build task job on the CI runner. That way contributors developing code could ignore security warnings / add their local suppression configuration, without affecting the end result – as GL CI will enforce the full rule set.

Of course we should also expect that code analysis tools are specific to language, and that we will need to configure the GitLab runners for each code base update. I’ve selected the following initial candidates for this category on a per language basis:

ToolLanguageNotes
Bandit PythonLooks like it’s maintained
currently, and could give good
coverage for issues in Python.

Needs a road test
Google Search DiggityN/AThink this may form part of
the post deployment scrutiny
rather than a codebase check.
It’s focused on assessing a
web application whilst running
Security Code Scan.NETThis looks like a good tool to
road-test for use within the
environment. Not sure if this
will work on VS Code on
Linux yet, though it has
NuGet package installation
instructions
LGTMMultiCould be the CI tool at the
GitLab runner end. Needs
investigation
Puma Scan.NETThe server-side flavour is
commercial so the end-user
(community) edition needs
careful comparison. Do I want
to use a commercially-funded
product which may later close
off, but be audited and funded
properly… or an open-source
(SCS) alternative which is
community dependant?
SonarLint.NETLike the look of this –
IDE-embedded live code analysis
tool. Works with VS Code OotB.
Maybe this for IDE and either SCS
or Puma Scan server-side?
It’s part of the overall SonarQube
platform which was listed in the
previous section

I’ll also take a look at OWASP O2 for the overall Ringo Dingo piece.

Other Thoughts

For now I’m going to say it’s “OK” to add overrides for warnings on exceptions relating to coding standards such as curly-braces-on-newline, and other such stuff. I’m a K&R-style fan and don’t care much for that – I’m likely to make that a standard for the repository submissions as well. I’m also not interested in debating that aspect.

I’m initially going to add suppression for IDE0063, so that standard using statements are not producing warnings. Whilst it’s ‘easier’ to use the newer notation I also see many defects being created where someone forgets the scope of something (or mixes scope at the same indentation level unintentionally).

Conversely it’s not ok to add a suppression just to get a section of code working. Coding style is one thing but likely that particular warning or error was there for a reason.

The problem area is going to be code analysis for PostgreSQL. Suspect there will be custom grep scripts for GitLab CI which search for particular problem syntax, and the need to try and create unit testing for procs to ensure they query within performance limits. No small task.

If I build that capability, I’m definitely going to open-source it as the community will be able to use the framework to do a far better job than I can at this stage.

It’s also worth periodically reviewing why warnings or exceptions might be explicitly (configured) overridden. Something you might do in an early phase to get the component working might not be appropriate later, especially approaching production.

Definitely need to review all suppression and exemptions before pre-prod deployment, and will have to think about what mechanisms will help achieve that – perhaps something linked between development branches and sitting in-between dev and master branches? We’ll see – normally current version is master, with development branches for each feature group in development.

So much to consider.

Comments are closed.