vale 2: vale Harder
They say “there’s nothing new under the sun” and “there are no original ideas”. That may be true but they rarely speak of restraint when you have what you think is a good idea. Marc Randolph is credited as saying “No one knows if it’s a good or bad idea until you try it,” but, Marc, sometimes people know. I already knew it was a bad idea to use this concept as the beginning of this post but I committed to the bit. It’s a similar situation with adding vale to the pipeline (or “build”–I’m certain I will begin using those two interchangeably at some point but it is “pipeline” in Azure DevOps vernacular).
Table Of Contents
Revved up like a deuce
This is the sequel to the (now) infamous (maybe?) post Spellcheck My Doc, Before It’s Too Late which also builds on prior posts. That’s ok. Maybe you’ve gone this far on your own. Maybe you’ve got vale set up just how you like it and you’re only looking to drop that in your build. This is certainly where you want to be.
Previously on…
There is some minutiae that I’d like to go over in the event that you are moving your wonderful existing vale setup over. As I’m using a Hugo site, and using the Azure DevOps Static Web App service and deployment task, there are some considerations with the path. The root of the Hugo site, where configuration.toml
is (assuming you only have one), is where the build takes place. styles
is in the root directory. It absolutely doesn’t need to be–in fact it should probably be in the assets
directory where the dictionary is so that it doesn’t actually get deployed to the website. Mark takes a note that will be another post.
styles
├── Microsoft
├── Readability
└── marktoso
├── Kiss.yml
├── Readability.yml
├── Spelling.yml
├── Substitute.yml
└── TresComas.yml
vale
, the command, was installed via Homebrew. This can be done in your pipeline on an on-demand basis or it can be copied over from some location however I’m pointing this out because I’m hosting this DevOps agent (build agent? runner? depends on which CI/CD product you’re using) on a Linux system that is in my possession. Azure DevOps agents run in containers and get built and destroyed every time they run so you’ll need to keep that in mind as it’s more likely that you’re using a DevOps agent than not. When running jobs from the DevOps agent they don’t load the user profile. Right now I’m using full paths to point to the binaries and takes another note that will also be another post.
“Lowrider”, Donny? Donny, “Lowrider”.
The Azure DevOps Pipeline is where the rubber meets the road. Or where the code meets the build. Well, it is the build. Where the code meets the deployment. But it’s also the deployment. So where the code meets the end result which is likely to be some other form ready for consumption by an agent that can be animal, vegetable, mineral, or other code. But let’s see exactly what it will take to get this built in sixty seconds.1
Scripty McScriptface
Yes, the ubiquitous bash
script. We call upon thee in time of utility. Shell scripts aren’t my preference–at least not anymore. I’ll say something nice about all of the scripting languages I’ve used.
- PowerShell makes life both fun and easy
- Batch is old and let me play TIE Fighter by editing autoexec.bat
- Shell is also old
- Python is excessivley cool and powerful right now
- JavaScript has C-like syntax
- VBScript
- Perl brought regex to the masses (this is also a drawback)
- PHP built a few websites
Bash is very portable in the sense that every platform I plan on building this static site on will have it installed. I could definitely install PowerShell at build time but it could mean unnecessary delays and a potential for errors that could have been avoided. Yes, I do have PowerShell installed on the system I’m using for builds but my plan is to move it to the cloud soon enough.
|
|
I call this process-vale.sh
and it lives with the other assets in, you guessed it, assets/utilities
. Line 1 is getting all the markdown files (yeah, I write my posts in markdown, big whoop wanna fight about it) in the path content/posts
which is where my posts are stored. That directory is actually flat at the moment and I may regret that but time will tell. Line 2 is just debugging and could easily go away. Line 3 is the wonderful for
statement which is actually the type of loop I use the most in scripting because, as it turns out, I usually have a large list of things I want to do something to or with.
This is an abjectly low-tech solution but it is quite effective. As the build environment is ephemeral I have no qualms in just tossing the requisite information right into the file itself–it doesn’t get pushed to the repository at any point and gets deleted when the next build rolls along. There are other options, genuinely. The template could have placeholder text that could be replaced after build. You could write a script or program to parse the DOM of the built HTML file and then “correctly” add in the information. I didn’t go this route because, honestly, it wouldn’t play quite as nice if I switched up the template significantly. That’s what I would tell people, at any rate. I chose to avoid editing the built files because it was just more complicated. Hugo is going to build the site for me so why even fight it? Let it do its job.
Lines 6 and 7 look interesting and that’s because of ansi2html.sh
and vale-prod-template.tmpl
. I’ll start with the one that doesn’t sound like a member of the Insane Clown Posse. vale-prod-template.tmpl
is an output template for vale
and what this means is that you can structure the output using the Go templating specification like Hugo.
|
|
Go templating is not complex so much as it can get complicated so I won’t get into it too much in this post. You may see more of that as I augment the template for this website. But it’s safe to say that there is a table object created, $table
, that stores the output in terms of category, content, and module that vale outputs by default. At the time of this build I did not desire the full output–just the totals. I removed one line from the example provided by vale in their documentation to get the desired effect. There is also the option of outputting JSON to do more complex operations, such as potentially interactively correcting issues, however that was not the scope of my ambition.
ansi2html.sh
is an open-source shell script by Pádraig Brady aka pixelbeat and is available on Github. I like color terminals and color output. I really like color terminals and color output. I didn’t want to go through the process of converting it myself but I was confident that someone else had already done it. I use the --body-only
argument to ensure I don’t get the unnecessary HTML tags. I then used the --css-only
tag to generate some CSS and incorporate it into my theme using the customCSS
parameter built in to the Archie theme.
It’s fairly apparent how color is used in the output template for vale
and I both liked it and wanted to incorporate it into the post page itself. So that’s how I did it. Piped it into a shell script. Appended it to a markdown file. Not exactly rocket surgery but it didn’t really need to be.
On one condition
How could this be translated into a green/red build status. So glad you asked. Will I be implementing this for my personal blog? Absolutely not. But if I was this would be how I would do it. As a thought experiment, I would certainly change a few things in process-vale.sh
to make that happen. Firstly, I would not use the modified output template for vale
so that, in my build log, I’d have the output. There may be a more clever way to do this, like perhaps publishing an artifact
Time may change me
Confession: I’ve changed a few things. I’ve adjusted and added on to a couple of things. Nothing terribly huge but it could have an impact. Most importantly, I’ve adjusted the azure-pipelines.yml
file to have various stages. It doesn’t really have an impact on anything but it is somehow more satisfying and allows for the generic reports in Slack to be a little more communicative if, also, a little more annoying.
trigger:
- main
- feature/*
schedules:
- cron: "0 6 * * *" # cron syntax defining a schedule
displayName: "Scheduled build" # friendly name given to a specific schedule
branches:
include: [ "main" ] # which branches the schedule applies to
exclude: [ ] # which branches to exclude from the schedule
always: true # whether to always run the pipeline or only if there have been source code changes since the last successful scheduled run. The default is false.
stages:
- stage: 'CheckoutCode'
displayName: 'Checking out code'
jobs:
- job:
pool:
name: Default
workspace:
clean: all
steps:
- checkout: self
displayName: 'Checkout repository including submodules'
submodules: true # true so Hugo theme submodule is checked out
- stage: 'ProcessVale'
displayName: 'Processing posts through vale'
jobs:
- job:
steps:
- checkout: none
- task: Bash@3
displayName: 'processing via vale script'
condition: eq(variables['Build.SourceBranch'], 'refs/heads/main')
inputs:
filePath: 'assets/utilities/process-vale.sh'
- task: Bash@3
displayName: 'processing via vale staging script'
condition: ne(variables['Build.SourceBranch'], 'refs/heads/main')
inputs:
filePath: 'assets/utilities/process-vale-staged.sh'
- stage: 'GenerateHugo'
displayName: 'Building Hugo site'
jobs:
- job:
steps:
- checkout: none
- task: CmdLine@2
displayName: 'Hugo build'
condition: eq(variables['Build.SourceBranch'], 'refs/heads/main')
#command line runs with --noprofile --norc and that's annoying
inputs:
script: '/home/vstsagent/.linuxbrew/bin/hugo --cleanDestinationDir --minify'
- task: CmdLine@2
displayName: 'Hugo Staging Build'
condition: ne(variables['Build.SourceBranch'], 'refs/heads/main')
#command line runs with --noprofile --norc and that's annoying
inputs:
script: '/home/vstsagent/.linuxbrew/bin/hugo -D -F --cleanDestinationDir --minify --baseURL=http://blah.net'
- stage: 'DeploymentStage'
displayName: "Deploy Static Web App"
jobs:
- job:
steps:
- checkout: none
- task: AzureStaticWebApp@0
condition: eq(variables['Build.SourceBranch'], 'refs/heads/main')
displayName: 'Deploying to marktoso.com'
inputs:
skip_app_build: true
app_location: '/public'
app_build_command: ''
output_location: ''
azure_static_web_apps_api_token: '$(TOKEN)'
- task: AzureStaticWebApp@0
displayName: 'deploying to staging site'
condition: ne(variables['Build.SourceBranch'], 'refs/heads/main')
inputs:
skip_app_build: true
app_location: '/public'
app_build_command: ''
output_location: ''
azure_static_web_apps_api_token: '$(STAGING_TOKEN)'
This change is primarily composed of adding stage
and job
entries however there is a very important component, if you were to go down this path, that you should be aware of. Every job
, by default, will 100% check out the code from the repo. I’m not entirely certain why this is desirable but in the pipeline that I have constructed it is definitively not desirable under any circumstances. This led to be troubleshooting why my changes to the vale
output template weren’t coming through any longer. I finally noticed, in the Azure Pipelines logs that every stage began with a checkout. An interesting choice, to me, but perhaps there is method to that madness that I am simply oblivious to.
Failure is an option
The second change was to process-vale-staged.sh
which has some added features to it. See for yourself:
|
|
Lines 14-16 add the additional content to the blurb at the bottom and there will be a clever CSS hack for that. Lines 18-21 actually throw an error to the Azure Pipeline. You’ll see that the condition there is that, if the output of vale for that file at any point contains marktoso.Spelling
to just outright fail and exit 1
. This fails the entire pipeline and doesn’t deploy the build. This also runs vale
twice on every file but my build times increased significantly from adding the stages to it (about an 100% increase or from 32-38 seconds to a minute and change) and increased hardly at all from running vale
twice as it is quite performant.
Quality of life update
Added a few lines to process-vale-staging.sh
which were massive quality of life improvements. To give you an idea, I have both the example posts from the theme as well as draft posts where I just type in random things to be filled out later. These have a plethora of spelling errors and going through and correcting them is of dubious value since they will never see the light of day as they are. Toss these three lines right at the top of the for
loop to save some struggle.
if head $POST | grep -q "draft: true"; then
continue
fi
Make sure that your front matter shows in head
. I was having an issue with the vale
output not showing up in this post and that’s because head
was originally cat
and of course “draft: true” showed up in the post because it was in that code block.
Task failed successfully
You may recall from Spellcheck My Doc, Before It’s Too Late that I set up a spelling rule, aptly named Spelling
, in the custom style named marktoso
. This is why the script looks for marktoso.Spelling
to fail. This text only shows up with spelling errors and that’s the failure condition that I’ve chosen to be the go/no go. If I have a spelling error that I haven’t addressed in one form or another then the site shouldn’t build. The other errors will normally come from the Microsoft
style which is geared towards professional documentation and I’m very comfortable with those staying in. They’re educational at worst.
That looks great, doesn’t it? Like so much is happening. But it is helpful to understand that a specific file triggered the error. No one is stopping anyone from creating utility scripts for their teams or other automation to run against the documents to then have a failed build–quite the opposite. This is the doc gate that should encourage teams and people to do this ahead of time. Before it even shows up in a pull request. I realize it’s steps short of a full test suite however it was relatively trivial to get going and does a not-too-dissimilar, and grantedly shallower, job.
Style me up
Yes, with process-vale-staging.sh
we are getting that delicious vale
output (and if you’re wondering what the difference is between vale and vale
is, vale
is the actual command while vale is the product or project). I’m going to do something perhaps ill-advised but just tack on some CSS to my ansi2html.css
file that was set up back in Spellcheck My Doc, Before It’s Too Late.
I’m actually struggling with the style sheet a little bit. The theme has some user-agent specific themes that may (or may not be) overriding the changes that I’m making to checkbox
. So you’ll see a checkbox down there. It’s helpful for accessibility so I don’t want to mess with it too much but it doesn’t look great.
.peekaboo{
display: none;
}
.vale-output input[type="checkbox"] {
display: none;
}
#toggle:checked ~ .peekaboo {
display: block;
}
I’m not a front end web person so I’m way out of date on this but you can check out “The Checkbox Hack” (and things you can do with it) for even more information. I’ll be looking around for a way to hide that checkbox from sight or refactor this to work with a different, but equally compatible, method. You will see the fruits of the labor below (unless I changed it already). Go forth and build. 👷
./content/posts/vale-2-vale-harder.md 10:50 error Punctuation should be inside Microsoft.Quotes the quotes. 10:107 warning Consider removing 'rarely'. Microsoft.Adverbs 10:394 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 10:534 warning Use first person (such as Microsoft.FirstPerson 'I'm') sparingly. 10:545 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 10:609 error Use 'it's' instead of 'it is'. Microsoft.Contractions 14:68 warning Use first person (such as Microsoft.FirstPerson 'My') sparingly. 14:133 warning Use first person (such as Microsoft.FirstPerson 'my') sparingly. 16:18 warning In general, don't use an Microsoft.Ellipses ellipsis. 17:49 warning Consider using 'if' instead of Microsoft.Wordiness 'in the event that'. 17:126 warning Use first person (such as Microsoft.FirstPerson 'I'm') sparingly. 29:158 warning Use first person (such as Microsoft.FirstPerson 'I'm') sparingly. 29:188 warning Use first person (such as Microsoft.FirstPerson 'I'm') sparingly. 29:212 warning Prefer 'personal digital Microsoft.Terms assistant' over 'agent'. 29:225 warning Prefer 'personal digital Microsoft.Terms assistant' over 'agent'. 29:303 error Use 'that's' instead of 'that Microsoft.Contractions is'. 29:314 warning Use first person (such as Microsoft.FirstPerson 'my') sparingly. 29:498 warning Prefer 'personal digital Microsoft.Terms assistant' over 'agent'. 29:598 warning Use first person (such as Microsoft.FirstPerson 'I'm') sparingly. 31:4 error Punctuation should be inside Microsoft.Quotes the quotes. 31:20 warning Don't use end punctuation in Microsoft.HeadingPunctuation headings. 31:30 error Punctuation should be inside Microsoft.Quotes the quotes. 32:104 error Use 'it's' instead of 'it is'. Microsoft.Contractions 32:293 warning Prefer 'personal digital Microsoft.Terms assistant' over 'agent'. 32:358 warning Try to avoid using Microsoft.We first-person plural like 'let's'. 35:36 warning Try to avoid using Microsoft.We first-person plural like 'We'. 35:95 warning Use first person (such as Microsoft.FirstPerson 'my') sparingly. 35:162 warning Consider using 'all' instead Microsoft.Wordiness of 'all of'. 37:28 warning Use first person (such as Microsoft.FirstPerson 'me') sparingly. 45:10 warning Consider removing 'very'. Microsoft.Adverbs 45:56 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 45:119 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 45:270 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 45:316 warning Use first person (such as Microsoft.FirstPerson 'I'm') sparingly. 45:341 warning Use first person (such as Microsoft.FirstPerson 'my') sparingly. 45:366 warning Prefer 'cloud' over 'the Microsoft.Terms cloud'. 57:1 warning Use first person (such as 'I Microsoft.FirstPerson ') sparingly. 57:162 warning Use first person (such as Microsoft.FirstPerson 'my') sparingly. 57:259 warning Use first person (such as Microsoft.FirstPerson 'my') sparingly. 57:329 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 57:403 warning Consider removing 'easily'. Microsoft.Adverbs 57:493 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 57:585 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 59:43 error Use 'it's' instead of 'it is'. Microsoft.Contractions 59:103 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 59:560 warning Consider removing 'honestly'. Microsoft.Adverbs 59:603 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 59:657 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 59:691 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 59:807 warning Use first person (such as Microsoft.FirstPerson 'me') sparingly. 104:56 error Use 'isn't' instead of 'is Microsoft.Contractions not'. 104:111 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 104:182 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 104:281 error More than 3 commas! marktoso.TresComas 104:413 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 104:416 error Use 'didn't' instead of 'did Microsoft.Contractions not'. 104:464 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 104:688 warning Consider removing Microsoft.Adverbs 'potentially'. 104:746 error Use 'wasn't' instead of 'was Microsoft.Contractions not'. 104:767 warning Use first person (such as Microsoft.FirstPerson 'my') sparingly. 106:250 warning Consider removing 'really'. Microsoft.Adverbs 106:364 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 106:462 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 106:501 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 106:580 warning Use first person (such as Microsoft.FirstPerson 'my') sparingly. 108:6 warning Consider removing 'fairly'. Microsoft.Adverbs 108:77 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 108:163 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 108:277 warning Consider removing 'really'. Microsoft.Adverbs 111:84 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 111:112 warning Use first person (such as Microsoft.FirstPerson 'my') sparingly. 111:152 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 111:176 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 111:216 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 111:305 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 111:374 warning Use first person (such as Microsoft.FirstPerson 'my') sparingly. 113:21 warning Use first person (such as Microsoft.FirstPerson 'me') sparingly. 114:98 warning Consider removing 'terribly'. Microsoft.Adverbs 114:239 warning Consider removing 'really'. Microsoft.Adverbs 114:277 error Use 'it's' instead of 'it is'. Microsoft.Contractions 205:91 warning Consider removing 'very'. Microsoft.Adverbs 205:250 warning Use first person (such as Microsoft.FirstPerson 'I'm') sparingly. 205:321 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 205:341 error Use 'it's' instead of 'it is'. Microsoft.Contractions 205:436 warning Use first person (such as Microsoft.FirstPerson 'my') sparingly. 205:625 warning Use first person (such as Microsoft.FirstPerson 'me') sparingly. 205:677 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 234:7 error Use an en dash in a range of Microsoft.RangeFormat numbers. 234:7 warning In most cases, use 'from' or Microsoft.Ranges 'through' to describe a range of numbers. 234:119 error Use an en dash in a range of Microsoft.RangeFormat numbers. 234:119 warning In most cases, use 'from' or Microsoft.Ranges 'through' to describe a range of numbers. 234:438 warning Use first person (such as Microsoft.FirstPerson 'my') sparingly. 234:538 warning In most cases, use 'from' or Microsoft.Ranges 'through' to describe a range of numbers. 234:538 error Use an en dash in a range of Microsoft.RangeFormat numbers. 234:633 error Use 'it's' instead of 'it is'. Microsoft.Contractions 236:191 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 236:390 error Use 'they're' instead of 'they Microsoft.Contractions are'. 243:50 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 245:33 warning Use first person (such as Microsoft.FirstPerson 'My') sparingly. 245:98 warning Use first person (such as Microsoft.FirstPerson 'my') sparingly. 245:138 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 245:406 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 245:435 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 245:628 warning Use first person (such as Microsoft.FirstPerson 'I'm') sparingly. 245:632 warning Consider removing 'very'. Microsoft.Adverbs 249:64 error Use 'it's' instead of 'it is'. Microsoft.Contractions 249:434 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 251:10 warning Use first person (such as Microsoft.FirstPerson 'me') sparingly. 252:37 error Use 'we're' instead of 'we Microsoft.Contractions are'. 252:37 warning Try to avoid using Microsoft.We first-person plural like 'we'. 252:225 warning Use first person (such as Microsoft.FirstPerson 'I'm') sparingly. 252:300 warning Use first person (such as Microsoft.FirstPerson 'my') sparingly. 252:360 warning Use first person (such as Microsoft.FirstPerson 'My') sparingly. 252:425 warning Use first person (such as Microsoft.FirstPerson 'my') sparingly. 254:1 warning Use first person (such as Microsoft.FirstPerson 'I'm') sparingly. 254:84 warning Prefer 'personal digital Microsoft.Terms assistant' over 'agent'. 254:159 warning Use first person (such as Microsoft.FirstPerson 'I'm') sparingly. 254:255 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 268:2 warning Use first person (such as Microsoft.FirstPerson 'I'm') sparingly. 268:36 warning Use first person (such as Microsoft.FirstPerson 'I'm') sparingly. 268:401 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly. 274:210 warning Use first person (such as ' I Microsoft.FirstPerson ') sparingly.✖ 20 errors, 102 warnings and 0 suggestions in 1 file.
not a guarantee batteries not included some assembly required your mileage may vary void where prohibited not valid in the state of solid, liquid, gas, or plasma ↩︎