Convert Jekyl files to Hugo files using Golang (You Know You Want To)
Although I’ve been programming professionally for many years now, this article deals with the kind of guilty-pleasure program that I most enjoy writing. It’s just a little script, not too polished, put together in a few hours on a weekend to do some idiosyncratic task while helping me to solidify my skills in language I’m wanting to polish up. This is the go source for a little go script I called blog_phoenix, since its purpose was to take a bunch of crusty old blog files that had once been in Wordpress and gosh-knows what and turn them into something that could appear on the latest iteration of CodeSolid.com.
These posts were last seen in a github repository in a folder with the redundant name of “OLD_CodeSolid/old_codesolid”, as if one “old_codesolid” weren’t ignominious enough.
This little program helped me take that sad gang of crusty old JEKYL files (in the “inputFiles” directory),and processes them into a respectable clique of new-hotness HUGO files in outputDir. That allowed me to polish them a bit more and move them into my main posts directory, so they can live on again.
package main
import (
"fmt"
"github.com/go-yaml/yaml"
"io/ioutil"
"log"
"regexp"
"strings"
)
“inputFiles” are where the old JEKYL files were. outputDir is where you want the processed files to go.
const outputDir = "/home/john/source/codesolid/old_blog/"
const inputFiles = "/home/john/source/OLD_CodeSolid/old_codesolid/source/_posts/"
Process the files in the old markdown directory. Read each one as a string and hand off to processFile
func main() {
files, err := ioutil.ReadDir(inputFiles)
check(err)
for _, fileInfo := range files {
filename := fileInfo.Name()
// Process only markdown files
if !strings.HasSuffix(filename, ".markdown") && !strings.HasSuffix(filename, ".md") {
continue
}
fmt.Println("Processing " + filename)
err := processFile(fileInfo.Name())
// Given "check" implementation, this case is unlikely at best
if err != nil {
fmt.Printf("%v error in file %s\n", err, fileInfo.Name())
}
}
}
Since this is just a handy script for my own use, bailing if anything untoward happens works well.
func check(err error) {
if err != nil {
log.Fatal(err)
}
}
getFileParts takes the contents of an existing file and returns either a YAML stucture representing the front matter a string containing the text of the rest of the file, or an error
func getFileParts(contents string) (map[string]interface{}, string, error) {
// Remove first yaml marker
trimmed := strings.TrimLeft(contents, "\n\t- ")
// Split on "2nd" yml marker
const sep = "---"
parts := strings.SplitAfterN(trimmed, sep, 2)
textYAML := parts[0]
// A place for the YAML front matter to live
var Y = make(map[string]interface{})
err := yaml.Unmarshal([]byte(textYAML), Y)
check(err)
textPost := strings.Trim(parts[1], "\n\t ")
return Y, textPost, nil
}
addExcerptSeparator puts the original markdown excerpt where it can do some good or failing that tries to find an appropriate paragraph break.
func addExcerptSeparator(text string, YAML map[string]interface{}) string {
excerpt, ok := YAML["excerpt"]; if ok {
return excerpt.(string) + "\n<!--more-->\n" + text
}
return strings.Replace(text, "\n\n", "\n<!--more-->\n", 1)
}
replacePrismWithMarkdownCodeblocks takes the prism formatting my old blog used and replaces it with simple markdown codeblocks.
func replacePrismWithMarkdownCodeblocks(text string) string {
prismCodeMarkerRegex := "{% +(end)?prism.*%}"
regex, err := regexp.Compile(prismCodeMarkerRegex)
check(err)
return string(regex.ReplaceAll([]byte(text),[]byte("'''")))
}
addYAML adds YAML front matter to the beginning of the post text and returns it.
func addYAML(text string, YAML map[string]interface{}) string {
s, err := yaml.Marshal(YAML)
check(err)
return "---\n" + string(s) + "\n---\n" + text
}
cleanupYAML removes front matter we no longer need for HUGO.
func cleanupYAML(YAML map[string]interface{}) {
doomed := [...]string {"wordpress_id", "excerpt", "layout", "author", "comments"}
for _, key := range(doomed) {
delete(YAML, key)
}
}
processFile takes the filename old file as a string, processes it, and outputs a new file in the outputDir. If this were not a quick-and-dirty script, this isn’t the sort of design one would choose, because it would be nice to take a string and return another, testable string for example. But for a quick weekend project, going file-to-file works fine.
The nice thing about Go is that it handles programming at Google scale and little one-off tasks like this one equally well.
func processFile(filename string) error {
// Check that we're dealing with markdown. If we're not, return a nil error to
// "continue" to the next file
if !strings.HasSuffix(filename, ".markdown") && !strings.HasSuffix(filename, ".md") {
return nil
}
// Get the file as a string
s, err := ioutil.ReadFile(inputFiles + filename)
check(err)
contents := string(s)
// Split it into parts
YAML, text, err := getFileParts(contents)
check(err)
// Process the text portion
text = addExcerptSeparator(text, YAML)
text = replacePrismWithMarkdownCodeblocks(text)
// Process the front matter
cleanupYAML(YAML)
// Add the two parts back together, front matter + text
text = addYAML(text, YAML)
// Write the file
outputFile := outputDir + strings.Replace(filename, ".markdown", ".md", 1)
err = ioutil.WriteFile(outputFile, []byte(text), 0644)
check(err)
return err
}