Convert Jekyl files to Hugo files using Golang (You Know You Want To)

Although I’ve been programming professionally for many years now, this article deals with the kind of guilty-pleasure program that I most enjoy writing. It’s just a little script, not too polished, put together in a few hours on a weekend to do some idiosyncratic task while helping me to solidify my skills in language I’m wanting to polish up.

This is the go source for a little go script I called blog_phoenix, since its purpose was to take a bunch of crusty old blog files that had once been in Wordpress and gosh-knows what and turn them into something that could appear on the latest iteration of CodeSolid.com.

These posts were last seen in a github repository in a folder with the redundant name of “OLD_CodeSolid/old_codesolid”, as if one “old_codesolid” weren’t ignominious enough.

This little program helped me take that sad gang of crusty old JEKYL files (in the “inputFiles” directory),and processes them into a respectable clique of new-hotness HUGO files in outputDir. That allowed me to polish them a bit more and move them into my main posts directory, so they can live on again.

package main

import (
	"fmt"
	"github.com/go-yaml/yaml"
	"io/ioutil"
	"log"
	"regexp"
	"strings"
)

“inputFiles” are where the old JEKYL files were. outputDir is where you want the processed files to go.

const outputDir = "/home/john/source/codesolid/old_blog/"
const inputFiles = "/home/john/source/OLD_CodeSolid/old_codesolid/source/_posts/"

Process the files in the old markdown directory. Read each one as a string and hand off to processFile

func main() {

	files, err := ioutil.ReadDir(inputFiles)
	check(err)

	for _, fileInfo := range files {

		filename := fileInfo.Name()

		// Process only markdown files
		if !strings.HasSuffix(filename, ".markdown") && !strings.HasSuffix(filename, ".md")  {
			continue
		}

		fmt.Println("Processing " + filename)

		err := processFile(fileInfo.Name())

		// Given "check" implementation, this case is unlikely at best
		if err != nil {
			fmt.Printf("%v  error in file %s\n", err, fileInfo.Name())
		}
	}
}

Since this is just a handy script for my own use, bailing if anything untoward happens works well.

func check(err error) {
	if err != nil {
		log.Fatal(err)
	}
}

getFileParts takes the contents of an existing file and returns either a YAML stucture representing the front matter a string containing the text of the rest of the file, or an error

func getFileParts(contents string) (map[string]interface{}, string, error) {

	// Remove first yaml marker
	trimmed := strings.TrimLeft(contents, "\n\t- ")

	// Split on "2nd" yml marker
	const sep = "---"
	parts := strings.SplitAfterN(trimmed, sep, 2)
	textYAML := parts[0]

	// A place for the YAML front matter to live
	var Y = make(map[string]interface{})
	err := yaml.Unmarshal([]byte(textYAML), Y)
	check(err)

	textPost := strings.Trim(parts[1], "\n\t ")

	return Y, textPost, nil
}

addExcerptSeparator puts the original markdown excerpt where it can do some good or failing that tries to find an appropriate paragraph break.

func addExcerptSeparator(text string, YAML map[string]interface{}) string {
	excerpt, ok := YAML["excerpt"]; if ok {
		return excerpt.(string) + "\n<!--more-->\n" + text
	}
	return strings.Replace(text, "\n\n", "\n<!--more-->\n", 1)
}

replacePrismWithMarkdownCodeblocks takes the prism formatting my old blog used and replaces it with simple markdown codeblocks.

func replacePrismWithMarkdownCodeblocks(text string) string {
	prismCodeMarkerRegex := "{% +(end)?prism.*%}"
	regex, err := regexp.Compile(prismCodeMarkerRegex)
	check(err)
	return string(regex.ReplaceAll([]byte(text),[]byte("'''")))
}

addYAML adds YAML front matter to the beginning of the post text and returns it.

func addYAML(text string, YAML map[string]interface{}) string {
	s, err := yaml.Marshal(YAML)
	check(err)
	return "---\n" + string(s) + "\n---\n" + text
}

cleanupYAML removes front matter we no longer need for HUGO.

func cleanupYAML(YAML map[string]interface{}) {
	doomed := [...]string {"wordpress_id", "excerpt", "layout", "author", "comments"}
	for _, key := range(doomed) {
		delete(YAML, key)
	}
}

processFile takes the filename old file as a string, processes it, and outputs a new file in the outputDir. If this were not a quick-and-dirty script, this isn’t the sort of design one would choose, because it would be nice to take a string and return another, testable string for example. But for a quick weekend project, going file-to-file works fine.

The nice thing about Go is that it handles programming at Google scale and little one-off tasks like this one equally well.

func processFile(filename string) error {

	// Check that we're dealing with markdown.  If we're not, return a nil error to
	// "continue" to the next file
	if !strings.HasSuffix(filename, ".markdown") && !strings.HasSuffix(filename, ".md")  {
		return nil
	}

	// Get the file as a string
	s, err := ioutil.ReadFile(inputFiles + filename)
	check(err)
	contents := string(s)

	// Split it into parts
	YAML, text, err := getFileParts(contents)
	check(err)

	// Process the text portion
	text = addExcerptSeparator(text, YAML)
	text = replacePrismWithMarkdownCodeblocks(text)

	// Process the front matter
	cleanupYAML(YAML)

	// Add the two parts back together, front matter + text
	text = addYAML(text, YAML)

	// Write the file
	outputFile := outputDir + strings.Replace(filename, ".markdown", ".md", 1)
	err = ioutil.WriteFile(outputFile, []byte(text), 0644)
	check(err)

	return err
}