CodeSolid

A Go Programming Notebook

Convert Jekyl files to Hugo files using Golang (You Know You Want To)

Although I’ve been programming professionally for many years now, this article deals with the kind of guilty-pleasure program that I most enjoy writing. It’s just a little script, not too polished, put together in a few hours on a weekend to do some idiosyncratic task while helping me to solidify my skills in language I’m wanting to polish up. This is the go source for a little go script I called blog_phoenix, since its purpose was to take a bunch of crusty old blog files that had once been in Wordpress and gosh-knows what and turn them into something that could appear on the latest iteration of CodeSolid.com.

These posts were last seen in a github repository in a folder with the redundant name of “OLD_CodeSolid/old_codesolid”, as if one “old_codesolid” weren’t ignominious enough.

This little program helped me take that sad gang of crusty old JEKYL files (in the “inputFiles” directory),and processes them into a respectable clique of new-hotness HUGO files in outputDir. That allowed me to polish them a bit more and move them into my main posts directory, so they can live on again.

package main

import (
	"fmt"
	"github.com/go-yaml/yaml"
	"io/ioutil"
	"log"
	"regexp"
	"strings"
)

“inputFiles” are where the old JEKYL files were. outputDir is where you want the processed files to go.

const outputDir = "/home/john/source/codesolid/old_blog/"
const inputFiles = "/home/john/source/OLD_CodeSolid/old_codesolid/source/_posts/"

Process the files in the old markdown directory. Read each one as a string and hand off to processFile

func main() {

	files, err := ioutil.ReadDir(inputFiles)
	check(err)

	for _, fileInfo := range files {

		filename := fileInfo.Name()

		// Process only markdown files
		if !strings.HasSuffix(filename, ".markdown") && !strings.HasSuffix(filename, ".md")  {
			continue
		}

		fmt.Println("Processing " + filename)

		err := processFile(fileInfo.Name())

		// Given "check" implementation, this case is unlikely at best
		if err != nil {
			fmt.Printf("%v  error in file %s\n", err, fileInfo.Name())
		}
	}
}


Since this is just a handy script for my own use, bailing if anything untoward happens works well.

func check(err error) {
	if err != nil {
		log.Fatal(err)
	}
}


getFileParts takes the contents of an existing file and returns either a YAML stucture representing the front matter a string containing the text of the rest of the file, or an error

func getFileParts(contents string) (map[string]interface{}, string, error) {

	// Remove first yaml marker
	trimmed := strings.TrimLeft(contents, "\n\t- ")

	// Split on "2nd" yml marker
	const sep = "---"
	parts := strings.SplitAfterN(trimmed, sep, 2)
	textYAML := parts[0]

	// A place for the YAML front matter to live
	var Y = make(map[string]interface{})
	err := yaml.Unmarshal([]byte(textYAML), Y)
	check(err)

	textPost := strings.Trim(parts[1], "\n\t ")

	return Y, textPost, nil
}



addExcerptSeparator puts the original markdown excerpt where it can do some good or failing that tries to find an appropriate paragraph break.

func addExcerptSeparator(text string, YAML map[string]interface{}) string {
	excerpt, ok := YAML["excerpt"]; if ok {
		return excerpt.(string) + "\n<!--more-->\n" + text
	}
	return strings.Replace(text, "\n\n", "\n<!--more-->\n", 1)
}


replacePrismWithMarkdownCodeblocks takes the prism formatting my old blog used and replaces it with simple markdown codeblocks.

func replacePrismWithMarkdownCodeblocks(text string) string {
	prismCodeMarkerRegex := "{% +(end)?prism.*%}"
	regex, err := regexp.Compile(prismCodeMarkerRegex)
	check(err)
	return string(regex.ReplaceAll([]byte(text),[]byte("'''")))
}

addYAML adds YAML front matter to the beginning of the post text and returns it.

func addYAML(text string, YAML map[string]interface{}) string {
	s, err := yaml.Marshal(YAML)
	check(err)
	return "---\n" + string(s) + "\n---\n" + text
}


cleanupYAML removes front matter we no longer need for HUGO.

func cleanupYAML(YAML map[string]interface{}) {
	doomed := [...]string {"wordpress_id", "excerpt", "layout", "author", "comments"}
	for _, key := range(doomed) {
		delete(YAML, key)
	}
}


processFile takes the filename old file as a string, processes it, and outputs a new file in the outputDir. If this were not a quick-and-dirty script, this isn’t the sort of design one would choose, because it would be nice to take a string and return another, testable string for example. But for a quick weekend project, going file-to-file works fine.

The nice thing about Go is that it handles programming at Google scale and little one-off tasks like this one equally well.

func processFile(filename string) error {

	// Check that we're dealing with markdown.  If we're not, return a nil error to
	// "continue" to the next file
	if !strings.HasSuffix(filename, ".markdown") && !strings.HasSuffix(filename, ".md")  {
		return nil
	}

	// Get the file as a string
	s, err := ioutil.ReadFile(inputFiles + filename)
	check(err)
	contents := string(s)

	// Split it into parts
	YAML, text, err := getFileParts(contents)
	check(err)

	// Process the text portion
	text = addExcerptSeparator(text, YAML)
	text = replacePrismWithMarkdownCodeblocks(text)

	// Process the front matter
	cleanupYAML(YAML)

	// Add the two parts back together, front matter + text
	text = addYAML(text, YAML)

	// Write the file
	outputFile := outputDir + strings.Replace(filename, ".markdown", ".md", 1)
	err = ioutil.WriteFile(outputFile, []byte(text), 0644)
	check(err)

	return err
}