meet lol, my new website generator!

2022-08-08 00:00

i've been pretty busy lately, so i haven't had a lot of time to explore computers, which is what i love doing. in the last few months, i've been trying to fight back against busyness by writing a new website generator after everyone goes to bed. i can't recommend doing this, because it gets exhausting after a while, but it gives me my kicks and makes me happy.

my old website generator, wg, was a wrapper around pandoc, and was written in fennel. i used separate fennel scripts to generate a list of posts and an rss feed as post-thoughts to the website generator. also, I didn't know about wireguard when I programmed wg. for those of you who don't know, wireguard uses the command-line name wg, so it was best that i didn't compete with that haha.

i still love pandoc and fennel, but I wanted to try to program something that had the following features:

i also wanted an excuse to make a bigger programming project in chicken scheme haha.

i think what I'm most proud of for this project is that I was able to implement string templates. for example, the {{im-an-example}} in the text below would be replaced with the value that corresponds to the im-an-example key in a config.scm file. 

Hey there, this a sentence, and my name is {{im-an-example}}.

though, this wasn't that easy.

first, i had to implement string replacement... okay, okay, string replacement exists in chicken scheme using the string-translate, string-translate*, irregex-replace, or irregex-replace/all procedures, but where's the fun in using those? i don't get to build anything!

my first step was to write a procedure that replaced the first occurrence of a string. i ended up using the string-append, substring, and string-length procedures to implement the following procedure: 

(define (str-replace str from-str to-str)
  (let* ((from-index (string-contains str from-str))
         (index-offset (+ from-index (string-length from-str))))
    (if from-index
      (string-append (substring str 0 from-index)
                     to-str
                     (substring str
                                index-offset
                                (string-length str)))
      str)))

which isn't very useful if you plan on having several of the same placeholder values in one string, so i also needed to write a procedure to replace all occurrences of the string. it will drop into an infinite loop if i try to replace l with ll, but this is personal programming, not some software that needs to be battle tested, so i settled with my implementation below: 

(define (str-replace-all str from-str to-str)
  (let* ((from-index (string-contains str from-str))
         (index-offset (+ from-index (string-length from-str))))
    (if from-index
      (let ((rest-of-string (substring str
                                       index-offset
                                       (string-length str))))
        (string-append
          (substring str 0 from-index)
          to-str
          (str-replace-all rest-of-string from-str to-str)))
      str)))

next, i needed somehow to take a list of pairs, convert the first item in each pair to a string, and then surround the string with {{ and }}, so it resembles one of the placeholder values that i mentioned earlier. after it changed the first element in each pair, i then took the first element of each pair, searched for it in the provided string, and then replaced it with the second element, using the str-replace-all procedure to ensure all instances of that placeholder were replaced.

i actually ended up having to split this algorithm into two procedures to keep things maintainable for myself in case i needed to go back to fix or update the code around this functionality. here are those two procedures: 

(define (key->mustached-key pair)
  (if (pair? pair)
    (let* ((key (symbol->string (car pair)))
           (mustached-key (string-append "{{" key "}}"))
           (value (cadr pair)))
    `(,mustached-key ,value))
    pair))

(define (string-populate str alist)
  (if (null? alist)
    str
    (let* ((mustached-keys (map key->mustached-key alist))
           (first-pair (car mustached-keys))
           (key (car first-pair))
           (val (cadr first-pair)))
      (string-populate
        (str-replace-all str key val)
        (cdr alist)))))

this ended up helping me get really good at quasiquoting in scheme as well!

apart from the string-populate procedure, and the core procedures that it's built on, most of the other features aren't anything special, though i did enjoy that i can just read arbitrary s-expressions from a string using scheme's read procedure. the read procedure made it super easy to read a configuration file that was all s-expressions. for example, all i needed to do was load an alist in a file with the following procedure: 

(define (load-config-file)
  (if (file-exists? config-file)
    (with-input-from-file config-file read)
    #f))

this procedure returns a quoted alist, so i wrote the following helper procedure to read it: 

(define (get alist key)
  (if (and (pair? alist)
           (pair? (car alist))
           (symbol? key))
    (cadr (assq key alist))
    alist))

functional programming purists will hate me for this, but this then allowed me set a globally mutated variable with (set! config-data (load-config-file)), and then read the variable with a (get config-data 'source-dir).

i've been using this method for reading and reloading configuration files for other projects as well, so that was a great learning experience.

as for generating my list of posts and rss feed, all i needed to do was parse each markdown file in a directory that's specified in the configuration file. to make things easy, the title of a post was extracted from the first line of a file, which should always be a markdown h1 heading. i would then take the markdown heading, for example, # hey i'm a heading, and remove the number sign and space proceeding the number sign, leaving me with hey i'm a heading.

the remaining string would be used as the title for each post in the list of posts page, and the title of each rss item. the way i generated links for my list of posts page was by converting the source path from, as an example, <source-dir>/path/to/post.md to https://<domain>/path/to/post.html.

because dates are pretty important to rss feeds, although not required, if you're following the spec, i chose to put dates on the third line of each post, in the format of yyyy-mm-dd, so i could convert yyyy-mm-dd to a number that resembled yyyymmdd, and then reverse sort by each number, resulting in a "latest post first, oldest post last" order.

to kind of finish this off, i think one of the major annoyances was converting all fenced code blocks to use indentation instead, because chicken scheme's lowdown egg replicates what the original markdown parser does. that, and replacing all of my pandoc-centric markdown stuff such as its markdown version of <div> blocks: 

:::{.im-a-class}
hey im a div
:::

the upside to using old school, feature-less markdown is that the markdown for my website will work on most markdown parsers i guess? haha.

the downside to using the lowdown markdown parser is that heading anchors aren't generated, so all of my links to heading anchors are broken, but i got to have fun with programming in scheme at least? plus, this isn't my professional website, so things are allowed to be broken here, and i don't want to get rid of old posts because they bring back good programming adventure memories for me.

i figured this blog could use a new post, so here it is!

have a good one!

if you want to check out the source code for my new website generator, you can view it here.