How I Generate My RSS Feed
I only just now started supplying an RSS feed to you fine people! You can subscribe to it at www.senders.io/blog/feed.rss!
I decided rather than manually generating the file contents I’d hook into my pre-existing publish scripts to be able to generate the RSS file.
Publishing blog posts - shell scripts ftw
In My Markdown -> HTML Setup I touch on how I publish my markdown files into HTML for this blog. But what I don’t really touch on is the shell scripts that tie the whole process together.
What I have is two, now three, scripts that feed the whole process:
publish-blog.sh
- the main scriptcompile-md.sh
- generates the HTML outputupdate-feed.sh
- generates/appends the RSS feed
The update-feed.sh
script is the new one I just
added.
publish-blog.sh
is the primary interface, I supply the
date of the post and the path to the md file and that calls compile and
update to automate the entire process.
Without going into TOO much detail you can view the latest versions of the scripts at git.senders.io/senders/senders-io/tree/.
But the gist of the scripts is I parse out the necessary details, find/replace some tokens in template files I have setup for headers and footers, and concat the outputs into the final output HTML files, and now RSS feed.
update-feed.sh
Source File: git.senders.io/senders/senders-io/tree/update-feed.sh
This script is pretty interesting. I didn’t want to deal with any XML parsers and libraries to just maintain a proper XML rss file and push items into the tree. Rather, I just follow a similar setup to my markdown generation. I leverage some temporary files to hold the contents, a static temp file for the previously generated content, and at the end swap the temp file with the real file.
I take in an input of the publish date (this is the date from the publish script), the title, and the HTML file path. These are all already variables in the publish script, but also something I can manually supply if I need to publish an older article, or something I wrote directly in HTML.
The core of the script is found here:
PUBDATE=$(date -d "$1" -R)
TITLE=$2
FILE_PATH=$3
PERMALINK=$(echo "${FILE_PATH}" | sed -e "s,${TKN_URL_STRIP},${URL_PREFIX},g")
LINK=$(echo "${PERMALINK}" | sed -e "s,${TKN_INDEX_STRIP},,g")
# Generate TMP FEED File Header
cat -s $FILE_RSS_HEADER > $FILE_TMP_FEED
sed -i -E "s/${TKN_BUILDDATE}/${BUILDDATE}/g" $FILE_TMP_FEED
sed -i -E "s/${TKN_PUBDATE}/${PUBDATE}/g" $FILE_TMP_FEED
# Generate TMP Item File
cat -s $FILE_RSS_ITEM_HEADER > $FILE_TMP_ITEM
sed -i -E "s~${TKN_TITLE}~${TITLE}~g" $FILE_TMP_ITEM
sed -i -E "s/${TKN_PUBDATE}/${PUBDATE}/g" $FILE_TMP_ITEM
sed -i -E "s,${TKN_PERMALINK},${PERMALINK},g" $FILE_TMP_ITEM
sed -i -E "s,${TKN_LINK},${LINK},g" $FILE_TMP_ITEM
sed -n "/<article>/,/<\/article>/p" $FILE_PATH >> $FILE_TMP_ITEM
cat -s $FILE_RSS_ITEM_FOOTER >> $FILE_TMP_ITEM
# Prepend Item to items list and overwrite items file w/ prepended item
## In order to "prepend" the item (so it's on top of the others)
## We need to concat the tmp item file with the existing list, then
## we can push the contents over the existing file
## We use cat -s to squeeze the blank lines
cat -s $FILE_ITEM_OUTPUT >> $FILE_TMP_ITEM
cat -s $FILE_TMP_ITEM > $FILE_ITEM_OUTPUT
# Push items to TMP FEED
cat -s $FILE_ITEM_OUTPUT >> $FILE_TMP_FEED
# Push RSS footer to TMP FEED
cat -s $FILE_RSS_FOOTER >> $FILE_TMP_FEED
echo $FILE_TMP_FEED
# Publish feed
cat -s $FILE_TMP_FEED > $FILE_RSS_OUTPUT
echo "Finished generating feed"
Some key takeaways are:
- sed lets you do regex with delimiters that AREN’T
/
so you can substitute something that shouldn’t actually ever show up in your regex. For me that is~
. - I always forget you can use sed to extract between tokens - which
is how I get the CDATA for the RSS:
sed -n "/<article>/,/<\/article>/p"
mktemp
is really REALLY useful - and I feel is under utilized in shellscripting
The obvious cracks are:
- I rely SO much on
sed
that it’s almost certainly going to break - I don’t have much other flag control to do partial generation - so if I need to do something either starting partway through or not finish the full process, I don’t have that.
- Sometimes things can break silently and it will go through, there is no verification or like manual checking along the way before publishing the feed.rss
The final two can easily be managed by writing the feed to a location
that isn’t a temp file and I can manually do the cat -s
$FILE_TMP_FEED > www/blog/feed.rss
myself after I check it
over.
But for now I’ll see if I ever have to redo it. I don’t think anyone will actually sub to this so I don’t really need to care that much if I amend the feed.
Where to put the feed URL
I never intended to provide an RSS feed. I doubt anyone but me reads this, and from my previous experience with gemini feed generation was a bit of a headache.
A quick aside: I really only decided thanks to Mastodon. I was thinking during the Twitter meltdown “what if twitter but RSS” (I know super unique idea). But basically like a true “microblog”. And some OSS tools to publish your blog. This got me reading the RSS spec and looking into it more - which then lead me down the using the RSS readers more (in conjunction with gemini, and Cortex podcast talking about using RSS more).
But I’ve decided to just put the RSS feed in the blog index, on my homepage, and that’s it. I don’t need it permanently in the header.
Conclusion
I didn’t have much to share here, it doesn’t make too much sense to write a big post on what can be explained better by just checking out the shell scripts in my git source. The code speaks better than I ever could.
I really, really like shell scripting.