This post has been replaced by Organize and Write Your Next Novel with Obsidian. I am leaving this post here for those who are interested in digging into the archives, but the most up to date information is in Organize and Write Your Next Novel with Obsidian – pdworkman.com.

Last updated: Dec 18, 2023

Jump over to my Linking Your Thinking session to see some of this in action.

As a prolific author, I am able to test and tweak my methods to come up with my own workflow and best practices more quickly than most.

I used Scrivener for drafting and formatting books for about seven years. That workflow involved generating a docx file which I then manually tweaked with drop caps and other formatting features I could not accomplish directly within Scrivener, then using that file to create the other sizes/bindings that I needed. Then, of course, save them as pdf files for upload to my distributors. If I had changes to make to a book, I had to change the three or more docx files individually, as well as the Scrivener file which I still used to generate the epub and mobi versions. It was quite intensive. I was also running into problems with files becoming corrupted, taking huge lengths of time to open/close, and not converting properly from Scrivener version 2.0 to version 3.0.

I finally broke down and purchased Vellum in the summer of 2020, and at that point, began to rejig my workflow to eliminate Scrivener. (I have not completely eliminated it, as I still use it as a tool for formatting large print and my WordPress portfolio files, but I no longer write in it, I just import a Vellum docx export file for that purpose.)

It took a few different attempts at using docx or odt files for my first draft before I landed on using Markdown files (one per chapter) for drafting, then compiling the Markdown files for my editor, and importing the finished file into Vellum for formatting and publishing. Now when I have changes to make, I only need to make them in Vellum, not in half a dozen different files, and I can produce my ebooks and print interiors with just a few clicks.

Initially, I was using Notebooks App to create the Markdown files and compile them, and have since made the shift to doing it all in Obsidian, using the Longform plugin. By my count, I have written 13 books in Markdown, so I have had 13 iterations to tweak and refine the process discussed in this article. (Updated — up to 52 now.)

Note: if you are moving a project from Notebooks to the Longform plugin, you need to do one tweak first. Notebooks creates a plist file for each of your MD files, which contains various internal information about that file. In the latest version of Longform, you can tell it to ignore plist files. Longform does not know to ignore plist files and does weird things with them. So before marking your folder as a Longform project, take a moment to either delete the plist files or to create another copy of your manuscript in another folder with only the MD files in it. If you can’t see the plist files, there is a setting in Notebooks to make them visible or invisible.

Why Markdown?

Markdown files are plain text, with a few added characters or symbols used to format it for conversation to HTML or other formats. Being plain text files, they are very small and not likely to become corrupted. Writing one scene or one chapter per file also keeps them very small and can be opened in an instant. Since they are written in plain text, they can be edited with any text processor or word processor on any platform and are future-proof. Plain text will never become obsolete.

What is outlined below is what works best for me at this time. That may change and be tweaked as other tools become available, but I intend to keep Markdown as the central file format for drafting. A different process or set of tools may work better for you, so take what you will and don’t worry about the rest.

Versioning and file backup

I use Dropbox to sync all of my data files across all of my devices, and the license that I pay for includes versioning, so I can roll any file back at any time if it becomes corrupted, overwritten, or something else disastrous happens, like a hard drive failure. I also use Mac OS Time Machine backups to four different hard drives so that I always have additional copies which can also be rolled back. And when I am actively drafting a new book, I also compile and save a backup every few days.

My file folder structure for a novel

• Writing
  • My books
    • Series folder (Obsidian vault)
      • Novel folder (Longform project folder)
        • archived (old drafts/versions of anything)
        • backup (compiled backups of book made every few days)
        • characters (character motivations/notes for this book, different than the general notes on the characters in the series wiki)
        • Drafts (Longform drafts folder)
          • draft 1 … 2 … 3 …
        • Editor or compiling notes (tasks about things I need to look at/write/review later, formatting, things to bring to the attention of my editor)
        • graphics (covers, promotional graphics, ads, etc.)
        • marketing metadata (back cover description, product listing, tags, tropes, comps, etc.)
        • prep – preplanning – snowflake, outline, character sketches, blurbs, etc.
        • published (Vellum output folder for this novel)
        • queries (saved search templates for Obsidian)
        • Research and Notes
      • templates (character notes, snowflake template, todo query template, word count tracking)
      • Series wiki (linked “series Bible”)

Writing in Markdown

Writing your manuscript in plain text is pretty simple and I assume if you are reading this article you have a basic understanding of Markdown. (ie. double-returns between paragraphs, no tab indent, use *asterisks* for italics.)

I am not going to give a tutorial in using Longform or any of the other plugins mentioned in this article. You can find that documentation elsewhere. Here is one comprehensive guide.

Best practices

These are my “best practices” for drafting a book using Markdown files for eventual use in Vellum. They may or may not work for you, but will maybe give you a starting point for your own system.

File names

I write one chapter per Markdown file. It may consist of 1-3 scenes. I like to use descriptive file name for the chapter so that I can quickly find what I am looking for and jump from one scene to another. They are numbered so that they can be sorted sequentially in any file viewer or editor. (If I look at them in the Mac Finder or on my iphone dropbox app, they are in order.) Some apps will let you manually arrange the chapters in the order you want them in, so you can drag them around until it suits you. I would suggest that once you are pretty sure of the order, you number them. I generally write in sequence, but sometimes I’ll write three or four chapters without a number, just in case I decide I want to add or move around a scene, and then number them once the sequence is solidified. Of course, you can always move things around and renumber again later. I would like to see the Longform app add auto-numbering at some point. Having chapters numbered sequentially in Obsidian means that you can sort them in order in Mac Finder, Windows Explorer, or any other file browser.

When I import into Vellum, I clear the title and just use let Vellum autonumber the chapters. If you use named chapters in your final copy, then use your chapter names as your file names.

Tip: If you find that Longform has reordered your scenes (which it seems to do sometimes when I switch between computers), it can be pretty time-consuming to drag them back into the right order again, especially with fifty or more chapters. If you have your files numbered and open the Longform index.md file and sort the chapters there, they will automatically be reodered in Longform. I use PopClip on the Mac and it has a sort extension, so I can just highlight the block of scenes and tell it to sort and I’m done. I use two-digit numbers so that 01-09 sort before 10. —I haven’t had this issue since the last version of Longform was released, but do occasionally find that Longform forgets my project and it needs to be redefined.

Front matter

I put searchable metadata in the front matter of each MD file. There are a number of different ways to do this, depending on which editor you want to use as your main Markdown editor. This metadata is “commented out” so that it will not appear in the final file, but can be used during the drafting stages to quickly find scenes or necessary information. Four methods of commenting out follow.

Commenting out

YAML

--- 
metadata 
---

Three hyphens are the first characters in the markdown file, followed by the metadata, followed by three more hyphens. YAML is a standard format for front matter, but is not recognized by all apps (like Notebooks). It is recognized by Obsidian.

The information normally included in a YAML header is things like title, author, and date (examples). Some apps or plugins (such as Obsidian Dataview) may make use of other variables to be included in the YAML header. You can put other Dataview fields anywhere in your file, so they can be inserted between any of the other pairings below as well.

If you are using the Obsidian Longform plugin to compile your scenes into one document, you can’t put a YAML header in each scene, only the first will actually convert to a header. Choose the Percents or HTML comments format below instead.

Updated: Longform was updated after this article was written, and you can now tell it to remove the YAML headers when compiling. I have two different compile workflows. I have one to back up my current wip to a compiled file, and I leave all of my metadata comments and YAML in that one. I have a separate compile process to produce the file file for my editor, and that one removes all YAML and comments. I am now using YAML headers rather than comments.

Updated: Obsidian has now added the Properties core plugin that gives you a nice editable sidebar view for your properties, which is nice if you want to make a change to your properties when you are down at the bottom of the file without scrolling back up to the top. (Screenshot below)

Percents

%% 
metadata 
%%

Recognized in some apps but not in others. Does not work for Notebooks App. Can be used anywhere in the document.

If you are using the Obsidian Word Count Dashboard to track your word count, use this method to comment out your metadata so that they will not be counted with your novel’s words.

Slashes

// 
metadata 
//

Recognized in some apps but not in others. Does not work for Obsidian. Can be used anywhere in the document.

HTML comments

<!-- 
metadata 
-->

Works in the majority of apps. (I won’t say all, because, who knows, but this one has worked pretty universally for me.) Can be used anywhere in the document.

This is the only method I could find that works for both Notebooks App and Obsidian. It is not recognized by the Obsidian Word Count Dashboard, but you can nest %% inside of the html comment bookends to make the counter ignore the metadata. Word Count Dashboard has been updated to recognize this format for comments. The word Count Dashboard will also not count words in YAML.

Metadata I Use

You can use the YAML header/Properties or other metadata section to add whatever tags or information you want to track, search, or filter. My metadata may look something like this.

I precede character names with ; and setting names with ~ so that I can also search/filter them quickly from any file explorer. Why would I not just search Malachi instead of ;Malachi? Because I don’t want to bring up every scene that Malachi is mentioned in, only the ones where he is a main character in the scene.

I only use POV in books that have multiple points of view. If there is only one POV in the book, I don’t need to track it. Similarly, if every scene in the book uses Joe Blow’s POV, then I don’t include ;Joe in the list of characters. He would be listed for every single scene and there’s never any need to filter on him.

Day helps me to keep track of day of the week or month, length of time between scenes, etc. so I don’t have to go back and count days later.

Note is anything that I need to remember for later.

Draft is the draft number

Grammarly indicates whether I have run the scene through Grammarly. Yes—you can copy and paste Markdown text into Grammarly and it will not screw up the paragraph spacing, italics, etc. like many formats do. You can check the file and paste the results back into your Markdown file, and everything will still be properly formatted.

Updated: You can also use the new Grammarly desktop app to check spelling and grammar within an Obsidian file rather than having to copy and paste in to the editor. It is still a little flaky, so I’ve gone back to copy and paste into the web app.

Critique means that the chapter has been critiqued and TTS that I have proofed it using Text to Speech.

As well as letting me search or filter by these variables in any file browser, I can also use the Obsidian Dataview plugin to produce a tabulated summary, like this:

Dataview query

Produces these results

Updated: I have updated my CSS to eliminate the bullets, so it is a bit cleaner now:

Other special formatting considerations

H1 Headers for chapters

H1 headers are automatically recognized by Vellum as chapter titles. If you use H1 headers, you will never fail in your chapters dividing in the correct place and retaining their names when imported into Vellum. Other methods will sometimes fail to divide every chapter correctly.

Notebooks has the option to make the first line of a file an H1 header (and by default synchronizes the first line of the file and the file name.) The Obsidian Longform plugin lets you add your Scene name as an H1 header in the compiling process.

If you are using other tools, create an H1 header by typing # followed by a space and your header.

# Header

Marking scene breaks

If you are importing into Vellum, then you want all of your scene breaks to be properly marked. Trust me, you don’t want to have to input all of your scene breaks by hand after the file has been imported!

If you use *** or --- to mark your scene breaks, then they will be converted to a horizontal line <hr> when you convert to docx. And horizontal lines are not recognized by Vellum as scene breaks.

If you use +++ to mark your scene breaks (you don’t need to center it), it will not be converted to a horizontal line when you convert to docx. It will stay as a +++. And +++ is recognized as as scene break by Vellum.

My Obsidian Longform plugin formatting

You can format your editing and preview screens in Obsidian using CSS formatting that only applies to Longform project files.

Update: I no longer do much CSS formatting, other than changing the colour of the YAML header. But I’ll keep this here just for reference if you want it. I thought that when I went back to read my books, it would be handy to read in the reading view (preview screen) shown below, but I actually never use that view.

My edit screen:

My preview screen:

My sample CSS

(Updated)

/* Font for the markdown source (edit) view */
.longform-leaf .cm-s-obsidian
{
    font-family: Atkinson Hyperlegible;
    font-size: 1.1em;
    line-height: 2em;
}

.longform-leaf .cm-hmd-frontmatter
{
    font-family: monospace;
    font-size: .8em;
    line-height: 1em;
    color: firebrick;
}

.longform-leaf .cm-comment
{
    font-family: monospace;
    font-size: .8em;
    line-height: 1em;
}

.longform-leaf .cm-header-1
{
  font-family: Garamond;
  font-size: 1.5em;
  line-height: 2em;
  font-weight: bold !important;
}

.longform-leaf .cm-header-2
{
  font-family: Garamond;
  font-size: 1.2em;
  line-height: 1.5em;
}

.longform-leaf .cm-header-4
{
  font-family: Garamond;
  font-size: 1em;
  line-height: 1em;
  font-weight: normal;
}

/* Font for the markdown preview view */
.longform-leaf .markdown-preview-view {
    font-family: Garamond;
    font-size: 1.2em;
		text-indent: 1em;
    text-align: justify;
}


.longform-leaf ul {
  list-style-position: outside;
}


.longform-leaf ol {
  list-style-position: outside;
}

.longform-leaf li {
    padding-left: 0em;
    padding-bottom: 2 em;
}

.longform-leaf h1 {
  font-family: Garamond;
  font-size: 2em;
  text-indent: 0em;
  margin-top: 1em;
}


.longform-leaf p {
  margin-bottom: -15px;
  line-height: 1.3em;
}

.longform-leaf .admonition-content p {
  margin-bottom: 0px;
  line-height: 1.3em;
  		text-indent: 0em;
}

.longform-leaf div.callout p {
  margin-bottom: 0px;
  line-height: 1.3em;
  		text-indent: 0em;
}

.longform-leaf .markdown-preview-sizer .markdown-preview-section {
    font-family: Garamond;
    font-size: .8em;
		text-indent: 0em;
    text-align: justify;
}

.longform-leaf {
  --background-primary: antiquewhite;
  --background-primary-alt: white;
  --background-secondary: white;
  --background-secondary-alt: white;
  --text-selection: #aaa;
  --text-normal: black;
	--text-faint: blue;
  color: black;
}

.longform-leaf .suggestion-item.is-selected {
  background-color: var(--text-accent);
}

.longform-leaf .markdown-preview-view blockquote {
    border-color: darkgrey;
    font-size: 95%;
    text-align: left;
    hyphens: auto;
    word-break: keep-all;
    color: black; # you need to define
    font-color: #aacdbe;
    line-height: 1.3;
    padding: 10px 2% 10px 2%;
    margin-top: 15pt;
    margin-bottom: 15pt;
}

/* Remove bullets from dataview table */

.dataview-result-list-li {
  list-style-type: none;
  padding: 0 !important;
  indent: 0em;
}

.list-view-ul {
  list-style-type: none;
  padding: 0 !important;
  indent: 0em;
}

.callout[data-callout-metadata="x"] .callout-title {
  display: none;
}

Old Workspace

My layout when planning is pretty fluid. I may have a couple of side-by-side panes/tab groups, along with a separate window showcasing the mindmap or kanban board for the book.

Here is my one workspace layout I have used while drafting or editing that Scrivener users may like.

Working clockwise around the workspace:

Top left – Explorer

The Obsidian Explorer, search, and Longform plugin panels. I switch between them as I like. Usually set for Obsidian Explorer.

Central – Editor

Pretty self-explanatory. The scene or scenes that I have open in the editor at the moment. Obsidian now allows these to be set up as tabs.

Top right – Unlinked Mentions

As I mention people, places, magical races, etc. that I have series wiki entries for, research articles, etc., they will pop up in the unlinked mentions panel. I would like to be able to just click to jump to a reference, but at the moment, you have to click to link, and then click to follow the link. You can either remove the link afterward or leave it in there and have Longform plugin remove it from the exported manuscript. I’d love to see more unlinked mentions functionality in Obsidian. To use this functionality, you need to have the Core plugin Outgoing links turned on.

Bottom right – Starred

My favorited documents. I can quickly jump to my to do list for the book, my Outline, scenes summary, open my Excel word count spreadsheet to record today’s count, etc. Cmd-click on a file to open it into a new tab. Update: I now often open these into another tab group rather than using favorites.

Bottom left – Metadata

This is my answer to the Scrivener Inspector. It is simply the document I am currently working on, scrolled to the top so I can see and modify the metadata in the yaml header. I can make or refer to notes, change dates or variables, etc. I don’t have a way to magically link so that the document in the main editor is always the on showing up in this panel. I just switch to it and Cmd-O, then press enter to select the last document that I worked on.

I often place checkbox task items beneath the yaml header if there is something in the scene I need to come back to check or fix later, as noted below. (If you put them inside the yaml header, a query will not find them.)

New Workspace

## Left Panel

File Explorer, Search, Bookmarks, and Longform scenes

Middle Panel

Editor

Right Panel

Most often, the Properties View.

Tasks within a project

I normally manage all of my projects and tasks with Todoist, and any tasks that I note in Obsidian, I migrate to Todoist. However, for book drafts, I keep the internal editing tasks inside Obsidian. I may have “Review ACB 20 tasks/edits” in Todoist, but the tasks I’ve made while drafting or editing live within Obsidian, embedded within the scene they apply to.

I have a Todo document for each draft, something like this:

The top section is tasks that apply to the whole book generally. Things I need to check for consistency or change after I am finished the current draft. I try to always keep things moving forward as I am writing the first draft and don’t go back to do corrections until that draft is completed.

The middle section is a query for unchecked tasks in each scene. You see the scene name and then the checkboxes found within that scene.

Update: I have a query here now for checked tasks too, because sometimes I accidentally leave them in the document before outputting to Word, and you can’t search them there. So now I make sure I have them all deleted before I export.

The bottom section is things I need to do before sending the final draft to my editor, notes of things to discuss with my editor, etc. and things to do when I get it back, before or after importing into Vellum as I get it ready for publication. The initial tasks are part of my Todo template and I add to them as I write.

I also have an audio section now which is words that need to be checked for proper pronunciation when I convert them to an autonarrated audiobook.

Other settings/plugins that I find helpful drafting novels in Obsidian

As well as the Longform and Pandoc plugins (below), I also find these to be useful when drafting novels.

Novel Word Count

I am now using Novel Word Count plugin, which automatically totals the number of words written in each file and folder. Sometimes this plugin has a memory problem and slows everything down. If it causes problems, I shut it off and only turn it on to check my word counts when I need to.

Collapse All

Quickly collapse all of your folders in the explorer.

Dataview Plugin

Needed to produce the data summaries you see above with my metadata and word count.

Smart Typography Plugin

I want curly quotes and apostrophes. I want my ellipses converted. Other than that, I turn the rest off.

I recommend that if you use Smart Typography, you turn off “auto pair brackets” in your editor preferences. Otherwise things can get a little wonky.

Templates Plugin (Core)

I use templates for my character notes and my snowflake planning. I could also use them to automatically insert my metadata template. Alternatively, you could use TextExpander or another utility for this.

Starred Plugin (Core)

I like to star certain reference documents, such as my outline, word count dashboard, etc.

Update: This is now the Bookmarks Plugin, which is a handy little tool.

Wikilinks (Preferences)

I use Wikilinks to set up my series wiki (series Bible).

Excalidraw

I am now using the core plugin Canvas rather than Excalidraw for storyboarding.

I have tried some storyboarding with Excalidraw, and it works out pretty nicely. Any object can be hyperlinked to a scene:

Kanban

Kanban is another way to visualize your plot, manage tasks, etc. I like it better than a regular outline because it is so easy to drag and reorder both the lists and cards. You can view it as either a Kanban or as a markdown file (headers with tasks).

Hover Editor

Hover editor will let you edit documents in a floating panel, so that you can, for instance, hover over a link in your Dataview tabular query and edit it on the screen without leaving the tabular results.

Update: I’m no longer using Hover Editor as I found it too annoying.

Text Generator and ChatGPT MD AI tools

Useful for summarizing, critiquing, generating names or titles, social media posts, ad copy, preliminary research questions, etc.

Note Refactor

Sometimes I do sprints in OhWrite. While it doesn’t say so anywhere on the site that I’ve seen, you can use Markdown within the OhWrite editor. So I mark my chapter breaks with ## for an H2 heading and insert my YAML header right in OhWrite, and then paste the whole thing into a note in Obsidian when I’m done. Using the Note Refactor plugin, I can instantly split this file into the multiple chapters without copying and pasting each manually.

Compiling and Converting to final format

If you are drafting one scene or chapter per file, as you do with the Longform plugin, then you are going to need tools that (a) combine those chapters into one long file and (b) convert that compiled file into docx format for your editor and/or for import into Vellum and/or to convert into epub and pdf for publication. I am going to focus on getting from Markdown to Vellum here, you will need to experiment and adapt for other formats.

Compiling tools

The two compiling tools that I have used are:

Notebooks App

You can compile all files in a folder to one HTML file by clicking the three dots at the top of the panel and selecting compile. By default, the first line of the file becomes an H1 header. By default, the first line of the file is the file name, and changing one changes the other. This prevents you from using a YAML header in Notebooks App, since the three dashes need to be the first three characters in the file.

Obsidian Longform Plugin

You can compile all files in the draft folder of a Longform project by going to the Longform panel compile tab and clicking compile. Prepend the title to the scene as shown below to add it as an H1 header.

Converting to your final file format

In order to import cleanly into Vellum, you want to convert to HTML first. Converting to HTML first will retain chapter names as H1 headers, which Vellum will recognize as chapter names. Converting directly to docx will not retain the H1 headers.

Notebooks App

You can compile directly to HTML as mentioned above.

Pandoc

The Obsidian Longform Plugin will compile all of the Scenes into one long MD (markdown) file.

To convert from MD to docx, you will want to use Pandoc. Pandoc is a command line file converter that lets you convert from one file format to another in Terminal. But don’t panic. If you are using Obsidian, install the Pandoc Plugin. (If you are using another Markdown editor, you will need to see what the options are available, learn Pandoc command line prompts, or use an online tool such as StackEdit to convert the file from MD to docx. Katrina commented below that she uses Ulysses to compile and convert her manuscript.)

To convert using Pandoc plugin in Obsidian: navigate to the compiled MD file and click on it. Pull up the command palette (cmd-p) and type docx. You should see “Pandoc Plugin: Export as docx” Select this option to convert your MD file to a docx file.

This new docx file can be imported into Vellum and all Chapter and Scene breaks will be properly converted. (Of course, you may want to send it to your editor first, and the import the finalized file into Vellum.) If you don’t want your chapters to have names, you can select all of your chapters on the left navigation panel and clear the titles. Then they will just show up as Chapter 1, Chapter 2, etc.

Vellum does not retain formatting such as block quotes, so if you have sections that will need to be converted to alignment blocks or other special formatting, I suggest that you use a symbol combination to mark each of those sections while working in Markdown, and then search for that combination in the Vellum file and format it accordingly.

eg. use “999 alignment block” and search for 999 in the text in Vellum after importing.

Vellum also does not import graphics embedded in your docx file. You will need to insert them manually.

Other formats

You can use Pandoc to create pdf files, epub files, etc. directly and not use Vellum or another formatting program. I have not experimented with this for novels. I have used Pandoc to create very nice formatted pdf files for printed newsletters (for people who want print copies rather than emails), but that is a topic for a separate article. It would take a little bit of work to get exactly the format you want for a book, but it would certainly be possible. Pandoc even has codes for running headers and footers.