Re-Organizing Obsidian.md into a Pseudo-Database Manager
Photo: Scott Coleman, 2023.
December was a busy month, as it is for many people. However, this year was more hectic for my partner and me. This was because of the revolving door of family and friends coming to see our newborn son. With course marking, family and friends, baby duties and Christmas holidays, I did not work on my Obsidian vault much. For the new year, I am diving right back into it.
Obsidian.md has numerous uses and plugins to customize your vault. Many people use it for note storage. Indeed, I primarily use it in this manner. However, I want to push it further and explore its viability for qualitative and quantitative analysis. Is this possible? Some have said this is different from what the software was designed for. It is not a Database (DB) manager. True, but this is a question of semantics. What do we consider data, and how do we represent it? How can we visualize data, and what digital tools can achieve effective visualization?
Visualizing relationships within Obsidian is simple, as the software has a graph view that facilitates visualization of your notes and their connections to other notes through nodes and edges, like Network Analysis. However, it is more of a gimmick and shows what you personally consider connected; it is not the software making these connections for you. Thus, you need to be mindful and critical about why you are making connections if you use Obsidian for any form of academic research. This is where Ryan Murphy's Obsidian template for Qualitative Analysis comes into play.
Fulcra.design by Ryan Murphy.
Since I have been wondering how to reorganize my notes and data from various projects and create a pseudo-database for the Inhabiting Byzantine Athens I participate in, I decided to reorganize my vault in a semi-Zettlekasten system (See Zettelkasten - Wikipedia for a quick overview. Yes! Its Wikipedia). What I mean by this is to use numerical sequences for note titles. I will explain this in more detail below. My supervisor, Dr. Shawn Graham, sent me a link to Ryan Murphy's Fulcra.design (thank you, Shawn). Murphy developed a template and method in Obsidian to aid in qualitatively analyzing open-ended questions about student experiences during COVID-19. Murphy converts notes into Data Points that are then coded with related themes. The data points start with the number 1 and continue sequentially (2, 3, 4, 5...you get the picture). The linking of data and themes is achieved by wiki links (double brackets [[]]). He sets up a folder in the vault where all newly created notes/data points are deposited when a hotkey is pressed (i.e., Command + Shift+ N). Furthermore, by creating a template via the Templater plugin, your note/datapoint framework is populated and ready for you to input your data.
Below is not Murphy's template but one I have modified over the years for my Zettlenotes. The key to Murphy's analysis is the sequential numbering system, coded themes, and an extremely close reading of notes produced by the students. Murphy extracts key ideas into data points and then applies Grounded Field Theory for his analysis. For my purpose, I am interested in using a more extensive, more quantitative approach for analyzing numismatic data (coins). I intend to borrow some fundamental structures from Murphy's vault (available here via his Github) and insert the Coin data from the 1930s Agora excavation legacy notes into a separate independent vault.
Zettle note and Zettle note template in my personal vault. On the left side are my notes converted into a numbering schema.
Photo: Scott Coleman, 2024.
Furthermore, I am now converting all my notes into numerical data points. However, instead of Murphy's 1, 2, 3, etc., schema, I am using the daily note core plugin, which uses the current date in YYYY-MM-DD format for the note title. However, If I need to create multiple notes on the same day, the daily note plugin will not make a new note but, instead, take me to the original existing note. Obsidian will not let you create a note with the same title/number. Thus, I cannot have two notes called 2023-12-02. To solve this problem, when I create a new note via the daily note plugin, I insert -01 at the end of YYYY-MM-DD to apply an easily trackable sequential numbering system. Any new note produced after the original will follow as 2023-12-02-01, 2023-12-02-02, 2023-12-02-03.
The reason for this method is simple: I cannot generate a code in Obsidian that identifies the previous note number and generates the following number for the new note. I want to save time looking for the last numbered note if I follow the 1, 2, 3 schema; that would be perpetually frustrating. Also, if a note that uses the 1, 2, 3 schema is deleted or changed for some reason, the code must detect the number associated with the note and adjust all other notes accordingly. I explored the Obsidian Discord channels and various other hubs of Obsidian.md dialogue, and the automated sequential numbering issue has not been addressed, as far as I know.
I chose the two-digit placement at the end of the dating sequence to give flexibility for numerous notes made in a single day. The unique number identification will likely build a more authentic relationship in notes than creating word-centred titles (Tagging and metadata menus will be critical for note-taking and data-entry features but will be explored in another post). The goal is to give each new note a distinct identity (like in Zettlekasten) and the potential to add more numbered positions to the notes as needed to create a more authentic visual relationship in Graph View. Converting all my notes into this schema is tedious as I have hundreds of notes. Existing notes will be dated according to the dates the note was initially created, and then I will append the two-digit schema to the date. All converted notes are placed into a parent folder called Inbox. I applied the above method to all my Zettle and Observation notes. They will be distinguished from each other in the properties metadata.
There are multiple hurdles I need to address in this process of converting my personal notes—for example, labelling images and other media files, process notes, and more. The above reorganization strictly applies to my Zettle and Observation notes. My goal is to force myself to think more critically about what themes and notes are linked together since the themes will be unidentifiable unless key terms are specifically queried (an Omnisearch plugin will be helpful to facilitate this process). I am creating a separate vault for the Agora data. However, I cannot publish any of the data or its contents without explicit consent from the project director, which I hope to obtain this coming summer. I will begin inputting the Agora legacy data into my Obsidian Agora Vault when I develop a more precise direction for managing all the metadata associated with the hundreds of coins that need to be processed. I will write about these processes via my personal vault, as it is a guinea pig for this experiment. Fingers crossed, I don't f@k up years of notes.