Hi all,
I am looking for a local database that is easily accessible via the command line.
It can be SQL or non-SQL
Whats my use case? I want to use it kinda like a second brain. A place to save my notes, my todo lists, my book reading lists, links / articles to read later, etc.
I want it to be a good CLI citizen so that I can script its commands to create simpler abstractions, rather than writing out the full queries every time.
Maybe sqlite is what I need, but is that ideal for my use case?
Edit: removed notes, as evidently they aren’t suitable for this and aren’t like the rest.
You must log in or # to comment.
I wouldn’t recommend a DB for note taking purpose. You can use Markdown with vim.
Agreed. You can still do tags with filenames like
===Category===Tag 1---Tag 2===Note name.mdor some such format.Perhaps I shouldn’t have said note taking. But the rest still applies.
Counter-proposal: Perhaps use Obsidian instead?
I second this, as it’s my use case.
Providing you lay out each note correctly with appropriate frontmatter, Dataview’s DQL and DataviewJS give you all the SQL-like functionality you could want.
Plus a load of useful functionality beyond a plain DB.
Add in the local REST API as well if you want to easily interact with your notes programmatically:
Had no idea about this. Very useful, thanks!
I want something I can use from the command line, so that I can integrate and interface it with other unix tools and scripts.
As far as I can tell, you can do.
Oh wow that’s pretty cool. From my understanding this is good for notes, but would it work for tasks / todos?
It has a huge number of plugins - this is a list of the productivity ones. You’d need to do some reading to see if it has what you require.
Is there a reason you’re not looking at tools explicitly built for this like orgmode, obsidian, task-warrior, etc? There’s a plethora of these tools and my experience with this is you really don’t want to over-engineer your productivity suite.
That said, if you go the SQL route, sqlite is the way to go. Other SQL databases must be run as a daemon whereas sqlite operates on a local file directly.
However any SQL database isnt going to have the CLI youre asking for. Its interface is… SQL, so you’re scripts are going to have a bunch of SQL code embedded that isnt easily reusable. A non-sql database will probably be better. I’m not familiar with them but I think there’s some that store their data as text files in a folder which is organized a certain way. But that starts looking like the tools I mentioned before.
Task warrior was close to what I wanted. I forgot what it lacked when I tried it. I think it was dependent tasks or sub tasks? If not that, then a tag system with flexible querying. I want querying by tag, due date, and other attributes to be possible.
I think orgmode may be what I want, but the learning curve was discouraging. I was also discouraged about the possibility for myself to build extensions for it, for example to use on android. That would be easier with sql.
Obsidian is gui from my understanding, so it wouldn’t fit what I’m looking for. I want something I can integrate with my scripts and other unix tools.
Yeah I only suggested obsidian because its so popular and is completely out-of-the-box.
If you want everything exactly as you want it you’ll need to spend time coding it all yourself. Otherwise you’re shopping around for different tools for specific things. Some editor plugin for notes. Another for tasks. Another for reminders etc.
My issue with task warrior was its syncing service
taskd. It required that you generate a self signed ssl certificate. You couldn’t host it behind caddy. But all the issues listed I’m pretty sure it covers. Its extremely robust.
Are you an emacs user?
Try org-roam. It’s a similar system to obsidian, but fully open source. You have all the note taking techniques of org-mode, and all the scripting power of emacs.
There is also a “web-version” of org-roam named “orgnote”: https://github.com/Artawower/orgnote
Using a database for notes is like going 2km in a plane.
You can use any relational database for this though but why would you subject yourself to this?
I think you’re right about notes, but what about the other use cases?
Obsidian has a huge number of plug-ins, that cover a wide range of uses. I also use Zettel Notes as a quick md file editor and that has the ability to capture pages in markdown to read later. And/or you can use Omnivore.
If you are generally interested in designing databases its a good exercise to try and build one yourself. That is the only situation I recommend you go this route
@matcha_addict Anything beyond SQLite is too heavy, but if it were me I would use a lightweight wiki like dokuwiki. Having to run SQL to do all these sorts of things sounds like it would just get in the way of both getting and consuming your thoughts.
I don’t think it would suffice. It would work for notes, but I want to also have a todo list in there, for example. Be able to check things off, query by due date, by assignee or task, etc.
I don’t think a database is a good fit for your use case at all, unless you’re willing to reinvent a lot of wheels.
What would then? I haven’t found anything that has command line interface and supports dependent tasks and a flexible, queryable tag system
I’m sorry, I don’t have a suggestion. If it were me I would not be looking at a CLI solution for this at all.
For todo stuff there’s the excellent “todo-txt” for cli use. Though not great for notes. Perhaps separate tools would be fine?
Todo txt lacked some features I needed, which would be trivial to implement as a SQL column.
I do not remember what it was… Maybe it was dependent tasks? I believe so.
Org mode, org roam, org agenda and org-ql
Joplin is a note taking app that stores its data in an sqlite database (easy to query but not a good idea to write to it) but there is also a command line version and both versions support access via a data API.
I think you would be better off using something like org-roam. It’s all text so script can still be used and it can be searched fast with ripgrep. Also org mode has loads of features that a homegrown system will never be able to catch up with
Is their any DB that doesn’t have a CLI?
I think what I’m looking for doesn’t exist, but what I meant by CLI is something I can pipe things into and interface with other unix tools easily.
But you’re right, they all have a way to open a session via CLI.
Unless your lists are 1000s of items long, just use text files in a folder.
Could even go fancy and use markdown.
I definitely considered this approach. The only problem is it is difficult to represent linking items to each other. For example, dependent tasks or sub tasks.
Duckdb has pretty good cli support https://duckdb.org/docs/api/cli/overview.html#non-interactive-usage
What if instead you used something that’s meant to be used to take notes but that also has querying capabilities?
I have been using Silverbullet for a while and I absolutely love it. It uses Markdown files in disk so it’s very easy to backup, have secondary instances running and even just edit files directly with any other program. But also provides some extra syntax to define objects and query them, so you can for example build a library of recipes and have a page that lists all of the ones that have a specific tag or take less than X time to cook or whatever.
Check out https://www.giuspen.net/cherrytree/, lightweight note-taking app with interesting scripting function built in.
Even if that’s not your cup of tea, it has the option to save your notebook to a single sqlite file, so I take that as good enough proof it’ll work for your similar purposes as well.
Definitely sounds like sqlite.
Hey!
I’ve wanted something very similar—specifically, a plain-text database. I recently came across GNU recutils, which I haven’t had time to play around with yet, but which seems like it fits the bill (at least for me). There’s a couple YouTube videos on it—I encourage you to check it out!
