Say I have a large txt or CSV file with data I want to search. And say I have several files.

What is the best way to index and make this data searchable? I’ve been using grep, but it is not ideal.

Is there any self hostable docker container for indexing and searching this? Or maybe should I use SQL?

  • h0bbl3s@lemmy.world
    12·
    3 months ago

    You can import CSV files directly into an SQLite database. Then you are free to do whatever sql searches or manipulations you want. Python and golang both have great SQLite libraries from personal experience, but I’d be surprised if there is any language that doesn’t have a decent one.

    If you are running Linux most distros have an SQLite gui browser in the repos that is pretty powerful.

    • Ephera@lemmy.ml
      1·
      3 months ago

      I’d be surprised if there is any language that doesn’t have a decent one.

      Yeah, SQLite provides a library implemented in C. Because C doesn’t require a runtime, it’s possible for other languages to call into this C library. All you need is a relatively thin wrapper library, which provides an API that feels good to use in the respective language.