Let’s assume I have a software application with millions of files. The files receive frequent updates.

Is there a way to distribute those updates across all those devices?

I’m thinking something like Torrents/IPFS/Resilio. . . But they all seem like overkill.

I’ve been looking at my Search Engine, and thought it would be cool to allow people to download the database themselves, and run a local UI, for maximum privacy.

I would just be creating records within the main database. I could offer searches through the main site and allow people to explore it on their desktops.

  • gole@lemmy.zip
    0·
    3 days ago

    Please take this as support: You need to look a little bit more into theories. You are reinventing things that don’t need reinventing (categorizing data), but are not thinking about the real pain points of building search that return relevant results (full text search performance, scalability, semantics, extracting user intention from queries).

    Here’s an article that I think is very good on the topic

    https://blog.wilsonl.in/search-engine

    • ki4jgt@feddit.orgOP
      0·
      3 days ago

      That’s because I don’t want that.

      I remember, as a kid, wandering the local library. Looking at all the books. Seeing the call numbers and author names. Exploring the shelves, as they transitioned from topic to topic to topic.

      My idea is to treat the Internet as a giant library, with domains being books on a shelf, and allowing my users to start at a random location, and then peruse the different listings as they transition from slight variations of the main topic.

      With all due respect, you don’t have the vision, man. To start looking at cultural anthropology, then have a link to a biological anthropology blog 3 lines down? That’s the goal. It’s not about a narrowed search. It’s about cataloging something, so people can go looking for it. To enjoy the hunt.