Thanks, I haven't heard of it before. I wonder if it's possible to use OrbitDB to download a file from IPFS though? Based on it's CID, I mean, because that was my intention with Helia. I thought that one of the nice thing about reading books is that it takes time, and if users could be seeding their files from IndexedDB to the network, they could automatically and effortlessly become "contributing citizens".

Another interesting use case would be to see if this can replace (or be an addition to) SQLite as the database in which the queries are ran.

Yes, https://bjesus.github.io/teatime/

> Do static sites built with sphinx-build or jupyter-book or hugo or other jamstack static site generators work with TeaTime?

I guess it depends on what you mean by "work with TeaTime". TeaTime itself is a static site, generated using Nuxt. Nothing that it does cannot be achieved with another stack - at the end it's just HTML, CSS and JS. I haven't tried sphinx-build or jupyter-book, but there isn't a technical reason why Hugo wouldn't be able to build a TeaTime like website, using the same databases.

> datasette-lite > Loading SQLite databases: https://github.com/simonw/datasette-lite#loading-sqlite-data...

I haven't seen datasette before. What are the biggest benefits you think it has over sql.js-httpvfs (which I'm using now)? Is it about the ability to also use other formats, in addition to SQLite? I got the impression that sql.js-httpvfs was a bit more of a POC, and later some possibly better solutions came out, but I haven't really went that rabbit hole to figure out which one would be best.

Edit: looking a little more into datasette-lite, it seems like one of the nice benefits of sql.js-httpvfs is that it doesn't download the whole SQLite database in order to query it. This makes it possible have a 2GB database but still read it in chunks, skipping around efficiently until you find your data.

That's a fair point - I guess it could be abused. Databases are sorted by their number of GitHub stars, so I was hoping that with the power of the crowds it will be possible to minimize the bad effect such actor might have, by simply not voting them up.

Libgen is one of the databases that are currently available (and was probably a good fit because they already had their files on IPFS), but I think that this architecture, in which the UI is decoupled from the database, the database doesn't hold any copyrighted materials, and the files are downloaded from IPFS, is quite resilient and could be used for serving all sorts of content.

Yeah - the desire to have TeaTime run as a normal website. BitTorrent doesn't run over HTTP, unless you use WebTorrent, which most BitTorrent clients aren't 100% compatible with. This means you can basically only download from other WebTorrent nodes - and there aren't many.

I do think this might change with the introduction of things like the Direct Sockets API, but for now they are too restricted and not widely supported.

It's my first time working with IPFS and I agree it hasn't always been 100% reliable, but I do hope that if I manage to get TeaTime users to also be contributing nodes, this might actually improve the reliability of the whole network. Once it's possible to use BitTorrent in the browser, I do think it would be a great addition (https://github.com/bjesus/teatime/issues/3).

IPFS contributor here.

IPFS certainly has its flaws, but it's been getting much better in the last year. If you want to do P2P retrieval in browsers, it's practically the most advanced from all protocols.

We just published an update on this topic https://blog.ipfs.tech/2024-shipyard-improving-ipfs-on-the-w...

Thanks, I shall have a look into that. Technically the index too can live on IPFS, but the real challenge was how to know which databases exist, and where. This is what I'm using the GitHub topics for - everyone can create a repo and assign a topic to it, and the GitHub free API lets you search for matching repos. In a way, what I'm looking is to be able to make a request and get a different result when a new database is up, and the immutable nature of IPFS for example isn't great for that. GitHub stars and accounts also give some basic concept of content moderation. I should see what Pears offers in this space.

Generally, I see the distributed nature of TeaTime as an instrument for being resilient, and less so a goal by itself. My thinking is that the GitHub database repositories are "resilient enough", especially if they can contain just a config.json file pointing to database files living elsewhere. But I'm not a lawyer and could be wrong.

Ha, that's a great point... I can probably work on a meta "table of content" for each database, shouldn't be hard to do. But generally the database descriptions say what they have - public domain books, library genesis non-fiction, etc.

I need to check about Pocket - do you have any relevant documentation? Note however that TeaTime does keep an offline cache, both a list of your recent books and the files themselves, but it does it using LocalStorage and IndexedDB because I really wanted to prevent the need for having user logins. Maybe I can add an option to download a "state dump" kind of thing, so that you can a file with your history and can always import it. I too always feel that LocalStorage could be a little flaky, in the sense that you don't always realize that switching a browser would mean losing all your data.

Thank you! I've seen quite a lot of similar projects as I was working on this, and they all served as much inspiration and also a proof that this can in fact be done.

Also, thanks to whoever came up with thebookbay.org instance :)

Thanks! I really wanted to use verified-fetch, but it seemed like it doesn't really support all types of CIDs? In my case, most of the files use blake2b-256, and the last time I checked verified-fetch didn't like that :( Did something change?

Edit: I now realised we previously talked on the Discord channel, thank you for the help there!

Small correction:

You can also pass [@helia/verified-fetch] gateways as a fallback for when it *cannot* connect to providers directly (due to transports).

It does require an IPFS node, i.e. server to provide tha data. The point is more that data isn't tied to any specific server. By content addressing with CIDs you can fetch from multiple IPFS nodes and even become a "provider" for a CID once your IPFS node downloads a copy.

I'm currently using public IPFS Gateways to fetch the files from IPFS.

For fetching files from IPFS, you can use a public IPFS Gateway, and the Gateway is essentially your server. That's what TeaTime is doing.

The CORS headers seem fine, though I'm a little surprised about the content-type:

  $ curl -I https://bafykbzacedvzdo4hru3wul5excjthwjuz5ggd6jcs4wg77tg7bkn3c7hlvohq.ipfs.dweb.link/
  HTTP/2 200
  date: Thu, 28 Nov 2024 16:34:03 GMT
  content-type: application/x-rar-compressed
  content-length: 192215114
  access-control-allow-headers: Content-Type
  access-control-allow-headers: Range
  access-control-allow-headers: User-Agent
  access-control-allow-headers: X-Requested-With
  access-control-allow-methods: GET
  access-control-allow-methods: HEAD
  access-control-allow-methods: OPTIONS
  access-control-allow-origin: *

Can you try with another file? I do think that once I implement Helia's verified-fetch, issues like this should be less common.