| 00:06:19 | | etnguyen03 quits [Client Quit] |
| 00:09:00 | | Sk1d joins |
| 00:18:06 | <kline> | is there anyone who still has an interest in chaingang? I now have a text serialised version of the bitcoin (blergh) blockchain, and wondering what the best architecture for it would be to upload |
| 00:19:05 | <nicolas17> | what |
| 00:19:42 | <nicolas17> | what is chaingang? |
| 00:19:57 | <kline> | https://wiki.archiveteam.org/index.php/ArchiveTeam_Chain_Gang |
| 00:20:16 | <nicolas17> | oh |
| 00:20:57 | <nicolas17> | isn't that like impossible to get lost? |
| 00:21:24 | <kline> | an otherwise abandoned project to back up blockchains used for cryptocurrencies. i dont personally like them, but i do think bitcoin and probably ethereum could be considered internet-culturally-important and especially right now has taken a bit of a dive with the allocation of computing resources towards LLMs |
| 00:22:42 | <kline> | i dont think its impossible to lose - there are some subset of users of the network who maintain fully copies of the chain, often service providers who want to sell analytics etc, but most users do not have a complete copy and only maintain the last-N blocks to allow a bit of history |
| 00:23:10 | <kline> | i think monero has already got some holes in its blockchain but i would need to find a source on that |
| 00:23:16 | <nicolas17> | ah |
| 00:23:30 | <nicolas17> | last time I used this there were two types of clients |
| 00:24:01 | <nicolas17> | full-history and thin-client-ish |
| 00:24:16 | <nicolas17> | guess they finally implemented history pruning |
| 00:25:04 | | wickedplayer494 quits [Ping timeout: 256 seconds] |
| 00:25:17 | <kline> | full history nodes (with indexes) are closing in on 1TB of storage space (about 650GB is binary data, the rest are fast indexes so you dont have to iterate back through the chain every time you want to find a given block) |
| 00:25:50 | <kline> | naturally most people aren't spending a week+ downloading this to get started |
| 00:26:02 | | wickedplayer494 (wickedplayer494) joins |
| 00:31:23 | <kline> | failing anyone specifically interested, a more generic question: i have 206 files of monthly data coming to a grand total of ~850GB. Would it be better to structure this as 206 single-file items under a collection, or 1 item with 206 files? |
| 00:31:45 | <kline> | the largest individual file is 11GB |
| 00:33:21 | <pokechu22> | I don't have an opinion, but it's worth considering how you'd handle adding new data next month |
| 00:34:00 | <@JAA> | Something inbetween like yearly or quarterly items might be worth considering, too. |
| 00:34:32 | <kline> | pokechu22, you can add new files to an item afterwards, no? |
| 00:34:43 | <@JAA> | How is this data distributed in the actual network? As in, when you set up a full history node, how does it obtain all the data? |
| 00:35:03 | <pokechu22> | Pretty sure you can, though I feel like that can be a bit weird in some situations |
| 00:35:20 | <@JAA> | You can add more files later, yes, but there is a hard item size limit of 1 TiB, so you'd probably run into that pretty soon. |
| 00:35:23 | <nicolas17> | JAA: from other nodes p2p via the custom bitcoin protocol |
| 00:35:38 | <kline> | JAA, it bootstraps itself into the p2p network finding other full nodes, then just iterates through every block in order until its up to date |
| 00:35:49 | <@JAA> | Ah |
| 00:36:03 | <@JAA> | And how does the bootstrapping work? |
| 00:36:36 | <kline> | thats a good question, let me check how it finds its first peer |
| 00:36:38 | <nicolas17> | peers can tell you about other peers, to find the first ones I think there's some hardcoded IPs or a DNS record? |
| 00:37:53 | | SootBector quits [Remote host closed the connection] |
| 00:38:02 | <@JAA> | Yeah, that's what I'd expect, I think. BitTorrent's DHT works like that. |
| 00:39:03 | | SootBector (SootBector) joins |
| 00:39:17 | <kline> | DNS and hardcoded IPs of promised long-term nodes as a backup, apparently |
| 00:43:40 | | nexussfan (nexussfan) joins |
| 00:47:08 | | useretail quits [Quit: Leaving] |
| 00:48:02 | | Sk1d quits [Read error: Connection reset by peer] |
| 01:00:47 | | wickedplayer494 quits [Ping timeout: 272 seconds] |
| 01:01:50 | | wickedplayer494 (wickedplayer494) joins |
| 01:06:12 | | Wohlstand quits [Quit: Wohlstand] |
| 01:14:12 | | etnguyen03 (etnguyen03) joins |
| 01:25:12 | <h2ibot> | PaulWise edited Obstacles (+44, Spigot poisoner): https://wiki.archiveteam.org/?diff=60473&oldid=60388 |
| 01:33:13 | <h2ibot> | PaulWise edited Obstacles (+55, sethrawall): https://wiki.archiveteam.org/?diff=60474&oldid=60473 |
| 01:34:59 | | wickedplayer494 quits [Ping timeout: 272 seconds] |
| 01:39:29 | | wickedplayer494 (wickedplayer494) joins |
| 02:32:33 | | LddPotato quits [Read error: Connection reset by peer] |
| 02:34:49 | | LddPotato (LddPotato) joins |
| 02:44:39 | | Bleo1826007227196234552220 quits [Ping timeout: 272 seconds] |
| 02:44:39 | | LddPotato quits [Read error: Connection reset by peer] |
| 02:46:02 | | LddPotato (LddPotato) joins |
| 02:51:45 | <Yakov> | Speaking of archiving archive.today captures, can't we make a system where users solve captchas towards a pool of browser sessions running at AT? Might not generate WBM-valid warcs but will at least be able to be preserved to some extent |
| 02:52:49 | <Yakov> | Thought of this for a while, where the captcha is streamed to the browser via novnc so people can contribute solving a captcha that's being operated on a remote server |
| 02:53:47 | <Yakov> | Then we can scrape popular sites (like wikis) which reference archive.today so we can build a nice priority list of urls |
| 02:56:56 | <Yakov> | Can technicality even be a valid warc if it's acceptable to omit the captcha request from the warc - not really sure about the standards of that |
| 03:03:14 | | LddPotato quits [Read error: Connection reset by peer] |
| 03:04:47 | | LddPotato (LddPotato) joins |
| 03:24:03 | | LddPotato quits [Read error: Connection reset by peer] |
| 03:26:00 | | LddPotato (LddPotato) joins |