| 00:19:57 | | HP_Archivist quits [Quit: Leaving] |
| 01:10:30 | <Arcorann> | https://teddit.net/r/HobbyDrama/comments/kmv685/webcomics_the_impending_death_of_smack_jeeves/ |
| 01:15:23 | <thuban> | https://www.teddit.net/r/HobbyDrama/comments/kmv685/webcomics_the_impending_death_of_smack_jeeves/ghi4amm/ a possibly relevant comment |
| 01:17:25 | <thuban> | (also https://www.reddit.com/r/DataHoarder/comments/jtdov6/webcomic_host_smackjeeves_will_shut_down_on/) |
| 01:19:46 | <OrIdow6> | I think this makes 4 or 5 separate groups (including AT) making copies, now? |
| 01:20:04 | <OrIdow6> | I'm glad to say that I think the AT copy is the most comprehensive and highest quality :) |
| 01:26:52 | <OrIdow6> | I think this is the most ridiculous example of this kind of duplication I've seen so far |
| 01:30:10 | <@arkiver> | OrIdow6: awesome :) |
| 01:30:22 | <@arkiver> | just us using WARCs gives us a better copy :P |
| 01:44:44 | <OrIdow6> | Not only that, but out all the IDs (as opposed to just using wget); our checking how high the IDs went (one other effort only went to 200k); our getting the users as well; etc. |
| 02:00:01 | | Dallas quits [Client Quit] |
| 02:03:37 | | Dallas (Dallas) joins |
| 02:37:17 | | anelki quits [Quit: servus!] |
| 02:45:09 | <brad> | Here’s a question for you — who decides when something will be an AB project versus an AT project? And by what criteria do they use to make that decision? Is there a FAQ somewhere that I’ve missed? |
| 02:54:01 | <brad> | I’m assuming that AB = ArchiveBot and AT = ArchiveTeam, e.g., the Warrior? |
| 02:54:24 | <@JAA> | Well, everything here is AT, and that includes AB and the DPoS projects. |
| 02:54:38 | <@JAA> | Between those, well they serve different purposes. |
| 02:55:00 | <@JAA> | AB does recursive crawls, DPoS projects require that tasks can be split up into work units of a reasonable size. |
| 02:55:10 | <@JAA> | So for example retrieve some forums in items of 10 or 100 thread IDs. |
| 02:55:39 | <@JAA> | AB is a generic crawler, DPoS requires custom code for each project. |
| 02:55:55 | <OrIdow6> | brad: Did you see my answer to your previous question 4 days ago? |
| 02:56:19 | <@JAA> | It's muddied a little since a few days because we now have the recursive .eu crawls as a DPoS project, but that's just due to the massive size of it, which AB can't handle. |
| 02:57:33 | <brad> | @OrIdow6: No, I don’t think I did see that answer. Let me go find it. |
| 02:58:54 | <@JAA> | Also, terminology: I call these distributed archival projects 'DPoS projects'. Some call them 'warrior projects', but since basically noone uses the warrior anymore (as it's mostly broken) and those projects were never warrior-exclusive, I don't find that accurate. |
| 02:59:37 | <@JAA> | The 'warrior' being both the VM and the wrapper around projects that communicates with the tracker to fetch the current project lists, can switch automatically, etc. |
| 03:01:41 | <brad> | Ahh, yes. Actually, I did see that answer, and I had forgotten it. I think I do now understand a bit better the difference between the technical aspects of the two. |
| 03:03:11 | <brad> | For my own part, I’d like to participate in both types of projects. But for now, I’m just working on getting my own copy of grab-site up and running. |
| 03:05:27 | <brad> | But I am also interested to learn more about the governance side of the equation, like who decides what kind of project goes where, and what constitutes a big enough project to require distributed teams of warriors (or warrior-likes), versus what is small enough that it could be handled by ArchiveBot (including private copies of grab-site, etc...). |
| 03:20:36 | <@JAA> | Ah, that's the artistic part of archival: getting a feel for how large a site is and how many resources it might need. Deadlines and personnel availability (as basically only one person has been writing the code for DPoS projects for the last couple years) also play into it of course. |
| 04:37:30 | | qw3rty_ joins |
| 04:41:04 | | qw3rty__ quits [Ping timeout: 240 seconds] |
| 04:48:13 | | HackMii_ (hacktheplanet) joins |
| 04:49:04 | | HackMii quits [Ping timeout: 240 seconds] |
| 04:49:22 | | HackMii_ is now known as HackMii |
| 04:59:49 | | etnguyen03 quits [Client Quit] |
| 06:23:52 | | Domaloma joins |
| 07:58:16 | | thuban quits [Quit: What price Glory ?] |
| 09:01:43 | | HackMii quits [Ping timeout: 240 seconds] |
| 09:25:22 | | hooway joins |
| 09:27:29 | | hooway_ joins |
| 09:30:22 | | thuban joins |
| 09:30:24 | | HackMii (hacktheplanet) joins |
| 09:31:24 | | hooway quits [Ping timeout: 240 seconds] |
| 09:47:24 | | Arcorann quits [Ping timeout: 240 seconds] |
| 09:50:19 | | Arcorann (Arcorann) joins |
| 09:51:05 | <Arcorann> | http://www.mezzacotta.net/garfield/?comic=4244 |
| 10:05:33 | | thuban quits [Client Quit] |
| 10:13:28 | | thuban joins |
| 10:14:44 | | HackMii quits [Ping timeout: 240 seconds] |
| 10:16:35 | | HackMii (hacktheplanet) joins |
| 10:29:38 | | HackMii quits [Client Quit] |
| 10:29:58 | | HackMii (hacktheplanet) joins |
| 12:02:43 | | honno joins |
| 12:14:33 | | spirit quits [Client Quit] |
| 12:36:55 | | Nullificator joins |
| 12:46:58 | | Nullificator quits [Remote host closed the connection] |
| 13:13:53 | | honno quits [Client Quit] |
| 14:04:54 | | Datechnom5 (Datechnom) joins |
| 14:06:04 | | Datechnom quits [Ping timeout: 240 seconds] |
| 14:06:04 | | Datechnom5 is now known as Datechnom |
| 14:16:44 | | Lord_Nightmare quits [Quit: ZNC - http://znc.in] |
| 14:30:05 | | etnguyen03 (etnguyen03) joins |
| 14:48:24 | | Arcorann quits [Ping timeout: 240 seconds] |
| 15:12:17 | | Ajay quits [Remote host closed the connection] |
| 15:23:27 | | HP_Archivist (HP_Archivist) joins |
| 15:49:12 | | systwi_ is now known as systwi |
| 16:06:38 | | spirit joins |
| 19:47:06 | | DogsRNice (Webuser299) joins |
| 19:49:32 | <DogsRNice> | unfortunate news but this youtuber has sadly passed away https://www.youtube.com/c/ApolloLegend It might be advisable to archive their content in case its taken down in the future |
| 19:51:40 | <@JAA> | That was discussed in -bs earlier. |
| 19:55:17 | <DogsRNice> | oh ok |
| 22:07:13 | | Lord_Nightmare (Lord_Nightmare) joins |
| 22:35:24 | | lukash7 quits [Ping timeout: 240 seconds] |
| 22:36:18 | | lukash7 joins |
| 23:00:03 | | HackMii quits [Remote host closed the connection] |
| 23:00:26 | | HackMii (hacktheplanet) joins |
| 23:09:10 | | wickedplayer494 quits [Remote host closed the connection] |
| 23:27:46 | | wickedplayer494 joins |
| 23:27:49 | | wickedplayer494 is now authenticated as wickedplayer494 |