| 00:18:00 | | Peetz0r (Peetz0r) joins |
| 00:31:58 | <Ryz> | Wondering if there's any way to archive https://www.humanbraincloud.com/ and it's contents? While it isn't in danger yet from the outside at least, it's been around since 2013 |
| 01:01:26 | | hooway quits [Client Quit] |
| 01:59:58 | <OrIdow6> | arkiver: All I did on SmackJeeves was a little more pre-discovery (to the conclusion that it doesn't need a discovery process) |
| 02:00:02 | | Dallas quits [Client Quit] |
| 02:00:29 | <@arkiver> | OrIdow6: alright sounds good |
| 02:00:37 | <OrIdow6> | https://logs.kiska.pw/archiveteam-bs/2020-12-06#lcc8a1cca - I basically just got further confirmation of these results - comic IDs go up to slightly over 300k, and user IDs slightly over 200k |
| 02:00:45 | <OrIdow6> | So it should be possible to just run them sequentially |
| 02:01:19 | <@arkiver> | yep |
| 02:03:59 | | Dallas (Dallas) joins |
| 04:12:31 | | HP_Archivist quits [Read error: Connection reset by peer] |
| 04:13:51 | | HP_Archivist (HP_Archivist) joins |
| 04:35:35 | | balrog quits [Quit: Bye] |
| 04:43:48 | | qw3rty_ joins |
| 04:47:04 | | qw3rty__ quits [Ping timeout: 240 seconds] |
| 06:52:14 | | nerdguy1138 quits [Remote host closed the connection] |
| 06:52:51 | | nerdguy1138 (nerdguy1138) joins |
| 07:16:19 | | themadpro quits [Read error: Connection reset by peer] |
| 07:16:20 | | @HCross quits [Read error: Connection reset by peer] |
| 07:16:20 | | Dragnog quits [Read error: Connection reset by peer] |
| 07:19:05 | | Dragnog joins |
| 07:19:30 | | HCross joins |
| 07:22:41 | | themadpro (themadpro) joins |
| 07:45:57 | | yauz joins |
| 07:46:22 | <HP_Archivist> | arkiver - thanks, just saw your message from earlier. I actually don't think it's possible. It seems like IA chooses video over photo in the preview box |
| 08:56:52 | | HCross is now authenticated as HCross |
| 08:56:53 | | @ChanServ sets mode: +o HCross |
| 09:01:27 | | Dragnog quits [Read error: Connection reset by peer] |
| 09:01:38 | | @HCross quits [Read error: Connection reset by peer] |
| 09:02:52 | | Dragnog joins |
| 09:04:11 | | HCross joins |
| 09:14:24 | | HCross quits [Ping timeout: 600 seconds] |
| 09:21:55 | | HCross joins |
| 09:30:17 | | yauz quits [Remote host closed the connection] |
| 11:25:06 | | VADemon joins |
| 13:09:34 | | spirit quits [Quit: Leaving] |
| 13:14:38 | | HCross is now authenticated as HCross |
| 13:14:38 | | @ChanServ sets mode: +o HCross |
| 13:43:34 | | hooway joins |
| 14:03:19 | | spirit joins |
| 14:07:00 | <purplebot> | Archive.today edited by Samanto Hermes (+167, Duplicated snapshots from other …) just now -- https://www.archiveteam.org/?diff=45975&oldid=41512 |
| 14:33:10 | | Ryz quits [Remote host closed the connection] |
| 14:34:44 | | Arcorann quits [Ping timeout: 240 seconds] |
| 16:30:54 | <@jrwr> | RIP Voat |
| 16:35:07 | | godane quits [Remote host closed the connection] |
| 16:35:47 | | godane (godane) joins |
| 17:04:43 | <themadpro> | Nah it's coming and going, not dead just yet |
| 17:04:56 | <themadpro> | Looks like we might get a near-complete archive |
| 17:41:24 | | mutantmonkey quits [Ping timeout: 240 seconds] |
| 17:41:55 | | mutantmonkey (mutantmonkey) joins |
| 17:44:06 | | mutantmonkey quits [Remote host closed the connection] |
| 17:44:25 | | mutantmonkey (mutantmonkey) joins |
| 17:54:18 | | godane quits [Remote host closed the connection] |
| 17:57:53 | | godane (godane) joins |
| 18:03:12 | | balrog joins |
| 18:03:13 | | balrog is now authenticated as balrog |
| 18:20:48 | | Ryz (Ryz) joins |
| 18:29:12 | <OrIdow6> | So my current approach for .eu domains, which is really just general web crawling, is to prioritize the URLs by order of "likeliness" to get a "good" result, then stop the crawl when the last N URLs fetched are "bad" |
| 18:30:18 | <OrIdow6> | Not implemented yet - "likeliness" as resulting from some sort of combination of proportion "good" for each (domain, path component) in each URL; what determines "good" (currently just status code) |
| 18:30:55 | <OrIdow6> | *(host, path component) |
| 18:31:56 | <OrIdow6> | s/when the last N URLs fetched are "bad"/when some large proportion the last N URLs fetched are "bad" |
| 18:32:10 | <OrIdow6> | / |
| 18:35:46 | <OrIdow6> | Thoughts on this from anyone? |
| 19:48:08 | <OrIdow6> | --------------------- |
| 19:48:16 | <OrIdow6> | betamax: I am looking into Sploder |
| 19:51:38 | <themadpro> | I thought Flashpoint had arranged something |
| 19:51:56 | <themadpro> | flashfire42: was it just games or did you have the forum covered as well? |
| 19:55:20 | <OrIdow6> | Oh |
| 19:57:00 | <purplebot> | ArchiveTeam Warrior edited by Tech234a (-176, Removed note about pinging for …) just now -- https://www.archiveteam.org/?diff=45976&oldid=45859 |
| 20:40:38 | <@Kaz> | ^^ thanks tech234a |
| 20:58:47 | <flashfire42> | themadpro Sploder was covered with all the games with a scrape by DarkMoe I do not know the state of the forums |
| 20:59:45 | <flashfire42> | But that scrape was months ago and still new games are being added so I dont know what he is doing there |
| 21:07:34 | <OrIdow6> | What did he grab? Description pages, the player, each games' XML file, comments? And where is this data? |
| 21:08:48 | <OrIdow6> | *game's |
| 21:11:42 | <betamax> | DarkMoe is grabbing all the games the previous grab (back in March) didnt grab |
| 21:12:28 | <betamax> | the forums are out of scope for Flashpoint, any work ypu could do on that would be really helpful |
| 21:12:58 | <betamax> | I will be doing the comments on the games, and making sure each game page is saved in wayback |
| 21:15:48 | <betamax> | OrIdow6: ^^^ |
| 21:16:07 | <OrIdow6> | What did he grab? Description pages, the player, each games' XML file? |
| 21:16:32 | <OrIdow6> | I assume the latter two, if they play back |
| 21:16:43 | <OrIdow6> | But user profile pictures? Etc. |
| 21:17:04 | | HackMii quits [Ping timeout: 240 seconds] |
| 21:19:23 | <betamax> | when I asked on 21st Dec, DarkMoe said their scraper "gets only the dynamic data, not the entire pages" |
| 21:20:02 | <betamax> | however I have scrapped all the games (can send a list shortly) and will be putting them into archivebot |
| 21:20:46 | <betamax> | this should include most of the metadata, although the comments are loaded dynamically hence why I'm grabbing them separately (they'll end up in wayback) |
| 21:21:28 | <betamax> | as of right now the forums have not been grabbed, and no one has "claimed" them |
| 21:28:50 | <OrIdow6> | Ok |
| 21:29:45 | <OrIdow6> | If you can, maybe DarMoe or you can generate/send a list of the per-game files as well? Would like to get those into the WBM, and from what I've seen they're fairly small |
| 21:30:10 | | HackMii (hacktheplanet) joins |
| 21:30:35 | <OrIdow6> | If it's easy |
| 21:35:22 | <betamax> | I can certainly ask |
| 21:37:35 | <OrIdow6> | Thanks - though again, if they're already downloaded, and it's difficult, I don't want to make extra work for anyone |
| 21:37:41 | <OrIdow6> | See #archivebot - forums put in |
| 21:37:47 | <OrIdow6> | Thanks Ryz |
| 21:49:20 | | VADemon quits [Client Quit] |
| 23:32:06 | | tzt joins |
| 23:34:53 | | hooway quits [Client Quit] |
| 23:42:51 | | Arcorann (Arcorann) joins |
| 23:43:48 | | Arcorann quits [Remote host closed the connection] |
| 23:44:12 | | Arcorann (Arcorann) joins |