| 00:04:30 | <@JAA> | If anyone wants the list for some reason: https://transfer.notkiska.pw/wiKWg/github-GrapheneOS-CopperheadOS-repos |
| 00:22:06 | <@JAA> | Ah yes, the typical fun of AOSP-like projects. Several repositories of the kernel with millions of objects each. Lovely. |
| 00:22:24 | | balrog quits [Ping timeout: 240 seconds] |
| 00:27:50 | | Wingy3 (Wingy) joins |
| 00:28:46 | | Wingy quits [Ping timeout: 252 seconds] |
| 00:28:46 | | Wingy3 is now known as Wingy |
| 00:29:07 | | balrog joins |
| 00:29:17 | | balrog is now authenticated as balrog |
| 00:55:39 | <@JAA> | GrapheneOS cloning is done. There are two empty repos, and only one repo has a wiki. |
| 01:10:55 | | HP_Archiv joins |
| 01:15:04 | | HP_Archivist quits [Ping timeout: 240 seconds] |
| 01:15:42 | | HP_Archiv quits [Ping timeout: 252 seconds] |
| 01:15:46 | | HP_Archivist (HP_Archivist) joins |
| 01:23:12 | | Doranwen joins |
| 01:23:14 | | Doranwen is now authenticated as Doranwen |
| 01:24:09 | <@hook54321> | the copperhead and grapheneos stuff seems like a mess |
| 01:25:53 | | Doranwen quits [Remote host closed the connection] |
| 01:26:07 | | Doranwen joins |
| 01:26:08 | | Doranwen is now authenticated as Doranwen |
| 01:27:08 | | Doranwen quits [Remote host closed the connection] |
| 01:53:08 | | Doranwen joins |
| 01:53:09 | | Doranwen is now authenticated as Doranwen |
| 02:00:10 | <@JAA> | CopperheadOS repositories are also done. One empty there, and no wikis. |
| 02:14:44 | | lunik1 quits [Ping timeout: 240 seconds] |
| 02:15:57 | | lunik1 joins |
| 02:22:16 | | lunik15 joins |
| 02:23:04 | | lunik1 quits [Ping timeout: 240 seconds] |
| 02:23:04 | | lunik15 is now known as lunik1 |
| 02:27:20 | | Jake6 (Jake) joins |
| 02:27:43 | | kiska0 (kiska) joins |
| 02:27:43 | | @ChanServ sets mode: +o kiska0 |
| 02:28:18 | | Ryz8 (Ryz) joins |
| 02:28:44 | | nerdguy1138 quits [Ping timeout: 240 seconds] |
| 02:28:44 | | kiskaLogBot quits [Ping timeout: 240 seconds] |
| 02:29:04 | | Ryz quits [Ping timeout: 240 seconds] |
| 02:29:24 | | Doranwen quits [Ping timeout: 240 seconds] |
| 02:29:24 | | @kiska quits [Ping timeout: 240 seconds] |
| 02:29:24 | | Jake quits [Ping timeout: 240 seconds] |
| 02:29:24 | | @kiska0 is now known as @kiska |
| 02:29:32 | | Doranwen joins |
| 02:30:20 | | kiskaLogBot joins |
| 02:30:43 | | nerdguy1138 (nerdguy1138) joins |
| 02:32:06 | | Ryz8 is now known as Ryz |
| 02:34:17 | | Jake6 is now known as Jake |
| 02:42:06 | | Doranwen is now authenticated as Doranwen |
| 02:45:41 | | lennier2 joins |
| 02:49:34 | | lennier1 quits [Ping timeout: 252 seconds] |
| 02:49:37 | | lennier2 is now known as lennier1 |
| 03:04:42 | <mgrandi> | https://twitter.com/williamlegate/status/1324512677033660416 |
| 03:05:23 | <mgrandi> | All of his tweets are probably already backed up, but might be worth to get them again just in case ™️ |
| 03:18:55 | <SketchTheCow> | JAA: FOS is now down to something normal |
| 03:19:09 | <@JAA> | Yay, nice! :-) |
| 03:19:18 | <SketchTheCow> | I'm going to start getting rid of some of the to-dos that are left, but archivebot crap is still flowing through |
| 03:19:36 | <SketchTheCow> | But one of the projects, 1,000 previously unavailable PS2 beta discs, needs to keep going |
| 03:19:46 | <SketchTheCow> | That's like 2tb |
| 03:19:50 | <@JAA> | Oh, neat. |
| 03:19:51 | <SketchTheCow> | And so on. |
| 03:19:59 | <SketchTheCow> | But I think I can get some other stuff out of there |
| 03:33:58 | <voltist> | mgrandi: I have quite a lot of experience with python, and a good amount with using it for scraping/mirroring |
| 03:34:07 | <voltist> | Mostly personal projects |
| 03:34:52 | <mgrandi> | You are in good shape then |
| 03:35:58 | <mgrandi> | Deathwatch has some projects that need attention probably, I'm not sure what the status is on all of them coming up to the end of the year |
| 03:40:21 | <voltist> | Any tips for finding out whats going on relating to a project? Could I just put a call out on IRC if there isn't a wiki page |
| 03:47:04 | | lunik1 quits [Client Quit] |
| 03:47:21 | | lunik1 joins |
| 04:03:48 | | qw3rty__ joins |
| 04:07:04 | | qw3rty_ quits [Ping timeout: 240 seconds] |
| 04:31:51 | | tar-xvf joins |
| 04:34:27 | | odemgi_ quits [Ping timeout: 240 seconds] |
| 04:39:55 | | vitzli (vitzli) joins |
| 05:12:57 | | vitzli quits [Client Quit] |
| 06:23:11 | <Lord_Nightmare> | trump twitter account could be suspended after the election... when did we last archive that "data"? |
| 06:37:19 | <nico_32> | via archivebot? look like never |
| 06:42:13 | <nico_32> | job queued |
| 06:44:04 | <OrIdow6> | It's well-covered in the WBM, to make an understatement |
| 06:45:09 | <nico_32> | the whole account? |
| 06:46:08 | <nico_32> | $ curl -s https://transfer.notkiska.pw/XiUK6/twitter-@realDonaldTrump |wc -l |
| 06:46:11 | <nico_32> | 64519 |
| 06:52:31 | <OrIdow6> | curl "https://web.archive.org/cdx/search?url=http://twitter.com/realdonaldtrump/*" | wc -l |
| 06:52:37 | <OrIdow6> | 1214021 |
| 06:53:51 | <OrIdow6> | Though it does look like some only have a few |
| 06:55:58 | | thuban quits [Ping timeout: 252 seconds] |
| 07:00:12 | <nico_32> | anyway the job is going to retrieve every tweet by trump |
| 07:00:20 | <nico_32> | that tweeter didn't took down |
| 07:45:26 | | godane1 quits [Read error: Connection reset by peer] |
| 07:49:58 | <Lord_Nightmare> | more likely it would be suspended after jan 20th, but good to have an archive now so stuff doesn't get 'scrubbed' or 'washed' etc |
| 08:02:02 | | HP_Archivist quits [Read error: Connection reset by peer] |
| 09:53:44 | <jodizzle> | We've archived Trump's twitter serveral times. Last time was apparently on Oct. 14th |
| 09:54:13 | <jodizzle> | And before that, back in May |
| 09:56:43 | <Ryz> | ...Wasn't there a stupid oddity when archiving that Twitter account that stumped socialbot/snscrape completely? |
| 09:56:46 | <jodizzle> | Hah, actually looks like both previous attempts I see in the logs failed |
| 09:57:01 | <jodizzle> | Yeah, was just seeing that Ryz |
| 09:57:53 | <jodizzle> | Looks like nico_32's job went through though. |
| 11:43:49 | <mgrandi> | does it actually attempt to go through the full account? |
| 11:44:18 | <mgrandi> | @voltist yeah, thats pretty much it, for major projects there usually is an IRC channel dedicated to it |
| 11:49:09 | <nico_32> | mgrandi: yes |
| 11:49:15 | <nico_32> | it tried to get every tweet |
| 11:49:24 | <nico_32> | and associated ressources (url, images) |
| 11:49:43 | <mgrandi> | nice, is it using the strategy that some project i dont have the link for that basically emulates a browser and uses their super secret private api |
| 11:50:13 | <mgrandi> | twitter limiting their official api to 4000 tweets per account is a travesty |
| 12:25:15 | <OrIdow6> | mgrandi: It uses J A A's "snscrape" script (which outputs the list you can see at that file on transfer.notkiska.pw); regular AB; and Chromebot |
| 12:39:13 | | VADemon joins |
| 14:18:32 | | VADemon quits [Ping timeout: 252 seconds] |
| 14:44:43 | | janedoe joins |
| 14:51:25 | | janedoe quits [Remote host closed the connection] |
| 15:13:15 | | Arcorann quits [Read error: Connection reset by peer] |
| 16:10:40 | | @arkiver quits [Excess Flood] |
| 16:11:10 | | arkiver (arkiver) joins |
| 16:11:10 | | @ChanServ sets mode: +o arkiver |
| 17:04:54 | <benjins> | Members of expected Biden transition team in US: https://www.democracyinaction.us/2020/chrntran/bidentransition.html |
| 17:12:03 | | thuban joins |
| 17:40:19 | <purplebot> | Coronavirus edited by Wessel1512 (+162, /* Information */) just now -- https://www.archiveteam.org/?diff=45745&oldid=45744 |
| 18:14:19 | <purplebot> | BNK48 edited by Gridkr (+229, /* Members that have "graduated" …) just now -- https://www.archiveteam.org/?diff=45746&oldid=45408 |
| 18:18:20 | <purplebot> | Deathwatch edited by JustAnotherArchivist (+358, /* Pining for the Fjords (Dying) …) just now -- https://www.archiveteam.org/?diff=45748&oldid=45741 |
| 18:19:24 | | sirvy quits [Ping timeout: 240 seconds] |
| 18:24:57 | | sirvy joins |
| 18:55:20 | <purplebot> | Mastodon edited by Zagra (+226) just now -- https://www.archiveteam.org/?diff=45749&oldid=41629 |
| 19:07:10 | <Stilett0> | I've got a really old and small but still operating website I'm interested in that has draconian country-based IP blocks - the rumor is that it's set to only Japan, and maybe even blocks TOR and some VPNs. The last I talked to JAA about it, he said that ArchiveTeam has a Japan endpoint but that it was completely busy with a large queue. Anyone? |
| 19:07:43 | <Stilett0> | oh, and they also blocked Wayback... |
| 19:08:37 | <@JAA> | Yeah, the Japanese AB pipeline should be available again sometime in the next couple weeks, actually. |
| 19:09:08 | <@JAA> | The running job is nearly done, and the queued one (which would've taken forever) won't work because the site switched to JS hell since it was queued. |
| 19:10:38 | <Stilett0> | To be more clear, Wayback archives it, but only sees the error page provided to those in blocked countries - so if we can archive it from the Japan endpoint, and provided that Japan endpoint isn't detected as a VPN to block, ArchiveTeam should be able to make a good grab for importing into Wayback evetually |
| 19:11:32 | <Stilett0> | Sounds good, JAA, I'll get with you guys about it in a couple of weeks :) |
| 19:12:23 | <@JAA> | Sounds good. :-) |
| 20:04:47 | <@kiska> | What is this site? And you actually gotten a valid response using the pipeline? |
| 20:26:04 | | lempamo_ joins |
| 20:27:11 | <@arkiver> | yeah which website is this? |
| 20:29:04 | | lempamo quits [Ping timeout: 240 seconds] |
| 20:29:09 | | lempamo_ is now known as lempamo |
| 20:49:24 | <voltist> | Is there an IRC channel for archiving US election related stuff? I'm interested in helping out but I don't really know where to start |
| 20:55:38 | <thuban> | voltist: most of the work is going on in #archivebot, although i don't believe there's a good onboarding process/document at the moment |
| 21:04:56 | <Ryz> | The discussion stuff I believe is on #vooterbooter |
| 21:06:04 | | nerdguy1138 quits [Ping timeout: 240 seconds] |
| 21:10:59 | <thuban> | voltist ^ |
| 21:20:46 | | nerdguy1138 (nerdguy1138) joins |
| 21:36:47 | | @dxrt quits [Quit: ZNC - http://znc.sourceforge.net] |
| 21:39:30 | | dxrt joins |
| 21:39:32 | | dxrt is now authenticated as dxrt |
| 21:39:32 | | dxrt quits [Changing host] |
| 21:39:32 | | dxrt (dxrt) joins |
| 21:39:32 | | @ChanServ sets mode: +o dxrt |
| 21:53:50 | | lempamo_ joins |
| 21:57:26 | | lempamo quits [Ping timeout: 252 seconds] |
| 22:00:10 | | lempamo_ quits [Ping timeout: 252 seconds] |
| 22:01:55 | | godane joins |
| 22:01:55 | | godane is now authenticated as godane |
| 22:10:09 | | Arcorann (Arcorann) joins |
| 22:10:47 | | Arcorann quits [Remote host closed the connection] |
| 22:11:10 | | Arcorann (Arcorann) joins |
| 22:22:46 | <nico_32> | Stilett0: what is the website and what the blocked page look like? |
| 22:23:06 | <Stilett0> | I messaged it to kiska. |
| 22:23:27 | <nico_32> | i still have some azure credit |
| 22:37:34 | | lunik1 quits [Ping timeout: 252 seconds] |
| 22:38:28 | | lunik1 joins |
| 22:50:54 | <@JAA> | Unfortunately, kiska's JP AB pipeline is also blocked from that site (assuming it's the one you mentioned to me in late September). |
| 22:51:22 | <@kiska> | Hrm... we could potentially use windscribe? |
| 23:09:09 | <Stilett0> | drat |
| 23:10:15 | <Stilett0> | I can maybe talk a Japan friend into making a warc but then I don't see how it'd ever make it into Wayback. :( |
| 23:37:00 | | britm0b joins |
| 23:40:16 | | britmob quits [Ping timeout: 252 seconds] |