| 00:13:19 | | Arcorann (Arcorann) joins |
| 00:16:16 | | EggplantN_ is now authenticated as EggplantN_ |
| 00:17:32 | | madmax (madmax) joins |
| 00:20:04 | | Arcorann_ joins |
| 00:22:00 | <lennier1> | 8kun is apparently down and going Tor only: https://twitter.com/fr_brennan/status/1348897011014258688 |
| 00:22:24 | | Arcorann quits [Ping timeout: 240 seconds] |
| 00:23:31 | <Dallas> | Is that like 4chan where the threads expire and delete? |
| 00:27:31 | <EggplantN_> | lennier1 urm, who was their provider |
| 00:27:40 | <EggplantN_> | I know NickLim was previously bit mitigate |
| 00:28:41 | | daalekz joins |
| 00:33:05 | <lennier1> | According to Wikipedia, it uses Russian service providers, including ddos-guard.net. |
| 00:33:24 | <lennier1> | Not too familiar with how it works, though I know its origin is with 4chan. |
| 00:35:00 | <EggplantN_> | Ah yes |
| 00:35:05 | <EggplantN_> | ddos-guard via spartanhost |
| 00:35:12 | | daalekz quits [Read error: Connection reset by peer] |
| 00:35:24 | <Dallas> | Is there any point archiving a site that deletes everything so often ? We’d have to do it every day surely |
| 00:45:13 | <atphoenix> | it'd need to be something like #shreddit, and otherwise worthwhile |
| 00:47:29 | | hex quits [Remote host closed the connection] |
| 00:48:12 | | Arcorann (Arcorann) joins |
| 00:50:53 | <@arkiver> | Dallas: which site |
| 00:51:29 | | Arcorann_ quits [Ping timeout: 265 seconds] |
| 00:51:42 | <EggplantN_> | https://usercontent.irccloud-cdn.com/file/4nfzHCqT/image.png |
| 00:51:47 | <EggplantN_> | the shit we do for archiveteam |
| 00:55:24 | <OrIdow6> | arkiver: I presum 8kun |
| 00:56:03 | <OrIdow6> | That may be too hot to touch, from what I've heard |
| 00:56:13 | | Yg84WJchYH87d3A5tF quits [Client Quit] |
| 00:59:02 | <@kiska> | At the same time... Parler is quite hot too |
| 00:59:56 | <crispyalice2> | Newbie here, but something being hot doesnt seem like a good reason to not archive it. Wouldnt that make it more desirable to archive? |
| 01:04:06 | <OrIdow6> | I suppose my question is whether IA will take it |
| 01:04:30 | <Billy549> | I mean if they're taking Parler content? :shrug: |
| 01:06:21 | | Arcorann quits [Ping timeout: 244 seconds] |
| 01:07:57 | | fuzzy8021 quits [Read error: Connection reset by peer] |
| 01:08:06 | <Ajay> | I think that's on another level |
| 01:08:59 | | fuzzy8021 joins |
| 01:08:59 | | fuzzy8021 is now authenticated as fuzzy8021 |
| 01:08:59 | | fuzzy8021 quits [Changing host] |
| 01:08:59 | | fuzzy8021 (fuzzy8021) joins |
| 01:09:40 | | @dxrt quits [Quit: ZNC - http://znc.sourceforge.net] |
| 01:10:17 | | fuzzy8021 quits [Read error: Connection reset by peer] |
| 01:10:44 | <@arkiver> | right |
| 01:10:45 | | fuzzy8021 joins |
| 01:10:45 | | fuzzy8021 is now authenticated as fuzzy8021 |
| 01:10:45 | | fuzzy8021 quits [Changing host] |
| 01:10:45 | | fuzzy8021 (fuzzy8021) joins |
| 01:11:14 | | dxrt joins |
| 01:11:16 | | dxrt is now authenticated as dxrt |
| 01:11:16 | | dxrt quits [Changing host] |
| 01:11:16 | | dxrt (dxrt) joins |
| 01:11:16 | | @ChanServ sets mode: +o dxrt |
| 01:15:48 | | trins_ joins |
| 01:17:03 | | fuzzy8021 quits [Read error: Connection reset by peer] |
| 01:18:20 | | fuzzy8021 joins |
| 01:18:20 | | fuzzy8021 is now authenticated as fuzzy8021 |
| 01:18:20 | | fuzzy8021 quits [Changing host] |
| 01:18:20 | | fuzzy8021 (fuzzy8021) joins |
| 01:19:38 | | fuzzy8021 quits [Read error: Connection reset by peer] |
| 01:20:46 | | fuzzy8021 joins |
| 01:20:46 | | fuzzy8021 is now authenticated as fuzzy8021 |
| 01:20:46 | | fuzzy8021 quits [Changing host] |
| 01:20:46 | | fuzzy8021 (fuzzy8021) joins |
| 01:21:27 | | VerifiedJ quits [Changing host] |
| 01:21:27 | | VerifiedJ (VerifiedJ) joins |
| 01:25:22 | <@ChanServ> | EggplantN_ now has flags +v in the group !archiveteam-core which communally has +AFORSVefiorstv on #archiveteam-bs. |
| 01:30:19 | | Arcorann (Arcorann) joins |
| 01:38:44 | | blue507 quits [Ping timeout: 240 seconds] |
| 01:39:04 | | fuzzy8021 quits [Read error: Connection reset by peer] |
| 01:40:00 | | fuzzy8021 joins |
| 01:40:00 | | fuzzy8021 is now authenticated as fuzzy8021 |
| 01:40:00 | | fuzzy8021 quits [Changing host] |
| 01:40:00 | | fuzzy8021 (fuzzy8021) joins |
| 01:40:55 | | nepeat quits [Remote host closed the connection] |
| 01:40:59 | | @dxrt quits [Client Quit] |
| 01:41:01 | | nepeat joins |
| 01:41:03 | | nepeat is now authenticated as nepeat |
| 01:42:12 | | fuzzy8021 quits [Read error: Connection reset by peer] |
| 01:42:26 | | fuzzy8021 joins |
| 01:42:26 | | fuzzy8021 is now authenticated as fuzzy8021 |
| 01:42:26 | | fuzzy8021 quits [Changing host] |
| 01:42:26 | | fuzzy8021 (fuzzy8021) joins |
| 01:42:55 | | dxrt joins |
| 01:42:57 | | dxrt is now authenticated as dxrt |
| 01:42:57 | | dxrt quits [Changing host] |
| 01:42:57 | | dxrt (dxrt) joins |
| 01:42:57 | | @ChanServ sets mode: +o dxrt |
| 01:43:56 | | blue507 joins |
| 02:17:05 | <PhantomTypist> | oddball question, but does anyone happen to have hoarded old versions of the Parler apk? |
| 02:18:14 | | ksand joins |
| 02:19:34 | <@JAA> | PhantomTypist: APKPure is your friend: https://apkpure.com/parler/com.parler.parler (And yes, that has been archived just in case.) |
| 02:20:35 | <purplebot> | MediaFire edited by Ajay (+137, Added URL Submission site) just now -- https://www.archiveteam.org/?diff=46161&oldid=46087 |
| 02:23:11 | <crispyalice2> | Going back to 8kun: they have an archive page set up, so it shouldn't be hard to mirror that, although i don't think every post gets to it bc while there are a lot, theres no way thats all of them. Also, the ids aren't sequential but they are just numbers and only have 4 (maybe 5) unique digits |
| 02:24:44 | | fuzzy8021 quits [Read error: Connection reset by peer] |
| 02:25:22 | | systwi quits [Read error: Connection reset by peer] |
| 02:25:22 | | some_donkus quits [Remote host closed the connection] |
| 02:25:55 | | fuzzy8021 joins |
| 02:25:56 | | fuzzy8021 is now authenticated as fuzzy8021 |
| 02:25:56 | | fuzzy8021 quits [Changing host] |
| 02:25:56 | | fuzzy8021 (fuzzy8021) joins |
| 02:26:06 | | systwi (systwi) joins |
| 02:26:35 | | soupsnakes joins |
| 02:26:52 | <PhantomTypist> | JAA, lol i see what they did |
| 02:26:57 | <PhantomTypist> | can search for it though |
| 02:27:00 | <PhantomTypist> | *cant |
| 02:27:08 | | soupsnakes quits [Remote host closed the connection] |
| 02:28:55 | <fuzzy8021> | anyone know someone at twelve99.net. seeems to be the common link of traffic that keeps dropping me :( |
| 02:29:13 | <EggplantN_> | That's Telia |
| 02:29:19 | <EggplantN_> | What issues you seeing fuzzy8021 |
| 02:30:02 | <fuzzy8021> | not sure but keep getting dropped from here one other irc net and a ping monitor keeps going off and thats the common link for a tracert |
| 02:30:33 | <fuzzy8021> | 5 4 ms 4 ms 4 ms omha-b1-link.telia.net [213.248.75.166] |
| 02:30:34 | <fuzzy8021> | 6 14 ms 14 ms 14 ms chi-b23-link.ip.twelve99.net [62.115.143.182] |
| 02:30:49 | <fuzzy8021> | only commonality of all 3 |
| 02:30:52 | <EggplantN_> | hrm. unaware of any AS1299 issues |
| 02:33:26 | <fuzzy8021> | ill add those to monitoring and see what happens |
| 02:38:56 | | TheFluffyCat joins |
| 02:43:38 | <PhantomTypist> | Someone is archiving feeds.... https://hub.docker.com/u/simon987 |
| 02:43:44 | | Doranwen quits [Ping timeout: 240 seconds] |
| 02:43:59 | | Doranwen joins |
| 02:44:00 | | Doranwen is now authenticated as Doranwen |
| 02:45:00 | | lmx quits [Client Quit] |
| 02:47:31 | <aleph> | PhantomTypist: Docker contains to archive feeds then? |
| 02:48:40 | <PhantomTypist> | yep |
| 02:48:46 | <PhantomTypist> | on all the "popular" sites |
| 02:48:57 | <PhantomTypist> | stumbled across it while doing work on #failwhale |
| 02:49:25 | <PhantomTypist> | maybe it's the same person as simon816 on this channel? |
| 02:51:57 | | ksand quits [Remote host closed the connection] |
| 02:53:01 | <aleph> | Heh nifty |
| 02:58:57 | <PhantomTypist> | i guess it's a personal thing because none of his github repos elaborate as to how to run them. looks like there is a volume mounted but i have no clue what files are supposed to go in there that get run by python |
| 02:59:16 | <PhantomTypist> | caught my eye he's also scraping all of 4chan and koal too |
| 02:59:35 | <PhantomTypist> | basically he's scraping all the alt-right stuff |
| 03:01:09 | <OrIdow6> | Koal? |
| 03:02:15 | <OrIdow6> | If that's just a typo of Poal, I'm suggesting it for the channel name of the latter, whenever it goes down |
| 03:08:28 | <PhantomTypist> | haha |
| 03:08:34 | <PhantomTypist> | android auto correct |
| 03:09:44 | <PhantomTypist> | i didn't even know it existed, but let's say it looks Parler-esq |
| 03:09:56 | <@JAA> | Lots of people from Voat moved there when it went down recently. |
| 03:10:27 | <PhantomTypist> | damn this guy has some nice repos/projects |
| 03:10:37 | <PhantomTypist> | i hope he's someone on here |
| 03:10:42 | <PhantomTypist> | or she |
| 03:10:47 | <PhantomTypist> | or whatever |
| 03:13:20 | <TheFluffyCat> | simon816 hey what's up! Cool repos :^) |
| 03:13:44 | <TheFluffyCat> | Scraping 4Chan might not be a bad idea |
| 03:13:52 | <TheFluffyCat> | And 8Kun. |
| 03:17:23 | <PhantomTypist> | TheFluffyCat might just be a coincidence and not that person |
| 03:17:51 | <@JAA> | TIL there's more than one person called 'Simon'. I know, amazing, right? |
| 03:18:44 | | etnguyen03 quits [Ping timeout: 240 seconds] |
| 03:18:57 | | etnguyen03 (etnguyen03) joins |
| 03:21:26 | <PhantomTypist> | is it really worth the disk space to save 8kun and poal? |
| 03:23:22 | | hex joins |
| 03:24:28 | <TheFluffyCat> | Hah. not wise name choice |
| 03:24:41 | <TheFluffyCat> | I'm sure if there's an API it'd be 100% worth saving the JSON, let me check |
| 03:25:08 | <PhantomTypist> | i'm not sure you'd want to save the images on those |
| 03:25:42 | <TheFluffyCat> | At least not publicly |
| 03:26:31 | | phantomtypist_ (phantomtypist) joins |
| 03:26:36 | | PhantomTypist quits [Remote host closed the connection] |
| 03:26:56 | | phantomtypist_ quits [Client Quit] |
| 03:27:33 | | PhantomTypist (phantomtypist) joins |
| 03:29:39 | | @Fusl quits [Changing host] |
| 03:29:39 | | Fusl (Fusl) joins |
| 03:29:39 | | guybrush.hackint.org sets mode: +o Fusl |
| 03:29:39 | | @Fusl_ quits [Changing host] |
| 03:29:39 | | Fusl_ (Fusl) joins |
| 03:29:39 | | guybrush.hackint.org sets mode: +o Fusl_ |
| 03:29:50 | | @Fusl quits [Changing host] |
| 03:29:50 | | Fusl (Fusl) joins |
| 03:29:50 | | guybrush.hackint.org sets mode: +o Fusl |
| 03:29:50 | | @Fusl_ quits [Changing host] |
| 03:29:50 | | Fusl_ (Fusl) joins |
| 03:29:50 | | guybrush.hackint.org sets mode: +o Fusl_ |
| 03:30:04 | | trins_ quits [Ping timeout: 240 seconds] |
| 03:34:35 | | kiiwiiwastaken (kiiwiiwastaken) joins |
| 04:01:49 | <aleph> | Wonder how much posts 4chan sees daily |
| 04:09:39 | <sierra_wright> | a lot, i used to spider several of the boards and it was crazy |
| 04:09:47 | <Arcorann> | Depends on the board |
| 04:18:50 | | etnguyen03 quits [Ping timeout: 265 seconds] |
| 04:19:02 | | etnguyen03 (etnguyen03) joins |
| 04:21:34 | | atphoenix quits [Changing host] |
| 04:21:34 | | atphoenix (atphoenix) joins |
| 04:23:25 | | qw3rty__ joins |
| 04:27:32 | | qw3rty_ quits [Ping timeout: 265 seconds] |
| 04:33:36 | | Doranwen quits [Changing host] |
| 04:33:36 | | Doranwen (Doranwen) joins |
| 04:43:03 | | DogsRNice quits [Read error: Connection reset by peer] |
| 04:45:51 | <@kiska> | Did someone say /b/ |
| 04:46:18 | <kiiwiiwastaken> | me |
| 04:48:07 | <Arcorann> | Speaking of which, someone should write up something on yuki.la for the 4chan wiki page |
| 04:50:35 | <purplebot> | Deathwatch edited by Nemo bis (+252, /* 2021 */ +Crowdmap) just now -- https://www.archiveteam.org/?diff=46162&oldid=46110 |
| 04:50:45 | <@JAA> | ^ https://crowdmap.com/ is shutting down on Friday. |
| 04:54:01 | | etnguyen03 quits [Client Quit] |
| 04:55:35 | | nomoreoragnes joins |
| 04:56:44 | <Billy549> | If it's shutting down Friday, would it be a good idea to get a warrior container ready for it? or is it already being put into the bot or similar |
| 04:57:14 | | nomoreoragnes quits [Remote host closed the connection] |
| 04:57:49 | <@JAA> | First, someone would have to look into it to determine what, if anything, can be archived and how. |
| 04:58:21 | <@JAA> | It's using cookies for switching between the 'classic' and 'beta' site, which complicates things. |
| 04:59:43 | <Billy549> | I see :P |
| 04:59:59 | <TheFluffyCat> | Is there an IRC + API? |
| 05:02:42 | <Billy549> | JAA: cURL'ing with the Cookie "cmgatekeeper=1E143D8A6764015D;" gives me classic |
| 05:05:00 | | Ajay quits [Remote host closed the connection] |
| 05:05:17 | | Ajay joins |
| 05:05:27 | | TheFluffyCat quits [Remote host closed the connection] |
| 05:05:34 | <Billy549> | but it seems like trying to search for any map just gives an empty response |
| 05:06:09 | <Billy549> | so im wondering if theyve already shut it down early and just keep it up for migration purposes |
| 05:08:27 | <Arcorann> | https://monsoonmonitoring.crowdmap.com/main <-- at least, this thing is still up |
| 05:09:49 | <Billy549> | https://monsoonmonitoring.crowdmap.com/json |
| 05:09:54 | <Billy549> | Uses that to get the map data at least |
| 05:10:20 | <Billy549> | if you just remove the arguments it gives all data |
| 05:12:26 | <@JAA> | Not all, descriptions of reports are not included for example. |
| 05:12:35 | <Billy549> | Ah yeah, that's true |
| 05:12:41 | <Billy549> | and it only seems to be for that specific map :p |
| 05:12:51 | <@JAA> | Yeah, each map seems to be on a subdomain. |
| 05:15:23 | <Billy549> | that's infuriating :P |
| 05:17:23 | | Ajay quits [Remote host closed the connection] |
| 05:17:49 | | Ajay joins |
| 05:17:58 | | kiiwiiwastaken quits [Ping timeout: 244 seconds] |
| 05:18:04 | | @SketchTheCow quits [Ping timeout: 240 seconds] |
| 05:20:59 | | Ajay quits [Remote host closed the connection] |
| 05:21:14 | | Ajay joins |
| 05:21:45 | | SketchTheCow joins |
| 05:23:14 | | fuzzy8021 quits [Read error: Connection reset by peer] |
| 05:23:42 | | systwi quits [Read error: Connection reset by peer] |
| 05:23:59 | | fuzzy8021 joins |
| 05:23:59 | | fuzzy8021 is now authenticated as fuzzy8021 |
| 05:23:59 | | fuzzy8021 quits [Changing host] |
| 05:23:59 | | fuzzy8021 (fuzzy8021) joins |
| 05:24:20 | | systwi (systwi) joins |
| 05:24:46 | | nepeat quits [Client Quit] |
| 05:24:46 | | @dxrt quits [Client Quit] |
| 05:24:54 | | nepeat joins |
| 05:25:05 | | dxrt joins |
| 05:25:07 | | dxrt is now authenticated as dxrt |
| 05:25:07 | | dxrt quits [Changing host] |
| 05:25:07 | | dxrt (dxrt) joins |
| 05:25:07 | | @ChanServ sets mode: +o dxrt |
| 05:27:55 | | blue507 is now authenticated as blue507 |
| 05:31:24 | | saizai quits [Ping timeout: 244 seconds] |
| 06:02:16 | | no112 quits [Ping timeout: 265 seconds] |
| 06:21:47 | | Mateon1 quits [Remote host closed the connection] |
| 06:22:01 | | Mateon1 joins |
| 06:24:59 | | sierra_wright quits [Ping timeout: 265 seconds] |
| 06:29:39 | | sierra_wright joins |
| 07:14:00 | | Interested quits [Read error: Connection reset by peer] |
| 07:28:48 | | HackMii quits [Remote host closed the connection] |
| 07:30:39 | | HackMii (hacktheplanet) joins |
| 07:40:38 | <systwi> | Ooh, HackMii, that name rings a bell. |
| 07:40:59 | <systwi> | Larsenv: ^ |
| 07:41:26 | <systwi> | I wonder if there was any relation to the BootMii creators. Probably not. Well, maybe. |
| 07:46:35 | | no112 joins |
| 07:59:21 | | hooway joins |
| 08:08:42 | | nickyyyyy joins |
| 08:18:33 | | D2A joins |
| 08:18:45 | | D2A quits [Remote host closed the connection] |
| 08:40:19 | | yawkat quits [Ping timeout: 265 seconds] |
| 08:46:01 | | yawkat (yawkat) joins |
| 08:54:17 | | fuzzy8021 quits [Read error: Connection reset by peer] |
| 08:54:44 | | fuzzy8021 joins |
| 08:54:44 | | fuzzy8021 is now authenticated as fuzzy8021 |
| 08:54:44 | | fuzzy8021 quits [Changing host] |
| 08:54:44 | | fuzzy8021 (fuzzy8021) joins |
| 09:00:18 | | britmob2 quits [Quit: britmob2] |
| 09:02:33 | | jon quits [Ping timeout: 265 seconds] |
| 09:02:38 | <HackMii> | systwi, Of course it is. Oh he's here? Interesting. |
| 09:06:15 | | doyleheaf quits [Client Quit] |
| 09:08:46 | | hellohello joins |
| 09:11:35 | | hellohello quits [Remote host closed the connection] |
| 09:13:16 | <systwi> | HackMii: :D |
| 09:13:32 | <systwi> | Huge fan, sorry, I thought it could be an impersonator |
| 09:14:05 | | jon joins |
| 09:14:32 | <systwi> | I also thought HackMii was a group of devs, and not a single entity. TIL |
| 09:17:58 | <HackMii> | It was a group of devs, I juah confirmed that it was a reference. |
| 09:19:43 | | jon is now authenticated as jon |
| 09:25:42 | | jon is now known as Jon- |
| 09:25:57 | | Jon (jon) joins |
| 09:27:07 | | Jon- quits [Client Quit] |
| 09:34:27 | | hex quits [Ping timeout: 265 seconds] |
| 09:54:16 | | hex joins |
| 10:07:42 | <OrIdow6> | From preliminary examination looks like CrowdMap is archiv-able to some extent |
| 10:08:03 | | FriskyandSickly quits [Remote host closed the connection] |
| 10:11:59 | <OrIdow6> | Don't know how well it would play back, my optimistic guess would be that the map would sort of work (you'd just get the report clusters on a white background) and everything else would - but there are a million ways for it to break - in any case looks like it's possible to get the data |
| 10:17:05 | <OrIdow6> | Someone should look into discovery |
| 10:31:04 | | hex quits [Client Quit] |
| 10:32:14 | <ccafe> | Has a channel been set up yet for crowdmap? I'm looking into how best to find out what subdomains are live atm |
| 11:04:04 | | LordThanatos joins |
| 11:04:20 | | LordThanatos quits [Remote host closed the connection] |
| 11:06:11 | <OrIdow6> | No channel, just -bs |
| 11:06:56 | <OrIdow6> | There are various public sources that can list subdomains (there's a wiki page somewhere), but ideally what would be found now is a comprehensive list from the site itself |
| 11:07:26 | <OrIdow6> | Unless you mean something other than doing discovery by "find out what subdomains are live" |
| 11:16:09 | <ccafe> | I figured seeing what subdomains are existing via osint and building a script to check if they were still accessible would be a start until someone smarter comes along since I can't see a way to get a list from the platform itself, but I'm gathering from your tone that I shouldn't be doing that. Sorry if I was being over-eager, I'll step back :) |
| 11:22:34 | <purplebot> | Template:Czech websites edited by Sanqui (+19, add hostuju.cz) just now -- https://www.archiveteam.org/?diff=46163&oldid=46145 |
| 11:32:12 | <OrIdow6> | There's nothing wrong with it |
| 11:32:18 | <OrIdow6> | It's just a question of return per unit labor |
| 11:36:50 | <OrIdow6> | If this is done by a warrior project, checking if online will probably be unnecessary |
| 11:37:08 | <OrIdow6> | As those items will fail, and can be examined after that happens |
| 11:51:18 | <OrIdow6> | https://crowdmap.com/explore/maps has now magically started working for me |
| 11:51:35 | <ccafe> | Fair enough, I'll leave it in that case. Got all day off today and will be back available tomorrow evening (5:30 GMT) so feel free to ping me to get some warriors spun up for it if needed |
| 11:51:58 | <OrIdow6> | Ok |
| 11:52:07 | <Aoede> | /explore/ works on classic site |
| 11:52:24 | <OrIdow6> | Still possible what you were doing could end up being useful |
| 11:52:32 | <OrIdow6> | If there turns out not to be a comprehensive way |
| 11:55:11 | <OrIdow6> | Anyone want to run this in AB? I assume https://crowdmap.com, with an early ignore for https://crowdmap.com/switch/new , might work |
| 12:02:47 | | daalekz joins |
| 12:13:29 | | Arcorann_ joins |
| 12:17:20 | | Arcorann quits [Ping timeout: 265 seconds] |
| 12:19:45 | | Mineroboter quits [Ping timeout: 265 seconds] |
| 12:28:07 | | EggplantN_ is now known as EggplantN |
| 12:30:35 | | EggplantN is now authenticated as * |
| 12:30:35 | | EggplantN is now authenticated as EggplantN |
| 12:34:38 | | Matthww quits [Quit: The Lounge - https://thelounge.chat] |
| 12:42:09 | | ccafe quits [Read error: Connection reset by peer] |
| 12:42:49 | | kor quits [Ping timeout: 244 seconds] |
| 12:45:49 | | ccafe (ccafe) joins |
| 12:48:41 | <mgrandi> | https://www.theverge.com/2021/1/12/22218757/youtube-trump-account-video-deleted-uploads-suspension-indefinite?utm_campaign=theverge&utm_content=chorus&utm_medium=social&utm_source=twitter |
| 12:49:16 | | groente_ quits [Remote host closed the connection] |
| 12:54:34 | | groente_ joins |
| 12:57:04 | | Matthww joins |
| 13:01:48 | | godane quits [Ping timeout: 265 seconds] |
| 13:04:42 | | ALTROSKY_mystique joins |
| 13:22:16 | | daalekz quits [Read error: Connection reset by peer] |
| 13:24:31 | | Datechnom quits [Ping timeout: 265 seconds] |
| 13:33:58 | | Datechnom (Datechnom) joins |
| 13:44:55 | | Yg84WJchYH87d3A5tF joins |
| 13:49:51 | | britmob2 joins |
| 13:57:57 | <Billy549> | OrIdow6: Does AB have support for cookies? |
| 13:58:12 | <Billy549> | iirc the "view classic site" is a small cookie, i sent a cURL request with it and I got classic site only |
| 13:58:21 | <OrIdow6> | Billy549: No |
| 13:58:25 | <Billy549> | i see |
| 13:58:45 | <OrIdow6> | But (unless something has changed) it does pick up cookies during the crawl |
| 13:59:03 | <@ChanServ> | EggplantN now has flags +v in the group !archiveteam-core which communally has +AFORSVefiorstv on #archiveteam-bs. |
| 13:59:17 | <EggplantN> | JAA & |
| 13:59:33 | | EggplantN quits [Changing host] |
| 13:59:33 | | EggplantN (EggplantN) joins |
| 13:59:40 | <Billy549> | I get 404 on https://crowdmap.com/explore/maps without cookie, sending "Cookie: cmgatekeeper=1E143D8A6764015D;" gives me the classic |
| 13:59:48 | <OrIdow6> | Yes |
| 14:00:49 | <OrIdow6> | My idea (basically to make things easier, and to get docs etc.) was to start AB in such a way that it would pick up on the cookie during the crawl |
| 14:00:59 | <Billy549> | That's fair enough :P |
| 14:01:14 | <OrIdow6> | In the interest of time, I think the list will have to be wgotten or similar |
| 14:01:18 | <Billy549> | some data (but not all) is in SUBDOMAIN.crowdmap.com/json but i assume that'll be picked up by AB |
| 14:01:26 | <OrIdow6> | Since apparently this thing is shutting down Friday |
| 14:01:38 | <OrIdow6> | I'm assuming that the project proper will be a warrior project |
| 14:01:45 | <OrIdow6> | Since there's a lot of dynamic stuff |
| 14:01:47 | <Billy549> | Yeah |
| 14:03:27 | <Billy549> | I'd be up for running a warrior as soon as one's ready (assuming it's a docker container) :P |
| 14:04:29 | <OrIdow6> | Thanks for finding the JSON thing by the way |
| 14:04:33 | <Billy549> | No prob c: |
| 14:04:49 | <Billy549> | I don't think it's all data (someone mentioned it doesnt have everything), but if you dont specify categories it will give ALL points |
| 14:04:57 | <Billy549> | *but* i don't think it tells you what category is which |
| 14:04:58 | | Arcorann (Arcorann) joins |
| 14:06:56 | <OrIdow6> | This will presumably get a bunch of stuff |
| 14:07:00 | <OrIdow6> | From the web interface etc. |
| 14:07:32 | | Arcorann_ quits [Ping timeout: 265 seconds] |
| 14:11:49 | <Aoede> | https://transfer.notkiska.pw/bixp2/crowdmapsub.txt |
| 14:12:11 | <Aoede> | 1300~ subdomains from /explore |
| 14:12:25 | <Aoede> | not sure if it's complete, I had some problems with my script |
| 14:15:18 | | nickyyyyy quits [Ping timeout: 244 seconds] |
| 14:21:36 | | etnguyen03 (etnguyen03) joins |
| 14:24:58 | <Billy549> | OrIdow6: that will be useful :P |
| 14:26:13 | <OrIdow6> | Yeah |
| 14:26:59 | <Billy549> | Aoede: what was your script doing? (and what probs) c: |
| 14:29:37 | <Aoede> | Billy549: scrapy crawler with regex to extract links. I messed the regex pattern the first time |
| 14:36:20 | <Billy549> | Ah ok, I see :P |
| 14:36:46 | <Billy549> | I wonder if with ~1300 to scrape if it'd even make sense to warrior it |
| 14:37:45 | <Jonboy3451> | Idea for an archiveteam project: MLB Filmroom: https://www.mlb.com/video Loads of archival footage free to view. |
| 14:39:26 | | Arcorann quits [Ping timeout: 265 seconds] |
| 14:40:11 | <Billy549> | Powered by Google Cloud, interesting |
| 14:41:19 | <AK> | How bad is it for a a task to be cancelled without waiting for it to finish? (touched STOP inside github container half an hour ago and it's still not finished). Happy to wait if it does cause problems |
| 14:47:08 | <@arkiver> | AK: feel free to kill it |
| 14:48:39 | <ccafe> | Billy549, OrIdow6 - may be of use relating to the /json endpoint, saw it having a quick look through github out of curiosity - /api?task=incidents&limit=[1-Whatever] - example https://worldushahidis.crowdmap.com/api?task=incidents&limit=1000 |
| 14:50:56 | <ccafe> | tried it with up to 10k and seems to work, albeit slowly. I assume the query is expensive on the backend so I'd use caution before hammering it |
| 14:55:18 | <Billy549> | that sounds useful :P |
| 14:59:01 | | AK quits [Client Quit] |
| 15:00:05 | | AK joins |
| 15:01:12 | | some_donkus joins |
| 15:08:55 | | Matthww quits [Ping timeout: 265 seconds] |
| 15:09:52 | | Ruthalas (Ruthalas) joins |
| 15:21:34 | | saizai joins |
| 15:31:15 | | saizai quits [Ping timeout: 244 seconds] |
| 15:58:00 | | katocala quits [Remote host closed the connection] |
| 16:01:33 | | Matthww joins |
| 16:04:30 | | katocala joins |
| 16:05:09 | | katocala is now authenticated as katocala |
| 16:05:47 | <atphoenix> | channel ideas, if you want some: crowdmap -> crowdead |
| 16:09:04 | | Matthww quits [Ping timeout: 240 seconds] |
| 16:11:45 | | Ruthalas quits [Ping timeout: 265 seconds] |
| 16:12:46 | | Matthww joins |
| 16:13:10 | | nepeat is now authenticated as nepeat |
| 16:15:03 | | Ruthalas (Ruthalas) joins |
| 16:26:24 | | LeGoupil joins |
| 16:44:44 | | OrIdow6 quits [Ping timeout: 240 seconds] |
| 16:45:22 | | OrIdow6 (OrIdow6) joins |
| 16:47:21 | | OrIdow6 quits [Remote host closed the connection] |
| 16:48:53 | | OrIdow6 (OrIdow6) joins |
| 16:50:01 | | kiiwiiwastaken (kiiwiiwastaken) joins |
| 16:53:08 | | prebala-publici quits [Client Quit] |
| 16:53:24 | | prebala-publici joins |
| 17:03:11 | | isthisthedarkweb quits [Changing host] |
| 17:03:12 | | isthisthedarkweb (isthisthedarkweb) joins |
| 17:12:11 | | groente_ quits [Remote host closed the connection] |
| 17:12:23 | | groente joins |
| 17:19:25 | | madmax quits [Client Quit] |
| 17:21:31 | | jut quits [Ping timeout: 252 seconds] |
| 17:26:09 | | jut (jut) joins |
| 17:26:26 | | madmax joins |
| 17:28:50 | <EggplantN> | https://usercontent.irccloud-cdn.com/file/OGOnJxGV/image.png |
| 17:28:54 | <EggplantN> | hrm new worker been found |
| 17:32:11 | <SketchTheCow> | Anything need my attention? |
| 17:33:05 | <EggplantN> | no fires today i dont believe |
| 17:33:07 | | groente quits [Remote host closed the connection] |
| 17:33:24 | | jut quits [Ping timeout: 240 seconds] |
| 17:34:02 | | lunik1 quits [Quit: Ping timeout (120 seconds)] |
| 17:34:05 | | madmax is now authenticated as madmax |
| 17:34:58 | | jut (jut) joins |
| 17:37:15 | | lunik1 joins |
| 17:41:32 | | godane joins |
| 17:41:32 | | godane is now authenticated as godane |
| 17:41:58 | | some_donkus quits [Ping timeout: 244 seconds] |
| 17:42:43 | | TheFluffyCat joins |
| 17:44:04 | | jut quits [Ping timeout: 252 seconds] |
| 17:45:16 | | jut (jut) joins |
| 17:45:41 | | spirit joins |
| 17:48:19 | | madmax quits [Client Quit] |
| 17:50:10 | | madmax joins |
| 17:50:10 | | madmax is now authenticated as madmax |
| 17:52:39 | | pgup quits [Client Quit] |
| 17:52:40 | | lunik1 quits [Read error: Connection reset by peer] |
| 17:54:11 | | lunik1 joins |
| 17:54:49 | | jut quits [Client Quit] |
| 17:54:55 | | pgup (pgup) joins |
| 18:03:35 | <sierra_wright> | so i've got a question about these url shortener projects. i'm watching the warrior and there are far more misses than hits... are the misses also being documented so they can be checked later? because it's not like the services aren't in active use |
| 18:05:20 | | Jonboy345 joins |
| 18:05:31 | | Jonboy3451 quits [Ping timeout: 252 seconds] |
| 18:05:58 | <phuzion> | sierra_wright: #urlteam, but no, we don't log the misses, but if we wanted to go back and re-check misses, we could just generate the inverse of the hits using the data we do collect (which is which URLs are hits, and what they coshorten to) |
| 18:12:50 | | AK is now authenticated as AK |
| 18:13:25 | | groente_ joins |
| 18:13:52 | | jut (jut) joins |
| 18:15:41 | | TheFluffyCat68 joins |
| 18:15:53 | <TheFluffyCat68> | Ha! Got disconnected and it didn't even let me keep my name. |
| 18:16:42 | | dsi joins |
| 18:18:39 | | TheFluffyCat quits [Ping timeout: 244 seconds] |
| 18:20:17 | <@JAA> | TheFluffyCat68: If you register with NickServ, you can kill your other connection to reassume the proper nick. Otherwise, you need to wait for the old connection to time out, as it just did. |
| 18:20:37 | <TheFluffyCat68> | Yeah, I can't message NickServ here though. It won't respond :/ |
| 18:29:04 | | jut quits [Ping timeout: 240 seconds] |
| 18:30:22 | | jut (jut) joins |
| 18:36:25 | | jut quits [Client Quit] |
| 18:42:06 | | jut (jut) joins |
| 19:01:40 | | LeGoupil quits [Client Quit] |
| 19:05:38 | | lmx joins |
| 19:05:49 | | kor joins |
| 19:23:40 | | TheFluffyCat68 quits [Remote host closed the connection] |
| 19:29:07 | | jut quits [Ping timeout: 252 seconds] |
| 19:30:21 | | jut (jut) joins |
| 19:46:24 | | jut quits [Ping timeout: 240 seconds] |
| 20:05:53 | | blue507 quits [Remote host closed the connection] |
| 20:10:59 | | madmax quits [Client Quit] |
| 20:11:57 | | sliccricc quits [Changing host] |
| 20:11:57 | | sliccricc (sliccricc) joins |
| 20:12:30 | | sliccricc quits [Client Quit] |
| 20:12:45 | | sliccricc joins |
| 20:13:04 | | sliccricc is now authenticated as sliccricc |
| 20:13:05 | | sliccricc quits [Changing host] |
| 20:13:05 | | sliccricc (sliccricc) joins |
| 20:13:08 | | spirit quits [Remote host closed the connection] |
| 20:13:26 | | sliccricc quits [Remote host closed the connection] |
| 20:13:40 | | sliccricc (sliccricc) joins |
| 20:26:37 | | atphoenix quits [Quit: Leaving] |
| 20:27:07 | | atphoenix (atphoenix) joins |
| 21:00:31 | | madmax joins |
| 21:00:31 | | madmax is now authenticated as madmax |
| 21:05:59 | | VerifiedJ quits [Quit: The Lounge - https://thelounge.chat] |
| 21:06:31 | | VerifiedJ (VerifiedJ) joins |
| 21:08:35 | | Nick62 joins |
| 21:09:01 | | Nick62 quits [Remote host closed the connection] |
| 21:15:05 | | madmax quits [Client Quit] |
| 21:17:32 | | godane1 joins |
| 21:20:07 | | godane quits [Ping timeout: 265 seconds] |
| 21:29:45 | | jut (jut) joins |
| 21:33:57 | | kiiwiiwastaken quits [Ping timeout: 244 seconds] |
| 21:38:29 | <Larsenv> | systwi: they probably are named that as a coincidence |
| 21:38:51 | <Larsenv> | nothing to do with Wii or Nintendo? |
| 21:41:12 | <Larsenv> | Ryz and JAA: so you remember how I was archiving https://gametdb.com/ ? I did it without the owner's permission, and I'm friends with the admin and I known him for 7 years. I told him about the Archive.org scrape and he says he noticed some higher server load because of the scrape, and he told me to ask before doing that next time lol. I didn't ask because I hesitated to and I didn't want him to say no |
| 21:42:05 | <Larsenv> | I'm sure a lot of webmasters wouldn't be happy about scraping, but I'd rather them be unhappy than have the site lost to time |
| 21:42:32 | <Ryz> | Mm, wild stuff~ >#<; |
| 21:44:50 | | VerifiedJ quits [Client Quit] |
| 21:44:52 | <Larsenv> | He didn't get mad at me for scraping |
| 21:45:07 | <Larsenv> | here's what he said |
| 21:45:08 | <Larsenv> | About archive.org I did notice heavier load on the server. How big is the total size of the archive? |
| 21:45:08 | <Larsenv> | I wish you would have told me first, it's not very cool to learn about it after the fact. In the future please tell me if you do anything like that, I would appreciate it. |
| 21:45:08 | <Larsenv> | I do keep complete backups and if anything happens to me a relative of mine would get the content out there. |
| 21:45:09 | <Larsenv> | That being said, the purpose of GameTDB was always to be an open database, so it's great that there's a backup :) |
| 21:45:09 | <Larsenv> | The way I see it, it's nice that it's preserved, but it makes it easier for people to add the whole content to their site without having to at least do some work, and the more people do that, the less people will contribute to GameTDB. |
| 21:45:18 | | VerifiedJ (VerifiedJ) joins |
| 21:45:24 | <Larsenv> | Ryz |
| 21:45:49 | <Ryz> | Ooo, interesting perspective from the webmaster~ |
| 22:06:01 | | ragu quits [Read error: Connection reset by peer] |
| 22:06:27 | | ragu joins |
| 22:08:39 | | VerifiedJ quits [Client Quit] |
| 22:09:09 | | VerifiedJ (VerifiedJ) joins |
| 22:13:52 | | Mopman (Mopman) joins |
| 22:21:28 | <Billy549> | Good perspective :P |
| 22:24:39 | | dd12 joins |
| 22:28:02 | <simon816> | PhantomTypist, not the same person :P |
| 22:32:06 | | AnotherBot quits [Remote host closed the connection] |
| 22:36:38 | | VerifiedJ quits [Client Quit] |
| 22:38:03 | | VerifiedJ (VerifiedJ) joins |
| 22:41:26 | | dd12 quits [Remote host closed the connection] |
| 22:47:10 | | VerifiedJ quits [Client Quit] |
| 22:48:40 | | VerifiedJ (VerifiedJ) joins |
| 22:55:35 | | dd12 joins |
| 23:06:53 | | dd12 quits [Remote host closed the connection] |
| 23:08:40 | | st-nick joins |
| 23:09:01 | | st-nick quits [Remote host closed the connection] |
| 23:12:25 | | godane1 quits [Ping timeout: 252 seconds] |
| 23:12:43 | | godane (godane) joins |
| 23:21:52 | | Arcorann (Arcorann) joins |
| 23:56:41 | | prebala-publici quits [Client Quit] |
| 23:56:57 | | prebala-publici joins |