| 00:00:00 | <maxfan8> | JAA: should we move to something like #couparchive (not sure if that's necessary) |
| 00:00:08 | <maxfan8> | Also: https://www.reddit.com/r/DataHoarder/comments/krx449/megathread_archiving_the_capitol_hill_riots/ |
| 00:00:26 | <Craigle> | Thanks jodizzle and JAA, I think I have it working now |
| 00:01:02 | <Craigle> | JAA: When you say post process the output, are you just talking about moving each link to a seperate line? |
| 00:01:24 | <jodizzle> | Yes, that's what J.AA means |
| 00:01:35 | <jodizzle> | Otherwise, AB won't process it correctly |
| 00:01:48 | <Craigle> | Gotcha, just want to make sure that was all I needed to do before I run it |
| 00:02:26 | | Arcorann (Arcorann) joins |
| 00:02:27 | <systwi> | maxfan8: Yeah, that's okay with me. Sorry, I was afk. |
| 00:04:50 | <@JAA> | Craigle: Yeah, basically `tr ' ' '\n'`. Ideally also dedupe, but that's not strictly necessary. |
| 00:05:41 | <maxfan8> | Awesome, sounds good -- also if anyone finds stuff on youtube worthy of archival (particularly if you see cops fraternizing with the coup plotters, throw it up on the etherpad as well under YouTube) |
| 00:09:43 | <OrIdow6> | Trump locked from Twitter for 12 hours |
| 00:10:06 | <OrIdow6> | Does ChromeBot get the old layout? If so could be used to save videos |
| 00:10:21 | <OrIdow6> | Maybe we can set up some sort of tubeup system |
| 00:13:22 | <phuzion> | OrIdow6: Trump's twitter lock is for 12 hours, if he removes the tweets. If he doesn't remove the tweets, he's stuck locked indefinitely. |
| 00:13:35 | <phuzion> | https://twitter.com/TwitterSafety/status/1346970431039934464 |
| 00:13:41 | <lennier1> | Rick Wilson just said on livestream that Twitter was considering kicking Trump off entirely. I assume that's the most archived account ever? |
| 00:13:59 | <jodizzle> | It's up there, for sure |
| 00:14:23 | <phuzion> | lennier1: I would be willing to bet that there are people in this channel that have a more complete archive of trump's twitter than the Library of Congress. |
| 00:14:41 | <OrIdow6> | maxfan8: Archiving shouldn't have the goal of punishing them, just of saving the information |
| 00:14:44 | <systwi> | OrIdow6: Use disable_polymer=true to force the old layout, but YT broke it ~ a month ago |
| 00:15:08 | <OrIdow6> | systwi: I'm talking about Twitter with that |
| 00:15:16 | <@JAA> | OrIdow6: I believe chromebot gets the new design. |
| 00:15:29 | <lennier1> | Has anyone archived TheDonald forum? I think a lot of the coup planning happened there. |
| 00:15:31 | <OrIdow6> | phuzion: I thought Twitter themselves removed his tweets? |
| 00:15:38 | <OrIdow6> | JAA: Good |
| 00:15:44 | <maxfan8> | Yeah, I agree Oridow6 |
| 00:15:48 | <@JAA> | lennier1: We've tried, but they didn't like us. |
| 00:15:51 | <OrIdow6> | Well, adding on to that, does it get videos? JAA |
| 00:15:55 | <OrIdow6> | As SPN did |
| 00:15:55 | <@JAA> | (Aka ban) |
| 00:15:56 | <phuzion> | OrIdow6: They blocked access to the tweets. Twitter never actually deletes a tweet, that's up to the user. |
| 00:16:02 | <OrIdow6> | phuzion: Oh |
| 00:16:13 | <@JAA> | Currently, TD is behind Cloudflare with captcha, so partically impossible to archive. |
| 00:16:15 | <maxfan8> | But I would like to point out that incriminating content is the most likely to be deleted, and therefore the most deserving/needing of archiving |
| 00:16:26 | <mgrandi> | Aren't there cloudflare solvers |
| 00:16:33 | <phuzion> | Like if I get suspended for calling some a fucking cocknozzle douchecanoe or something, I am forced to delete the tweet to get my account back. |
| 00:16:39 | <@JAA> | OrIdow6: Maybe, but not reliably. |
| 00:16:42 | <mgrandi> | Isn't it just a JS math challenge and then it sets a cookie and then you don't get challenged again |
| 00:16:49 | <OrIdow6> | JAA: Oh |
| 00:16:58 | <@JAA> | mgrandi: Attack mode != captcha. |
| 00:17:09 | <OrIdow6> | phuzion: Oh, thanks for explaining |
| 00:17:23 | <mgrandi> | Oh they actually set attack mode when we tried it? |
| 00:17:32 | <@JAA> | You misunderstand. |
| 00:17:53 | <mgrandi> | Cloudflare has different protection levels you set in the dashboard, right |
| 00:17:54 | <@JAA> | They've had Attack mode for a while I believe. They *currently* have captcha mode, which is just a shitty interface around reCAPTCHA. |
| 00:17:56 | <phuzion> | Yup. No problem. I've personally never been suspended, but I have a number of friends who have been suspended numerous times, including one person who has literally made the Wikipedia "Notable Twitter suspensions" page lol |
| 00:18:06 | <phuzion> | OrIdow6: ^^ |
| 00:18:10 | <mgrandi> | Ok they just leave it on then, interesting |
| 00:18:41 | <mgrandi> | Also way back machine is super backed up , 138 minute wait geez |
| 00:20:12 | <OrIdow6> | Wow, I didn't even know SPN could get backed up |
| 00:20:41 | <mgrandi> | https://usercontent.irccloud-cdn.com/file/yJG4sGfS/Screenshot_20210106-162027.png |
| 00:25:20 | <@kiska> | I guess I'll say this for pad.notkiska.pw please export regularly since I am working on the instance and may break it :D |
| 00:26:36 | <OrIdow6> | Ok, currently exporting once every min |
| 00:26:51 | <OrIdow6> | If it goes down & anyone needs it, ping me |
| 00:27:00 | <@kiska> | I do recommend once per 5 mins |
| 00:28:03 | <OrIdow6> | Ok, slowed to once every 5m |
| 00:33:29 | <lennier1> | I've used paid captcha solvers before. It's quite cheap per captcha, but obviously depends on how frequently they require one. |
| 00:36:29 | | jlu5` quits [Quit: Client exited] |
| 00:43:43 | <OrIdow6> | Thinking of doing videos - "#stopthesteal filter:native_video min_retweets:100 since:2021-01-05" look good for Twitter? |
| 00:43:57 | <OrIdow6> | I know IA can get fairly cautious about TUbeup |
| 00:44:56 | <jodizzle> | OrIdow6: That seems fine to me. |
| 00:52:44 | | etnguyen03 (etnguyen03) joins |
| 00:53:04 | <OrIdow6> | Hoping IA doesn't ban me for that |
| 00:54:45 | <flashfire42> | OrIdow6 make an alt account for tubeup is my advice |
| 00:55:51 | <Craigle> | Agreed. If they ban your account, at least it won't be your main account |
| 01:00:49 | <@kiska> | Craigle: I might have stolen your colour on the pad :D |
| 01:08:59 | <Craigle> | HA, I thought I was doing it wrong, and got very confused |
| 01:09:10 | <Craigle> | With that said, I may still be doing it wrong |
| 01:25:12 | <@kiska> | I really need to get this working... https://server8.kiska.pw/uploads/6e2812281dc1a40b/image.png |
| 01:57:42 | <OrIdow6> | maxfan8: I see you ahve https://www.youtube.com/watch?v=bE2-VnBd4ps , since removed, marked as finished - what was this, and where did you save it to? |
| 01:59:49 | | Guest69 joins |
| 02:00:01 | | Dallas quits [Client Quit] |
| 02:00:51 | | Guest69 is now known as bithippo |
| 02:01:06 | | bithippo is now known as rumbles |
| 02:04:04 | | Dallas (Dallas) joins |
| 02:11:19 | | Doranwen quits [Ping timeout: 252 seconds] |
| 02:27:24 | | Doranwen joins |
| 02:27:26 | | Doranwen is now authenticated as Doranwen |
| 02:28:11 | <Jake> | in relation to the other message: https://twitter.com/Twitch/status/1347003152222937088 |
| 02:29:44 | <maxfan8> | OrIdow6 #youtubearchive |
| 02:36:15 | <OrIdow6> | maxfan8: Oh |
| 02:37:05 | <OrIdow6> | If you have any other videos (or if anyone else does), feel free to tell me as well - TubeUp -> they get public instead of going into a Google Drive whose data may or may not last |
| 03:02:54 | | Ruthalas (Ruthalas) joins |
| 03:04:05 | | rumbles quits [Client Quit] |
| 03:04:26 | <OrIdow6> | Does anyone have any links to livestreams, etc. made? |
| 03:07:54 | <atphoenix> | livestreams of the turmoil-in-america coverage and/or Congress events? |
| 03:08:33 | <atphoenix> | https://www.pbs.org/newshour/politics/congress-reconvenes-electoral-votes |
| 03:08:51 | <atphoenix> | https://www.youtube.com/watch?v=BKy84YmL-sU&feature=emb_title |
| 03:09:32 | <atphoenix> | CNBC: https://www.youtube.com/watch?v=NIPgXM8bRtk |
| 03:09:58 | <OrIdow6> | Livestreams made by the rioters |
| 03:16:43 | <atphoenix> | Some folks on Twitter are reposting clips hosted on Infowars. Appears they have a series of clips from around the time when the woman was shot |
| 03:24:08 | <Craigle> | Yeah, I am wondering if we should be trying to grab places like poal and some of the other know sites. Not sure what we already have on InfoWars |
| 03:33:12 | <atphoenix> | OrIdow6, livestream from RSBN: https://www.youtube.com/watch?v=vTXOkcwYxv |
| 03:34:17 | <OrIdow6> | atphoenix: Says it's "unavailable" |
| 03:35:14 | <atphoenix> | oh? it plays for me... |
| 03:35:36 | <OrIdow6> | Did you copy & paste the whole thing? Youtube-dl says URL "looks truncated" |
| 03:35:54 | <atphoenix> | https://www.youtube.com/watch?v=vTXOkcwYxvE |
| 03:36:06 | <atphoenix> | I had lost a char |
| 03:37:34 | <OrIdow6> | Thank you |
| 03:37:39 | <atphoenix> | there are some other videos from today here too: https://www.youtube.com/c/RightSideBroadcastingNetwork/videos |
| 03:38:31 | <OrIdow6> | It looks like IA is blocking the creation of items that Tubeup creates from Youtube, I assume by looking for the "youtube-" prefix; in contrast Twitter is fine |
| 03:39:31 | <atphoenix> | Trump rally in DC today: https://www.youtube.com/watch?v=HrGJfQzUrnY |
| 03:45:43 | <atphoenix> | https://www.youtube.com/watch?v=AWU7PW3Q3xA Live DC 2021 from The Sun |
| 03:47:17 | <OrIdow6> | I can find the ones from big media outlets easily, and in any case those are not likely to be removed |
| 03:59:27 | <jodizzle> | OrIdow6: Someone was apparently streaming on this channel, but idk how this site works: https://dlive.tv/bakedalaska |
| 04:00:03 | <OrIdow6> | jodizzle: I saw - apparently it got taken down, but I found 2 copies on Youtube |
| 04:00:50 | <OrIdow6> | That's the sort of thing I'm looking for - can't imagine that that's the only one |
| 04:02:56 | | britm0b joins |
| 04:04:24 | <@kiska> | Maybe this https://www.youtube.com/watch?v=bARiQZo2JSk :D |
| 04:06:16 | | britmob quits [Ping timeout: 252 seconds] |
| 04:11:55 | | britmob joins |
| 04:13:55 | <Craigle> | OrIdow6: This has links to a bunch of different streams/streamers. Might be some gold in there. https://www.disclose.tv/ |
| 04:14:44 | | britm0b quits [Ping timeout: 240 seconds] |
| 04:17:42 | | britm0b joins |
| 04:18:04 | | Arcorann quits [Ping timeout: 240 seconds] |
| 04:21:40 | | britmob quits [Ping timeout: 265 seconds] |
| 04:25:46 | <OrIdow6> | kiska: From the video I have (I think) |
| 04:25:56 | <OrIdow6> | Craigle: Thank you, got some from that |
| 04:29:11 | | qw3rty__ joins |
| 04:32:44 | | qw3rty_ quits [Ping timeout: 240 seconds] |
| 04:52:18 | <@dxrt> | FTP of some capitol material https://twitter.com/_IntelligenceX/status/1346967229187952644 |
| 04:52:35 | | etnguyen03 quits [Client Quit] |
| 05:01:54 | | Doran joins |
| 05:02:16 | | Doranwen quits [Ping timeout: 265 seconds] |
| 06:57:23 | | thuban joins |
| 06:58:08 | | lunik1 quits [Client Quit] |
| 07:23:28 | | lunik1 joins |
| 08:20:45 | | Arcorann (Arcorann) joins |
| 08:49:46 | | burnmjd joins |
| 09:31:02 | | Ryz quits [Remote host closed the connection] |
| 09:31:56 | | Ryz (Ryz) joins |
| 12:37:00 | | Peetz0r quits [Quit: Het is 13:37, tijd om ook maar eens te rebooten dan :p] |
| 12:39:42 | | Peetz0r (Peetz0r) joins |
| 13:24:21 | | britmob joins |
| 13:27:21 | | britm0b quits [Ping timeout: 265 seconds] |
| 14:04:10 | | Ryz quits [Remote host closed the connection] |
| 14:13:04 | | Arcorann quits [Ping timeout: 240 seconds] |
| 14:26:00 | | Ryz (Ryz) joins |
| 14:28:54 | | sec^nd quits [Remote host closed the connection] |
| 14:29:21 | | sec^nd (second) joins |
| 14:32:47 | | Hackerpcs quits [Quit: Hackerpcs] |
| 14:34:42 | | Hackerpcs (Hackerpcs) joins |
| 15:03:39 | <mgrandi> | Has anyone thrown this into archive bot? It's apparently one of the people who died at the capitol yesterday https://twitter.com/Ashli_Babbitt |
| 15:04:25 | <Craigle> | Yep, that was grabbed last night |
| 15:05:41 | <mgrandi> | Cool |
| 15:11:04 | | treora quits [Ping timeout: 240 seconds] |
| 15:51:50 | | treora joins |
| 16:05:23 | | AlsoHP_Archivist quits [Client Quit] |
| 16:05:38 | | HP_Archivist (HP_Archivist) joins |
| 16:29:27 | | etnguyen03 (etnguyen03) joins |
| 17:09:38 | | VerifiedJ quits [Quit: The Lounge - https://thelounge.chat] |
| 17:10:28 | | VerifiedJ (VerifiedJ) joins |
| 17:19:46 | <HP_Archivist> | Past day or so the WBM Chrome Extension continuously shows job fails, anyone else having this issue? |
| 17:21:59 | <@JAA> | SPN has been overwhelmed, and SPN2 had wait times of several hours last night. So yeah. |
| 17:23:48 | <Craigle> | I've been using it. Sometimes you have to be persistent, or come back and try again a little later |
| 17:25:15 | | Doran quits [Remote host closed the connection] |
| 17:25:31 | | Doran joins |
| 17:31:32 | <atphoenix> | HP_Archivist, are you using the Chrome store version, or the GitHub version? Github version is newer and has more features. |
| 17:32:32 | <HP_Archivist> | atphoenix: Chrome store, think I'll switch over to the GH version though |
| 17:32:49 | <atphoenix> | however it was also not saving every page that I visited (I had turned on autosave). Some I had to retry/reload. |
| 17:32:54 | <HP_Archivist> | And yeah Craigle I've been trying that but kept getting job fails regardless |
| 17:32:59 | <atphoenix> | well, autosave if not already saved |
| 17:33:19 | <HP_Archivist> | JAA it's understandable |
| 17:45:17 | <HP_Archivist> | atphoenix: This is it, right https://github.com/internetarchive/wayback-machine-webextension |
| 17:46:12 | <Craigle> | That's what I'm using. Make sure to follow the directions to get the latest version. I find it vastly improved |
| 17:46:52 | <HP_Archivist> | Alright will do, Craigle |
| 17:46:57 | <Craigle> | Also make sure you're logged in. They may be rate limiting harder on anonymous accounts ro something |
| 17:47:42 | <HP_Archivist> | Erm, will being logged in save the captures under web captures in the account? |
| 17:48:47 | <HP_Archivist> | For example, my IA account is Harry Potter focused, and I'd rather not have non-HP items show up under the 'web captures' tab when signed in/connected to that account |
| 17:49:05 | <HP_Archivist> | web archives* tab |
| 17:51:23 | <Craigle> | Pretty sure no, but let me double check real quick |
| 17:52:58 | <Craigle> | I don't see anything captured using the extension in my web archive, so you should be good |
| 17:53:25 | <HP_Archivist> | Alright cool will give it a try |
| 17:59:22 | <thuban> | i've been using SPN via waybackpy and overall i have about a 67% success rate :/ |
| 18:00:07 | <thuban> | (not counting the tumblr image thing which i was never able to work out... i should write them about that) |
| 18:07:57 | <thuban> | hm, apparently package has been released three times in the last week. that's definitely not concerning or anything |
| 18:18:55 | <atphoenix> | yes, that is the extension. I also like the federated archive concept I saw mentioned in one of the archivebox.io threads as that would allow for a broader saving of "what's out there" |
| 18:28:23 | <Dallas> | Are the warcs archivebox produces to spec enough to go in to ia ??? |
| 18:33:53 | <@JAA> | According to the readme, it uses wget for the WARCs, so probably, except for the angle brackets bug. |
| 18:34:09 | <@JAA> | Still don't know whether IA works around that yet. |
| 18:34:30 | <@JAA> | Reminds me to poke the devs about the new release that will fix this. |
| 18:43:12 | <atphoenix> | FYI, other capitol turmoil archiving efforts: https://twitter.com/_IntelligenceX/status/1346967229187952644 and https://www.reddit.com/r/DataHoarder/comments/krx449/megathread_archiving_the_capitol_hill_riots/ |
| 18:49:17 | | VerifiedJ quits [Client Quit] |
| 18:55:54 | <atphoenix> | here's the specific archivebox federated archive concept thread I was referring to https://old.reddit.com/r/selfhosted/comments/an2368/archivebox_the_opensource_selfhosted_web_archive/efv0ssa/ . Multiple references to IA. |
| 18:57:09 | <Dallas> | Okay that sounds pretty cool if it can be pulled off |
| 18:59:28 | | VerifiedJ (VerifiedJ) joins |
| 19:06:28 | <Ryz> | Any more stuff related to yesterday's BS to archive via AB? |
| 19:08:00 | <atphoenix> | Ryz, maybe look at the 2 projects I mentioned? |
| 19:24:05 | | HP_Archivist quits [Client Quit] |
| 19:24:54 | <Ryz> | Do they have their own IRC channels, atphoenix? |
| 19:36:07 | | DogsRNice (Webuser299) joins |
| 19:42:01 | <atphoenix> | idk, I have not dug into those links further. The first one mentions an FTP site of collected material. |
| 19:42:02 | | Wingy quits [Read error: Connection reset by peer] |
| 19:42:42 | <atphoenix> | OrIdow6, https://twitter.com/hannahgais/status/1346913339000156162 may lead you to more livestreams or reuploads of livestreams |
| 19:44:02 | <@JAA> | Yeah, the FTP was mentioned last night, and I ran it through AB. As its size increases though, so does the chance of an AB crash, so rerunning that isn't reasonable. |
| 19:44:19 | | Wingy (Wingy) joins |
| 19:45:31 | <atphoenix> | https://www.youtube.com/watch?v=pdgDLwNXM3g "Baked Alaska DC Stream (Ft. MAGA Hulk) |
| 19:45:32 | <atphoenix> | " |
| 19:47:44 | | Wingy quits [Client Quit] |
| 19:49:18 | | Wingy (Wingy) joins |
| 19:56:53 | | Wingy quits [Client Quit] |
| 19:58:56 | <atphoenix> | https://www.youtube.com/watch?v=7EpHSOGUIL4 #washingtondc #theellipse #live #irl LIVE: in DC where Trump has called in the PATRIOTS #civilwar (from 1/6) |
| 20:07:05 | | Wingy (Wingy) joins |
| 20:30:01 | | holbrooke_ quits [Client Quit] |
| 20:31:24 | | holbrooke joins |
| 20:31:57 | | holbrooke is now authenticated as holbrooke |
| 21:32:11 | <mgrandi> | Archive bot request: this guy apparently died from a heart attack at the capitol, 50 tweets https://twitter.com/kevinGreeson5 |
| 21:44:11 | <Barto> | mgrandi: queued in archivebot |
| 21:44:33 | <mgrandi> | Thanks :) |
| 21:45:16 | <Dallas> | Nice, a 91% on Bot Sentinel https://botsentinel.com/profile/1175793002788401152 |
| 22:58:03 | | spirit quits [Quit: Leaving] |
| 23:43:55 | | HackMii quits [Remote host closed the connection] |
| 23:44:15 | | HackMii (hacktheplanet) joins |
| 23:54:53 | | Arcorann (Arcorann) joins |