HomeSearchPrevious dayNext day

00:03:57<nicolas17>ugh I wish I could just rsync the NCBI data server to a local disk
00:05:51<nicolas17>but I don't have anywhere near enough local storage, and also the server feels like it's running on a highly contended magnetic disk
00:06:01<nicolas17>high latency for metadata ops
00:06:40<@JAA>nicolas17: → #UncleSamsArchive ?
00:07:51<nicolas17>my current level of "fucking around with unclear goal" is almost worth -ot, but sure
00:07:55<nicolas17>:p
00:20:21<nicolas17>JAA: what do you think is the best approach for https://wiki.archiveteam.org/index.php/Foro_3DJuegos ? would we need DPoS?
00:21:04<@JAA>nicolas17: I haven't looked at it yet as other things are more urgent.
00:22:18<nicolas17>I haven't looked at the size yet, maybe a single-computer wget-lua can do it, but I have no experience writing such lua scripts
00:23:46<Vokun>we've got about 32 days left on it. Probably enough time, hopefully, if unclesam doesn't take 100% of dev time for the 3 weeks
00:23:47<@JAA>Yeah, or qwarc. The IDs seem sequential enough, but I'm guessing they're using post IDs for topics, too, since 54M topics seems a bit too large.
00:24:03<@JAA>And it seems you need to know which forum the post is in, too.
00:24:06<pabs>c3manu: I have a browser console based scraper for the API that the securitytrails.com frontend uses, sec
00:24:42<nicolas17>oh, I didn't realize if you go to the ID alone it redirects to the correct slug
00:25:04<nicolas17>that makes sequential scraping much easier
00:25:07<@JAA>Ah no, /foros/temas/$ID/0/ does work, actually.
00:25:17<@JAA>/foros/tema/$ID/0/ *
00:25:49<nicolas17>yeah I just thought you needed the correct slug too, and then you'd have no option but crawling links
00:25:50<pabs>c3manu: you have to adjust the domain at the top and the API URL further down (it changes): https://transfer.archivete.am/5f2XC/securitytrails-domain-scraper.js
00:25:51<eggdrop>inline (for browser viewing): https://transfer.archivete.am/inline/5f2XC/securitytrails-domain-scraper.js
00:25:56etnguyen03 quits [Client Quit]
00:26:21<pabs>c3manu: I only use it when there are more than one page of results though :)
00:26:25<@JAA>Yeah, I thought you needed the forum slug (foros or community-foros or maybe others), but I made a typo.
00:26:36<@JAA>Which would be less bad but still annoying.
00:27:00<Webuser849303>Thanks JAA, got the docker working with your help
00:27:15<nicolas17>throwing 54M URLs into archivebot is probably a bad idea (:
00:27:51drfish joins
00:28:18<@JAA>nicolas17: Also wouldn't work properly for pagination URLs.
00:28:19<Vokun>there've been bigger projects
00:28:46<@JAA>Anyway, I'll take a closer look in a couple days, it's just not top priority for me right now.
00:28:52<nicolas17>yeah
00:29:05<Vokun>if only copy pasting another #y project worked
00:29:11<@JAA>If you want to poke around, it'd be good to know whether there's any content that isn't in topics.
00:31:16<h2ibot>PaulWise edited Finding subdomains (+184, add securitytrails-domain-scraper.js): https://wiki.archiveteam.org/?diff=54352&oldid=53857
00:32:30drfish quits [Client Quit]
00:50:24Dango360_ (Dango360) joins
00:53:33_Dango360 quits [Ping timeout: 260 seconds]
00:58:18sec^nd quits [Remote host closed the connection]
00:58:34spirit quits [Ping timeout: 250 seconds]
00:58:45sec^nd (second) joins
01:11:30spirit joins
01:18:30cascode quits [Ping timeout: 250 seconds]
01:19:05cascode joins
01:28:33cascode quits [Ping timeout: 260 seconds]
01:28:57cascode joins
01:32:23gust joins
01:39:22Naruyoko joins
01:41:39etnguyen03 (etnguyen03) joins
01:41:58Naruyoko5 quits [Ping timeout: 260 seconds]
01:45:29notarobot19 joins
01:49:33notarobot1 quits [Ping timeout: 260 seconds]
01:49:34notarobot19 is now known as notarobot1
01:49:42cascode quits [Read error: Connection reset by peer]
01:50:01cascode joins
01:54:48cascode quits [Ping timeout: 260 seconds]
01:57:43cascode joins
01:59:03monoxane (monoxane) joins
02:05:45cascode quits [Read error: Connection reset by peer]
02:05:53cascode joins
02:13:21etnguyen03 quits [Client Quit]
02:20:58ramsey (ramsey) joins
02:24:43ramsey quits [Changing host]
02:24:43ramsey (ramsey) joins
02:27:26etnguyen03 (etnguyen03) joins
02:27:28Barto quits [Ping timeout: 260 seconds]
02:31:54Wohlstand (Wohlstand) joins
02:36:16gust quits [Read error: Connection reset by peer]
02:39:57BornOn420 quits [Remote host closed the connection]
02:40:35BornOn420 (BornOn420) joins
02:41:24Wohlstand quits [Client Quit]
02:45:01<h2ibot>Nulldata uploaded File:Dailymotion logo 2023.png: https://wiki.archiveteam.org/?title=File%3ADailymotion%20logo%202023.png
02:56:03<h2ibot>Nulldata edited Dailymotion (+898, Added project and information regarding…): https://wiki.archiveteam.org/?diff=54356&oldid=49222
02:59:05<h2ibot>Nulldata edited Current Projects (+86, Added Dailymotion to upcoming projects): https://wiki.archiveteam.org/?diff=54357&oldid=54315
02:59:32etnguyen03 quits [Client Quit]
03:18:59HP_Archivist (HP_Archivist) joins
03:19:08etnguyen03 (etnguyen03) joins
03:19:22<pabs>https://sublimemusic.app/ is EOL
03:19:30<pabs>http://sumnerevans.com/posts/projects/sublime-music-eom/
03:19:47Retroity joins
03:22:13<Retroity>Hi all, I'm currently running the warrior on my Windows PC using Docker. I'm currently running the US Government project. The CheckIP step keeps failing, with the error noting "AssertionError: Invalid return code 4 on http://legacy-api.arpa.li/now." Not sure what the issue is. Any help would be appreciated. Thanks!
03:22:51Wohlstand (Wohlstand) joins
03:24:03scurvy_duck quits [Ping timeout: 260 seconds]
03:25:13<TheTechRobo>Retroity: Are you using a VPN?
03:26:04<Retroity>No. I'm on my university's network but that's the only thing I can think of that might be causing issues
03:32:17<that_lurker>can you open that url on the computer?
03:33:14<Retroity>Yes, opening it on my web browser displays "1738812779.243"
03:47:25HP_Archivist quits [Client Quit]
03:49:41icedice quits [Quit: Leaving]
03:53:19<h2ibot>PaulWise edited Dailymotion (+134, add dailymotion-dl): https://wiki.archiveteam.org/?diff=54358&oldid=54356
03:53:20<h2ibot>PaulWise edited Dailymotion (+0, format typo): https://wiki.archiveteam.org/?diff=54359&oldid=54358
03:55:38etnguyen03 quits [Remote host closed the connection]
03:58:00Retroity quits [Client Quit]
03:58:03Craigle quits [Quit: The Lounge - https://thelounge.chat]
03:58:20<h2ibot>PaulWise edited Mailing Lists (+76, groups.io supports custom domains :(): https://wiki.archiveteam.org/?diff=54360&oldid=54252
03:58:41@imer quits [Quit: Oh no]
03:58:56Craigle (Craigle) joins
03:59:12imer (imer) joins
03:59:13@ChanServ sets mode: +o imer
04:00:21<h2ibot>PaulWise edited Mailing Lists (+165, fix up message-id text, add mapping/search idea): https://wiki.archiveteam.org/?diff=54361&oldid=54360
04:00:44Webuser809883 joins
04:01:12Webuser809883 quits [Client Quit]
04:05:11Craigle quits [Client Quit]
04:06:18Craigle (Craigle) joins
04:11:19Wohlstand quits [Client Quit]
04:13:03Krownest (Krownest) joins
04:13:28<TheTechRobo>Retroity: Your university is likely blocking custom DNS, unfortunately. We use Quad9 in our projects.
04:13:33<TheTechRobo>RIP
04:15:06<@JAA>Perhaps we should do the DNS check before the time check so it's more obvious what's wrong.
04:17:30BlueMaxima quits [Read error: Connection reset by peer]
05:06:22ahm258 joins
05:09:50scurvy_duck joins
05:22:12HP_Archivist (HP_Archivist) joins
05:22:12TerritoryJazz quits [Quit: Ooops, wrong browser tab.]
05:27:30Webuser741750 joins
05:35:07Webuser741750 quits [Client Quit]
05:35:26abahbob joins
06:18:01ArchivalEfforts joins
06:22:51fuzzy8021 (fuzzy80211) joins
06:23:42fuzzy80211 quits [Read error: Connection reset by peer]
06:44:44spirit quits [Quit: Leaving]
07:12:06ahm258 quits [Ping timeout: 250 seconds]
07:18:10monika quits [Ping timeout: 250 seconds]
07:19:11<c3manu>pabs: i was looking to get a subdomain list for gatech.edu (since c99 and subdomain.center both hat obvious ones missing). securitytrails just said "10,000+" or sth ^^"
07:19:17<c3manu>so yeah, more than one page
07:19:46<pabs>hmm, seems like too many domains?
07:20:02<c3manu>no idea how i would use that js file you sent me
07:20:08<c3manu>too many for what?
07:20:16<pabs>too many to be realistic
07:20:47<pabs>basically, open your browser console, paste the code in there and run it. you will get a urls.txt file saved with the URLs
07:20:49<c3manu>it’s a university. i would of course expect some junk in it, but also an awful lot of valid ones
07:20:56monika (boom) joins
07:20:59<pabs>hmm
07:22:03LunarianBunny11474 (LunarianBunny1147) joins
07:23:48LunarianBunny1147 quits [Ping timeout: 250 seconds]
07:23:48LunarianBunny11474 is now known as LunarianBunny1147
07:26:54<c3manu>pabs: might be a little late now anyways, some search results for "DEI" google has indexed return 404s already. i found hits on other subdomains, but going through them one by one is being tedious, at best
07:27:06<c3manu>that was the reason for grabbing those: https://cyberplace.social/@GossiTheDog/113921481331311737/
07:27:32<pabs>ah yeah
07:31:23abirkill quits [Ping timeout: 260 seconds]
07:33:33Webuser849303 quits [Quit: Ooops, wrong browser tab.]
07:36:36abirkill (abirkill) joins
07:40:44ahm258 joins
08:18:03scurvy_duck quits [Ping timeout: 260 seconds]
08:40:53Webuser108513 joins
08:41:39Webuser108513 quits [Client Quit]
09:03:36myself quits [Read error: Connection reset by peer]
09:03:58myself (myself) joins
09:41:25<@JAA>So Equinix Metal, we should maybe start a list of projects that are in danger. Not all of them may be able to find adequate hosting that allows them to transfer all existing data.
09:46:32<@JAA>Freedesktop, Alpine Linux, and WireGuard are the ones I'm aware of so far.
09:51:11<monoxane>JAA is something going on with Equinix Metal?
09:51:41<monoxane>oh fuck I see it ignore me
09:51:44<monoxane>thats a big rip
09:52:31<@JAA>Yeah, https://deploy.equinix.com/blog/sunsetting-equinix-metal/
09:53:04<monoxane>particularly unfortunate for us because the 100tb storage box for $500/mo would make a wonderful target 😆
09:53:05<@JAA>And those projects who were given free hosting resources are getting kicked within a couple months.
10:00:08Hackerpcs quits [Ping timeout: 260 seconds]
10:03:40icedice (icedice) joins
10:17:45Hackerpcs (Hackerpcs) joins
10:21:43threedeeitguy (threedeeitguy) joins
10:50:53Church quits [Ping timeout: 260 seconds]
11:05:54Church (Church) joins
11:19:06abahbob quits [Quit: Ooops, wrong browser tab.]
11:22:06@arkiver quits [Remote host closed the connection]
11:23:18arkiver (arkiver) joins
11:23:18@ChanServ sets mode: +o arkiver
11:41:41Webuser048517 joins
11:42:15Webuser048517 quits [Client Quit]
11:46:10loug8318142 joins
11:46:35yasomi quits [Read error: Connection reset by peer]
11:49:00yasomi (yasomi) joins
11:55:04cascode quits [Ping timeout: 250 seconds]
11:55:17cascode joins
12:00:02Bleo18260072271962345 quits [Quit: The Lounge - https://thelounge.chat]
12:02:53Bleo18260072271962345 joins
12:05:03mls (mls) joins
12:10:01@arkiver quits [Remote host closed the connection]
12:10:58arkiver (arkiver) joins
12:10:58@ChanServ sets mode: +o arkiver
12:18:55moth_ quits [Remote host closed the connection]
12:27:28Freiner quits [Quit: Ooops, wrong browser tab.]
12:35:32SkilledAlpaca418962 quits [Quit: SkilledAlpaca418962]
12:36:03SkilledAlpaca418962 joins
12:55:25nomead joins
13:06:26Naruyoko5 joins
13:10:02Naruyoko quits [Ping timeout: 250 seconds]
13:13:09pixel (pixel) joins
13:31:09Barto (Barto) joins
13:44:15tmob joins
13:59:14pixel leaves
14:18:53<h2ibot>Imer edited Deathwatch (+170, /* 2025 */ add splits.io): https://wiki.archiveteam.org/?diff=54362&oldid=54349
14:21:54<h2ibot>Imer edited Deathwatch (+327, /* Frozen Solid */ add evga forums): https://wiki.archiveteam.org/?diff=54363&oldid=54362
14:56:46<Hans5958>What is the "web server" on the Docker containers? Are there any benefits on turning them on for monitoring purposes?
15:06:11tmob quits [Read error: Connection reset by peer]
15:07:23tmob joins
15:17:33<Hans5958>Btw Livestream is brewing (minus videos)
15:18:50<@arkiver>oh yeah
15:21:32caylin quits [Read error: Connection reset by peer]
15:21:49caylin (caylin) joins
15:24:19scurvy_duck joins
15:28:53Wohlstand (Wohlstand) joins
15:54:18Webuser182716 joins
15:54:24Webuser182716 quits [Client Quit]
15:55:06Naruyoko joins
15:58:18Naruyoko5 quits [Ping timeout: 260 seconds]
15:58:53Wohlstand quits [Remote host closed the connection]
15:59:15Wohlstand (Wohlstand) joins
16:01:23ahm258 quits [Quit: The Lounge - https://thelounge.chat]
16:03:31VickoSaviour joins
16:05:21ahm258 joins
16:06:26Wohlstand quits [Client Quit]
16:07:55<VickoSaviour>I just randomly saw this while browsing on answers.microsoft.com
16:07:55<VickoSaviour>"Windows Client Forum Moving to Microsoft Q&A
16:07:55<VickoSaviour>We are excited to announce that soon, the Windows Client for IT Pros forum will be available exclusively in the Microsoft Q&A.. This change will help us provide a more streamlined and efficient experience for all your questions and discussions...
16:07:55<VickoSaviour>Starting February 21, you will no longer be able to create new questions here in the Microsoft Support Community. However, you can continue to participate in ongoing discussions and create new questions in the Microsoft Q&A."
16:07:55<VickoSaviour>This includes the Windows Server forum, and i'm afraid that maybe the rest of the forums may be included and inaccessible in the near future. It's up to you if you would archive Microsoft Support Community.
16:08:17<@arkiver>whenever it goes "we are excited [...]", it's bad news
16:26:48breadbrix (breadbrix) joins
17:19:40lennier2_ joins
17:22:53lennier2 quits [Ping timeout: 260 seconds]
17:30:43SootBector quits [Remote host closed the connection]
17:31:05SootBector (SootBector) joins
17:41:38lennier2 joins
17:44:20lennier2_ quits [Ping timeout: 250 seconds]
18:10:50lennier2_ joins
18:11:18scurvy_duck quits [Ping timeout: 260 seconds]
18:13:22lennier2 quits [Ping timeout: 250 seconds]
18:13:30<that_lurker>ohh microsoft changing stuff. That means millions of links that will take you Dictionary
18:13:41<that_lurker>s/Dictionary/nowhere
18:18:54that_lurker quits [Remote host closed the connection]
18:18:59that_lurker (that_lurker) joins
18:39:32scurvy_duck joins
18:45:00benjins3 quits [Ping timeout: 250 seconds]
18:49:13NatTheCat quits [Ping timeout: 260 seconds]
18:56:13HP_Archivist quits [Ping timeout: 260 seconds]
18:57:08Sluggs quits [Ping timeout: 250 seconds]
19:01:26NatTheCat joins
19:04:36Sluggs joins
19:09:18VickoSaviour quits [Quit: Ooops, wrong browser tab.]
19:14:18wyatt8740 quits [Ping timeout: 260 seconds]
19:15:18wyatt8740 joins
19:23:55scurvy_duck quits [Client Quit]
19:42:20khaoohs quits [Read error: Connection reset by peer]
19:45:34lennier2 joins
19:48:43lennier2_ quits [Ping timeout: 260 seconds]
19:54:14Webuser544416 joins
19:55:39khaoohs joins
20:01:16loug8318142 quits [Ping timeout: 250 seconds]
20:04:00<nicolas17>brickshelf reached 1TB
20:04:10loug8318142 joins
20:06:17loug8318142 quits [Read error: Connection reset by peer]
20:06:33loug8318142 joins
20:07:34HP_Archivist (HP_Archivist) joins
20:17:55AlsoHP_Archivist joins
20:21:38HP_Archivist quits [Ping timeout: 250 seconds]
20:24:40loug8318142 quits [Ping timeout: 250 seconds]
20:32:50Webuser544416 quits [Client Quit]
20:37:28<@JAA>answers.microsoft.com is fun. It has an OAuth flow even for anonymous access.
20:44:30<h2ibot>JustAnotherArchivist edited ArchiveTeam Warrior (+231, /* Can I use whatever internet access for the…): https://wiki.archiveteam.org/?diff=54364&oldid=53514
20:45:37DogsRNice joins
20:57:48loug8318142 joins
20:59:58SootBector quits [Remote host closed the connection]
21:15:49Webuser312814 joins
21:16:54Webuser312814 quits [Client Quit]
21:21:24hyenatown joins
21:26:29lennier2_ joins
21:29:14lennier2 quits [Ping timeout: 250 seconds]
21:46:33tmob quits [Ping timeout: 260 seconds]
21:53:18AlsoHP_Archivist quits [Client Quit]
21:53:34HP_Archivist (HP_Archivist) joins
21:59:31Webuser705947 joins
22:00:20<Webuser705947>Hello
22:04:09<Webuser705947>JAA Hello
22:04:25<@JAA>Hi
22:06:03<Webuser705947>JAA So, is there a reason why you can't list the URL that I just requested to archive including the others?
22:06:58etnguyen03 (etnguyen03) joins
22:07:06<Webuser705947>JAA i mean't to say URL's contents. My bad
22:07:26<nicolas17>how can we possibly know what files are in wdig-2.ocs.llnw.net?
22:08:58<pokechu22>do *you* have a list of all URLs on http://wdig-2.ocs.llnw.net/? If you do we can run that, but I'm not seeing any way to do it
22:09:19<pokechu22>that page is a 403 and https://duckduckgo.com/?t=ffab&q=site%3Awdig-2.ocs.llnw.net&ia=web and https://www.google.com/search?hl=en&q=site%3Awdig%2D2.ocs.llnw.net give no results
22:11:45<Webuser705947>maybe you can ask a user who is good at archiving websites that have 403 forbidden codes on them, or maybe use FileZilla application. And no, i don't have a list of all URL's for that URL.
22:13:57<nicolas17>FileZilla is for FTP
22:13:59<nicolas17>this is not an FTP server
22:14:32<Webuser705947>Oops
22:14:40<@JAA>Well, it actually is an FTP server, but not an open one we can access.
22:15:29nomead quits [Quit: Leaving]
22:15:29<@JAA>If you have a login, maybe that can be used to list it.
22:15:50<@JAA>If you know of *any* content on there, maybe it's possible to guess more URLs.
22:16:17<@JAA>We don't have a magic wand, unfortunately.
22:16:35<nicolas17>bizarre, the unencrypted-rsync port is open too, but you can't retrieve anything from there either
22:17:51SootBector (SootBector) joins
22:19:38benjins3 joins
22:55:05<that_lurker>They do have some interesting subdomains https://subdomainfinder.c99.nl/scans/2025-02-06/llnw.net
22:55:51territoryjazz joins
22:55:52territoryjazz quits [Client Quit]
22:56:13territoryjazz joins
22:57:39<@JAA>TIL they're called Edgio now. It was Limelight, but they acquired/merged with Edgecast recently.
23:01:14<that_lurker>and now they are part of akamai
23:01:36<that_lurker>https://learn.microsoft.com/en-us/azure/cdn/edgio-retirement-faq
23:03:03loug8318142 quits [Client Quit]
23:05:25<@JAA>Akamai--
23:05:25<eggdrop>[karma] 'Akamai' now has -83 karma!
23:07:15<pokechu22>https://github.com/ufs-community https://github.com/NOAA-EMC https://github.com/NOAA-OWP https://github.com/NOAA-GFDL https://github.com/NOAA-GSL https://github.com/NOAA-PMEL https://github.com/NOAA-ORR-ERD https://github.com/NOAA-FIMS https://github.com/NCAR
23:07:25<pokechu22>err, that's supposed to be #gitgud
23:46:08cascode quits [Ping timeout: 260 seconds]
23:46:35cascode joins

HomeSearchPrevious dayNext day