HomeSearchPrevious dayNext day

00:06:15Hackerpcs quits [Quit: Hackerpcs]
00:08:08Hackerpcs (Hackerpcs) joins
00:13:16vantec quits [Changing host]
00:13:16vantec (vantec) joins
01:11:05icedice2 joins
01:11:31icedice quits [Read error: Connection reset by peer]
01:12:46MRX joins
01:12:46icedice2 quits [Read error: Connection reset by peer]
01:13:44MRX quits [Read error: Connection reset by peer]
01:14:22icedice (icedice) joins
01:17:46icedice2 joins
01:18:26icedice2 quits [Read error: Connection reset by peer]
01:19:39icedice2 joins
01:20:06icedice quits [Ping timeout: 265 seconds]
01:23:26icedice (icedice) joins
01:24:27icedice2 quits [Ping timeout: 265 seconds]
01:25:12icedice quits [Read error: Connection reset by peer]
01:25:23icedice (icedice) joins
01:27:23icedice quits [Read error: Connection reset by peer]
01:27:38icedice (icedice) joins
01:34:21icedice2 joins
01:34:22icedice quits [Read error: Connection reset by peer]
01:36:54icedice2 quits [Read error: Connection reset by peer]
01:37:02icedice (icedice) joins
01:40:21icedice quits [Read error: Connection reset by peer]
01:41:04icedice (icedice) joins
01:42:08icedice quits [Read error: Connection reset by peer]
01:42:40icedice (icedice) joins
01:46:34icedice quits [Read error: Connection reset by peer]
01:46:58icedice (icedice) joins
01:49:15icedice quits [Read error: Connection reset by peer]
01:49:45icedice (icedice) joins
01:51:50icedice quits [Read error: Connection reset by peer]
01:52:01icedice (icedice) joins
01:54:23icedice quits [Read error: Connection reset by peer]
01:54:28icedice (icedice) joins
01:57:29icedice quits [Read error: Connection reset by peer]
01:57:41icedice (icedice) joins
02:00:14icedice2 joins
02:00:31icedice quits [Read error: Connection reset by peer]
02:02:43icedice2 quits [Read error: Connection reset by peer]
02:02:49icedice (icedice) joins
02:05:28icedice quits [Read error: Connection reset by peer]
02:05:54icedice (icedice) joins
02:09:16icedice2 joins
02:09:20icedice quits [Read error: Connection reset by peer]
02:11:40icedice (icedice) joins
02:14:14icedice2 quits [Ping timeout: 265 seconds]
02:15:10icedice quits [Read error: Connection reset by peer]
02:43:29xkey quits [Quit: xkey]
02:51:16sec^nd quits [Ping timeout: 245 seconds]
03:06:14sec^nd (second) joins
03:20:45nostalgebraist_ quits [Read error: Connection reset by peer]
03:22:27Ryz quits [Quit: The Lounge - https://thelounge.chat]
03:22:40IDK_ joins
03:32:03IDK_ quits [Ping timeout: 265 seconds]
03:34:14IDK_ joins
03:35:02Mateon1 quits [Remote host closed the connection]
03:35:15Mateon1 joins
03:35:57Ryz (Ryz) joins
03:36:10Ryz quits [Excess Flood]
03:36:46Ryz (Ryz) joins
03:40:59hlgs|m joins
03:49:07nicolas17 quits [Client Quit]
04:03:49nostalgebraist joins
04:04:00<hlgs|m>hope this is the right channel for this: i've got some links (and their immediate outlinks) i want to archive to the wayback machine that are only accessible when logged in. any options for that? i have access to the session cookie and have used it before while scraping
04:04:22<hlgs|m>and ideally the backups would not have my username there, lol
04:09:45nostalgebraist quits [Ping timeout: 265 seconds]
04:10:19nostalgebraist joins
04:11:20<pokechu22>Being logged in is a bit of a pain, most tools aren't designed for that (e.g. archivebot can't do it)
04:11:47<pokechu22>Pretty sure anything that involves being logged in would have your username too (or *a* username, at least)
04:13:58<hlgs|m>hmmmm right. i can't make a throwaway account quickly due to a waiting list
04:14:54<hlgs|m>can backups of sites be uploaded directly, or does that create a risk of decreased veracity?
04:23:47Island quits [Read error: Connection reset by peer]
04:44:06<pokechu22>You can upload warcs directly, but they won't generally be displayed on web.archive.org (only available for manual download from archive.org/details/youritemname).
04:45:35<pokechu22>(and if you have it in a format other than WARC, it can't be displayed on web.archive.org at all, but formats other than WARC are generally more user-friendly for other navigation)
04:54:13nostalgebraist quits [Ping timeout: 265 seconds]
04:54:28nostalgebraist joins
04:56:38dumbgoy quits [Ping timeout: 265 seconds]
05:20:52nostalgebraist quits [Client Quit]
05:38:10<hlgs|m>good to know, i'll consider my options. manual download is better than no archive but it might be a lot of work
05:44:14decky_e joins
06:00:54BlueMaxima quits [Read error: Connection reset by peer]
06:31:31nicolas17 joins
06:45:40xkey (xkey) joins
07:44:55Arcorann (Arcorann) joins
07:57:14decky_e quits [Client Quit]
07:59:17umgr036 joins
08:00:07umgr036 quits [Remote host closed the connection]
08:00:20umgr036 joins
08:15:46umgr036 quits [Ping timeout: 265 seconds]
08:39:31<AK>Mineplex is shutting down: https://www.mineplex.com/
08:54:13<tech234a>^ announcement is in their Discord server, seems like an abrupt ending
08:55:35<@arkiver>didn't mineplex have something to do with enjin
08:59:57<tech234a>perhaps it was hosted there at one point?
09:00:58<tech234a>http://webcache.googleusercontent.com/search?q=cache:IqNlYK7sx2IJ:https://www.mineplex.com/threads/is-there-anywhere-the-old-enjin-forums-can-be-accessed.146647/&hl=en&gl=us&strip=1&vwsrc=0
09:01:44<@arkiver>yeah
09:01:51<@arkiver>so their shutdown may be related to enjin shutting down
09:15:03<masterx244|m><tech234a> "^ announcement is in their..." <- and thats why discord sucks for anything above a private friend-circle. info not accessible without wasting a "slot" on your account
09:48:57<lasdkfj|m><hlgs|m> "good to know, i'll consider my..." <- Maybe https://archive.ph/ can do it. I use it sometimes to get around paywalls, it seems, kind people with a subscription save the webpage they see.
10:57:56Megame (Megame) joins
11:20:09icedice (icedice) joins
11:42:21nstrom|m joins
11:52:39sonick quits [Client Quit]
11:57:23CreaZyp154 joins
12:05:51CreaZyp154 quits [Client Quit]
12:15:17<TheTechRobo>I think the wbm supports adding custom cookies in Save Page Now though I'm not sure
12:15:50<TheTechRobo>https://docs.google.com/document/d/1Nsv52MvSjbLb2PCpHlat0gkzw0EvtSgpKHu4mk0MnrA/edit
12:17:47Ruthalas5 quits [Ping timeout: 252 seconds]
12:55:38Unholy23611 (Unholy2361) joins
12:59:02Unholy2361 quits [Ping timeout: 252 seconds]
12:59:02Unholy23611 is now known as Unholy2361
14:14:53Arcorann quits [Ping timeout: 265 seconds]
14:47:11hitgrr8 joins
15:27:10nostalgebraist joins
15:48:45Island joins
15:56:40dumbgoy joins
16:00:49nic0 (nic) joins
16:01:38nic quits [Ping timeout: 252 seconds]
16:01:38nic0 is now known as nic
16:13:44Megame quits [Ping timeout: 252 seconds]
16:24:06x-56k-mo1 (x-56k-modem) joins
16:24:44x-56k-modem quits [Ping timeout: 252 seconds]
16:58:03Hans5958|m quits [Quit: Reconnecting]
16:58:09Hans5958|m joins
16:58:46Hans5958|m is now known as Hans5958
16:58:50Hans5958 quits [Client Quit]
16:58:54Hans5958 (Hans5958) joins
16:59:23Hans5958 quits [Client Quit]
16:59:30Hans5958 (Hans5958) joins
17:01:16<Hans5958><arkiver> "didn't mineplex have something..." <- I don't think it is related... Mineplex already had a dwindling player base so its demise is inevitable
17:01:47<Hans5958>In fact, they already moved from Enjin to their own forums ages ago IIRC
17:02:03<@arkiver>right
17:02:18<@arkiver>they were still mentioned on the enjin list of sites
17:02:28<@arkiver>but yeah i guess it is not really related
17:13:57<hlgs|m>enjin is shutting down? what about curseforge, i remember those being related...? i got an account when some mergers and acquisitions happened that i tried to delete but wasn't allowed to, hopefully it'll be gone if some of this is shutting down...
17:14:21<hlgs|m>(i think it was... minecraft forums -> curseforge -> enjin, maybe?)
17:14:25<hlgs|m>ah, this is probably off-topic
17:14:55<Hans5958>All three were actually unrelated
17:16:00<Hans5958>Enjin was a separate entity of all two, Curse had both Minecraft Forum and CurseForge before it being sold out
17:16:34<@arkiver>hlgs|m: enjin is already gone. we had a project for it
17:17:58<hlgs|m>oh yes, curse was different from curseforge, thank you for the reminder. i'm trying to remember where enjin came in because i recall it being somewhat related to this...
17:18:18<hlgs|m>oh cool, thanks for that project. i'll see if i can figure out if i still have an account or if it's thankfully gone now
17:39:10<hlgs|m>hm. did some backups and ran into some issues
17:40:00birdjj quits [Client Quit]
17:40:09test___ (decky_e) joins
17:40:15birdjj joins
17:40:33<hlgs|m>namely, sometimes links would get a 302 redirect, and they would then lose a specific query that the original link had, which would mean that outlinks would not get captured as the only outlink on the non-query page would be to the page i actually wanted to back up
17:40:49datechnoman7 (datechnoman) joins
17:40:53that_lurker quits [Quit: my throat's getting sore from humming modem tones into my phone]
17:41:15<hlgs|m>any way to... back up the version of an url that gets redirected to with an url query?
17:42:12<hlgs|m>so, it would go...
17:42:12<hlgs|m>example.com/id?query, redirect to example.com/id/1, and that page would sometimes only have one outlink, whereas example.com/id/1?query has the outlinks i wanted
17:42:18datechnoman quits [Ping timeout: 252 seconds]
17:42:18datechnoman7 is now known as datechnoman
17:42:23<hlgs|m>hope i've explained that well
17:43:31that_lurker (that_lurker) joins
17:45:07<hlgs|m>hm, i guess i should clarify, i used the wayback machine spn script, logged in and with its native outlinks option to limit them to just 1. i suppose i could retry the backups limiting outlinks to 2 steps, hmm...
17:50:07<hlgs|m>i wouldn't want to save eveything again really, i'd just want to back up only the links that result in a 302, but with an url query instead
17:50:35<hlgs|m>ah, i do have access to the saved urls so i can find ones that got redirected to, but that is going to be extra manual work, getting to automate it would be nice
18:28:52test___ quits [Ping timeout: 252 seconds]
18:29:14test___ (decky_e) joins
18:29:47test___ quits [Remote host closed the connection]
18:30:11rr quits [Quit: Ping timeout (120 seconds)]
18:30:19test___ (decky_e) joins
18:30:54rr (rr) joins
18:33:18Megame (Megame) joins
18:36:19rr quits [Client Quit]
18:36:39rr (rr) joins
18:48:43rr quits [Client Quit]
18:49:23rr (rr) joins
18:52:34test___ quits [Remote host closed the connection]
19:13:38spirit joins
19:19:06<spirit>hiyo, a friend of mine asked for an art community site to be archived. it is probably low single-digit million URLs but might be quite big (quick estimate by me would be ~600GB). what is the best approach to get this into archive.org? still archivebot even with that size?
19:19:20<spirit>he is supportive and happy to help as much as he can
19:20:26<pokechu22>ArchiveBot can probably do that, though it is on the higher end of what's doable with it
19:22:31<spirit>i am mostly worried about the size if it would end up on a pipeline server that cannot handle it
19:23:47<pokechu22>AB supports --pipeline to pick a pipeline with space; that shouldn't be a problem (especially since jap-addax and jap-puaiohi each have ~700GB)
19:29:56<spirit>excellent! maybe i will add a pipeline myself to make sure it is sized well :)
19:29:57<spirit>thanks!
19:30:05<spirit>i'll be back some soonish day
19:30:13spirit quits [Client Quit]
19:38:36<hlgs|m>with the SPN scripts https://github.com/overcast07/wayback-machine-spn-scripts, is there a quick way to check links that have been archived and get a list of which ones get 302 redirects and to what urls?
19:39:19<hlgs|m>hmm, actually, never mind, i might have a better option
19:40:07ubuntu joins
19:40:19ubuntu leaves
19:45:12rr quits [Client Quit]
19:45:31rr (rr) joins
20:02:24<hlgs|m>https://www.blog.google/technology/safety-security/updating-our-inactive-account-policies/
20:02:24<hlgs|m>oh dammit
20:28:54icedice quits [Client Quit]
20:34:00icedice (icedice) joins
20:46:27<@JAA>spirit, pokechu22: Total size rarely matters; number of URLs and size of the largest files does.
20:47:36<Dango360>"<hlgs|m> with the SPN scripts https://github.com/overcast07/wayback-machine-spn-scripts," are you telling me that i could've just used this instead of creating an entire juypiter notebook based around waybackpy
20:48:26<hlgs|m>possibly XD i only just discovered it last night, was using the batch google sheets thing before then
20:49:12<Dango360>i made it before i knew about the sheets thing; not gonna forget to mention that my method isn't multitasking at all, it's one after the other
20:50:18<@JAA>A long time ago, I wrote a script to take a list of URLs and slowly open SPN for each in tabs. Ahh, memories.
20:51:12<Dango360>it didn't matter how long it took, it was hope at the end of the marathon that guided me through
20:51:29<programmerq>I had a script that would SPN via curl for a list of URLs. it was a few hundred. I didn't know what the ratelimit was, so I implemented retry, and binary search to find the optimal interval. It settled in at 1 every 30 seconds (I think)
20:54:29<hlgs|m>nice
20:54:42<hlgs|m>i used the addon once, opening tabs and having it automatically back up when i did
20:55:06<hlgs|m>worked okay since i had to open them anyway so making a list would have taken the same amount of time
20:56:26BearFortress quits [Ping timeout: 252 seconds]
20:56:30test___ (decky_e) joins
21:07:05<programmerq>yeah, I'll fire up the browser addon if I'm researching stuff.
21:18:07hitgrr8 quits [Client Quit]
21:20:37BearFortress joins
21:33:35test___ quits [Read error: Connection reset by peer]
21:36:41test___ (decky_e) joins
21:38:09woans (WOANS) joins
22:08:39icedice quits [Client Quit]
22:10:48icedice (icedice) joins
22:11:26rr quits [Client Quit]
22:11:46rr (rr) joins
22:19:04nostalgebraist_ joins
22:19:11Megame quits [Ping timeout: 265 seconds]
22:19:40nostalgebraist quits [Ping timeout: 265 seconds]
22:20:33nostalgebraist joins
22:23:54<@JAA>Newgrounds recently (2023-03-22) discontinued the 'General' and 'Politics' forums: https://www.newgrounds.com/bbs/topic/1522847 They're no longer listed on the homepage but remain accessible at https://www.newgrounds.com/bbs/forum/1 and https://www.newgrounds.com/bbs/forum/4 respectively.
22:24:01nostalgebraist_ quits [Ping timeout: 265 seconds]
22:24:54nostalgebraist quits [Client Quit]
22:40:23test___ quits [Ping timeout: 252 seconds]
22:40:43test___ (decky_e) joins
22:54:19<kiska>> Sonatype will retire our public-facing Jira instances on May 25, 2023. This includes https://issues.sonatype.org/browse/NEXUS. The decision is necessitated by a need for increased security as well as to replace a system that is almost at “end of life” with Atlassian.
22:54:19<kiska>RIP issue tracker?
22:59:25<pokechu22>You can export 1000 issues at a time from the search page (but it looks like `KEY > "NEXUS-34000" AND KEY < "NEXUS-35000"` doesn't work here for some reason - maybe dates or something would work if we want to go that path? documentation at https://confluence.atlassian.com/jiracoreserver0904/advanced-searching-1188766820.html)
23:00:50icedice quits [Client Quit]
23:12:21test___ quits [Ping timeout: 265 seconds]
23:12:50test___ (decky_e) joins
23:24:01aGerman2 (aGerman) joins
23:24:03lunik1735 joins
23:24:27AlsoTheTechRobo (TheTechRobo) joins
23:24:37Gereon8 (Gereon) joins
23:24:38Justin[home] joins
23:24:42graham4 joins
23:24:44driib5 (driib) joins
23:24:47birdjj2 joins
23:24:48apache2 quits [Remote host closed the connection]
23:24:48balrog quits [Quit: Bye]
23:24:48driib quits [Quit: Ping timeout (120 seconds)]
23:24:48driib5 is now known as driib
23:24:48Gereon quits [Client Quit]
23:24:48nic quits [Client Quit]
23:24:48graham quits [Quit: Ping timeout (120 seconds)]
23:24:48birdjj quits [Client Quit]
23:24:48apache2 joins
23:24:48@Fusl quits [Excess Flood]
23:24:48systwi_ quits [Quit: Ping timeout (120 seconds)]
23:24:48lunik173 quits [Quit: Ping timeout (120 seconds)]
23:24:48aGerman quits [Quit: Ping timeout (120 seconds)]
23:24:48aGerman2 is now known as aGerman
23:24:48lunik1735 is now known as lunik173
23:24:48IDK_ quits [Client Quit]
23:24:48TheTechRobo quits [Remote host closed the connection]
23:24:48rr quits [Client Quit]
23:24:48eroc1990 quits [Quit: Ping timeout (120 seconds)]
23:24:48DopefishJustin quits [Remote host closed the connection]
23:24:48graham4 is now known as graham
23:24:48Gereon8 is now known as Gereon
23:24:48birdjj2 is now known as birdjj
23:24:52rr9 (rr) joins
23:24:53nic (nic) joins
23:24:54IDK_ joins
23:24:57eroc1990 (eroc1990) joins
23:25:04Fusl (Fusl) joins
23:25:04@ChanServ sets mode: +o Fusl
23:25:06balrog (balrog) joins
23:25:53systwi_ joins
23:31:44jess9123 quits [Ping timeout: 252 seconds]
23:35:43jess9123 joins
23:40:14BlueMaxima joins
23:43:38jess9123 quits [Ping timeout: 252 seconds]
23:49:07jess9123 joins
23:49:59<pokechu22>Is there a way to specify the Accept header with SPN? https://quotes.darkscience.net/quotes/all/ (from https://www.darkscience.net/quotes/) only works with Accept: application/json. I tried doing SPN with screenshot on https://www.darkscience.net/quotes/ but it looks like that used the wrong Accept header :|
23:51:24sonick (sonick) joins
23:54:42needlenose joins
23:55:30woans quits [Client Quit]

HomeSearchPrevious dayNext day