HomeSearchPrevious dayNext day

00:05:18<Ryz>It's a new day for the WBM SPN, and since hitting UTC, the counter has been reset~ <#>;
00:11:37<nicolas17>let's see if parallel uploads help...
00:12:36<nicolas17>hmm not really, I'm doing 7Mbps across 4 uploads
00:16:22Pedrosso joins
00:16:23ScenarioPlanet (ScenarioPlanet) joins
00:18:00TheTechRobo (TheTechRobo) joins
02:13:58qw3rty__ joins
02:14:05katia- (katia) joins
02:14:40driib8 (driib) joins
02:14:41PredatorIWD_ joins
02:14:48s-crypt9 (s-crypt) joins
02:14:52threedeeitguy395 (threedeeitguy) joins
02:15:02sepro3 (sepro) joins
02:31:44Fusl quits [Excess Flood]
02:31:44@arkiver quits [Read error: Connection reset by peer]
02:31:44s-crypt quits [Read error: Connection reset by peer]
02:31:44Larsenv quits [Read error: Connection reset by peer]
02:31:44driib quits [Read error: Connection reset by peer]
02:31:44that_lurker quits [Read error: Connection reset by peer]
02:31:44Fusl (Fusl) joins
02:31:44geezabiscuit quits [Read error: Connection reset by peer]
02:31:44nicolas17 quits [Read error: Connection reset by peer]
02:31:44drin joins
02:31:44Larsenv (Larsenv) joins
02:31:44Doran (Doranwen) joins
02:31:44tbc18872 (tbc1887) joins
02:31:44flashfire42 quits [Read error: Connection reset by peer]
02:31:44nicolas17 joins
02:31:44qw3rty joins
02:31:44atphoenix_ (atphoenix) joins
02:31:44rewby quits [Read error: Connection reset by peer]
02:31:44Lord_Nightmare2 (Lord_Nightmare) joins
02:31:44drin is now known as geezabiscuit
02:31:44Pedrosso5 joins
02:31:44systwi__ (systwi) joins
02:31:44ScenarioPlanet quits [Ping timeout: 262 seconds]
02:31:44Pedrosso quits [Ping timeout: 262 seconds]
02:31:44datechnoman quits [Ping timeout: 262 seconds]
02:31:44Lord_Nightmare quits [Ping timeout: 262 seconds]
02:31:44magmaus3 quits [Ping timeout: 262 seconds]
02:31:44nulldata quits [Ping timeout: 262 seconds]
02:31:44sepro quits [Ping timeout: 262 seconds]
02:31:44angenieux quits [Ping timeout: 262 seconds]
02:31:44katia quits [Ping timeout: 262 seconds]
02:31:44pabs quits [Ping timeout: 262 seconds]
02:31:44PredatorIWD quits [Ping timeout: 262 seconds]
02:31:44SF quits [Ping timeout: 262 seconds]
02:31:44Doranwen quits [Ping timeout: 262 seconds]
02:31:44project10 quits [Ping timeout: 262 seconds]
02:31:44DLoader quits [Ping timeout: 262 seconds]
02:31:44threedeeitguy39 quits [Ping timeout: 262 seconds]
02:31:44qw3rty_ quits [Ping timeout: 262 seconds]
02:31:44balrog quits [Ping timeout: 262 seconds]
02:31:44systwi_ quits [Ping timeout: 262 seconds]
02:31:44tbc1887 quits [Ping timeout: 262 seconds]
02:31:44G4te_Keep3r34924 quits [Ping timeout: 262 seconds]
02:31:44Pedrosso5 is now known as Pedrosso
02:31:44tbc18872 is now known as tbc1887
02:31:44arkiver (arkiver) joins
02:31:44@ChanServ sets mode: +o arkiver
02:31:44DLoader_ (DLoader) joins
02:31:44datechnoman (datechnoman) joins
02:31:44ScenarioPlanet (ScenarioPlanet) joins
02:31:44IDK_ (IDK) joins
02:31:44DLoader_ is now known as DLoader
02:31:44PredatorIWD joins
02:31:44nulldata (nulldata) joins
02:31:44Lord_Nightmare2 is now known as Lord_Nightmare
02:31:44Connection closed.
02:31:58atirclog (atirclog) joins
02:31:58Topic: This channel is 100% not affiliated with archive.org. We will help if we can.
02:31:58Topic set by hook54321 at 2020-10-13 23:40:17Z
02:32:04Current users: atirclog (atirclog), line, pokechu22 (pokechu22), Jake (Jake), eggdrop (eggdrop), angenieux (angenieux), Nemo_bis (Nemo_bis), @AlsoJAA (JAA), AK (AK), katia (katia), OrIdow6 (OrIdow6), ArchivalEfforts, nyany (nyany), tzt (tzt), JTL (JTL), zhongfu (zhongfu), Ryz5 (Ryz), @JAA (JAA), fionera (Fionera), imer (imer), monika (boom), sknebel (sknebel), cm, that_lurker (that_lurker), project10 (project10), TheTechRobo (TheTechRobo), sepro (sepro), kiska (kiska), G4te_Keep3r34924, magmaus3 (magmaus3), dxrt_, Sluggs_, Craigle (Craigle), DigitalDragons (DigitalDragons), balrog (balrog), jodizzle (jodizzle), threedeeitguy39 (threedeeitguy), SF, Justin[home] (DopefishJustin), @kaz (Kaz), pabs (pabs), systwi_, rewby (rewby), flashfire42, s-crypt (s-crypt), driib (driib), nulldata (nulldata), PredatorIWD, IDK (IDK), ScenarioPlanet (ScenarioPlanet), datechnoman (datechnoman), DLoader (DLoader), @arkiver (arkiver), systwi__ (systwi), Pedrosso, Lord_Nightmare (Lord_Nightmare), atphoenix_ (atphoenix), qw3rty, nicolas17, tbc1887 (tbc1887), Doran (Doranwen), Larsenv (Larsenv), geezabiscuit, Fusl (Fusl), SootBector (SootBector), qwertyasdfuiopghjkl (qwertyasdfuiopghjkl), Sanqui (Sanqui), Terbium, Barto (Barto), yano (yano), c3manu (c3manu), igloo22225 (igloo22225), fireonlive (fireonlive), Jonimus, simon816 (simon816), kpcyrd (kpcyrd), murmur, tech234a (tech234a), ats (ats), mattwright324|m, theblazehen|m, thermospheric (Thermospheric), hlgs|m, tomodachi94 (tomodachi94), x9fff00 (x9fff00), qyxojzh|m, britmob|m, schwarzkatz|m, DigitalDragon (DigitalDragon), igneousx (igneousx), Thibaultmol, yzqzss (yzqzss), s-crypt|m|m, Sanqui|m (Sanqui), audrooku|m, Vokun (Vokun), Exorcism (exorcism), @ChanServ, [42] (N4Y), @hook54321 (hook54321), rewby|backup (rewby), justcool393 (justcool393), mgrandi (mgrandi), HCross (HCross), masterX244 (masterX244)
02:32:20f_ (funderscore) joins
02:32:21Matthww joins
02:33:22wessel1512 joins
02:35:40erenrich joins
02:57:04<nicolas17>damn, I tried uploading from my VPS and it's similarly slow
03:08:24nothere joins
03:34:11Ryz5 is now known as Ryz
03:37:22BearFortress joins
03:43:59BearFortress_ joins
03:47:01BearFortress quits [Ping timeout: 255 seconds]
06:09:13systwi__ quits [Ping timeout: 255 seconds]
06:19:07nulldata quits [Ping timeout: 255 seconds]
06:19:11andrew (andrew) joins
06:25:48nulldata (nulldata) joins
06:28:59systwi (systwi) joins
06:38:13<yzqzss>+1
06:55:34<@JAA>It's weird. From the same machine, I have one upload going at 8 MiB/s and the other at 1.7 s/MiB...
07:04:41Arcorann (Arcorann) joins
07:23:31qwertyasdfuiopghjkl quits [Remote host closed the connection]
07:24:50qwertyasdfuiopghjkl (qwertyasdfuiopghjkl) joins
07:24:57<pabs>Ryz: tried with the SPN2 email interface?
07:25:30<Ryz>...Does that require an account on the Internet Archive?
07:25:39<pabs>no. I expect it would allow more URLs than the web
07:26:05<pabs>just mail HTML or plain text to savepagenow@archive.org
07:26:29<pabs>HTML it looks for a hrefs, plain text it has some parsing
07:26:44<Ryz>Just to post a list of links in the body text or a text file as an attachment?
07:26:53<pabs>body
07:27:17<pabs>you'll get multiple mails back with the status of each batch (it splits up into a bunch) with the status of each link (first save etc)
07:27:42pabs wonders how much spam that interface archives :)
07:28:08<Ryz>How many links at a time? Hmm~
07:28:47<pabs>ISTR each batch split is like 30, let me check
07:29:58<pabs>ok, the max per batch is 150.
07:30:00<pabs>so if you send one mail with 300 URLs, you should get 2 mails back
07:30:59<pabs>I did all of https://kdd.cs.ksu.edu/TEUNC/Banazir/Pics/ this way
07:31:24<pabs>it grabs images/etc but I don't think it can do outlinks
07:32:29<@JAA>> FATAL ERROR: server must neither be primary nor secondary as it has no mate!
07:32:33<@JAA><this_is_fine.png>
07:32:40<pabs>Oo
07:33:21<@JAA>I've seen this before, and I'm sure someone (Hank?) will fix it as soon as they're awake. :-)
07:45:45systwi quits [Ping timeout: 272 seconds]
07:51:31systwi (systwi) joins
07:54:24<@arkiver>can confirm those things are seen and fixed ^
08:29:10pabs quits [Ping timeout: 255 seconds]
08:48:04pabs (pabs) joins
09:23:52pabs quits [Remote host closed the connection]
09:25:25pabs (pabs) joins
09:54:19nulldata quits [Ping timeout: 272 seconds]
09:59:46nulldata (nulldata) joins
10:48:29driib quits [Client Quit]
11:11:38qwertyasdfuiopghjkl quits [Remote host closed the connection]
11:19:50qwertyasdfuiopghjkl (qwertyasdfuiopghjkl) joins
11:42:48nulldata quits [Client Quit]
11:46:43nulldata (nulldata) joins
11:46:43nulldata quits [Excess Flood]
11:47:37nulldata (nulldata) joins
11:48:41driib (driib) joins
13:06:51Arcorann quits [Ping timeout: 272 seconds]
13:43:16qwertyasdfuiopghjkl quits [Client Quit]
13:49:30qwertyasdfuiopghjkl (qwertyasdfuiopghjkl) joins
14:27:17systwi quits [Ping timeout: 272 seconds]
14:35:37systwi (systwi) joins
14:42:24DLoader_ (DLoader) joins
14:44:28DLoader quits [Ping timeout: 255 seconds]
14:44:33DLoader_ is now known as DLoader
15:57:13nulldata quits [Ping timeout: 272 seconds]
16:04:30nulldata (nulldata) joins
17:00:27<fireonlive>IA: making servers less lonely
18:17:41qwertyasdfuiopghjkl quits [Client Quit]
18:34:47<nicolas17>632MB in 1:09:39
18:38:23<nicolas17>538MB in 1:02:07
18:41:50<fireonlive>does IA accept HDDs by mail? :P
20:52:32<Ryz>Mmm, I re-encountered the 300 captures per day limit, but I never did that many via https://web.archive.org/save if at all; I'm seriously confused now
20:55:55<@JAA>Frames, images, etc. might count as well.
21:15:20<Ryz>...Does it? Uhhh...
21:16:31<@JAA>They certainly count for concurrent saves, so it wouldn't surprise me if they did there as well.
21:18:14<Ryz>Uhh, arkiver, any input or thoughts on this? This is seriously new territory that I never encountered before
21:19:27<pokechu22>Ryz: I'd recommend just creating an account; it's pretty easy (and it also lets you borrow books from the library which has been really helpful for me). Plus it lets you use outlinks on web.archive.org/save which can be helpful
21:19:54<fireonlive>do note that anything you upload to IA (not via SPN) has your email in it
21:20:07<fireonlive>so choose something you don't mind being public if you intend to upload items
21:20:29<nicolas17>...you have no IA account?
21:20:54<fireonlive>AIUI just using SPN and the other stuff shouldn't expose your email
21:21:12<Ryz>I'm more confused that me just browsing a bunch of archived pages and not explicitly archiving via WBM SPN seems to hit that limit
21:21:48<pokechu22>Might be from advertizements and beacons and other scripts that correspond to URLs that haven't been saved before?
21:21:59<pokechu22>You could look at it with browser devtools open
21:22:27<Ryz>If that counts, whaaa...?
21:23:40<pokechu22>If something is used on the page and hasn't been saved before it'll generally automatically be saved and then the newly-saved version will be used on the page
21:24:41<Ryz>nicolas17, I mentioned before that I'm iffy on making accounts unnecessarily only for them to be unused immediately after; I may have a feeling on creating an Internet Archive account sooner
21:25:20<Ryz>I'm not sure if there are any volunteer opportunities to do with the account too, I know there's the Open Library, but that's a different set of stuff admittedly~
21:32:47<Ryz>Yeah, gonna may have to get an account; just that, reason for checking archived pages individually whether or not it was by me who did the archiving via WBM SPN is that sometimes either ArchiveBot/WBM SPN did the grabbing, but it didn't grab completely, and requires checking the pages themselves for the grabbing to make it happen
21:33:26<nicolas17>"only for them to be unused immediately after" how long have you been involved in archiveteam? :P
21:34:21<Ryz>nicolas17, even I was cagey on joining in Archive Team for some time before I decided to dive in~
21:36:24<Ryz>I've been around...for half a decade now?
21:37:21<Ryz>So WBM SPN, when logged in, does the captures go back to 6 per minute?
21:39:01<pokechu22>I'm pretty sure the limit is much higher, yes. I'm not sure if it resets it if you've already hit the limit while signed out or not
21:43:28<Ryz>Hmm >#<;
21:45:11<Ryz>Besides uploading (which is unlikely), some of the WBM SPN perks, and the Open Library goodies, what else I would have access to? o.o;
21:48:53<pokechu22>Writing reviews on items, I guess?
21:49:34<fireonlive>favouriting items as well, or making lists (unsure if those lists can contain others' items)
21:50:04<fireonlive>yep they can
21:50:39<fireonlive>so like sorted favourites, kinda. https://dl.fireon.live/irc/bef1c4190b704f15/lists.png
21:50:45<fireonlive>can be public/private
21:51:05<fireonlive>can also post on their forums, but.. i don't generally go there
21:51:35<fireonlive>and they don't seem to be emphasized anymore
21:51:57<fireonlive>https://archive.org/iathreads/forums.php
21:52:55<fireonlive>but it's not like your account is going to be a burden or anything
21:56:59<Ryz>Oh yeah, the forums were used to be a more prominent thing; there were a bunch of odd discussions and maybe scams like a decade ago
21:57:12<Ryz>Seems to be de-emphasized, yeah
22:02:20<Ryz>Well I made an account; it'll most likely be used soon, not too strongly certain
22:20:52<Ryz>Yeah, I did a random capture save, after browsing a bunch of archived pages, and checking https://web.archive.org/save/status/user - it ticked from 0 to 1, the 'daily_captures' attribute
22:37:11PredatorIWD quits [Read error: Connection reset by peer]
22:41:23PredatorIWD joins
22:42:04<Ryz>...Well, this is a bit disappointing, I can't do https://web.archive.org/save/https://www.resetera.com/threads/pc-era-and-dev-era-how-do-you-feel-about-keyshops-unofficial-resellers-read-op.816099/ while logged in, it's only when I have to go through through the Save Page Now way of putting in the link in the box and then save it through that :/
22:43:13<Ryz>I do it the other way because it is less friction overall
22:44:08<Ryz>I didn't ask for that at all :c
22:52:55Arcorann (Arcorann) joins
22:53:55<@JAA>Huh, what error does it give?
22:54:36<Ryz>It just only says with the header 'Sorry' followed by body text 'Job failed' when I explicitly tried to archive it via https://web.archive.org/save/https://www.resetera.com/threads/pc-era-and-dev-era-how-do-you-feel-about-keyshops-unofficial-resellers-read-op.816099/ while logged in
22:54:37<@JAA>I thought they were unified under the hood by now, the `GET /save/$url` just not providing feedback other than success or fail.
22:55:10<@JAA>Try with some more URLs (from different hosts etc.). I see that error all the time.
22:55:37<pokechu22>for what it's worth you might still find the form useful since it offers outlinks and such
22:56:05<pokechu22>How does GET /save/$url handle 4XX and 5XX responses? Does it default to failing the job in that case?
22:56:12<@JAA>Correct
22:56:37<@JAA>It's like anonymous access to the form with that checkbox unchecked as far as I'm aware.
22:57:54that_lurker quits [Client Quit]
22:58:57that_lurker (that_lurker) joins
22:59:04that_lurker quits [Client Quit]
23:00:14<Ryz>I'll reserve outlink grabbing in certain cases, but otherwise, I consider this a fair bit of a downgrade on how I do things when using WBM SPN, it means I have to do extra clicks or actions which is additional friction
23:01:46that_lurker (that_lurker) joins
23:02:34<@JAA>Yeah, I also use /save/$url most of the time. I have a keyword so I can enter 'spn $url' into the address bar and it goes there. Not possible to do with the form, annoyingly.
23:03:53<pokechu22>I'd recommend you continue trying the GET version since it does sound like you ran into a random error (that you probably also would have ran into when not signed in)
23:06:36<Ryz>No, that's the odd part, I never ran this type of error before, before the new 300 captures limit, which would've shown that instead~
23:07:30<Ryz>I don't think it's a random error at all
23:08:03<Ryz>Random error as in by chance
23:08:33<@JAA>Never? That seems really weird.
23:11:47<Ryz>Before making an account, I always use https://web.archive.org/save/somerandomURLhere to do my archives since I find it frictionless; ever since the new limit of 300 captures, upon making an account, and doing the same thing, it just gives 'Sorry' followed by 'Job failed', so yeah, this is baffling
23:12:15<Ryz>To note, I reached that limit moments earlier (which I'm still confused about), I wonder if that conflicts with me being logged in
23:13:36<Ryz>Logged out, did the https://web.archive.org/save/https://www.resetera.com/threads/pc-era-and-dev-era-how-do-you-feel-about-keyshops-unofficial-resellers-read-op.816099/ - 'Sorry' followed by 'Job failed'
23:14:01<Ryz>If I do explicitly through the form, it's 'Sorry', followed by 'You cannot make more than (300,) captures per day. Please email us at "info@archive.org" if you would like to discuss this more.'
23:14:55<@JAA>Ah, so that was just the limit, yeah, makes sense.
23:15:26<@JAA>Still confused why you never saw 'Job failed' before. Maybe you just hit the right time windows where SPN is actually working and aren't archiving unstable sites.
23:15:52<Ryz>Yeah, so I'm thinking, for some reason, even while logged in, when doing it my way, it still thinks I hit that limit when logged out
23:16:22<Ryz>Mmm, I think usually if it is a 'Job failed', it would've taken a while until it gave me a message like that
23:16:22<@JAA>On an unrelated note, I was surprised to find the other day that SPN (via /save/$url) grabbed an HTTPS URL with a self-signed cert. I thought it blocked those.
23:16:43<Ryz>Or a different error entirely
23:17:48<Ryz>I never got an instance where it gave me 'Job failed' nearly instantly before now
23:22:30<@JAA>Ah, right. Yeah, instant ones are rare.
23:23:42<Ryz>Well, about less than 40 minutes left until a new day UTC-wise~
23:24:07<Ryz>When it happens, I'll log in and do the usual save thing without the form to see if that makes a difference
23:24:15<Ryz>Additionally to watch how many captures I have left
23:25:00<Ryz>Because I only used 7...and I have a feeling that doing https://web.archive.org/save/somerandomURLhere while not logged in, it counts individual images and scripts as a capture x___X;
23:25:14<Ryz>Which can rack up to 300 captures really quickly
23:50:16<nicolas17>776/921 [52:04<07:24, 3.06s/MiB]
23:50:18<nicolas17>x_x

HomeSearchPrevious dayNext day