HomeSearchPrevious dayNext day

00:12:37<michaelblob>wow been a hot sec since i've been on irc and it looks like h2ibot is alive and well
00:12:59<nicolas17>JAA: I'm now monitoring for changes in opensource.samsung in a pretty efficient way
00:13:15<nicolas17>once a minute I grab 1 random page from that list you AB'd yesterday
00:13:56<h2ibot>*beep boop*
00:14:40<nicolas17>if a file is added or removed from any of the 255 pages in the "mobile" category, it will change the <p class="txt-result con-wrap">Result : 2,533</p on all of the pages
00:15:54<nicolas17>if they delete an old one and add a new one, the result count will be the same, but all the pages in between will have their items shifted, so I should notice changes pretty soon anyway
00:18:13<@JAA>nicolas17: Neat!
00:18:46<nicolas17>if I grab all 600 URLs every few minutes with high concurrency, they'll ban my ass pretty quick, but this is super slow :D
00:58:55Hackerpcs quits [Quit: Hackerpcs]
01:00:53Hackerpcs (Hackerpcs) joins
01:28:13eroc1990 (eroc1990) joins
02:22:38icedice quits [Client Quit]
03:40:25Island quits [Read error: Connection reset by peer]
03:42:23Island joins
03:51:21nic quits [Ping timeout: 240 seconds]
03:55:21nulldata quits [Ping timeout: 240 seconds]
04:03:40nic (nic) joins
04:15:01<pabs>for anyone monitoring the AB websocket to get a *lot* of data, this script is useful for splitting it by domain https://transfer.archivete.am/DjNee/split-archivebot-log-on-domain
04:15:01<eggdrop>inline (for browser viewing): https://transfer.archivete.am/inline/DjNee/split-archivebot-log-on-domain
04:15:11<pabs>(needs gawk)
04:16:21BearFortress quits [Client Quit]
04:18:46<@JAA>Please don't run anything against the WebSocket for the fun of it though; the server is already struggling at times.
04:25:42BearFortress joins
04:26:51nic quits [Ping timeout: 240 seconds]
04:27:36nic (nic) joins
04:34:08ell1 quits [Read error: Connection reset by peer]
04:34:16jacksonchen666 (jacksonchen666) joins
04:38:57jacksonchen666 quits [Ping timeout: 255 seconds]
04:39:20<pabs>its ok to do it for archiving projects though? in my case code and Mailman/2
04:40:04<pabs>also briefly while I am doing a domain
04:40:47<@JAA>Yeah, that's fine, as is fireonlive's monitoring. Just try to do it with a single stream, not multiple.
04:44:06<pabs>currently its separate streams, will have to re-architect stuff to do it in one :(
04:49:51nic quits [Ping timeout: 240 seconds]
04:57:03nic (nic) joins
04:58:48<@JAA>Every additional stream adds load to an already strained server, and it also increases the chance for *all* clients to miss messages because the server drops them when it can't keep up.
04:59:41<michaelblob>is it possible to create a relay for those logs?
05:00:11<@JAA>At the moment, the WS server is 'fine' and only dropping the messages for one overly slow client, it seems.
05:00:28<@JAA>That should be possible in principle, yeah.
05:01:27<michaelblob>could you point me in the direction you would implement a relay? i may do some exploration after my exams
05:04:15<@JAA>Well, you need a WS client to read from the upstream server and a WS server to broadcast each line to its clients. Such a server already exists in the AB repo because that's exactly what happens there (though the data doesn't come from a WS, but it just reads from stdin).
05:04:34<@JAA>It's clearly not the best implementation though given the performance.
05:24:32lennier2_ quits [Read error: Connection reset by peer]
05:24:51lennier2_ joins
06:00:31jacksonchen666 (jacksonchen666) joins
06:03:26Arcorann (Arcorann) joins
06:05:21jacksonchen666 quits [Ping timeout: 255 seconds]
06:16:48<@OrIdow6>What's it bottlenecking on currently?
06:21:05<@JAA>Not sure. I should profile it again when it's slamming into the wall next time.
06:21:58<fireonlive>there was the 'turn down compression' patch
06:27:35<h2ibot>Pokechu22 edited Jira (+12226, /* Status */ WIP): https://wiki.archiveteam.org/?diff=51753&oldid=51750
06:32:17<HP_Archivist>JAA RE: Hasselblad. I grabbed those PDFs into the WBM. Saw only 6 in those list with the //
06:32:57<HP_Archivist>This one was captured wrong or it's just a temp redirect and WBM needs time to update the most recent capture https://cdn.hasselblad.com/manuals//2.0.0/Hasselblad_Brochure_fr_0126.pdf
06:33:40<HP_Archivist>It will redirect to a capture with a url that has only one, e.g. https://cdn.hasselblad.com/manuals/2.0.0/Hasselblad_Brochure_fr_0126.pdf
06:35:00<HP_Archivist>This capture is correct: https://web.archive.org/web/20240213183742/https://cdn.hasselblad.com/manuals//2.0.0/Hasselblad_Brochure_fr_0126.pdf
06:35:07<HP_Archivist>But it didn't capture the pdf.
06:35:50<HP_Archivist>This capture I did earlier tonight has one of 2 '/' for some reason https://web.archive.org/web/20240213183742/https://cdn.hasselblad.com/manuals/2.0.0/Hasselblad_Brochure_fr_0126.pdf
06:36:37jacksonchen666 (jacksonchen666) joins
06:38:05<@JAA>What a mess... :-|
06:38:55jacksonchen666 quits [Remote host closed the connection]
06:40:38<h2ibot>Pokechu22 edited CurseForge (+573, /* API Archival */ update status): https://wiki.archiveteam.org/?diff=51754&oldid=51640
06:41:34<HP_Archivist>JAA: Yeah. Oddly enough, maybe I've never noticed or gave it much thought. Haven't come across to many other examples like that in the wild
06:41:58<HP_Archivist>I mean with seeing // in an URL path
06:45:01<@JAA>It causes problems all the time in AB.
06:46:37<@JAA>Usually though, it's something like an image resizing service where the full URL of an image appears in the path, and wpull butchering .../https://example.org/... into .../http:/example.org/... breaks things.
07:37:37jacksonchen666 (jacksonchen666) joins
07:39:59jacksonchen666 quits [Remote host closed the connection]
07:51:47Island quits [Read error: Connection reset by peer]
07:55:57<h2ibot>Pokechu22 edited Jira (+16528, /* Status */ fill out the rest of the list): https://wiki.archiveteam.org/?diff=51755&oldid=51753
08:37:42eyes joins
08:38:31jacksonchen666 (jacksonchen666) joins
08:38:59jacksonchen666 quits [Remote host closed the connection]
08:54:21DLoader quits [Ping timeout: 272 seconds]
08:57:50beastbg8 quits [Read error: Connection reset by peer]
09:00:57beastbg8 (beastbg8) joins
09:05:45magmaus3 quits [Ping timeout: 272 seconds]
09:28:43<that_lurker>Could someone grab http://www.airovtech.com/
09:28:47<that_lurker>Alot is missing from the IA
09:39:33jacksonchen666 (jacksonchen666) joins
09:40:00JustThatNerdyNerd joins
09:41:03jacksonchen666 quits [Remote host closed the connection]
09:41:27JustThatNerdyNerd quits [Remote host closed the connection]
09:43:46DLoader (DLoader) joins
10:00:01Bleo18260 quits [Client Quit]
10:07:21decky_e quits [Ping timeout: 240 seconds]
10:32:35pixel leaves [Disconnected: Replaced by new connection]
10:32:36pixel (pixel) joins
10:40:28jacksonchen666 (jacksonchen666) joins
10:45:15jacksonchen666 quits [Ping timeout: 255 seconds]
11:14:24jacksonchen666 (jacksonchen666) joins
11:50:33jacksonchen666 quits [Client Quit]
12:24:08@dxrt quits [Quit: ZNC - http://znc.sourceforge.net]
12:24:37xarph quits [Ping timeout: 272 seconds]
12:25:08Ryz quits [Quit: Ping timeout (120 seconds)]
12:25:22xarph joins
12:26:51anarcat quits [Ping timeout: 240 seconds]
12:26:56anarcat (anarcat) joins
12:33:32dxrt joins
12:33:34dxrt quits [Changing host]
12:33:34dxrt (dxrt) joins
12:33:34@ChanServ sets mode: +o dxrt
12:35:46Ryz (Ryz) joins
12:40:15nulldata (nulldata) joins
12:42:21Arcorann quits [Ping timeout: 272 seconds]
12:43:33nicolas17 quits [Remote host closed the connection]
13:45:15SootBect1 quits [Ping timeout: 255 seconds]
13:46:51SootBector (SootBector) joins
13:49:51nic quits [Ping timeout: 240 seconds]
13:50:31nic (nic) joins
13:56:07etnguyen03 (etnguyen03) joins
14:58:54<thuban>that_lurker: looks js-dependent
15:00:22<that_lurker|m>Damn. I did not have time to check the site throughly
15:05:12Mika (Maika) joins
15:14:21nic quits [Ping timeout: 240 seconds]
15:41:21etnguyen03 quits [Ping timeout: 240 seconds]
16:28:22nicolas17 joins
16:34:47pabs quits [Ping timeout: 272 seconds]
16:37:26nicolas17 quits [Client Quit]
16:40:46magmaus3 (magmaus3) joins
16:43:21nicolas17 joins
17:02:53nic (nic) joins
17:06:08qwertyasdfuiopghjkl quits [Remote host closed the connection]
17:08:44etnguyen03 (etnguyen03) joins
17:14:23etnguyen03 quits [Client Quit]
17:18:28etnguyen03 (etnguyen03) joins
17:41:17Mika quits [*.net *.split]
17:41:17Ryz quits [*.net *.split]
17:41:17@dxrt quits [*.net *.split]
17:41:17anarcat quits [*.net *.split]
17:41:17xarph quits [*.net *.split]
17:41:17beastbg8 quits [*.net *.split]
17:41:17lennier2_ quits [*.net *.split]
17:41:18wickedplayer494 quits [*.net *.split]
17:41:18michaelblob quits [*.net *.split]
17:41:18Aoede quits [*.net *.split]
17:41:18midou quits [*.net *.split]
17:41:18vukky quits [*.net *.split]
17:41:18Lord_Nightmare quits [*.net *.split]
17:41:18lukash98 quits [*.net *.split]
17:41:18itachi1706 quits [*.net *.split]
17:41:18JohnnyJ quits [*.net *.split]
17:41:18rbmarliere quits [*.net *.split]
17:41:18balrog quits [*.net *.split]
17:41:18mcint quits [*.net *.split]
17:41:18nepeat quits [*.net *.split]
17:41:18tzt quits [*.net *.split]
17:41:18project10 quits [*.net *.split]
17:41:18systwi_ quits [*.net *.split]
17:41:18skyrocket quits [*.net *.split]
17:41:18HP_Archivist quits [*.net *.split]
17:41:18benjins quits [*.net *.split]
17:41:18imer quits [*.net *.split]
17:41:18klg quits [*.net *.split]
17:41:18bilboed quits [*.net *.split]
17:41:18Carnildo quits [*.net *.split]
17:41:18riku quits [*.net *.split]
17:41:18that_lurker quits [*.net *.split]
17:41:18lea quits [*.net *.split]
17:41:18ThetaDev quits [*.net *.split]
17:41:18@AlsoJAA quits [*.net *.split]
17:41:18threedeeitguy39 quits [*.net *.split]
17:41:18datechnoman quits [*.net *.split]
17:41:18ScenarioPlanet quits [*.net *.split]
17:41:18VerifiedJ quits [*.net *.split]
17:41:18Larsenv quits [*.net *.split]
17:41:18fangfufu quits [*.net *.split]
17:41:18TheTechRobo quits [*.net *.split]
17:41:18Pedrosso quits [*.net *.split]
17:41:18ThreeHM quits [*.net *.split]
17:41:18hackbug quits [*.net *.split]
17:41:18ymgve quits [*.net *.split]
17:41:18tbc1887 quits [*.net *.split]
17:41:18Jens quits [*.net *.split]
17:41:18le0n quits [*.net *.split]
17:41:18emberquill080 quits [*.net *.split]
17:41:18Letur quits [*.net *.split]
17:41:18xkey quits [*.net *.split]
17:41:18h3ndr1k quits [*.net *.split]
17:41:18Gereon quits [*.net *.split]
17:41:18kiska quits [*.net *.split]
17:41:18s-crypt quits [*.net *.split]
17:41:18Craigle quits [*.net *.split]
17:41:18razul quits [*.net *.split]
17:41:18Billy549 quits [*.net *.split]
17:41:18maxfan8 quits [*.net *.split]
17:41:18@rewby quits [*.net *.split]
17:41:19DrasticActions quits [*.net *.split]
17:41:19Matthww quits [*.net *.split]
17:41:19@Sanqui quits [*.net *.split]
17:41:19AK quits [*.net *.split]
17:41:19Doomaholic quits [*.net *.split]
17:41:19simon816 quits [*.net *.split]
17:41:19Kitty quits [*.net *.split]
17:41:19redbees quits [*.net *.split]
17:41:19@arkiver quits [*.net *.split]
17:41:19murb quits [*.net *.split]
17:41:19tertu quits [*.net *.split]
17:41:19murmur quits [*.net *.split]
17:41:19mgrytbak quits [*.net *.split]
17:41:19nickofnicks quits [*.net *.split]
17:41:19Doranwen quits [*.net *.split]
17:41:19treora quits [*.net *.split]
17:41:19koon quits [*.net *.split]
17:41:19thuban quits [*.net *.split]
17:41:19fredgido quits [*.net *.split]
17:41:19fluke quits [*.net *.split]
17:41:19fireonlive quits [*.net *.split]
17:41:19cm quits [*.net *.split]
17:41:19TastyWiener954 quits [*.net *.split]
17:41:19raxxy-137409 quits [*.net *.split]
17:41:19CraftByte quits [*.net *.split]
17:41:19pie_ quits [*.net *.split]
17:41:19flashfire42 quits [*.net *.split]
17:41:19HugsNotDrugs` quits [*.net *.split]
17:41:19angenieux quits [*.net *.split]
17:41:19Ryz2 quits [*.net *.split]
17:41:19useretail_ quits [*.net *.split]
17:41:19aismallard quits [*.net *.split]
17:41:19monika quits [*.net *.split]
17:41:19lumidify quits [*.net *.split]
17:41:19kallsyms quits [*.net *.split]
17:41:19kiskaLogBot quits [*.net *.split]
17:41:19kpcyrd quits [*.net *.split]
17:41:19cronfox quits [*.net *.split]
17:41:19G4te_Keep3r34924 quits [*.net *.split]
17:41:19lindowsME quits [*.net *.split]
17:41:19mikael quits [*.net *.split]
17:41:19FalconK_ quits [*.net *.split]
17:41:19mattx433 quits [*.net *.split]
17:41:19@Fusl quits [*.net *.split]
17:41:19colona quits [*.net *.split]
17:41:19mrfooooo quits [*.net *.split]
17:41:19Suika quits [*.net *.split]
17:41:19Jonimus quits [*.net *.split]
17:41:19kdy quits [*.net *.split]
17:41:19@chfoo quits [*.net *.split]
17:41:19apache2 quits [*.net *.split]
17:41:20apache2 joins
17:41:20raxxy-137409 joins
17:41:20treora joins
17:41:21ThetaDev joins
17:41:21murmur joins
17:41:21DrasticActions joins
17:41:24fangfufu joins
17:41:25AlsoJAA (JAA) joins
17:41:25@ChanServ sets mode: +o AlsoJAA
17:41:25koon joins
17:41:26mgrytbak joins
17:41:26Suika joins
17:41:26redbees joins
17:41:26mikael joins
17:41:26cm joins
17:41:26mattx433 (mattx433) joins
17:41:27Gereon (Gereon) joins
17:41:27Ryz2 (Ryz) joins
17:41:28Pedrosso joins
17:41:28midou joins
17:41:28Larsenv (Larsenv) joins
17:41:29kiskaLogBot joins
17:41:29aismallard joins
17:41:29s-crypt (s-crypt) joins
17:41:29Letur joins
17:41:30bilboed joins
17:41:30kiska (kiska) joins
17:41:30CraftByte (DragonSec|CraftByte) joins
17:41:30that_lurker (that_lurker) joins
17:41:31Carnildo joins
17:41:31lea (lea_) joins
17:41:31murb (murb) joins
17:41:31colona (colona) joins
17:41:32G4te_Keep3r34924 joins
17:41:32Jens (JensRex) joins
17:41:32h3ndr1k (h3ndr1k) joins
17:41:32kpcyrd (kpcyrd) joins
17:41:32VerifiedJ (VerifiedJ) joins
17:41:32Kitty (Kitty) joins
17:41:33ScenarioPlanet (ScenarioPlanet) joins
17:41:33xkey (xkey) joins
17:41:33lindowsME joins
17:41:33xarph joins
17:41:33HugsNotDrugs joins
17:41:33kallsyms joins
17:41:33fluke joins
17:41:33Mika (Maika) joins
17:41:33Doomaholic (Doomaholic) joins
17:41:34fredgido joins
17:41:34thuban (thuban) joins
17:41:34FalconK_ (FalconK) joins
17:41:34Aoede (Aoede) joins
17:41:34rewby (rewby) joins
17:41:34mrfooooo joins
17:41:35@ChanServ sets mode: +o rewby
17:41:35vukky (vukky) joins
17:41:35razul joins
17:41:35Matthww joins
17:41:35Fusl (Fusl) joins
17:41:35JohnnyJ joins
17:41:35wickedplayer494 joins
17:41:35klg (klg) joins
17:41:36lumidify (lumidify) joins
17:41:36Billy549 (Billy549) joins
17:41:36dxrt joins
17:41:36@ChanServ sets mode: +o Fusl
17:41:36simon816 (simon816) joins
17:41:37lukash98 joins
17:41:37hackbug (hackbug) joins
17:41:37chfoo (chfoo) joins
17:41:37@ChanServ sets mode: +o chfoo
17:41:37TastyWiener954 (TastyWiener95) joins
17:41:37Lord_Nightmare (Lord_Nightmare) joins
17:41:37kdy (kdy) joins
17:41:37emberquill080 (emberquill) joins
17:41:38mcint joins
17:41:38le0n (le0n) joins
17:41:38dxrt quits [Changing host]
17:41:38dxrt (dxrt) joins
17:41:38@ChanServ sets mode: +o dxrt
17:41:38riku joins
17:41:38useretail_ joins
17:41:38ymgve joins
17:41:39benjins joins
17:41:40Jonimus joins
17:41:40tbc1887 (tbc1887) joins
17:41:40rbmarliere (rbmarliere) joins
17:41:40nickofnicks (nickofnicks) joins
17:41:41maxfan8 (maxfan8) joins
17:41:41ThreeHM (ThreeHeadedMonkey) joins
17:41:41tzt (tzt) joins
17:41:41datechnoman (datechnoman) joins
17:41:41michaelblob (michaelblob) joins
17:41:42flashfire42 joins
17:41:42Sanqui joins
17:41:42imer (imer) joins
17:41:42project10 (project10) joins
17:41:42systwi_ joins
17:41:44HP_Archivist (HP_Archivist) joins
17:41:44Sanqui quits [Changing host]
17:41:44Sanqui (Sanqui) joins
17:41:44@ChanServ sets mode: +o Sanqui
17:41:44beastbg8 (beastbg8) joins
17:41:44Doranwen (Doranwen) joins
17:41:46angenieux (angenieux) joins
17:41:47Craigle (Craigle) joins
17:41:47monika (boom) joins
17:41:48itachi1706 (itachi1706) joins
17:41:51TheTechRobo (TheTechRobo) joins
17:41:56AK (AK) joins
17:42:01threedeeitguy39 (threedeeitguy) joins
17:42:03Ryz (Ryz) joins
17:42:05fireonlive (fireonlive) joins
17:42:10balrog (balrog) joins
17:42:11lennier2_ joins
17:42:37IDK (IDK) joins
17:42:40nepeat (nepeat) joins
17:42:44qwertyasdfuiopghjkl (qwertyasdfuiopghjkl) joins
17:43:24cronfox (Cronfox) joins
17:43:33skyrocket joins
17:43:54nic1 (nic) joins
17:44:13tertu (tertu) joins
17:45:05nic quits [Ping timeout: 272 seconds]
17:45:05nic1 is now known as nic
17:46:22anarcat (anarcat) joins
17:54:02pie_ joins
17:57:19eyes quits [Client Quit]
18:24:05<h2ibot>John123521 edited Talk:List of lost Twitter accounts (+220, /* @theliamnissan */ new section): https://wiki.archiveteam.org/?diff=51756&oldid=49391
18:38:11<fireonlive>we should probably murder that page
18:40:33<fireonlive>the scope and amount of work for that stillborn listicle is boundless and efforts are probably better spent elsewhere/there’s probably a better source
18:43:57Mika quits [Client Quit]
19:04:26Island joins
19:07:13Island quits [Read error: Connection reset by peer]
19:10:51Island joins
19:16:11<DigitalDragons>there are a lot of "list of X" pages on the wiki that are probably better served in other ways
19:42:58andrew quits [Quit: Ping timeout (120 seconds)]
19:43:24andrew (andrew) joins
20:20:09IDK quits [Client Quit]
20:24:41etnguyen03 quits [Ping timeout: 272 seconds]
20:41:35<fireonlive>indeed
20:41:43<fireonlive>like list of events on the internet
20:42:10<fireonlive>we do have a full WBM capture of the wiki as of ... sept 2023? maybe
20:42:34<fireonlive>wbm-browsable that is, capped by archivebot
20:59:31nulldata quits [Ping timeout: 272 seconds]
21:02:53<fireonlive>am i misremembering or did the warc size for archivebot change from 50GB to 5GB?
21:03:22<fireonlive>it's different by 10x so it sounds like a 'fireonlive remembered a conversion error long ago'
21:07:13nulldata (nulldata) joins
21:20:34<@JAA>It did not.
21:20:56<@JAA>Originally, it was 10 GiB WARCs, but only for a few months in 2014.
21:21:39<@JAA>The size is configurable via an env variable, but it hasn't been used in at least 6-7 years.
21:37:58BlueMaxima joins
21:43:36<fireonlive>ah! thanks :)
22:00:34lennier2_ quits [Client Quit]
22:00:48ThetaDev quits [Client Quit]
22:01:01ThetaDev joins
22:05:34^ quits [Remote host closed the connection]
22:05:53lennier1 (lennier1) joins
22:10:47qwertyasdfuiopghjkl quits [Client Quit]
22:14:20^ (^) joins
22:19:55icedice (icedice) joins
22:22:15Lambro_D joins
22:23:43toss (toss) joins
22:26:02<michaelblob>looking to use tmpfs in my containers, what's the storage path for inside the container?
22:28:22<Barto> /grab/data ?
22:33:51<fireonlive>looks like it :)
22:40:33toss quits [Client Quit]
22:43:35<michaelblob>thanks!
22:47:32<fireonlive>=]
23:24:28qwertyasdfuiopghjkl (qwertyasdfuiopghjkl) joins
23:47:31Island quits [Read error: Connection reset by peer]
23:47:54Island joins
23:51:18etnguyen03 (etnguyen03) joins
23:56:39qwertyasdfuiopghjkl quits [Remote host closed the connection]
23:56:49Earendil7 (Earendil7) joins

HomeSearchPrevious dayNext day