HomeSearchPrevious dayNext day

00:02:35genofire quits [Ping timeout: 276 seconds]
00:02:42Jonboy345 joins
00:03:53pcr quits [Ping timeout: 276 seconds]
00:07:26Jonboy345 quits [Read error: Connection reset by peer]
00:08:33genofire joins
00:09:54Jonboy345 joins
00:09:57Jonboy345 quits [Remote host closed the connection]
00:23:19Arcorann_ joins
00:59:03Iki joins
01:00:43dm4v quits [Read error: Connection reset by peer]
01:02:45dm4v joins
01:02:47dm4v quits [Changing host]
01:02:47dm4v (dm4v) joins
01:09:48pcr joins
01:13:01onetruth quits [Read error: Connection reset by peer]
01:30:48driib798943 (driib) joins
01:34:11driib79894 quits [Ping timeout: 258 seconds]
01:34:11driib798943 is now known as driib79894
02:02:33dm4v_ joins
02:03:25dm4v quits [Ping timeout: 244 seconds]
02:03:25dm4v_ is now known as dm4v
02:03:26dm4v quits [Changing host]
02:03:26dm4v (dm4v) joins
02:56:17qwertyasdfuiopghjkl quits [Client Quit]
03:59:15Arcorann_ quits [Ping timeout: 265 seconds]
04:09:50sonick quits [Client Quit]
04:22:51<@JAA>https://gitorious.org/ is back online.
04:25:47astrid (astrid) joins
04:30:02<astrid>lol when someone posted gitorious dot org in here like fifteen clients fetched the page from the server in the same minute
04:30:14<astrid>(idk if it was here exactly but who else)
04:30:40<@JAA>Yeah, it was here, and it's the people who use The Lounge and have automatic fetching enabled. Causes trouble in #archivebot all the time with big !ao < URLs.
04:31:08<@JAA>Or well, it caused trouble in the past and is now prevented.
04:31:47<@JAA>Also, hi :-)
04:31:57<astrid>hi
04:32:17<astrid>so yeah, uh, its back on line, should be a bit less fiddly than before
04:32:41<astrid>used to be some special snowflake bullshit, now it's just Another Ceph Volume
04:33:03<@JAA>Lovely, thanks a lot!
04:33:23<astrid>bundle per repo is the long term plan, just ... that's a fuckload of bundling
04:34:07<astrid>bundle per repo, ia item per username (so each item would have somewhere between one repo, and a few gigabytes of bundles in it)
04:34:18<@JAA>Yeah, and it has to be done right to avoid missing certain edge cases of branches.
04:34:25<@JAA>I.e. --all
04:34:38<astrid>if you want i can give you a ssh login to the server so you can start experimenting
04:34:43<astrid>the thing is
04:34:55<astrid>the volume is extremely hardlink-dense
04:35:13<astrid>because when you forked a repo it would do on the serverside "git clone" but not copy the data, because no need
04:35:31<@JAA>Yeah, makes sense.
04:35:33<astrid>so heavily-forked repos have a shitload of hardlinks
04:36:05<astrid>have also been considering tarfile-per-username, which would not need weird fuckery and would preserve the hardlinks
04:36:36<astrid>because e.g., if 'astrid' forked 'jaa/repo.git' the newly-forked repo would be called 'jaa/astrid-repo.git'
04:36:36<@JAA>How would that work when the forks are under different usernames?
04:36:42<@JAA>Oh
04:36:43<@JAA>lol
04:36:45<astrid>it's Weird
04:37:52<astrid>also um
04:37:54<@JAA>Any idea what the size would roughly be without the hardlink dedupe?
04:38:12<astrid>gitorious stored everything in random hex named directories
04:38:34HP_Archivist quits [Ping timeout: 258 seconds]
04:39:24<astrid>before handover they tried to rename everything to usable names following the scheme i metioned earlier
04:39:33<astrid>but ran into some issues
04:39:35<astrid>so there's a mix
04:39:44<@JAA>What a surprise.
04:40:05<astrid>anyway i made a directory on a different filesystem with symlinks named like what they were hoping to accomplish
04:41:35<astrid>e.g.:
04:41:37<astrid>$ ls -ld z3:z3.git geeqie:g2p-geeqie.git
04:41:39<astrid>lrwxrwxrwx 1 root root 49 Jun 30 2015 geeqie:g2p-geeqie.git -> /mnt/gitorious/repositories/geeqie/g2p-geeqie.git
04:41:41<astrid>lrwxrwxrwx 1 root root 74 Jun 30 2015 z3:z3.git -> /mnt/gitorious/repositories/b37/715/d301bef079361b5dfa4d90392c50465bd1.git
04:42:41qw3rty_ joins
04:43:00<astrid>and then there's a apache rule to transform "/z3/z3" to whatever makes cgit look inside of "/srv/gitorious/repositories/z3:z3.git/"
04:43:28<astrid>just a bunch of dumb unix shit
04:43:28<@JAA>So regarding bundling it all up, I don't have any intentions of tackling this in the near term, but maybe sometime early next year would be nice. I want to get that general project up and running anyway. Still needs a fair bit of work though, and I keep distracting myself with stuff like collecting billions of YouTube video IDs...
04:43:37<astrid>sure lmk whenever
04:43:44<astrid>shit isn't going anywhere
04:43:48<@JAA>:-)
04:45:54<astrid>i have zero, zero idea how big this would be after expansion
04:46:14qw3rty__ quits [Ping timeout: 258 seconds]
04:46:41<astrid>i did a randomly chosen trial on geeqie/* earlier today, which was 75 megs on disk and became about 180 megs after duplication, the pile of bundles was about 120 megs
04:48:46<astrid>the whole filesystem is 4.4T
04:49:14<astrid>i've been estimating (pessimistically) a 4x expansion
04:51:49<@JAA>Alright, not too horrible, but quite a bit, yeah.
04:53:01<@JAA>Simple per-repo bundles are probably not the best option though. My thing will handle deduping at the commit level, but that obviously means that things are spread across multiple bundles.
04:53:45<astrid>fwiw it is on owned hardware in a datacenter and bandwidth charges are not an issue
04:54:19<@JAA>That sounds good. Maybe we can just slowly run it through that then once it exists.
04:58:05Arcorann_ joins
04:58:43<astrid>ha shit, it's a whole 3ms from www.archive.org
04:59:58<@JAA>Nice
05:05:28sonick (sonick) joins
05:39:37wyatt8740 quits [Remote host closed the connection]
05:51:38driib79894 quits [Client Quit]
05:53:11wyatt8740 joins
06:02:01wyatt8740 quits [Ping timeout: 265 seconds]
06:04:53wyatt8740 joins
06:21:03Matthww quits [Client Quit]
06:21:19Matthww joins
06:30:37Megame (Megame) joins
06:30:59Connection closed.
06:31:12atirclog (atirclog) joins
06:31:12Topic: Lengthy ArchiveTeam-related discussions, questions here | Offtopic: #archiveteam-ot | https://twitter.com/textfiles/status/1069715869994020867
06:31:12Topic set by JAA at 2021-11-02 03:43:57Z
06:31:13colona (colona) joins
06:31:17Current users: colona (colona), atirclog (atirclog), yawkat (yawkat), Lord_Nightmare (Lord_Nightmare), s-crypt (s-crypt), jtagcat (jtagcat), @EggplantN (EggplantN), summerisle (summerisle), Suika_, fionera_ (Fionera), Megame (Megame), Matthww, wyatt8740, sonick (sonick), Arcorann_, qw3rty_, astrid (astrid), dm4v (dm4v), pcr, Iki, genofire, TastyWiener95 (TastyWiener95), thetechrobo_, tzt (tzt), lunik1, Hackerpcs (Hackerpcs), Daloader_ (Daloader_), klg (klg), appledash (appledash), monoxane4 (monoxane), vukky (Vukky), lennier1 (lennier1), Ruthalas (Ruthalas), HackMii (hacktheplanet), ragu (ragu), Zerote_, shoghicp (shoghicp), xkey (eyo), FileplanetHunter, flashfire42 (flashfire42), igloo22225 (igloo22225), kiska (kiska), h2ibot (h2ibot), @Fusl_ (Fusl), tech234a (tech234a), Aoede (Aoede), nothere, luckcolors (luckcolors), treora, psy (psy), lukash7, MrRadar (MrRadar), sknebel (sknebel), dominikh, @hook54321 (hook54321), Ctrl-S, betamax (betamax), avoozl1, masterX244 (masterX244), billy549 (Billy549), Mayk78, Ajay, russss (russss), Larsenv (Larsenv), Eighty (Eighty), wickedplayer494 (wickedplayer494), meowe (meowe), revi (revi), leo60228 (leo60228), FalconK (FalconK), @arkiver (arkiver), Soulflare, endrift|ZNC, balrog (balrog), Ryz2 (Ryz), mgrytbak, celestial_, Nay (JeDa), jspiros_ (jspiros), datechnoman (datechnoman), mrfooooo, Dj-Wawa (Dj-Wawa), notbasetwo, ]42[ (N4Y), nyany (nyany), coderobe (coderobe), pie_, @JAA (JAA), Hecz (Hecz), Zeklyn, @chfoo (chfoo), girst (girst), Kenshin, Sanqui (Sanqui), atphoenix (atphoenix), Dallas (Dallas), yano (yano), monika (boom), voltagex_, Minkafighter, Mateon1, godane (godane), Myself (myself), britmob25636477, nepeat (nepeat), Stiletto, wizards, Nulo, Gereon62 (Gereon), wolfin (wolfin), G4te_Keep3r, ThreeHM (ThreeHeadedMonkey), sec^nd (second), pabs (pabs), mikael, IDK (IDK), Sluggs, @Fusl (Fusl), katocala (katocala), xit78, knecht420 (knecht420), AK (AK), Craigle (Craigle), themadpro (themadpro), @jrwr (jrwr), aarchi (aarchi), devsnek (devsnek), thuban, TappyToes, swebb, Gaelan (Gaelan), aismallard, Dragnog, thelounge31, asie, @rewby (rewby), nimaje, mgrandi (mgrandi), ats (ats), JensRex (JensRex), jonty (jonty), murmur, ave (ave), jodizzle (jodizzle), @Kaz (Kaz), phuzion (phuzion), nico_32 (nico), IDK_, HugsNotDrugs, systwi (systwi), holbrooke, @OrIdow6 (OrIdow6), sembiance (sembiance), maxfan8 (maxfan8), apache2_, kpcyrd (kpcyrd), HotSwap, cpina_, Frogging101, starship_8601 (starship_8601), @HCross (HCross), ghuntley, mutantmnky (mutantmonkey), aleph, Doranwen (Doranwen), Jonimus, justcool393 (justcool393), NotEggplant, thermospheric (Thermospheric), Atom--, duce1337 (duce1337), mjh (mjh), pekster (pekster), SketchTh1Cow, hexa- (hexa-), Barto (Barto), BPCZ (BPCZ), h3ndr1k (h3ndr1k), @dxrt (dxrt), robbi5 (robbi5), Ryz (Ryz), kiskaLogBot, ArchivalEfforts, Pingerfowder (Pingerfowder), Deewiant (Deewiant), djsrv (djsrv), @ChanServ, simon816 (simon816), Muad-Dib, Jon (jon), pmlo3, bleb, @AlsoJAA (JAA), kallsyms
06:31:17adamus1red_ (adamus1red) joins
06:31:17@OrIdow6 quits [Write error: Broken pipe]
06:31:17sembiance quits [Write error: Broken pipe]
06:31:17maxfan8 quits [Write error: Broken pipe]
06:31:17kpcyrd quits [Write error: Broken pipe]
06:31:17HotSwap quits [Write error: Broken pipe]
06:31:17Frogging101 quits [Write error: Broken pipe]
06:31:17@HCross quits [Write error: Broken pipe]
06:31:17Sluggs quits [Write error: Broken pipe]
06:31:17Dallas quits [Write error: Broken pipe]
06:31:17Sanqui quits [Write error: Broken pipe]
06:31:17Kenshin quits [Write error: Broken pipe]
06:31:17Hecz quits [Write error: Broken pipe]
06:31:17mrfooooo quits [Write error: Broken pipe]
06:31:17balrog quits [Write error: Broken pipe]
06:31:17revi quits [Write error: Broken pipe]
06:31:17Ctrl-S quits [Write error: Broken pipe]
06:31:17TastyWiener95 quits [Write error: Broken pipe]
06:31:17Arcorann_ quits [Write error: Broken pipe]
06:31:17psy quits [Write error: Broken pipe]
06:31:17Ctrl-S joins
06:31:17apache2_ quits [Write error: Broken pipe]
06:31:17cpina_ quits [Write error: Broken pipe]
06:31:17Nulo quits [Write error: Broken pipe]
06:31:17starship_8601 quits [Write error: Broken pipe]
06:31:17endrift|ZNC quits [Write error: Broken pipe]
06:31:17apache2 joins
06:31:19cpina joins
06:31:19DopefishJustin joins
06:31:22VerifiedJ (VerifiedJ) joins
06:31:23Kenshin joins
06:31:23revi (revi) joins
06:31:25mrfooooo joins
06:31:25Viniter6927 (Viniter) joins
06:31:26fuzzy8021 (fuzzy8021) joins
06:31:27starship_8601 (starship_8601) joins
06:31:27kpcyrd (kpcyrd) joins
06:31:28Sanqui joins
06:31:29gazorpazorp (gazorpazorp) joins
06:31:31Frogging101 joins
06:31:33JTL (JTL) joins
06:31:35Arcorann_ joins
06:31:35HotSwap joins
06:31:36Jake (Jake) joins
06:31:36psy (psy) joins
06:31:37OrIdow6 (OrIdow6) joins
06:31:37@ChanServ sets mode: +o OrIdow6
06:31:39maxfan8 (maxfan8) joins
06:31:39JSharp (JSharp) joins
06:31:41TastyWiener95 (TastyWiener95) joins
06:31:44adamus1red_ is now known as adamus1red
06:31:45lun4 (lun4) joins
06:31:56Hecz joins
06:31:57Hecz quits [Changing host]
06:31:57Hecz (Hecz) joins
06:32:14Dallas (Dallas) joins
06:32:15endrift joins
06:32:15Nulo joins
06:32:27HCross (HCross) joins
06:32:27@ChanServ sets mode: +o HCross
06:32:30balrog (balrog) joins
06:32:31benjins joins
06:33:23Sluggs joins
06:44:23thermospheric quits [Changing host]
06:44:23thermospheric (Thermospheric) joins
07:06:09Mateon1 quits [Ping timeout: 258 seconds]
07:41:19tzt quits [Ping timeout: 244 seconds]
08:00:16knecht420 quits [Quit: The Lounge - https://thelounge.chat]
08:00:41knecht420 (knecht420) joins
09:55:35Megame quits [Client Quit]
09:55:35ragu quits [Read error: Connection reset by peer]
10:23:16Mateon1 joins
10:30:05Mateon1 quits [Ping timeout: 258 seconds]
10:57:43Mateon1 joins
11:04:46Mateon1 quits [Read error: Connection reset by peer]
11:05:27Mateon1 joins
11:08:35ragu joins
11:51:04Mateon2 joins
11:51:52Mateon1 quits [Read error: Connection reset by peer]
11:51:53Mateon2 is now known as Mateon1
12:00:03xit78 quits [Quit: The Lounge - https://thelounge.chat]
12:03:32xit78 joins
12:14:20Mateon1 quits [Read error: Connection reset by peer]
12:20:10Mateon1 joins
12:55:53TastyWiener95 quits [Client Quit]
13:49:53vukky quits [Ping timeout: 265 seconds]
13:52:55LeGoupil joins
14:06:39Matthww quits [Client Quit]
14:07:46Arcorann_ quits [Ping timeout: 265 seconds]
14:31:49vukky (Vukky) joins
14:34:50lunik1 quits [Client Quit]
14:47:58lunik1 joins
15:06:04Matthww joins
15:37:49HP_Archivist (HP_Archivist) joins
15:39:48mutantmnky quits [Remote host closed the connection]
15:40:06mutantmnky (mutantmonkey) joins
15:51:48qwertyasdfuiopghjkl joins
16:56:25spirit joins
16:59:52HackMii quits [Remote host closed the connection]
17:00:31HackMii (hacktheplanet) joins
17:19:52PyryV joins
17:20:43<PyryV>I'm sorry, i don't know if this is the right place but what is the image address for youtube-dislike? I want to run a docker image.
17:25:23<@rewby>PyryV: The image you're looking for is atdr.meo.ws/archiveteam/youtube-dislikes-grab:latest and it's better to ask in #down-the-tube for questions on that project.
17:33:39tzt (tzt) joins
17:51:01HackMii quits [Remote host closed the connection]
17:51:30<@OrIdow6>Bob Dole died https://www.politico.com/news/2021/12/05/bob-dole-republican-presidential-nominee-advance-obit-033611
17:52:36HackMii (hacktheplanet) joins
17:53:59HP_Archivist quits [Ping timeout: 258 seconds]
18:07:40HP_Archivist (HP_Archivist) joins
18:11:08Megame (Megame) joins
18:35:56HackMii quits [Remote host closed the connection]
18:36:17HackMii (hacktheplanet) joins
18:38:43HackMii quits [Remote host closed the connection]
18:39:05HackMii (hacktheplanet) joins
18:49:25onetruth joins
19:02:21sec^nd quits [Remote host closed the connection]
19:03:00sec^nd (second) joins
19:03:19PyryV quits [Ping timeout: 244 seconds]
19:03:45mutantmnky quits [Ping timeout: 258 seconds]
19:10:20HP_Archivist quits [Ping timeout: 265 seconds]
19:11:04mutantmnky (mutantmonkey) joins
19:17:33HackMii quits [Ping timeout: 258 seconds]
19:18:01thetechrobo_ quits [Client Quit]
19:18:55HackMii (hacktheplanet) joins
19:25:56TheTechRobo joins
19:40:57Megame quits [Client Quit]
20:39:25qwertyasdfuiopghjkl quits [Ping timeout: 244 seconds]
20:40:26qwertyasdfuiopghjkl joins
20:56:04spirit quits [Client Quit]
21:13:45vukky quits [Client Quit]
21:53:58qwertyasdfuiopghjkl53 joins
21:54:16LeGoupil quits [Client Quit]
21:55:53qwertyasdfuiopghjkl quits [Ping timeout: 244 seconds]
21:56:01qwertyasdfuiopghjkl53 is now known as qwertyasdfuiopghjkl
22:19:31thelounge31 quits [Quit: Ping timeout (120 seconds)]
22:19:58thelounge31 joins
22:23:40BlueMaxima joins
22:24:03thelounge31 quits [Client Quit]
22:29:13HackMii quits [Ping timeout: 258 seconds]
22:30:21HackMii (hacktheplanet) joins
22:33:43Arcorann_ joins
22:36:16thelounge31 joins
22:42:35thelounge31 quits [Client Quit]
22:45:22thelounge31 joins
22:48:27thelounge31 quits [Client Quit]
23:25:15thelounge31 joins
23:46:57Hackerpcs quits [Client Quit]
23:51:22Hackerpcs (Hackerpcs) joins

HomeSearchPrevious dayNext day