00:10:53<@JAA>RAID failure on the secondary and some unspecified filesystem issue on the primary, apparently. Recovery is at least 14 hours away per their numbers posted at 21:39.
00:11:05<@JAA>https://mastodon.matrix.org/@matrix/115136245785561439
00:13:05<nukke>Weirdly enough... https://github.com/openzfs/zfs/issues/17635
00:13:17<nukke>>I've experienced permanent errors/corruption in a 3-disk raidz-1 and I do not believe the corruption is caused by hardware.
00:14:20<nstrom>yeah I just get error "M_UNKNOWN was returned while trying to access the room or space." from any of the hackint rooms on matrix.org
00:14:34<nukke>>So we're having to do a full 55TB DB snapshot restore from last night, which will take >10h to recover the data, and then >4h to actually restore, and then >3h to catch up on missing traffic.
00:14:55<nukke>55TB sounds both smaller and bigger than I expected
00:15:09<nicolas17>I guess rooms that only exist on matrix.org will have data loss?
00:15:52<@JAA>> We believe the incremental DB traffic since last night is intact however.
00:16:20<nukke>Is it buffered? How does that work?
00:16:41<nukke>Or some weird multinode setup with warm and hot nodes?
00:16:46<@JAA>¯\_(ツ)_/¯
00:17:45<nicolas17>ah
00:17:53<nicolas17>yeah if they backup transaction logs
00:18:30<nicolas17>postgresql replication can be done by sending transaction logs from primary to secondary and applying them there
00:19:19<nicolas17>if you also *save* those logs long term, you can apply them on top of your last full backup
00:32:52etnguyen03 quits [Client Quit]
01:12:31etnguyen03 (etnguyen03) joins
01:13:53<hexa->mdadm raid10, lvm, ext4
01:14:18lunik1 quits [Quit: :x]
01:14:19<hexa->so no checksumming?
01:14:51lunik1 joins
01:15:09<hexa->possibly running with --data-checksums, but I doubt it
01:15:52<hexa->setup via https://www.postgresql.org/message-id/flat/CAPo1J60Vcu%2B5G0EvvAZtYgTn6U6ADij3aVJ8WFVz77jP%2BBd_Tw%40mail.gmail.com
01:32:35<nukke>dear lord pls no
01:33:11<nukke>also *two* physical servers with a primary and secodnary? why not patroni or percona for *true* replication?
01:33:30<nukke>I guess bandwidth and hardware costs would skyrocket
01:38:19<nukke>also I was talking to the owner of 0x0 and they said: "someone at facebook is getting paid to write a 0x0 uploader. i’m seeing a bunch of test uploads from their ipv6 nets"
01:39:24<nicolas17>what's 0x0?
01:39:38<nukke>popular file host https://0x0.st/
01:39:43<nicolas17>wtf
01:39:46<nukke>"the null pointer"
01:42:36<BlankEclair>now i wonder what they're uploading...
01:48:17<hexa->lol
01:48:22<hexa->poor 0x0
01:49:09<hexa->I think weechat-android defaults to 0x0.st as well?
01:49:57<nukke>I think it defaults to x0.at
01:54:48<hexa->easily confused :D
01:57:47<steering>BlankEclair: I mean, if you get a thrill from sending them without encryption, wouldn't you get an even bigger thrill by posting them publicly? Shame you can
01:57:56<steering>can't use imgur or twitter for that anymore :(
01:58:08<steering>or tumblr
02:04:00beardicus9 (beardicus) joins
02:08:59<nukke>BlankEclair: re: 0x0, "just test/health check files apparently. no idea what they’re up to"
02:26:54<hexa->probably seeding porn for kids again
02:26:58<hexa->/s
02:30:05pabs quits [Read error: Connection reset by peer]
02:31:04pabs (pabs) joins
02:32:21etnguyen03 quits [Client Quit]
02:34:19unknownsrc quits [Ping timeout: 260 seconds]
02:35:27etnguyen03 (etnguyen03) joins
02:38:40unknownsrc joins
02:43:49etnguyen03 quits [Remote host closed the connection]
03:13:26<steering>https://en.wikipedia.org/wiki/Morecambe_Bay_cockling_disaster cockling... cockler... cockles.
03:21:42<steering>postgresql link: "One coincidence is that we started seeing the first symptoms of this around the same time as libicu was updated with a security patch. However, postgres hasn’t been restarted and doesn’t reference the new version" hmm, effective security :D
03:44:36<nicolas17>https://mas.to/@Ask_aubry/115135835166066873
03:51:07<hexa->10/10 if the vows were a list of items
03:51:26<steering>also I like the last message in the thread: "This is a generic symptom of corruption." but the "tools to verify table and index consistency" can't detect it??
03:53:37<steering>uhhhhhhhhhhh
03:55:34<steering>how would that guy have expected his spouse not to notice that it sounded nothing like him :D
04:05:37<steering>Me and my wife had a pretty simple wedding, we didn't do any speeches or anything, just did the whole "I do" thing and then had a dinner in the church gym. According to some of the comments on that thread we should probably be divorced by now, 7 years later.
04:09:49<nicolas17>no
04:09:54<nicolas17>"not doing speeches" is valid
04:12:27<nicolas17>I don't have the reddit link handy
04:12:29<nicolas17>steering: https://cdn.discordapp.com/attachments/1065047083104608388/1412565405654057020/IMG_5927.png?ex=68b8c18c&is=68b7700c&hm=f3122fd89a917edf4e95724bf7fb17595b3c73aa2995656a5b2a29b503692835&
04:13:10<BlankEclair>i blame the patriarchy
04:14:35<BlankEclair>but like
04:14:49<BlankEclair>ughhhhhhhhhhhhh this is making me really appalled
04:16:22<@JAA>Here's the actual thread: https://old.reddit.com/r/relationship_advice/comments/1n6errf/i_35m_was_caught_using_ai_to_write_wedding_vows/
04:17:56<BlankEclair>at least this sucked the horniness out of me
04:18:32<nicolas17>>sucked
04:19:16<BlankEclair>oh my god
04:28:55<steering>nicolas17: erm... ok?
04:31:18<steering>I was referring to something along the lines of "my [ex-]husband's vows seemed off and it turned out he recycled them from his old emails and now we're divorced. it was a sign!"
04:45:54<nukke>netcat me ur nudes to my public IP'd printer
04:51:35steering makes it print "I'm a teapot" repeatedly instead
05:03:19HackMii quits [Remote host closed the connection]
05:03:19SootBect1 quits [Remote host closed the connection]
05:03:46HackMii (hacktheplanet) joins
05:05:19SootBector (SootBector) joins
06:17:42hexagonwin quits [Read error: Connection reset by peer]
06:18:57hexagonwin joins
06:27:47hexagonwin quits [Ping timeout: 258 seconds]
06:28:16hexagonwin joins
07:18:22SootBector quits [Remote host closed the connection]
07:18:22HackMii quits [Remote host closed the connection]
07:18:31HackMii (hacktheplanet) joins
07:19:22SootBector (SootBector) joins
07:29:54BearFortress quits [Read error: Connection reset by peer]
07:30:24BearFortress joins
08:05:25tzt quits [Read error: Connection reset by peer]
08:07:03tzt (tzt) joins
09:11:05zhongfu (zhongfu) joins
09:13:05<murb>make the display say: "PC LOAD LETTICE"
09:17:40Dada joins
09:21:37<masterx244|m>(giggle giggle)
09:25:39JTL quits [Ping timeout: 258 seconds]
09:35:32JTL (JTL) joins
10:14:00Umbire quits [Remote host closed the connection]
10:14:30Umbire (Umbire) joins
10:17:19wessel1512 quits [Quit: Ping timeout (120 seconds)]
10:27:04wessel1512 joins
10:42:28Dada quits [Remote host closed the connection]
10:52:46Exorcism|irc quits [Quit: Ping timeout (120 seconds)]
10:53:04DigitalDragons quits [Quit: Ping timeout (120 seconds)]
11:00:02Bleo182600722719623455222 quits [Quit: The Lounge - https://thelounge.chat]
11:02:50Bleo182600722719623455222 joins
11:10:15SootBector quits [Ping timeout: 255 seconds]
11:11:24SootBector (SootBector) joins
11:21:20DigitalDragons (DigitalDragons) joins
11:25:54Exorcism|irc (exorcism) joins
11:55:03SootBector quits [Remote host closed the connection]
11:56:16SootBector (SootBector) joins
12:52:07<steering>huh, cool. I just realized I apparently have a DVD-RAM drive.
12:53:04<steering>was trying to figure out why it says "RW" with "DVD+R DL" write below it (i.e. instead of DVD+RW), which I guess is just... a thing
12:59:27<steering>(was trying to figure out whether to keep that one or the drive that says DVD+ReWritable, lol)
13:27:04<Dango360>some pretty cool DVD-RAM vids:
13:27:05<Dango360>https://www.youtube.com/watch?v=ecH3OU0R4ls
13:27:08<Dango360>https://www.youtube.com/watch?v=U9aZuFOTHtQ
13:45:43<nukke>https://arstechnica.com/gadgets/2025/09/google-wont-have-to-sell-chrome-judge-rules/
13:46:56MrMcNuggets (MrMcNuggets) joins
13:56:34SootBector quits [Remote host closed the connection]
13:57:03Dango360 quits [Ping timeout: 258 seconds]
13:57:44SootBector (SootBector) joins
14:24:33<steering>well, bye Dango360. but yeah, those're good videos. god, 6 years ago?!
14:25:09<steering>TBH before those I knew nothing about DVD-RAM except having heard of it in passing
14:48:57SootBector quits [Ping timeout: 255 seconds]
14:50:45SootBector (SootBector) joins
15:12:21grill (grill) joins
15:13:49Umbire quits [Ping timeout: 260 seconds]
15:22:56Gadelhas5628737844 joins
15:30:18SootBector quits [Remote host closed the connection]
15:31:20SootBector (SootBector) joins
16:02:08Dada joins
16:25:47anonymoususer852 quits [Ping timeout: 258 seconds]
16:27:48anonymoususer852 (anonymoususer852) joins
16:32:51SootBector quits [Remote host closed the connection]
16:33:56SootBector (SootBector) joins
16:47:24<phillipsjk>I actually bought a pair to handle incremental backups until they get large enough to fill a DVD+R. In theory the write speed was faster than the USB 1.1 port in that machine. Never did get DVD backups working. I think I was foiled by buffer underrun protection taking up 2MB each time.
17:02:19nicolas17 quits [Ping timeout: 260 seconds]
17:04:04grill quits [Ping timeout: 260 seconds]
17:07:13nicolas17 joins
17:20:13<nstrom|m>Matrix.org is back
17:54:01<alexlehm>DVD-RAM was a pretty rare format, most drives were DVD+RW
17:57:55PAARCLiCKS quits [Quit: The Lounge - https://thelounge.chat]
18:04:53<steering>lmao security@charter.com sends a bounce, not that it doesn't exist, but that it doesn't allow external emails :D
18:05:24<steering>anyway nice of them to send out a please-click-on-phishing training to their customers from feedback@spectrumcustomersurvey.com
18:28:45<steering>hmm. accidentally plugged one of my keys into a USB port (i was plugging my yubikey into its C port and one of the other keys on the ring fell enough into the A port next to it i guess). computer didn't seem very happy. bluetooth (mouse) dropped and dmesg reported 21 over-current conditions in about 1.2s. :D
18:41:12MrMcNuggets quits [Quit: WeeChat 4.3.2]
19:04:29<phillipsjk>Unplugging everything, waiting 2 minutes, then plugging things back in should restore service. (This advice assumes you need to let a PTC fuse to cool down.)
19:09:16<masterx244|m>murphy's law striking at its finsest :(
19:16:18<anonymoususer852>Inb4 I jinx it, my Warrior 4.1 instances doesn't appear to be automatically rebooting on the scheduled 7 days uptime.
19:19:14<steering>phillipsjk: it started working again at approximately the same time as i removed the key from the usb port :P
19:20:25<steering>(actually for some reason the mouse's USB receiver won't work, but that's probably unrelated, as it shows up in lsusb and the mouse indicates it has established a connection to it)
19:26:48<phillipsjk>I think actual PTC devices are more of a SCSI ting.
19:28:18<phillipsjk>Apparently made the standard safe/fun to experiment with. (Was taken aback seeing SCSI ports on all the lab computers studying electronics in 2006)
19:31:22steering thinks back to the SCSI disk the he poofed back in high school
19:31:38<steering>protip: dont set drives with the pcb facing down on squishy conductive surfaces
19:33:13<phillipsjk>Even a non-conductive squishy surface is asking for trouble due to ESD.
19:46:44Dango360 (Dango360) joins
21:06:34<Guest>!kfind eclair
21:06:35<eggdrop>[karma] 3 matches for 'eclair': BlankEclair: '17' BlankEclair bonk: '6' BlankEclair book: '1'
21:06:57<Guest>whats with the bonk?
21:09:08<BlankEclair>Guest: let's just say that i've... been a _little_ bit horny
21:09:16<BlankEclair>the book was also because of that
21:09:37<Guest>seems like some of the others around here have been too
21:09:42<Guest>asking for your pics over fax
21:09:54<BlankEclair>steering: > <steering> BlankEclair: I mean, if you get a thrill from sending them without encryption, wouldn't you get an even bigger thrill by posting them publicly?
21:09:56<BlankEclair>here >~<
21:10:03<BlankEclair>https://catgirl.center/notes/ac816yc28t6s0vab
21:23:56<hexa->250ms latency :o
21:24:19<hexa->ah sure, nudes
21:26:22steering wonders whether its worth clicking
21:27:02<steering>if is joke: yes
21:27:21<steering>if is newd: thats what archivebot is for
21:27:22<steering>i mean wait what
21:31:04etnguyen03 (etnguyen03) joins
22:14:07Umbire (Umbire) joins
22:14:33Umbire is now known as RJHacker65774
22:14:39Umbire (Umbire) joins
22:18:29RJHacker65774 quits [Ping timeout: 260 seconds]
22:30:50<BlankEclair>steering: you meant mnbot
22:30:57<BlankEclair>unless you're !aing the direct image url
22:31:11<BlankEclair>but idt mnbot would reveal the cw without custom js
22:46:10Dada quits [Remote host closed the connection]
22:48:32Umbire quits [Client Quit]
22:48:51Umbire (Umbire) joins
22:54:42Umbire quits [Client Quit]
23:12:40<steering>I mean... if you say so
23:13:10<BlankEclair>aiwlubefiawfluiwabefwauibfawuiefweudiulwaefbuiwedbWEfiluwaefawjfeweljf
23:13:42<steering>I haven't used either bot \o/
23:16:27<BlankEclair>gg xD
23:17:41<BlankEclair>https://mnbot.very-good-quality-co.de/item/6832c3d1-8ac1-4ec0-a16f-9bba35e142ff
23:17:46<BlankEclair>bwalefbwealfawefuawuefuliawlibfe JAA already archived it
23:22:11nstrom quits [Quit: Ooops, wrong browser tab.]
23:22:16<nicolas17>weird keysmash
23:25:28<BlankEclair>how so?
23:25:33<BlankEclair>also, i archived the direct image url >~<
23:31:12etnguyen03 quits [Client Quit]